Subject: RE: [ipfix] iesg comment on draft-ietf-ipfix-reqs-09.txt

From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" <[email-masked]>
Date: Fri, 11 Apr 2003 09:24:55 -0700

Hi,

  I'm not rehashing this, simply pointing out that a reliable
transport is not the same thing as guaranteed delivery.  For
some reason people don't seem to accept this.  Don't take my
word for it, see RFC2975 section 2.1.5 "Application acknowledgements".

  Since NF/IPFIX as defined today is unidirectional, the server
has NO way of knowing what may have been lost in transit.

  One can wish or pretend this isn't a problem, but it just
won't go away.

  For some reasons, providers who bill on this accounting information
like to have a higher level of confidence that things don't just
"fall on the floor".

  Note: I'm resigned to the fact that IPFIX will not enable this
capability, and one could argue that this is a different class
of accounting (or telemetry), and shouldn't be bothered with
such details.  Fine.  My only point is that there is a significant
market out there that does care about this, and RADIUS and Diameter
are too high overhead to address.  So a middle ground between
IPFIX's apparent charter and the AAA groups is needed.  Hence, IPDR.

  Alternatively, this could be flagged as "issue #1" in the 
joint work on creating an IPFIX protocol.

Regards,

  Jeff Meyer

> -----Original Message-----
> From: Mark Fullmer [mailto:[email-masked]]
> Sent: Thursday, April 10, 2003 9:15 PM
> To: MEYER,JEFFREY D (HP-Cupertino,ex1)
> Cc: [email-masked]
> Subject: Re: [ipfix] iesg comment on draft-ietf-ipfix-reqs-09.txt
> 
> 
> Why are we rehashing this again?  NetFlow v9 can run over a 
> reliable transport
> such as TCP or SCTP.  The only part that's missing is a way 
> for the collector
> to inform the exporter at failover time what flows to resend, 
> which _must_
> be an optional feature for the exporter.
> 
> Consider:
> 
>   Exporter sends a "I expect a hello every N seconds" to the 
> collector 
>   as an option, using the existing NetFlow v9 option template.
> 
>   Collector sees the above message and ACK's with the highest sequence
>   number PDU it has processed every N seconds for each 
> observation domain.
>   Collector could even use the same packet format.
> 
>   If the exporter doesn't get an ACK in N+holdown seconds it 
> switches to
>   the secondary and retransmits PDU's that have not been ACK'd, iff it
>   has the resources to buffer them.
> 
> There's no negotiation, this feature is entirely controlled 
> by the exporter
> (routers probably won't send this option, stand alone probes that have
>  the resources probably will), and it's done within the 
> current framework
> of NetFlow v9.
> 
> NetFlow v9 != IPFIX.  There will be some _small_ changes.
> 
> The burden is going to be on the collector to support 
> multiple transports,
> a reliable transport would allow something like above, an unreliable
> transport obviously won't.
> 
> mark
> 
> On Tue, Apr 08, 2003 at 07:47:58PM -0400, MEYER,JEFFREY D 
> (HP-Cupertino,ex1) wrote:
> > Hi,
> > 
> > As to the last point, (accounting requirements), if we are 
> talking about
> > carrier grade accounting for billing purposes, then definitely NFv9
> > continues
> > to have the same problems as all previous versions, i.e. if 
> a packet is lost
> > in transit it's lost, too bad.  The fact that many carriers 
> do bill with NF
> > doesn't mean they are happy about this behavior.
> > 
> > So, I do think it is important to point this out.  Since the lack of
> > addressing
> > this requirement implies IPDR will need to go ahead 
> independently to address
> > this important requirement.
> > 
> > Also I'd include references to the two RFC's I mentioned 
> earlier in the
> > references section as they include some useful discussions of prior
> > learnings
> > in this area:
> >   
> >     RFC2975 - Introduction to Accounting Management
> >     RFC2924 - Accounting Attributes and Record Formats
> > 
> > Regards,
> > 
> >   Jeff Meyer
> > 
> > > -----Original Message-----
> > > From: Juergen Quittek [mailto:[email-masked]]
> > > Sent: Friday, April 04, 2003 8:07 AM
> > > To: Randy Bush; [email-masked]
> > > Subject: Re: [ipfix] iesg comment on draft-ietf-ipfix-reqs-09.txt
> > > 
> > > 
> > > Hi all,
> > > 
> > > I think we got many good comments. Most of them can be 
> > > committed quickly.
> > > However, there are some that might require some discussion:
> > > 
> > >   - Does the Appendix need to be moved up in the document?
> > >   - Shall we be more clear on byte counter, multicast 
> > > replication factor,
> > >     and BGP AS#s?
> > >   - Do we agree that we need to add a statement on that 
> > > accounting requirements
> > >     can only be met with higher reliability?
> > > 
> > > I would answer 'no' on all three questions.  Please see some 
> > > more detailed
> > > statement on the comments below.
> > > 
> > >     Juergen
> > > -- 
> > > Juergen Quittek        [email-masked]        Tel: +49 
> > > 6221 90511-15
> > > NEC Europe Ltd.,       Network Laboratories        Fax: +49 
> > > 6221 90511-55
> > > Kurfuersten-Anlage 36, 69115 Heidelberg, Germany   
> > > http://www.ccrle.nec.de
> > > 
> > > 
> > > -- Randy Bush wrote on 03 April 2003 09:01 -0800:
> > > >
> > > > comments from iesg reviewers
> > > >
> > > > randy
> > > >
> > > > ---
> > > >
> > > > I see:
> > > >    5.4.  Timestamps
> > > >
> > > >      The metering process MUST be able to generate 
> > > timestamps for the
> > > >      first and the last observation of a packet of a flow at the
> > > >      observation point. The timestamp resolution MUST be at 
> > > least the one
> > > >      of the sysUpTime [RFC1213], which is one centisecond.
> > > >
> > > > I think I'd rather see a reference to RFC3418 instead 
> of RFC1213.
> > > > We can do so by RFC-Editor note. May want to check with 
> authors too.
> > > 
> > > Certainly we should follow this suggestion.
> > > 
> > > > On page 14 I see:
> > > >
> > > >
> > > >       9. type of service octet (in case of IPv4), traffic class
> > > >          octet (in case of IPv6)
> > > >
> > > > Is that not better state as
> > > >
> > > >       9. DSCP, which is set in the type of service octet 
> > > (in case of IPv4),
> > > >          or in the traffic class octet (in case of IPv6)
> > > 
> > > We discussed this before: According to RFC 2474, the DSCP 
> > > covers only 6 bit
> > > of the type of service octet, and in some cases you want to 
> > > see the entire byte.
> > > 
> > > >
> > > > ---
> > > >
> > > >    I would prefer to see the paragraph referencing RFC 2119 
> > > in section 2.
> > > 
> > > Makes sense.
> > > 
> > > >    Last paragraph in section 6.3.3 needs to be reworded.
> > > 
> > > Let's try to find another wording.
> > > 
> > > >    In section 10.2, an important point is left unsaid.  If 
> > > the injection of
> > > > IPFIX traffic fool the network provided into thinking that 
> > > a DOS attack is
> > > > underway, the countermeasures employed by the network 
> > > provider may actually
> > > > deny service to real customers.
> > > 
> > > Good point. We should discuss this threat also in some 
> > > additional text.
> > > 
> > > >    The appendix needs to be moved up in the document.
> > > 
> > > Can anybody explain why?
> > > 
> > > > ---
> > > >
> > > > Overall, I like it.  A couple of things jumped out at 
> me, though.
> > > > This document goes to a lot of trouble to say exactly what is
> > > > exported (e.g. what fields, etc), except for two cases:
> > > >
> > > >       8. byte counter
> > > >          Which bytes of a packet are counted MUST be 
> > > defined exactly.
> > > >
> > > >      20. multicast replication factor
> > > >          the number of outgoing packets originating 
> from a single
> > > >          incoming multicast packet. This is a dynamic 
> property of
> > > >          multicast flows, that may change over time. For 
> > > unicast flows
> > > >          it has the constant value 1. The computation of 
> > > the factor MUST
> > > >          be clearly defined.
> > > >
> > > > This document is defining what fields are being reported on;
> > > > why can't it also define these two things that must also be
> > > > defined?  Is it useful to have different ipfix 
> protocols exporting
> > > > different measurements?
> > > 
> > > This is a requirements document.  It is a preparation for defining
> > > a single IPFIX standard protocol. Therefore, I do not see 
> a problem in
> > > leaving some (potentially tricky) issues to be defined by the 
> > > protocol.
> > > 
> > > Also, this list was intended to name and briefly explain the 
> > > attributes.
> > > For the multicast replication factor, a definition takes 
> > > several more words:
> > > It is a dynamic property. What would be its value, if during 
> > > the time since
> > > the last report there were five outgoing packets per incoming 
> > > packet and
> > > just before the report was made, one receiver disconnected?
> > > Would still 4 be correct for this report? Should it be 4.96? Or 5?
> > > We has this discussion and found no requirement to prefer one 
> > > or the other.
> > > 
> > > >
> > > > Also, at the end of 6.1 it kind of says "Plus other stuff 
> > > related to inter-AS
> > > > routing if you want".  It's already in the MAY section; 
> > > can't they just
> > > > list some concrete items related to inter-AS routing as 
> 27, 28, ...?
> > > > I don't think it's useful to say what amounts to "plus 
> > > anything else that
> > > > you might feel like adding that you think is relevant", 
> > > since that doesn't
> > > > encourage different people to make the same choices.
> > > 
> > > Well, the statement was not THAT general.  When discussing 
> > > the BGP issues, we
> > > found that source AS#, destination AS#, previous hop AS#, and 
> > > next hop AS# could
> > > be useful. Shall we list all of them as 27, 28, 29, 30?
> > > 
> > > > ---
> > > >
> > > > This is a good document overall.
> > > >
> > > > The applications include usage-based accounting.  They 
> > > should cite RFC 2975,
> > > > and indicate that the particular niche for usage-based 
> > > accounting would not
> > > > be met if reliability was not very high.  Section 5.1 
> > > reliability requirements
> > > > do not meet the needs, nor do the timestamping of first and 
> > > last packet of
> > > > a flow, because the flow may have been mischaracterized and 
> > > that is first and
> > > > last of different flows.  The way to satisfy this issue is 
> > > to caveat the
> > > > discussion of usage-based accounting with strong words on 
> > > reliability that
> > > > counter the comments on lesser reliability and more 
> > > probabilistic techniques
> > > > for other applications of ipfix.
> > > 
> > > Citing RFC 2975 is fine.
> > > 
> > > But the WG could not agree on a statement, that this niche 
> > > would not be met
> > > if reliability was not very high. We found non-neglectible 
> > > counter-examples.
> > > 
> > > I think there will be consensus on a statement like "Some 
> > > accounting applications
> > > require high reliability".
> > > 
> > > > Requirements on the ipfix implementation - the document is 
> > > long and I wonder if
> > > > the working group really meant the protocol's 
> > > confidentiality and anonymization
> > > > features to be so optional -  SHOULD confidentiality, MAY 
> > > anonymization.
> > > > Just for implementation.
> > > 
> > > Sorry, what is the proposal here?
> > > 
> > >     Juergen
> > > 
> > > > -30-
> > > >
> > > >
> > > > --
> > > > Help        mailto:[email-masked] and say 
> > > "help" in message body
> > > > Unsubscribe mailto:[email-masked] and say
> > > > "unsubscribe ipfix" in message body
> > > > Archive     http://ipfix.doit.wisc.edu/archive/
> > > 
> > > 
> > > 
> > > --
> > > Help        mailto:[email-masked] and say "help" 
> > > in message body
> > > Unsubscribe mailto:[email-masked] and say
> > > "unsubscribe ipfix" in message body
> > > Archive     http://ipfix.doit.wisc.edu/archive/
> > > 
> > 
> > --
> > Help        mailto:[email-masked] and say 
> "help" in message body
> > Unsubscribe mailto:[email-masked] and say
> > "unsubscribe ipfix" in message body
> > Archive     http://ipfix.doit.wisc.edu/archive/
> 

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: Mark Fullmer <[email-masked]>
Date: Sat, 12 Apr 2003 19:40:25 -0400

You're referring to in-flight data that the exporter has sent with a write()/sendto()
that the collector hasn't processed yet, correct?  The issue being the exporter
can not know the data has made it to the collector without application level ACK's?

My read of the bidirectional argument stemmed from people wanting IPFIX to include
the ability to configure the export and metering process.

Assuming a reliable transport that guarantees in-order delivery of records,
do you feel an optional periodic keep-alive from the collector which includes
the packet sequence number as an indicator of processed (stored to disk, whatever)
data sufficient?  This doesn't preclude an implementation from not using the
keep-alive's and/or using an unreliable transport.

mark