Subject: [ipfix] General comments on the IPFIX drafts

From: Nevil Brownlee <[email-masked]>
Date: Mon, 14 Jul 2003 20:14:35 +1200

Hi all:

These are some comments from me covering issues which don't fit neatly
into one or other of the IPFIX drafts.  At this point we shouldn't be
raising 'big' new issues; I hope these are small-scale enough for us
to reach consensus on fairly quickly!

Cheers, Nevil

-----------------------------------------------------------------------
   Nevil Brownlee                   Director, Technology Development
   Phone: +64 9 373 7599 x88941     ITSS, The University of Auckland
   FAX: +64 9 373 7021      Private Bag 92019, Auckland, New Zealand

Ng1: Timestamp precision:
     flowCreationTime and flowEndTime, in -info draft, are in dateTime
     units, i.e. 'seconds normalized to GMT' (why not to UTC?).
     Is this sufficient precision?  For example, RTFM used SNMP TimeTicks,
     i.e. centiseconds (of sysUpTime).
     There are three questions here:
     a) How many bytes do we really need?  If seconds is precise enough,
        4 bytes is fine.  If we need more, or should we could go to 8?
     b) If we feel that we can afford 8-byte precision, what time base
        should we use, sysUpTime or time of day.  Time of day would mean
        that collectors didn't have to convert from sysUpTime.  Can we
        assume that Metering Processes will have a reliable time-of-day
        clock?
     c) Format of 8-byte times.  The most common Unix one is (sec, usec).
        (sec, nsec) is also widely available.  My favourite is
        (sec, binary fraction of sec) [NTP 64-bit timestamp, Section
        3.1 of RFC 1305], because you
        can compute time differences with a single 64-bit subtraction.

Ng2: Data representation in IPFIX packets.
     The info model defines the data types for each field (IPFIX attribute),
     I think we need the protocol document to specify how they're mapped
     into the packet, e.g. 'data values are placed in their fields within
     the packet in network byte order, i.e. most-significant byte first.'
     Which data types can be variable-length?
     If - as I suspect - it's only strings, could we agree to only have
     variable-length strings (i.e. no fixed-length strings)?

Ng3: IANA Namespace for XML Namespace URIs.  WG Chairs (probably Nevil)
     will follow up on this for the Info Model I-D.

Ng4: Flow Sampling.  This is mentioned in both the Requirements I-D
     and the AS I-D.  We need to decide how it should be covered in the
     IPFIX drafts.

----------------------------------------------------------------------

-------------------------------------------------
This mail sent through University of Auckland
http://www.auckland.ac.nz/

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: "Carter Bullard" <[email-masked]>
Date: Fri, 18 Jul 2003 02:23:51 -0400

Buon Giorno Fulvio,
   Millie gracie!!!  Precision is not really the number
one thing here, IMHO.  The real issue is that you need a
non-recurring counter that can be used to order
network events.  This could be accomplished using a
simple packet counter, but if you go that route,
you lose the additional properties that you get
from basing the counter on time.

   If you're microsecond delay in getting to the clock
is a constant, then the number is valid for this ordering
purpose.  It may not be sync'd but its good enough to
do nSec interpacket arrival times, and good enough to
indicate that the ICMP packet came a few 100 nSecs
after the TCP RST packet in the stream.

   At least one vendor is shipping nSec GPS sync'd
packet capture devices today.  A little pricey, yes,
but if you want to do good stuff, you have to spend
some money.  IPFIX should at least support the current
state of the art in flow record generation and reporting.

   Hope all is well!!! Ciao!

Carter

> -----Original Message-----
> From: majordomo listserver 
> [mailto:[email-masked]] On Behalf Of Fulvio Risso
> Sent: Friday, July 18, 2003 1:38 AM
> To: Carter Bullard; [email-masked]; 'MEYER,JEFFREY D 
> (HP-Cupertino,ex1)'
> Cc: 'Nevil Brownlee'; [email-masked]
> Subject: RE: [ipfix] General comments on the IPFIX drafts 
> 
> 
> Hi.
> 
> > -----Original Message-----
> > From: majordomo listserver 
> [mailto:[email-masked]]On Behalf
> > Of Carter Bullard
> > Sent: giovedi 17 luglio 2003 16.53
> > To: [email-masked]; 'MEYER,JEFFREY D (HP-Cupertino,ex1)'
> > Cc: 'Nevil Brownlee'; [email-masked]
> > Subject: RE: [ipfix] General comments on the IPFIX drafts
> >
> >
> > Hey Olav,
> >    We've used uSec precision UTC in Argus IP flow
> > records for over 8 years now, and the customer base
> > is pushing us to use nSec precision, especially
> > when measuring at contemporary network rates.
> > Today, Argus is used by some to measure intra-LAN
> > network events, such as single hop one-way delay
> > and RTT.  At common speeds (10/100 Mbps eth and
> > 802.11a/b), these events are in the 20-100 uSec
> > time domains.
> >
> >    Epoch time is not a good idea, as it is pretty
> > much impossible to correlate records with any
> > other event that is generated by any other application,
> > and eliminates any benefit that vendors could have
> > if they were using GPS to sync up devices.
> >
> >    At 10Gbps, interpacket arrival times can easily
> > be in the order of once every 0.0000000768 seconds
> > (48 byte packets, 50% link utilization), and at
> > OC-768, we're looking at network flow behavior that
> > manifests itself well below uSec granularity.
> > since there are so many 1 packet flows out there
> > in the big bad Internet, it would be nice to at least
> > be able to sort flows in arrival order.  At least
> > that is my opinion.
> >
> >    While widely available clocks are just barely accurate
> > at this level, consider that even Windows 2000 provides
> > a nanoSecond clock to kernel and applications, so it's
> 
> Yes, but the cost for getting a timestamp with nanoSec 
> precision is several
> microseconds.
> So, it is pretty much useless.
> There is no way to get precise nanoseconds in PCs, unless you 
> have a GPS
> receiver.
> 
> In any case, I agree with you that the standard should 
> specify this kind of
> precision, no matter of how precixe can be workstations and routers in
> getting the timestamp.
> 
> Cheers,
> 
> 	fulvio
> 
> > a simple metric and should be readily available.  The
> > cost of goods for a crystal that can provide nSec level
> > precision is around $2.5 US, so switch and router
> > manufacturers should have no quarrel with the requirement,
> > and the Unix timeval data structure, has enough space
> > to hold the value.
> >
> >
> > Carter
> >
> >
> >
> > > -----Original Message-----
> > > From: majordomo listserver
> > > [mailto:[email-masked]] On Behalf Of
> > > [email-masked]
> > > Sent: Thursday, July 17, 2003 8:46 AM
> > > To: MEYER,JEFFREY D (HP-Cupertino,ex1)
> > > Cc: 'Nevil Brownlee'; [email-masked]
> > > Subject: Re: [ipfix] General comments on the IPFIX drafts
> > >
> > >
> > >
> > > [email-masked] said:
> > > >      Is this sufficient precision?
> > >
> > > [email-masked] said:
> > > >   Ng-1:  I would suspect that second resolution time is 
> insufficient
> > > > for
> > > >      some applications.
> > >
> > > A large proportions of flow will be lasting in the order 
> of a second.
> > > I would like to measure performance per flow to diagnose the
> > > traffic at an observation point.
> > >
> > >
> > > [email-masked] said:
> > > > Millisecond resolution represented as a 64-bit
> > > >      integer counter since EPOCH is a type currently
> > > defined by IPDR.
> > >
> > > At Gigabits per second there could be millions of packets 
> a second.
> > > User response times should be subsecond (csec) and our 
> measures one
> > > order less = msec.
> > >
> > > Since some devices have GPS clocks aming for microsec
> > > timestamps is not
> > > that utopic.
> > >
> > > A solution to the accuracy problem could be to give the length of
> > > the flow in microsecs explisitely in a own field, as precision
> > > of the internal clock could be better than the clock accuracy ?
> > > (would keep reporting from wondering when changing to DST ?).
> > >
> > > For other reports a msec precision flowCreationTime  could be ok
> > > which is roughly about the accuracy of NTP.
> > > flowEndTime (if needed) can be when the record was concluded
> > > (inlcuding
> > > timeout)
> > > and the length of the flow could be computed by time of the
> > > last packet ?
> > >
> > >
> > > regards
> > > Olav
> > >
> > > > Nevil,
> > > >
> > > >   Some comments on the first three issues:
> > > >
> > > >    Ng-1:  I would suspect that second resolution time is
> > > insufficient for
> > > >      some applications.  Millisecond resolution represented
> > > as a 64-bit
> > > >      integer counter since EPOCH is a type currently
> > > defined by IPDR.
> > > >
> > > >       Having this information item encoded explicitly would
> > > certainly
> > > >      be more convenient from a collection (and info
> > > modelling) perspective
> > > >      than the current NF model, which is relative to boot time.
> > > >
> > > >    Ng-2:  Having the protocol spec define the mappings 
> from the base
> > > >      type space in the info model makes sense.  Network
> > > byte ordering
> > > >      would be my expectation for fixed width 
> quantities.  For the
> > > >      variable length fields, there seemed to be some
> > > discussion around
> > > >      the use of 1,2 or 4 byte length indicators.   There
> > > also remains
> > > >      the question of whether or not padding is used for 
> any sort of
> > > >      word alignment.
> > > >
> > > >    Ng-3:  The namespace URI would be worth advancing.
> > > RFC3470 which talks
> > > >      about use of XML in IETF drafts has some discussion in
> > > sections 4.9
> > > >      and 4.17.  Although it sounds like policy here is 
> still being
> > > >      defined.  Let me know if you would like me to contact
> > > the authors
> > > >      to better understand the current state of affairs 
> w/ IANA or if
> > > >      you would like to do this.
> > > >
> > > >
> > > > Regards,
> > > >
> > > >   Jeff Meyer
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: Nevil Brownlee [mailto:[email-masked]]
> > > > Sent: Monday, July 14, 2003 1:15 AM
> > > > To: [email-masked]
> > > > Subject: [ipfix] General comments on the IPFIX drafts
> > > >
> > > >
> > > >
> > > > Hi all:
> > > >
> > > > Help        mailto:[email-masked] and say
> > > "help" in message
> > > > body