Subject: RE: [ipfix] General comments on the IPFIX drafts

From: "Carter Bullard" <[email-masked]>
Date: Thu, 17 Jul 2003 10:53:12 -0400

Hey Olav,
   We've used uSec precision UTC in Argus IP flow
records for over 8 years now, and the customer base
is pushing us to use nSec precision, especially
when measuring at contemporary network rates.
Today, Argus is used by some to measure intra-LAN
network events, such as single hop one-way delay
and RTT.  At common speeds (10/100 Mbps eth and
802.11a/b), these events are in the 20-100 uSec
time domains.

   Epoch time is not a good idea, as it is pretty
much impossible to correlate records with any
other event that is generated by any other application,
and eliminates any benefit that vendors could have
if they were using GPS to sync up devices.

   At 10Gbps, interpacket arrival times can easily
be in the order of once every 0.0000000768 seconds
(48 byte packets, 50% link utilization), and at
OC-768, we're looking at network flow behavior that
manifests itself well below uSec granularity.
since there are so many 1 packet flows out there
in the big bad Internet, it would be nice to at least
be able to sort flows in arrival order.  At least
that is my opinion.

   While widely available clocks are just barely accurate
at this level, consider that even Windows 2000 provides
a nanoSecond clock to kernel and applications, so it's
a simple metric and should be readily available.  The
cost of goods for a crystal that can provide nSec level
precision is around $2.5 US, so switch and router
manufacturers should have no quarrel with the requirement,
and the Unix timeval data structure, has enough space
to hold the value.

Carter

> -----Original Message-----
> From: majordomo listserver 
> [mailto:[email-masked]] On Behalf Of 
> [email-masked]
> Sent: Thursday, July 17, 2003 8:46 AM
> To: MEYER,JEFFREY D (HP-Cupertino,ex1)
> Cc: 'Nevil Brownlee'; [email-masked]
> Subject: Re: [ipfix] General comments on the IPFIX drafts 
> 
> 
> 
> [email-masked] said:
> >  Ng1: Timestamp precision:
> >      flowCreationTime and flowEndTime, in -info draft, are 
> in dateTime
> >      units, i.e. 'seconds normalized to GMT' (why not to UTC?).
> >      Is this sufficient precision?  
> 
> [email-masked] said:
> >   Ng-1:  I would suspect that second resolution time is insufficient
> > for
> >      some applications. 
> 
> A large proportions of flow will be lasting in the order of a second.
> I would like to measure performance per flow to diagnose the
> traffic at an observation point. 
> 
> 
> [email-masked] said:
> > Millisecond resolution represented as a 64-bit
> >      integer counter since EPOCH is a type currently 
> defined by IPDR.
> 
> At Gigabits per second there could be millions of packets a second. 
> User response times should be subsecond (csec) and our measures one
> order less = msec.
> 
> Since some devices have GPS clocks aming for microsec 
> timestamps is not
> that utopic.
> 
> A solution to the accuracy problem could be to give the length of
> the flow in microsecs explisitely in a own field, as precision
> of the internal clock could be better than the clock accuracy ?
> (would keep reporting from wondering when changing to DST ?).
> 
> For other reports a msec precision flowCreationTime  could be ok
> which is roughly about the accuracy of NTP.
> flowEndTime (if needed) can be when the record was concluded 
> (inlcuding 
> timeout)
> and the length of the flow could be computed by time of the 
> last packet ?
> 
> 
> regards
> Olav
> 
> > Nevil,
> > 
> >   Some comments on the first three issues:
> > 
> >    Ng-1:  I would suspect that second resolution time is 
> insufficient for
> >      some applications.  Millisecond resolution represented 
> as a 64-bit
> >      integer counter since EPOCH is a type currently 
> defined by IPDR.
> > 
> >       Having this information item encoded explicitly would 
> certainly
> >      be more convenient from a collection (and info 
> modelling) perspective
> >      than the current NF model, which is relative to boot time.
> > 
> >    Ng-2:  Having the protocol spec define the mappings from the base
> >      type space in the info model makes sense.  Network 
> byte ordering
> >      would be my expectation for fixed width quantities.  For the 
> >      variable length fields, there seemed to be some 
> discussion around
> >      the use of 1,2 or 4 byte length indicators.   There 
> also remains
> >      the question of whether or not padding is used for any sort of
> >      word alignment.
> > 
> >    Ng-3:  The namespace URI would be worth advancing.  
> RFC3470 which talks
> >      about use of XML in IETF drafts has some discussion in 
> sections 4.9
> >      and 4.17.  Although it sounds like policy here is still being 
> >      defined.  Let me know if you would like me to contact 
> the authors
> >      to better understand the current state of affairs w/ IANA or if
> >      you would like to do this.
> > 
> > 
> > Regards,
> > 
> >   Jeff Meyer
> > 
> > 
> > -----Original Message-----
> > From: Nevil Brownlee [mailto:[email-masked]]
> > Sent: Monday, July 14, 2003 1:15 AM
> > To: [email-masked]
> > Subject: [ipfix] General comments on the IPFIX drafts
> > 
> > 
> > 
> > Hi all:
> > 
> > These are some comments from me covering issues which don't 
> fit neatly
> > into one or other of the IPFIX drafts.  At this point we 
> shouldn't be
> > raising 'big' new issues; I hope these are small-scale enough for us
> > to reach consensus on fairly quickly!
> > 
> > Cheers, Nevil
> > 
> > 
> --------------------------------------------------------------
> ---------
> >    Nevil Brownlee                   Director, Technology Development
> >    Phone: +64 9 373 7599 x88941     ITSS, The University of Auckland
> >    FAX: +64 9 373 7021      Private Bag 92019, Auckland, New Zealand
> > 
> > 
> > 
> > Ng1: Timestamp precision:
> >      flowCreationTime and flowEndTime, in -info draft, are 
> in dateTime
> >      units, i.e. 'seconds normalized to GMT' (why not to UTC?).
> >      Is this sufficient precision?  For example, RTFM used 
> SNMP TimeTicks,
> >      i.e. centiseconds (of sysUpTime).
> >      There are three questions here:
> >      a) How many bytes do we really need?  If seconds is 
> precise enough,
> >         4 bytes is fine.  If we need more, or should we 
> could go to 8?
> >      b) If we feel that we can afford 8-byte precision, 
> what time base
> >         should we use, sysUpTime or time of day.  Time of 
> day would mean
> >         that collectors didn't have to convert from 
> sysUpTime.  Can we
> >         assume that Metering Processes will have a reliable 
> time-of-day
> >         clock?
> >      c) Format of 8-byte times.  The most common Unix one 
> is (sec, usec).
> >         (sec, nsec) is also widely available.  My favourite is
> >         (sec, binary fraction of sec) [NTP 64-bit timestamp, Section
> >         3.1 of RFC 1305], because you
> >         can compute time differences with a single 64-bit 
> subtraction.
> > 
> > Ng2: Data representation in IPFIX packets.
> >      The info model defines the data types for each field 
> (IPFIX attribute),
> >      I think we need the protocol document to specify how 
> they're mapped
> >      into the packet, e.g. 'data values are placed in their 
> fields within
> >      the packet in network byte order, i.e. 
> most-significant byte first.'
> >      Which data types can be variable-length?
> >      If - as I suspect - it's only strings, could we agree 
> to only have
> >      variable-length strings (i.e. no fixed-length strings)?
> > 
> > Ng3: IANA Namespace for XML Namespace URIs.  WG Chairs 
> (probably Nevil)
> >      will follow up on this for the Info Model I-D.
> > 
> > Ng4: Flow Sampling.  This is mentioned in both the Requirements I-D
> >      and the AS I-D.  We need to decide how it should be 
> covered in the
> >      IPFIX drafts.
> > 
> > 
> ----------------------------------------------------------------------
> > 
> > 
> > -------------------------------------------------
> > This mail sent through University of Auckland
> > http://www.auckland.ac.nz/
> > 
> > --
> > Help        mailto:[email-masked] and say 
> "help" in message
> > body
> > Unsubscribe mailto:[email-masked] and say
> > "unsubscribe ipfix" in message body
> > Archive     http://ipfix.doit.wisc.edu/archive/
> > 
> > --
> > Help        mailto:[email-masked] and say 
> "help" in message body
> > Unsubscribe mailto:[email-masked] and say
> > "unsubscribe ipfix" in message body
> > Archive     http://ipfix.doit.wisc.edu/archive/
> 
> 
> 
> 
> 
> --
> Help        mailto:[email-masked] and say "help" 
> in message body
> Unsubscribe mailto:[email-masked] and say
> "unsubscribe ipfix" in message body
> Archive     http://ipfix.doit.wisc.edu/archive/
> 

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: "Fulvio Risso" <[email-masked]>
Date: Fri, 18 Jul 2003 07:38:29 +0200

Hi.

> From: majordomo listserver [mailto:[email-masked]]On Behalf
> Of Carter Bullard
> Sent: giovedi 17 luglio 2003 16.53
> To: [email-masked]; 'MEYER,JEFFREY D (HP-Cupertino,ex1)'
> Subject: RE: [ipfix] General comments on the IPFIX drafts
>
>
> Hey Olav,

Yes, but the cost for getting a timestamp with nanoSec precision is several
microseconds.
So, it is pretty much useless.
There is no way to get precise nanoseconds in PCs, unless you have a GPS
receiver.

In any case, I agree with you that the standard should specify this kind of
precision, no matter of how precixe can be workstations and routers in
getting the timestamp.

Cheers,

	fulvio