Subject: [ipfix] Flow timestamps.

From: Mark Fullmer <[email-masked]>
Date: Tue, 18 Nov 2003 14:14:57 -0500

NetFlow pre v9 provided for millisecond flow start and end timestamps 
by using a
combination of a 32 bit timestamp in milliseconds since device boot and 
a 96 bit
synchronization record that ties the device boot time to real time.

Effectively for each export packet you have in the header.

   u_int32 sysUpTime;	/* Current time in millisecs since router booted */
   u_int32 unix_secs;	/* Current seconds since 0000 UTC 1970 */
   u_int32 unix_nsecs; 	/* Residual nanoseconds since 0000 UTC 1970 */

Then for each flow:

     u_int32 First;      /* SysUptime at start of flow */
     u_int32 Last;       /* and of last packet of flow */

Note that sysUpTime here is in milliseconds and not centiseconds like 
SNMP.
Why the unix timestamp is in nanoseconds not milliseconds I don't know.

The advantage of doing this is you get millisecond resolution 
timestamps with
32 bit fields in the flow.  With 30 flows/packet (NetFlow V5 over UDP 
w. 1500
byte packets) this saves a bunch of overhead.  The cost is a small 
calculation
on the exporter side to get "real time" timestamps.  Another potential 
benefit
is the real time timestamps added on the exporter side to each packet.  
This
can be useful in post processing flows, but it's probably good enough 
to add
this on the collector side.

NetFlow v9 / the IPFIX draft broke this by not including the unix_nsecs 
in
the packet header, so we now only have 1 second real time resolution.

At the WG meeting the consensus was for microsecond timestamps.  The 
above
procedure doesn't work too well for this because the counters wrap too 
fast.

I see we have a few choices.

   1) Resurrect the nanosecond (change it to millisecond) field.

   2) Change all timestamps to 64 bits.

   3) Allow for both.  The exporter gets to decide, collectors must be 
able
      to decode both types.

For an export PDU with 30 flows this saves 8*30=240 (extra bits for 64 
bit
timestamps) - 12 (bits for synch) = 228 bytes per PDU.  The cost is the
extra calculation at the collector side and millisecond resolution.

With a TCP/SCTP transport there's also the potential of not sending
the 96 bit synch message for each PDU since the collector now has 
connection
state.

We're trying to get the protocol draft moved along and this is one of 
the
issues that needs resolved.  Comments?

For the curious:

/*
  * function: ftltime
  *
  * Flow exports represent time with a combination of uptime of the
  * router, real time, and offsets from the router uptime.  ftltime
  * converts from the PDU to a standard unix seconds/milliseconds
  * representation
  *
  * returns: struct fttime
  */
struct fttime ftltime(u_int32 sys, u_int32 secs, u_int32 nsecs, u_int32 
t)
{

   u_int32 sys_s, sys_m;
   struct fttime ftt;

   /* sysUpTime is in milliseconds, convert to seconds/milliseconds */
   sys_s = sys / 1000;
   sys_m = sys % 1000;

   /* unix seconds/nanoseconds to seconds/milliseconds */
   ftt.secs = secs;
   ftt.msecs = nsecs / 1000000L;

   /* subtract sysUpTime from unix seconds */
   ftt.secs -= sys_s;

   /* borrow a second? */
   if (sys_m > ftt.msecs) {
     -- ftt.secs;
     ftt.msecs += 1000;
   }
   ftt.msecs -= sys_m;

   /* add offset which is in milliseconds */
   ftt.secs += t / 1000;
   ftt.msecs += t % 1000;

   /* fix if milliseconds >= 1000 */
   if (ftt.msecs >= 1000) {
     ftt.msecs -= 1000;
     ftt.secs += 1;
   }

   return ftt;

} /* ftltime */

mark

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/