Subject: RE: [ipfix] [issue] INFO-24 High-res time encoding

From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" <[email-masked]>
Date: Tue, 9 Dec 2003 02:57:29 -0500

Resend to list...

-----Original Message-----
From: MEYER,JEFFREY D (HP-Cupertino,ex1) 
Sent: Monday, December 08, 2003 11:57 PM
To: 'Mark Fullmer'
Subject: RE: [ipfix] [issue] INFO-24 High-res time encoding

Mark,

  You're right.  It's too late at night and microseconds give you a
reasonable window.

  So the two IPFIX information types would be:

  dateTimeUsec - signed microsecond reslolution, yielding a +/- 2147 second
window in IPFIX encoding model.

  dateTimeNsec - nanosecond resolution, encoded using 64-bit NTP format with
about 136 year rollover.

  Both formats have a relatively narrow (on a geologic scale) time window.  

  Java's 64-bit dateTimeMsec format increases the 64-bit rollover interval
to +/- 292M years approximately.  I.e. A range of 2**64 msec, treating this
as a signed quantity means it should roll over in  +/-
(10**-3)*(2**63)/60/60/24/365 ~= 292,471,208.

  So as a general purpose time representation, dateTimeMsec has some
desirable properties.  But it may be that time for IPFIX is typically sent
in dateTimeUsec or dateTimeNsec, but is persisted or transformed to
dateTimeMsec for further processing, dependent on application.  It still may
make sense to define dateTimeMsec encoding according to Java time.  

  I would agree that basic dateTime (second resolution) could logically be
transmitted using the dateTimeUsec encoding and simply stripping off the
subsecond resolution.  So there may be no need to define an encoding for
this in IPFIX, but it would satisfy completeness.

  How about recommending the use of dateTimeUsec or dateTimeNsec, but
defining encodings for all four resolutions?

  In particular, the default flowCreationTime and flowEndTime would be
redefined in the current information model as dateTimeUsec.

-- Jeff

-----Original Message-----
From: Mark Fullmer [mailto:[email-masked]]
Sent: Monday, December 08, 2003 11:19 PM
To: MEYER,JEFFREY D (HP-Cupertino,ex1)
Cc: ''Ipfix Wg' (E-mail)'
Subject: Re: [ipfix] [issue] INFO-24 High-res time encoding

I think you're off by one SI unit, or I'm sending e-mail too late at
night...

Millisecond : 10^-3 seconds
Microsecond : 10^-6 seconds
Nanosecond  : 10^-9 seconds

2^32 * 10^-3 = 4294967 seconds
2^32 * 10^-6 = 4294 seconds
2^32 * 10^-9 = 4 seconds

I'm not sure running down the list of flows per export packet to fix
the timestamps is that big a deal, but it probably doesn't matter.  An
implementation that wants the full range can put the header/sentinel
time in the middle of the range.

mark

On Tue, Dec 09, 2003 at 01:58:04AM -0500, MEYER,JEFFREY D (HP-Cupertino,ex1)
wrote:
> Hi,
> 
>   My preference would be milliseconds with a signed quantity.
> 
>   This gives you about +/- 2147 seconds on either side of the sentinel
time.
> If you do not use a signed offset it forces a flow record to be adjusted
> "backward" so that all time offsets are positive (could be tricky for
> FlowCreationTime).
> 
>   Using microseconds would yield only +/- 2.14 seconds which is a pretty
> narrow window for grouping flows.
> 
> 
> -- Jeff
> 
> -----Original Message-----
> From: Mark Fullmer [mailto:[email-masked]]
> Sent: Monday, December 08, 2003 10:18 PM
> To: MEYER,JEFFREY D (HP-Cupertino,ex1)
> Cc: ''Ipfix Wg' (E-mail)'
> Subject: Re: [ipfix] [issue] INFO-24 High-res time encoding
> 
> 
> Microsecond might be okay...
> 
> mark
> 
> On Mon, Dec 08, 2003 at 10:04:48PM -0800, MEYER,JEFFREY D
(HP-Cupertino,ex1)
> wrote:
> > OK,
> > 
> >   I'll get it right yet...
> > 
> >   It should say for dateTimeMsec   "...32-bit integer count of (signed)
> >   millisecond offset... [not microsecond]"

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: Maurizio Molina <[email-masked]>
Date: Tue, 09 Dec 2003 12:52:27 +0100

>as a signed quantity means it should roll over in  +/-
I agree with supporting this basic 32 bit encoding.

>defining encodings for all four resolutions?
For the other 3 "high-res" 64 bit discussed encoding schemes, the 
"desirable" characteristics I saw discussed so far were
1) Allowed maximum resolution
2) possibility to "split" into 32bit (header) + 32 bit (flow record) so 
that to save space in export msg
3) maximum flow lifetime

According to these 3 creteria dateTimeNsec is certainly better than 
dateTimeUsec.
dateTimeNsec is better than dateTimeMsec  for  the first two, worse for 
the third. But I think that a maximum flow duration time of 292Myears is 
not really an advantage wrt a maximum flow duration of 136 years, while 
I see a flow duration of 2147 secs as a limitation.
My conclusion would be that on this basis there is no advantage in 
supporting also dateTimeMsec and dateTimeUsec.
However, there's of course a fourth factor, which is the difficulty of 
transforming internal time representations in that format. This 
difficulty depends on the specific HW/SW of the exporter/collector and 
this may be a good reason for supporting also the other two formats.
Therefore, I think IPFIX should support them only if somebody loudly 
says "yes, I want them because with my implementation it's easier". That 
would be a good reason. Even in that case, my preference would be to 
specify dateTimeNsec as the default in the info model.
Maurizio

---

From: Sebastian Zander <[email-masked]>
Date: Tue, 09 Dec 2003 12:52:27 +0100

Jeff,

some questions on the delta timestamps:

Why should the microsecond deltas be _signed_?

What happens when the flow is longer than 2147 seconds? Is a new flow
started? Is there a timestamp refreshment e.g. every x export includes
an absolute (sentinel) timestamp or is there an absolute timestamp
if the exporters detects the duration is > 2147?

What happens in case of unreliable transport when packets with
sentinel time get lost?

In my view delta timestamp support seems to be a bit more than just simple
encoding and I don't fully understand how it works now.

At the moment I tend to agree in keeping all four encodings.

Cheers,

Sebastian

-- 
Sebastian Zander                         E-mail: [email-masked]
Fraunhofer FOKUS / METEOR                Tel: +49-30-3463-7287
Kaiserin-Augusta-Allee 31                Fax: +49-30-3463-8287
D-10589 Berlin, Germany                  www.fokus.fraunhofer.de/usr/sebastian.zander

---

From: Mark Fullmer <[email-masked]>
Date: Tue, 9 Dec 2003 13:53:24 -0500

How do you envision an exporter implementing "split time?"

Foreach flow the metering could use a unix timeval structure internally,
ie tv_sec and tv_usec.  Then when a collection of flows is ready for
export the exporter would need to run down the list of flows, pick
a suitable "header time" then adjust all the flow Start and End times
to the correct usec offset.

This seems like a lot of extra work compared to the existing NetFlow
method where the flow times are not offsets and no special work
needs to be done during export.

Do you see an easier way to implement this?

mark

---

From: Maurizio Molina <[email-masked]>
Date: Wed, 10 Dec 2003 16:58:44 +0100

> 2) possibility to "split" into 32bit (header) + 32 bit (flow record) 
> so that to save space in export msg
> 3) maximum flow lifetime
>

I went through this thread once again, and I realize that this my 
statement is clearly incorrect, as dateTimeNsec (NTP encoding) cannot be 
split  between the header and the flow record. Then it is a good reason 
to support all 4 types of encoding.
Maurizio