From majordomo@mil.doit.wisc.edu Mon Dec 1 00:57:47 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA26817 for ; Mon, 1 Dec 2003 00:57:47 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 1AQgiR-0007ay-00 for ipfix-list@mil.doit.wisc.edu; Sun, 30 Nov 2003 23:35:23 -0600 Received: from palrel10.hp.com ([156.153.255.245]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 1AQgiQ-0007at-00 for ipfix@net.doit.wisc.edu; Sun, 30 Nov 2003 23:35:22 -0600 Received: from xparelay2.ptp.hp.com (xparelay2.ptp.hp.com [15.1.28.65]) by palrel10.hp.com (Postfix) with ESMTP id 77D731C0180F; Sun, 30 Nov 2003 21:35:21 -0800 (PST) Received: from xpabh3.ptp.hp.com (xpabh3.ptp.hp.com [15.1.28.63]) by xparelay2.ptp.hp.com (Postfix) with ESMTP id 6E9081C00C04; Sun, 30 Nov 2003 21:35:21 -0800 (PST) Received: by xpabh3.ptp.hp.com with Internet Mail Service (5.5.2657.72) id ; Sun, 30 Nov 2003 21:35:21 -0800 Message-ID: <1758A044D46A8A4CB320429F9462D6C248F782@xsun03.ptp.hp.com> From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" To: "'Sebastian Zander'" , Falko Dressler Cc: ipfix@net.doit.wisc.edu Subject: RE: [ipfix] [issue] INFO-24 High-res time encoding Date: Sun, 30 Nov 2003 21:35:20 -0800 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2657.72) Content-Type: text/plain; charset="iso-8859-1" Precedence: bulk Sender: majordomo listserver Hi, The most widely supported (in terms of number of platforms) representation of time is that used by both J2SE and J2ME, defined by the java.util.Date object. Specifically the methods get and setTime() available in both environments interprets a 64-bit integer value as the number of milliseconds since January 1, 1970, 00:00:00 GMT. For thorough documentation of behavior see: http://java.sun.com/j2se/1.4.1/docs/api/java/util/Date.html - and the introduction describing GMT vs. UTC concerns (i.e. leap seconds) and getTime() and setTime() methods describe the usage of the 64-bit quantity. This integer value proves quite handy for doing direct millisecond granularity math, i.e. duration_msec = end_time_msec - start_time_msec. Compare this simple model which has no roll over for 10,000 years, with alternatives at second resolution or finer resolution. I think msec is the most practical. So, I would argue that this time format is the optimal for rollover free 64-bit time. I would agree that NTP format optimizes capacity of 64-bits for a wide variety of time, providing resolution as fine as .232 nanoseconds. I question the need for microsecond based resolution, but for completeness using timeval format for 64-bit microsecond resolution would be my recommended encoding. So I would propose encoding formats of: - dateTime (second resolution, using 32-bits representing unsigned seconds since 1970 EPOCH) - dateTimeMsec (msec resolution, using 64-bits represening unsinged msecs since 1970 EPOCH) - dateTimeUsec (usec resolution using 2 32-bit quantities representing seconds since 1970 EPOCH and another for the number of usec, i.e. range of 0-.999999) - dateTimeNsec (nsec resolution, using 2 32-bit quantities representing seconds since 1900 (not 1970) EPOCH and another 32-bit for the fractional component. I.e. the 32-bit value 0x00000001 = 1*2**-32. And the 32-bit value of 0xFFFFFFFF = (2**32 - 1) * 2**-32, assuming standard C language operator precedence. I still believe there is a desire among some WG participants to leverage per "flow group" compacting via the use of integral offsets from some time stamp stored in a common format in the header. E.g. header has dateTimeUsec format of some reference time and flows in that group specify times as 32-bit offset + or - from this value. If the group feels this tradeoff in complexity vs. compactness is worthwile (it may be) then I'm OK. However some consensus on default time encodings and addressing all four of the time resolutions shown above is necessary. Regards, Jeff Meyer -----Original Message----- From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu]On Behalf Of Sebastian Zander Sent: Wednesday, November 26, 2003 6:21 AM To: Falko Dressler Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] [issue] INFO-24 High-res time encoding Hi, common time formats are: - time_t: seconds since epoch (4 byte) - struct timeval: seconds, microseconds since epoch (8 byte) - struct timespec (POSIX): seconds, nanoseconds since epoch (8 byte) - NTP: seconds, 1/2^32 fraction since epoch (8 byte) Unix epoch is 1970 whereas NTP epoch is 1900. Therefore some conversion is needed to convert from NTP to one of the others. The size of the first 3 depend on the architecture. On my machine (Intel) they have the sizes above but on other platforms this may be different AFAIK on Alpha a long is 8 bytes. I'd like to see a timestamp format with second resolution which can be used where a higher granularity is not required (e.g. accounting) and saves some bytes on the wire. Another type with at least microsecond resolution should be available for cases where a higher granularity is required (e.g. delay measurement) NTP makes the most of the bits (in terms of resolution ;-) and is an IETF standard. Coversion to time_t etc. is simple. Cheers, Sebastian Falko Dressler wrote: > Issue: INFO-24 Encoding options for high-resolution timestamps > Description: > http://ipfix.doit.wisc.edu/archive/2214.html > > Various encoding options for high-resolution timestamps for recommended > or optional usage. > > Available types: > > * NTP format (8 byte) > 2 Parts: seconds (4 byte) / 1/2**32 fraction of seconds) > > * Posix timeval > 2 Parts: seconds (4 byte) / nanoseconds (4 byte) > > * Milliseconds sinch EPOCH (8 byte) > > * Micro/Nanoseconds since EPOCH (8 byte, rollover!) > > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message > body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > -- Sebastian Zander E-mail: zander@fokus.fraunhofer.de Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 D-10589 Berlin, Germany www.fokus.fraunhofer.de/usr/sebastian.zander -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Dec 3 17:19:40 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA12662 for ; Wed, 3 Dec 2003 17:19:39 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 1AReyc-0006NV-00 for ipfix-list@mil.doit.wisc.edu; Wed, 03 Dec 2003 15:56:06 -0600 Received: from dplonka by mil.doit.wisc.edu with local (Exim 3.13 #1) id 1AReyb-0006NO-00 for ipfix-info@net.doit.wisc.edu; Wed, 03 Dec 2003 15:56:05 -0600 Date: Wed, 3 Dec 2003 15:56:05 -0600 From: Dave Plonka To: ipfix-info@net.doit.wisc.edu Subject: [ipfix-info] I-D ACTION:draft-ietf-ipfix-info-02.txt Message-ID: <20031203155605.B23075@doit.wisc.edu> Reply-To: plonka@doit.wisc.edu Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0.1i X-Organization: University of Wisconsin-Madison, DoIT Network Services X-Organization-Too: Wisconsin Advanced Internet Laboratory (WAIL) X-URL: http://net.doit.wisc.edu/~plonka/ X-VMS-Error: %SYSTEM-E-PRIMNOSTP, PRIMARY CPU cannot be stopped X-Shakespearean-Insult: Thou surly ill-nurtured ratsbane Precedence: bulk Sender: majordomo listserver The new information model draft the Jeff mentioned is now available on the IETF web site as mentioned below. I've mirrored it here: http://ipfix.doit.wisc.edu/info/ Dave ----- Forwarded message from Internet-Drafts@ietf.org ----- To: IETF-Announce: ; Cc: ipfix@net.doit.wisc.edu From: Internet-Drafts@ietf.org Reply-to: Internet-Drafts@ietf.org Subject: I-D ACTION:draft-ietf-ipfix-info-02.txt Date: Wed, 03 Dec 2003 15:28:52 -0500 Precedence: bulk A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the IP Flow Information Export Working Group of the IETF. Title : Information Model for IP Flow Information Export Author(s) : P. Calato Filename : draft-ietf-ipfix-info-02.txt Pages : 42 Date : 2003-12-3 This document defines and information and data model for the IP Flow Information export (IPFIX) protocol. It is used by the IPFIX protocol for encoding measured traffic information and information related to the traffic measurement process. Although developed for the IPFIX protcol, the model is defined in an open way that easily allows using it in other protocols, interfaces, and applications. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-ipfix-info-02.txt To remove yourself from the IETF Announcement list, send a message to ietf-announce-request with the word unsubscribe in the body of the message. Internet-Drafts are also available by anonymous FTP. Login with the username "anonymous" and a password of your e-mail address. After logging in, type "cd internet-drafts" and then "get draft-ietf-ipfix-info-02.txt". A list of Internet-Drafts directories can be found in http://www.ietf.org/shadow.html or ftp://ftp.ietf.org/ietf/1shadow-sites.txt Internet-Drafts can also be obtained by e-mail. Send a message to: mailserv@ietf.org. In the body type: "FILE /internet-drafts/draft-ietf-ipfix-info-02.txt". NOTE: The mail server at ietf.org can return the document in MIME-encoded form by using the "mpack" utility. To use this feature, insert the command "ENCODING mime" before the "FILE" command. To decode the response(s), you will need "munpack" or a MIME-compliant mail reader. Different MIME-compliant mail readers exhibit different behavior, especially when dealing with "multipart" MIME messages (i.e. documents which have been split up into multiple messages), so check your local documentation on how to manipulate these messages. Below is the data which will enable a MIME compliant mail reader implementation to automatically retrieve the ASCII version of the Internet-Draft. ----- End forwarded message ----- -- plonka@doit.wisc.edu http://net.doit.wisc.edu/~plonka ARS:N9HZF Madison, WI -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Dec 8 00:11:38 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA13687 for ; Mon, 8 Dec 2003 00:11:38 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 1ATDAe-0006eM-00 for ipfix-list@mil.doit.wisc.edu; Sun, 07 Dec 2003 22:38:56 -0600 Received: from mailhost2.auckland.ac.nz ([130.216.1.4]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 1ATDAd-0006cm-00 for ipfix@net.doit.wisc.edu; Sun, 07 Dec 2003 22:38:55 -0600 Received: from mailhost.auckland.ac.nz (mailhost [130.216.191.61]) by mailhost2.auckland.ac.nz (8.12.9/8.12.9/8.12.9-ua) with ESMTP id hB84cmKU029351; Mon, 8 Dec 2003 17:38:48 +1300 (NZDT) Received: from localhost (mailhost.auckland.ac.nz [127.0.0.1]) by mailhost.auckland.ac.nz (Postfix) with ESMTP id 2A12133E7E; Mon, 8 Dec 2003 17:37:13 +1300 (NZDT) Received: from mailhost.auckland.ac.nz ([127.0.0.1]) by localhost (mailhost.auckland.ac.nz [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 12388-13; Mon, 8 Dec 2003 17:37:10 +1300 (NZDT) Received: from motoko.itss.auckland.ac.nz (motoko.itss.auckland.ac.nz [130.216.191.146]) by mailhost.auckland.ac.nz (Postfix) with ESMTP id 0E89733E8E; Mon, 8 Dec 2003 17:37:10 +1300 (NZDT) Received: (from apache@localhost) by motoko.itss.auckland.ac.nz (8.11.6/8.11.6) id hB84aNR28215; Mon, 8 Dec 2003 17:36:23 +1300 Received: from dyn42.caida.org (dyn42.caida.org [192.172.226.42]) by webmail.auckland.ac.nz (Horde) with HTTP for ; Mon, 8 Dec 2003 17:36:23 +1300 Message-ID: <1070858183.fa4a0a2cfe225@webmail.auckland.ac.nz> Date: Mon, 8 Dec 2003 17:36:23 +1300 From: Nevil Brownlee To: "MEYER,JEFFREY D (HP-Cupertino,ex1)" Cc: ipfix@net.doit.wisc.edu Subject: RE: [ipfix] [issue] INFO-24 High-res time encoding References: <1758A044D46A8A4CB320429F9462D6C248F782@xsun03.ptp.hp.com> In-Reply-To: <1758A044D46A8A4CB320429F9462D6C248F782@xsun03.ptp.hp.com> MIME-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Internet Messaging Program (IMP) 4.0-cvs X-Originating-IP: 192.172.226.42 X-Virus-Scanned: by amavisd-new at mailhost.auckland.ac.nz Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hi Jeff: Thanks for your description of the Java 64-bit ms since 1 Jan 1970, that does seem like a good compromise - to me, ms for flow times is about right (I need us for packets, but not flows), and I like its 'no rollover' feature. > So I would propose encoding formats of: > > - dateTime (second resolution, using 32-bits representing unsigned > seconds since 1970 EPOCH) > - dateTimeMsec (msec resolution, using 64-bits represening unsinged msecs > since 1970 EPOCH) > - dateTimeUsec (usec resolution using 2 32-bit quantities representing > seconds since 1970 EPOCH and another for the number of usec, i.e. range of > 0-.999999) > - dateTimeNsec (nsec resolution, using 2 32-bit quantities representing > seconds since 1900 (not 1970) EPOCH and another 32-bit for the fractional > component. I.e. the 32-bit value 0x00000001 = 1*2**-32. And the 32-bit > value of 0xFFFFFFFF = (2**32 - 1) * 2**-32, assuming standard C language > operator precedence. I agree we should support the first two, the others seem less useful to me. > I still believe there is a desire among some WG participants to leverage > per "flow group" compacting via the use of integral offsets from some time > stamp stored in a common format in the header. E.g. header has dateTimeUsec > format of some reference time and flows in that group specify times as > 32-bit offset + or - from this value. I can see that it would save space to do this. Would it be enough to have a dateTime (seconds) in the header, then 32-bit ms offsets from that in the flow groups? Cheers, Nevil ----------------------------------------------------------------------- Nevil Brownlee Director, Technology Development Phone: +64 9 373 7599 x88941 ITSS, The University of Auckland FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand ------------------------------------------------- This mail sent through University of Auckland http://www.auckland.ac.nz/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Dec 8 10:52:52 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA15560 for ; Mon, 8 Dec 2003 10:52:51 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 1ATNLB-0002J2-00 for ipfix-list@mil.doit.wisc.edu; Mon, 08 Dec 2003 09:30:29 -0600 Received: from tokyo.netlab.nec.de ([195.37.70.2] helo=tokyo.ccrle.nec.de) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 1ATNLA-0002Iw-00 for ipfix@net.doit.wisc.edu; Mon, 08 Dec 2003 09:30:28 -0600 Received: from venus.office (venus.office [10.1.1.11]) by tokyo.ccrle.nec.de (8.12.10/8.12.9) with ESMTP id hB8FUOCx080785 for ; Mon, 8 Dec 2003 16:30:24 +0100 (CET) Received: from ccrle.nec.de (molina.office [10.1.1.126]) by venus.office (Postfix on SuSE Linux eMail Server 3.0) with ESMTP id EC7A868361 for ; Mon, 8 Dec 2003 16:30:23 +0100 (CET) Message-ID: <3FD4990F.4000708@ccrle.nec.de> Date: Mon, 08 Dec 2003 16:30:23 +0100 From: Maurizio Molina User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.2) Gecko/20030208 Netscape/7.02 X-Accept-Language: en-us, en MIME-Version: 1.0 To: "'ipfix@net.doit.wisc.edu'" Subject: Re: [ipfix] [issue] INFO-24 High-res time encoding References: <1758A044D46A8A4CB320429F9462D6C248F782@xsun03.ptp.hp.com> <1070858183.fa4a0a2cfe225@webmail.auckland.ac.nz> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.35 Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Nevil Brownlee wrote: >Hi Jeff: > >Thanks for your description of the Java 64-bit ms since 1 Jan 1970, >that does seem like a good compromise - to me, ms for flow times >is about right (I need us for packets, but not flows), and I like its >'no rollover' feature. > > > >> So I would propose encoding formats of: >> >> - dateTime (second resolution, using 32-bits representing unsigned >>seconds since 1970 EPOCH) >> - dateTimeMsec (msec resolution, using 64-bits represening unsinged msecs >>since 1970 EPOCH) >> - dateTimeUsec (usec resolution using 2 32-bit quantities representing >>seconds since 1970 EPOCH and another for the number of usec, i.e. range of >>0-.999999) >> - dateTimeNsec (nsec resolution, using 2 32-bit quantities representing >>seconds since 1900 (not 1970) EPOCH and another 32-bit for the fractional >>component. I.e. the 32-bit value 0x00000001 = 1*2**-32. And the 32-bit >>value of 0xFFFFFFFF = (2**32 - 1) * 2**-32, assuming standard C language >>operator precedence. >> >> > >I agree we should support the first two, the others seem less useful to me. > Hi Nevil, I think we should be careful with excluding encodings for resolutions higher than the millisecond. As Sebastian pointed out in some previous mail, "Another type with at least microsecond resolution should be available for cases where a higher granularity is required (e.g. delay measurement)" Delay measurements are actually considered in the IPFIX applicability draft. Moreover, PSAMP is considering of using IPFIX as a transport, and PSAMP based applications are likely to include delay measurements. Also, at the last meeting there was a hummmm in favour of microseconds. I think this sentence from Jeff is the one that best captures the issue: - dateTimeUsec and Nsec - micro and nano second resolution are too exotic for the majority of my customers, but there appears to be interest from the group. All that said, I'd be in favour of also keeping the support of encodings for high time resolutions. Cheers, Maurizio > > > > >> I still believe there is a desire among some WG participants to leverage >>per "flow group" compacting via the use of integral offsets from some time >>stamp stored in a common format in the header. E.g. header has dateTimeUsec >>format of some reference time and flows in that group specify times as >>32-bit offset + or - from this value. >> >> > >I can see that it would save space to do this. Would it be enough to >have a dateTime (seconds) in the header, then 32-bit ms offsets from that >in the flow groups? > >Cheers, Nevil > >----------------------------------------------------------------------- > Nevil Brownlee Director, Technology Development > Phone: +64 9 373 7599 x88941 ITSS, The University of Auckland > FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand > > >------------------------------------------------- >This mail sent through University of Auckland >http://www.auckland.ac.nz/ > >-- >Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body >Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >"unsubscribe ipfix" in message body >Archive http://ipfix.doit.wisc.edu/archive/ > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Dec 8 12:03:03 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA18267 for ; Mon, 8 Dec 2003 12:03:02 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 1ATORy-0004V4-00 for ipfix-list@mil.doit.wisc.edu; Mon, 08 Dec 2003 10:41:34 -0600 Received: from smtp02.mrf.mail.rcn.net ([207.172.4.61]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 1ATORx-0004Uy-00 for ipfix@net.doit.wisc.edu; Mon, 08 Dec 2003 10:41:33 -0600 Received: from 207-237-36-98.c3-0.avec-ubr10.nyr-avec.ny.cable.rcn.com ([207.237.36.98] helo=osiris) by smtp02.mrf.mail.rcn.net with esmtp (Exim 3.35 #4) id 1ATORw-0004oC-00; Mon, 08 Dec 2003 11:41:32 -0500 Reply-To: From: "Carter Bullard" To: "'Maurizio Molina'" , Subject: RE: [ipfix] [issue] INFO-24 High-res time encoding Date: Mon, 8 Dec 2003 11:41:29 -0500 Organization: QoSient, LLC Message-ID: <5C8959A16A71B449AE793CF52FBBED661AB4FB@ptah.newyork.qosient.com> X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.4510 Importance: Normal In-Reply-To: <5C8959A16A71B449AE793CF52FBBED66251DCC@ptah.newyork.qosient.com> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Precedence: bulk Sender: majordomo listserver Gentle people, I have customers today that are using uSec granularity exclusively and we're moving to nSec granularity for some applications shortly. So, these resolutions are not exotic for my customers, they are the norm. Carter -----Original Message----- From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu] On Behalf Of Maurizio Molina Sent: Monday, December 08, 2003 10:30 AM To: 'ipfix@net.doit.wisc.edu' Subject: Re: [ipfix] [issue] INFO-24 High-res time encoding Nevil Brownlee wrote: >Hi Jeff: > >Thanks for your description of the Java 64-bit ms since 1 Jan 1970, >that does seem like a good compromise - to me, ms for flow times >is about right (I need us for packets, but not flows), and I like its >'no rollover' feature. > > > >> So I would propose encoding formats of: >> >> - dateTime (second resolution, using 32-bits representing unsigned >>seconds since 1970 EPOCH) >> - dateTimeMsec (msec resolution, using 64-bits represening unsinged msecs >>since 1970 EPOCH) >> - dateTimeUsec (usec resolution using 2 32-bit quantities representing >>seconds since 1970 EPOCH and another for the number of usec, i.e. range of >>0-.999999) >> - dateTimeNsec (nsec resolution, using 2 32-bit quantities representing >>seconds since 1900 (not 1970) EPOCH and another 32-bit for the fractional >>component. I.e. the 32-bit value 0x00000001 = 1*2**-32. And the 32-bit >>value of 0xFFFFFFFF = (2**32 - 1) * 2**-32, assuming standard C language >>operator precedence. >> >> > >I agree we should support the first two, the others seem less useful to me. > Hi Nevil, I think we should be careful with excluding encodings for resolutions higher than the millisecond. As Sebastian pointed out in some previous mail, "Another type with at least microsecond resolution should be available for cases where a higher granularity is required (e.g. delay measurement)" Delay measurements are actually considered in the IPFIX applicability draft. Moreover, PSAMP is considering of using IPFIX as a transport, and PSAMP based applications are likely to include delay measurements. Also, at the last meeting there was a hummmm in favour of microseconds. I think this sentence from Jeff is the one that best captures the issue: - dateTimeUsec and Nsec - micro and nano second resolution are too exotic for the majority of my customers, but there appears to be interest from the group. All that said, I'd be in favour of also keeping the support of encodings for high time resolutions. Cheers, Maurizio > > > > >> I still believe there is a desire among some WG participants to leverage >>per "flow group" compacting via the use of integral offsets from some time >>stamp stored in a common format in the header. E.g. header has dateTimeUsec >>format of some reference time and flows in that group specify times as >>32-bit offset + or - from this value. >> >> > >I can see that it would save space to do this. Would it be enough to >have a dateTime (seconds) in the header, then 32-bit ms offsets from that >in the flow groups? > >Cheers, Nevil > >----------------------------------------------------------------------- > Nevil Brownlee Director, Technology Development > Phone: +64 9 373 7599 x88941 ITSS, The University of Auckland > FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand > > >------------------------------------------------- >This mail sent through University of Auckland >http://www.auckland.ac.nz/ > >-- >Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body >Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >"unsubscribe ipfix" in message body >Archive http://ipfix.doit.wisc.edu/archive/ > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Dec 8 13:46:15 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA23630 for ; Mon, 8 Dec 2003 13:46:15 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 1ATQFZ-0007Ws-00 for ipfix-list@mil.doit.wisc.edu; Mon, 08 Dec 2003 12:36:53 -0600 Received: from mailhost2.auckland.ac.nz ([130.216.191.4]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 1ATQFX-0007Wl-00 for ipfix@net.doit.wisc.edu; Mon, 08 Dec 2003 12:36:52 -0600 Received: from mailhost.auckland.ac.nz (mailhost [130.216.191.61]) by mailhost2.auckland.ac.nz (8.12.9/8.12.9/8.12.9-ua) with ESMTP id hB8IahLQ005929; Tue, 9 Dec 2003 07:36:43 +1300 (NZDT) Received: from localhost (mailhost.auckland.ac.nz [127.0.0.1]) by mailhost.auckland.ac.nz (Postfix) with ESMTP id A411533F3F; Tue, 9 Dec 2003 07:34:57 +1300 (NZDT) Received: from mailhost.auckland.ac.nz ([127.0.0.1]) by localhost (mailhost.auckland.ac.nz [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 18673-07; Tue, 9 Dec 2003 07:34:45 +1300 (NZDT) Received: from motoko.itss.auckland.ac.nz (motoko.itss.auckland.ac.nz [130.216.191.146]) by mailhost.auckland.ac.nz (Postfix) with ESMTP id A0E7833F4C; Tue, 9 Dec 2003 07:34:45 +1300 (NZDT) Received: (from apache@localhost) by motoko.itss.auckland.ac.nz (8.11.6/8.11.6) id hB8IY4e01924; Tue, 9 Dec 2003 07:34:04 +1300 Received: from dyn42.caida.org (dyn42.caida.org [192.172.226.42]) by webmail.auckland.ac.nz (Horde) with HTTP for ; Tue, 9 Dec 2003 07:34:04 +1300 Message-ID: <1070908444.f24cd8b1bce6a@webmail.auckland.ac.nz> Date: Tue, 9 Dec 2003 07:34:04 +1300 From: Nevil Brownlee To: carter@qosient.com, "'Maurizio Molina'" Cc: ipfix@net.doit.wisc.edu Subject: RE: [ipfix] [issue] INFO-24 High-res time encoding References: <5C8959A16A71B449AE793CF52FBBED661AB4FB@ptah.newyork.qosient.com> In-Reply-To: <5C8959A16A71B449AE793CF52FBBED661AB4FB@ptah.newyork.qosient.com> MIME-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Internet Messaging Program (IMP) 4.0-cvs X-Originating-IP: 192.172.226.42 X-Virus-Scanned: by amavisd-new at mailhost.auckland.ac.nz Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hi Carter and Maurizio > I have customers today that are using uSec granularity > exclusively and we're moving to nSec granularity for some > applications shortly. So, these resolutions are not exotic > for my customers, they are the norm. OK, by all means let's keep all four of Jeff's timestamp formats. Now what about the notion of having a dateTimeUsec timestamp in the header and 32-bit offsets from that for timestamps in flow groups? Would that save enough space to be useful? CHeers, Nevil ----------------------------------------------------------------------- Nevil Brownlee Director, Technology Development Phone: +64 9 373 7599 x88941 ITSS, The University of Auckland FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand ------------------------------------------------- This mail sent through University of Auckland http://www.auckland.ac.nz/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Dec 8 14:12:25 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA25263 for ; Mon, 8 Dec 2003 14:12:24 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 1ATQew-0000lz-00 for ipfix-list@mil.doit.wisc.edu; Mon, 08 Dec 2003 13:03:06 -0600 Received: from mx5.informatik.uni-tuebingen.de ([134.2.12.32]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 1ATQev-0000lu-00 for ipfix@net.doit.wisc.edu; Mon, 08 Dec 2003 13:03:05 -0600 Received: from localhost (loopback [127.0.0.1]) by mx5.informatik.uni-tuebingen.de (Postfix) with ESMTP id 5678112B; Mon, 8 Dec 2003 20:03:03 +0100 (NFT) Received: from mx3.informatik.uni-tuebingen.de ([134.2.12.26]) by localhost (mx5 [134.2.12.32]) (amavisd-new, port 10024) with ESMTP id 18232-04; Mon, 8 Dec 2003 20:03:01 +0100 (NFT) Received: from informatik.uni-tuebingen.de (rouen.Informatik.Uni-Tuebingen.De [134.2.11.152]) by mx3.informatik.uni-tuebingen.de (Postfix) with ESMTP id AD5D1139; Mon, 8 Dec 2003 20:03:00 +0100 (NFT) Message-ID: <3FD4CAE4.5070808@informatik.uni-tuebingen.de> Date: Mon, 08 Dec 2003 20:03:00 +0100 From: Falko Dressler Organization: University of Tuebingen User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.5) Gecko/20031007 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Maurizio Molina Cc: "'ipfix@net.doit.wisc.edu'" Subject: Re: [ipfix] [issue] INFO-24 High-res time encoding References: <1758A044D46A8A4CB320429F9462D6C248F782@xsun03.ptp.hp.com> <1070858183.fa4a0a2cfe225@webmail.auckland.ac.nz> <3FD4990F.4000708@ccrle.nec.de> In-Reply-To: <3FD4990F.4000708@ccrle.nec.de> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new (McAfee AntiVirus) at informatik.uni-tuebingen.de Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit I think there *is* a consensus for keeping high resolution time stamps. The remaining question is about its encodings. Do we really need "a number of" different encodings or is it possible to find a consensus on a particular encoding? My favorite is still NTP ;-) Cheers, Falko. Maurizio Molina wrote: > > > Nevil Brownlee wrote: > >> Hi Jeff: >> >> Thanks for your description of the Java 64-bit ms since 1 Jan 1970, >> that does seem like a good compromise - to me, ms for flow times >> is about right (I need us for packets, but not flows), and I like its >> 'no rollover' feature. >> >> >> >>> So I would propose encoding formats of: >>> >>> - dateTime (second resolution, using 32-bits representing unsigned >>> seconds since 1970 EPOCH) >>> - dateTimeMsec (msec resolution, using 64-bits represening unsinged >>> msecs >>> since 1970 EPOCH) >>> - dateTimeUsec (usec resolution using 2 32-bit quantities representing >>> seconds since 1970 EPOCH and another for the number of usec, i.e. >>> range of >>> 0-.999999) >>> - dateTimeNsec (nsec resolution, using 2 32-bit quantities >>> representing >>> seconds since 1900 (not 1970) EPOCH and another 32-bit for the >>> fractional >>> component. I.e. the 32-bit value 0x00000001 = 1*2**-32. And the 32-bit >>> value of 0xFFFFFFFF = (2**32 - 1) * 2**-32, assuming standard C language >>> operator precedence. >>> >> >> >> I agree we should support the first two, the others seem less useful >> to me. >> > Hi Nevil, > I think we should be careful with excluding encodings for resolutions > higher than the millisecond. > As Sebastian pointed out in some previous mail, > "Another type with at least microsecond resolution should > be available for cases where a higher granularity is required (e.g. delay > measurement)" > Delay measurements are actually considered in the IPFIX applicability > draft. Moreover, PSAMP is considering of using IPFIX as a transport, > and PSAMP based applications are likely to include delay measurements. > Also, at the last meeting there was a hummmm in favour of microseconds. > > I think this sentence from Jeff is the one that best captures the issue: > > - dateTimeUsec and Nsec - micro and nano second resolution are too exotic > for the majority of my customers, but there appears to be interest from > the group. > > All that said, I'd be in favour of also keeping the support of encodings > for high time resolutions. > Cheers, > Maurizio > >> >> >> >> >>> I still believe there is a desire among some WG participants to >>> leverage >>> per "flow group" compacting via the use of integral offsets from some >>> time >>> stamp stored in a common format in the header. E.g. header has >>> dateTimeUsec >>> format of some reference time and flows in that group specify times as >>> 32-bit offset + or - from this value. >>> >> >> >> I can see that it would save space to do this. Would it be enough to >> have a dateTime (seconds) in the header, then 32-bit ms offsets from that >> in the flow groups? >> >> Cheers, Nevil >> >> ----------------------------------------------------------------------- >> Nevil Brownlee Director, Technology Development >> Phone: +64 9 373 7599 x88941 ITSS, The University of Auckland >> FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand >> >> >> ------------------------------------------------- >> This mail sent through University of Auckland >> http://www.auckland.ac.nz/ >> >> -- >> Help mailto:majordomo@net.doit.wisc.edu and say "help" in >> message body >> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >> "unsubscribe ipfix" in message body >> Archive http://ipfix.doit.wisc.edu/archive/ >> >> > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message > body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Dr.-Ing. Falko Dressler Computer Networks and Internet Wilhelm-Schickard-Institute for Computer Science University of Tuebingen, Germany Phone: +49 7071 29-70522 / Fax: +49 7071 29-5220 EMail: dressler@informatik.uni-tuebingen.de / fd@acm.org http://net.informatik.uni-tuebingen.de/members/dressler/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Dec 8 14:42:27 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA26409 for ; Mon, 8 Dec 2003 14:42:26 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 1ATR2q-0001Tm-00 for ipfix-list@mil.doit.wisc.edu; Mon, 08 Dec 2003 13:27:48 -0600 Received: from smtp02.mrf.mail.rcn.net ([207.172.4.61]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 1ATR2q-0001Tg-00 for ipfix@net.doit.wisc.edu; Mon, 08 Dec 2003 13:27:48 -0600 Received: from 207-237-36-98.c3-0.avec-ubr10.nyr-avec.ny.cable.rcn.com ([207.237.36.98] helo=osiris) by smtp02.mrf.mail.rcn.net with esmtp (Exim 3.35 #4) id 1ATR2p-0005Ok-00; Mon, 08 Dec 2003 14:27:47 -0500 Reply-To: From: "Carter Bullard" To: "'Nevil Brownlee'" , "'Maurizio Molina'" Cc: Subject: RE: [ipfix] [issue] INFO-24 High-res time encoding Date: Mon, 8 Dec 2003 14:27:43 -0500 Organization: QoSient, LLC Message-ID: <5C8959A16A71B449AE793CF52FBBED6607A734@ptah.newyork.qosient.com> X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.4510 Importance: Normal In-Reply-To: <5C8959A16A71B449AE793CF52FBBED66252FE0@ptah.newyork.qosient.com> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Precedence: bulk Sender: majordomo listserver Hey Nevil, This is an interesting question, as it assumes that all the records are coming from a single IPFIX data source. What happens if IPFIX is transporting data from multiple sources as they are received, but the timestamps in the various records are not synchronized? Carter -----Original Message----- From: Nevil Brownlee [mailto:n.brownlee@auckland.ac.nz] Sent: Monday, December 08, 2003 1:34 PM To: carter@qosient.com; 'Maurizio Molina' Cc: ipfix@net.doit.wisc.edu Subject: RE: [ipfix] [issue] INFO-24 High-res time encoding Hi Carter and Maurizio > I have customers today that are using uSec granularity > exclusively and we're moving to nSec granularity for some > applications shortly. So, these resolutions are not exotic > for my customers, they are the norm. OK, by all means let's keep all four of Jeff's timestamp formats. Now what about the notion of having a dateTimeUsec timestamp in the header and 32-bit offsets from that for timestamps in flow groups? Would that save enough space to be useful? CHeers, Nevil ----------------------------------------------------------------------- Nevil Brownlee Director, Technology Development Phone: +64 9 373 7599 x88941 ITSS, The University of Auckland FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand ------------------------------------------------- This mail sent through University of Auckland http://www.auckland.ac.nz/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Dec 8 17:22:23 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA07238 for ; Mon, 8 Dec 2003 17:22:22 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 1ATTZj-0005ad-00 for ipfix-list@mil.doit.wisc.edu; Mon, 08 Dec 2003 16:09:55 -0600 Received: from mailhost2.auckland.ac.nz ([130.216.1.4]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 1ATTZi-0005a4-00 for ipfix@net.doit.wisc.edu; Mon, 08 Dec 2003 16:09:54 -0600 Received: from mailhost.auckland.ac.nz (mailhost [130.216.191.61]) by mailhost2.auckland.ac.nz (8.12.9/8.12.9/8.12.9-ua) with ESMTP id hB8M9pLQ011577; Tue, 9 Dec 2003 11:09:52 +1300 (NZDT) Received: from localhost (mailhost.auckland.ac.nz [127.0.0.1]) by mailhost.auckland.ac.nz (Postfix) with ESMTP id 23AD933E7B; Tue, 9 Dec 2003 11:08:03 +1300 (NZDT) Received: from mailhost.auckland.ac.nz ([127.0.0.1]) by localhost (mailhost.auckland.ac.nz [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 08004-01; Tue, 9 Dec 2003 11:07:58 +1300 (NZDT) Received: from motoko.itss.auckland.ac.nz (motoko.itss.auckland.ac.nz [130.216.191.146]) by mailhost.auckland.ac.nz (Postfix) with ESMTP id 07BDD33F5C; Tue, 9 Dec 2003 11:07:55 +1300 (NZDT) Received: (from apache@localhost) by motoko.itss.auckland.ac.nz (8.11.6/8.11.6) id hB8M7Km03850; Tue, 9 Dec 2003 11:07:20 +1300 Received: from dyn42.caida.org (dyn42.caida.org [192.172.226.42]) by webmail.auckland.ac.nz (Horde) with HTTP for ; Tue, 9 Dec 2003 11:07:20 +1300 Message-ID: <1070921240.a6939d48ea695@webmail.auckland.ac.nz> Date: Tue, 9 Dec 2003 11:07:20 +1300 From: Nevil Brownlee To: carter@qosient.com Cc: ipfix@net.doit.wisc.edu Subject: RE: [ipfix] [issue] INFO-24 High-res time encoding References: <5C8959A16A71B449AE793CF52FBBED6607A734@ptah.newyork.qosient.com> In-Reply-To: <5C8959A16A71B449AE793CF52FBBED6607A734@ptah.newyork.qosient.com> MIME-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Internet Messaging Program (IMP) 4.0-cvs X-Originating-IP: 192.172.226.42 X-Virus-Scanned: by amavisd-new at mailhost.auckland.ac.nz Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hello Carter: > This is an interesting question, as it assumes that > all the records are coming from a single IPFIX data > source. What happens if IPFIX is transporting data > from multiple sources as they are received, but the > timestamps in the various records are not synchronized? Sounds like a good reason not to do it. That would simply mean that timestamps in an IPFIX record would all be in whichever of the four encodings the meter was configured to send, right? Cheers, Nevil ----------------------------------------------------------------------- Nevil Brownlee Director, Technology Development Phone: +64 9 373 7599 x88941 ITSS, The University of Auckland FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand ------------------------------------------------- This mail sent through University of Auckland http://www.auckland.ac.nz/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Dec 8 18:09:15 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA10658 for ; Mon, 8 Dec 2003 18:09:14 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 1ATUMP-00078j-00 for ipfix-list@mil.doit.wisc.edu; Mon, 08 Dec 2003 17:00:13 -0600 Received: from smtp02.mrf.mail.rcn.net ([207.172.4.61]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 1ATUMO-00078U-00 for ipfix@net.doit.wisc.edu; Mon, 08 Dec 2003 17:00:12 -0600 Received: from 207-237-36-98.c3-0.avec-ubr10.nyr-avec.ny.cable.rcn.com ([207.237.36.98] helo=osiris) by smtp02.mrf.mail.rcn.net with esmtp (Exim 3.35 #4) id 1ATUMN-0005ze-00; Mon, 08 Dec 2003 18:00:11 -0500 Reply-To: From: "Carter Bullard" To: "'Nevil Brownlee'" Cc: Subject: RE: [ipfix] [issue] INFO-24 High-res time encoding Date: Mon, 8 Dec 2003 18:00:04 -0500 Organization: QoSient, LLC Message-ID: <5C8959A16A71B449AE793CF52FBBED6607A735@ptah.newyork.qosient.com> X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.4510 Importance: Normal In-Reply-To: <5C8959A16A71B449AE793CF52FBBED66253645@ptah.newyork.qosient.com> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Precedence: bulk Sender: majordomo listserver I would hope so. Carter -----Original Message----- From: Nevil Brownlee [mailto:n.brownlee@auckland.ac.nz] Sent: Monday, December 08, 2003 5:07 PM To: carter@qosient.com Cc: ipfix@net.doit.wisc.edu Subject: RE: [ipfix] [issue] INFO-24 High-res time encoding Hello Carter: > This is an interesting question, as it assumes that > all the records are coming from a single IPFIX data > source. What happens if IPFIX is transporting data > from multiple sources as they are received, but the > timestamps in the various records are not synchronized? Sounds like a good reason not to do it. That would simply mean that timestamps in an IPFIX record would all be in whichever of the four encodings the meter was configured to send, right? Cheers, Nevil ----------------------------------------------------------------------- Nevil Brownlee Director, Technology Development Phone: +64 9 373 7599 x88941 ITSS, The University of Auckland FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand ------------------------------------------------- This mail sent through University of Auckland http://www.auckland.ac.nz/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Dec 8 21:44:13 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA21615 for ; Mon, 8 Dec 2003 21:44:13 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 1ATXWH-0004Vf-00 for ipfix-list@mil.doit.wisc.edu; Mon, 08 Dec 2003 20:22:37 -0600 Received: from palrel11.hp.com ([156.153.255.246]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 1ATXWG-0004Va-00 for ipfix@net.doit.wisc.edu; Mon, 08 Dec 2003 20:22:37 -0600 Received: from xparelay2.ptp.hp.com (xparelay2.ptp.hp.com [15.1.28.65]) by palrel11.hp.com (Postfix) with ESMTP id 438701C01F93 for ; Mon, 8 Dec 2003 18:22:36 -0800 (PST) Received: from xpabh3.ptp.hp.com (xpabh3.ptp.hp.com [15.1.28.63]) by xparelay2.ptp.hp.com (Postfix) with ESMTP id 39AE01C00AD3 for ; Mon, 8 Dec 2003 18:22:36 -0800 (PST) Received: by xpabh3.ptp.hp.com with Internet Mail Service (5.5.2657.72) id ; Mon, 8 Dec 2003 18:22:36 -0800 Message-ID: <1758A044D46A8A4CB320429F9462D6C248F7B4@xsun03.ptp.hp.com> From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" To: "'Ipfix Wg' (E-mail)" Subject: [ipfix] [issue] Information Model References in ipfix-protocol-01 Date: Mon, 8 Dec 2003 18:22:34 -0800 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2657.72) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C3BDFA.9E5F8B62" Precedence: bulk Sender: majordomo listserver This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C3BDFA.9E5F8B62 Content-Type: text/plain; charset="iso-8859-1" The current draft does not have pointers to the information model. Proposed Text: In section 3 "Terminology" add the term: Information Model - IPFIX has a formal description of IPFIX information elements (fields), their name, type and additional semantic information. A Flow Record is a collection of information elements defined in the information model. Templates identify information items by their Field Type identifier. The IPFIX Information Model is defined in [ipfix-info]. -- Jeff ------_=_NextPart_001_01C3BDFA.9E5F8B62 Content-Type: text/html; charset="iso-8859-1"

The current draft does not have pointers to the information model.

Proposed Text:

In section 3 "Terminology" add the term:

Information Model - IPFIX has a formal description of IPFIX information elements (fields), their name, type and additional semantic information. A Flow Record is a collection of information elements defined in the information model. Templates identify information items by their Field Type identifier. The IPFIX Information Model is defined in [ipfix-info].

-- Jeff

------_=_NextPart_001_01C3BDFA.9E5F8B62-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Dec 8 21:50:33 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA21740 for ; Mon, 8 Dec 2003 21:50:33 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 1ATXcw-0004kt-00 for ipfix-list@mil.doit.wisc.edu; Mon, 08 Dec 2003 20:29:30 -0600 Received: from atlrel6.hp.com ([156.153.255.205]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 1ATXcv-0004kn-00 for ipfix@net.doit.wisc.edu; Mon, 08 Dec 2003 20:29:29 -0600 Received: from xatlrelay1.atl.hp.com (xatlrelay1.atl.hp.com [15.45.89.190]) by atlrel6.hp.com (Postfix) with ESMTP id 1F5EB1C01965 for ; Mon, 8 Dec 2003 21:29:29 -0500 (EST) Received: from xatlbh2.atl.hp.com (xatlbh2.atl.hp.com [15.45.89.187]) by xatlrelay1.atl.hp.com (Postfix) with ESMTP id 1955C1C0009B for ; Mon, 8 Dec 2003 21:29:29 -0500 (EST) Received: by xatlbh2.atl.hp.com with Internet Mail Service (5.5.2657.72) id ; Mon, 8 Dec 2003 21:29:28 -0500 Message-ID: <1758A044D46A8A4CB320429F9462D6C248F7B5@xsun03.ptp.hp.com> From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" To: "'Ipfix Wg' (E-mail)" Subject: [ipfix] [issue] need for MTU size in protocol-01 Date: Mon, 8 Dec 2003 21:29:25 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2657.72) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C3BDFC.43204A46" Precedence: bulk Sender: majordomo listserver This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C3BDFC.43204A46 Content-Type: text/plain; charset="iso-8859-1" Issue: section 5.2.3.1 talks about concerns around IP fragmentation, but these should not be issues unless UDP is used. It may be recommended if *DP is to be used, that MTU size SHOULD be considered. But for SCTP and TCP, this doesn't seem like a useful constraint. Suggested wording: In situations where the underlying protocol may not directly address messages greater than MTU size, such as UDP, implementations MAY chose to restrict Export Packets to a size equal to or less than the local MTU size. In situations where multiple hops are taking place, both SCTP and TCP automatically manage concerns around IP fragmentation and since congestion aware protocols MUST be used, IP fragmentation should not be an issue. -- Jeff http://www.inetpix.com/dare ------_=_NextPart_001_01C3BDFC.43204A46 Content-Type: text/html; charset="iso-8859-1"

Issue: section 5.2.3.1 talks about concerns around IP fragmentation, but these should not be issues unless UDP is used. It may be recommended if *DP is to be used, that MTU size SHOULD be considered. But for SCTP and TCP, this doesn't seem like a useful constraint.

Suggested wording:

In situations where the underlying protocol may not directly address messages greater than MTU size, such as UDP,

implementations MAY chose to restrict Export Packets to a size equal to or less than the local MTU size. In situations where multiple hops are taking place, both SCTP and TCP automatically manage concerns around IP fragmentation and since congestion aware protocols MUST be used, IP fragmentation should not be an issue.

-- Jeff

http://www.inetpix.com/dare

------_=_NextPart_001_01C3BDFC.43204A46-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Dec 8 22:54:24 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA24658 for ; Mon, 8 Dec 2003 22:54:23 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 1ATYnu-0006nW-00 for ipfix-list@mil.doit.wisc.edu; Mon, 08 Dec 2003 21:44:54 -0600 Received: from palrel10.hp.com ([156.153.255.245]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 1ATYnt-0006nQ-00 for ipfix@net.doit.wisc.edu; Mon, 08 Dec 2003 21:44:53 -0600 Received: from xparelay2.ptp.hp.com (xparelay2.ptp.hp.com [15.1.28.65]) by palrel10.hp.com (Postfix) with ESMTP id 6780A1C02ACB for ; Mon, 8 Dec 2003 19:44:52 -0800 (PST) Received: from xpabh2.ptp.hp.com (xpabh2.ptp.hp.com [15.1.28.61]) by xparelay2.ptp.hp.com (Postfix) with ESMTP id 5A0781C000A7 for ; Mon, 8 Dec 2003 19:44:52 -0800 (PST) Received: by xpabh2.ptp.hp.com with Internet Mail Service (5.5.2657.72) id ; Mon, 8 Dec 2003 19:44:52 -0800 Message-ID: <1758A044D46A8A4CB320429F9462D6C248F7B9@xsun03.ptp.hp.com> From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" To: "'Ipfix Wg' (E-mail)" Subject: [ipfix] [issue] templates vs. option templates Date: Mon, 8 Dec 2003 19:44:47 -0800 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2657.72) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C3BE06.CA050F1A" Precedence: bulk Sender: majordomo listserver This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C3BE06.CA050F1A Content-Type: text/plain; charset="iso-8859-1" Issue: Collapse concept of templates vs. option templates to a single flag in order to simplify protocol and be more consistent in terms of information model usage. See also: http://ipfix.doit.wisc.edu/archive/2148.html Proposal Overview: proto-01 defines in section 8.2 a Template FlowSet. Later in section 9.1 an Options Template FlowSet is defined. The semantic difference between the two mechanisms is the concept of a template being for an "option" and the concept of "scoping". It is proposed that these two qualities be represented by a flag in the protocol and appropriate annotation in the Information model respectively. This enhanced consistency will make the discussion of the basic IPFIX protocol simpler and the subsequent extension of its information model more straightforward. Specific Wording (proto-01): (sec. 8.2): +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |O| FlowSet ID = 0 | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |R| Template ID 1 | Field Count | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Field Type 1 | Field Length 1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Field Type 2 | Field Length 2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ... | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Field Type N | Field Length N | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |R| Template ID 2 | Field Count | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ... O The "O" flag indicates if this set of templates describes options information. Options information are special FlowSet records which describe metadata about the flow meter. Options records have special "scoping" information items which must be specified as the first set of fields. ... R Template ID's are 15 bits long. The high order bit is reserved to distinguish Option templates from basic flow information templates. See also section 8.3. (sec 8.2) 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |O| FlowSet ID = Template ID | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Record 1 - Field Value 1 | Record 1 - Field Value 2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Record 1 - Field Value 3 | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Record 2 - Field Value 1 | Record 2 - Field Value 2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Record 2 - Field Value 3 | ... | ... O The "O" flag indicates if this set of records contains options information. Options information are special FlowSet records which describe metadata about the flow meter. Options records have special "scoping" information items which must be specified as the first set of fields in the template message. Logically there are two sets of template identifiers, those for basic flow information and those for meta data (options). See also section 8.2. (section 9) Remove all. Note that some details are transferred to the Information Model described below. Specific Wording (info-02) Add section 7 "Flow Option Attributes" 7 Flow Attribute Options In the process of metering and observation point, it may prove useful to provide details not just about the individual metered flows, but also details about the metering process itself or various aggregate counters maintained by the meter. Such values can allow for basic observation of system status, and coarser grained information about the overall set of flows passing by the observation point. The following sections identify the various meta flow details which are relevant as a separate information model. Both models are extensible, but it is assumed that each may have unique rates of extension and modification, making their segregation useful. Information items which have the identifier semantic should always appear at the beginning of an option template decalaration. These act to scope the details presented. 7.1 system Description: describe details... Semantics: identifier. Field Id: 1 7.2 interface Description: describe details... Semantics: identifier. Field Id: 2 7.3 lineCard Description: describe details... Semantics: identifier. Field Id: 4 7.4 cache Description: describe details... Semantics: identifier. Field Id: 4 7.5 template Description: describe details... Semantics: identifier. Field Id: 5 7.6 numFlows Description: describe details... Semantics: counter Field Id: 6 7.7 numPackets Description: describe details... Semantics: counter Field Id: 7 7.8 numBytes Description: describe details... Semantics: counter Field Id: 8 ------_=_NextPart_001_01C3BE06.CA050F1A Content-Type: text/html; charset="iso-8859-1"

Issue: Collapse concept of templates vs. option templates to a single flag in order to simplify protocol and be more consistent in terms of information model usage.

Proposal Overview:

proto-01 defines in section 8.2 a Template FlowSet. Later in section 9.1 an Options Template FlowSet is defined. The semantic difference between the two mechanisms is the concept of a template being for an "option" and the concept of "scoping".

It is proposed that these two qualities be represented by a flag in the protocol and appropriate annotation in the Information model respectively. This enhanced consistency will make the discussion of the basic IPFIX protocol simpler and the subsequent extension of its information model more straightforward.

Specific Wording (proto-01):

(sec. 8.2):

   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |O|     FlowSet ID = 0          |          Length               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |R|     Template ID 1           |         Field Count           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |        Field Type 1           |         Field Length 1        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |        Field Type 2           |         Field Length 2        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |             ...               |              ...              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |        Field Type N           |         Field Length N        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |R|     Template ID 2           |         Field Count           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

...

The "O" flag indicates if this set of templates describes options information. Options information are special FlowSet records which describe metadata about the flow meter. Options records have special "scoping" information items which must be specified as the first set of fields.

...

Template ID's are 15 bits long. The high order bit is reserved to distinguish Option templates from basic flow information templates. See also section 8.3.

(sec 8.2)

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |O| FlowSet ID = Template ID   |          Length               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |   Record 1 - Field Value 1    |   Record 1 - Field Value 2    |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |   Record 1 - Field Value 3    |             ...               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |   Record 2 - Field Value 1    |   Record 2 - Field Value 2    |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |   Record 2 - Field Value 3    |             ...               |

...

The "O" flag indicates if this set of records contains options information. Options information are special FlowSet records which describe metadata about the flow meter. Options records have special "scoping" information items which must be specified as the first set of fields in the template message. Logically there are two sets of template identifiers, those for basic flow information and those for meta data (options). See also section 8.2.

(section 9) Remove all. Note that some details are transferred to the Information Model described below.

Specific Wording (info-02)

Add section 7 "Flow Option Attributes"

7 Flow Attribute Options

In the process of metering and observation point, it may prove useful to provide details not just about the individual metered flows, but also details about the metering process itself or various aggregate counters maintained by the meter.

Such values can allow for basic observation of system status, and coarser grained information about the overall set of flows passing by the observation point.

The following sections identify the various meta flow details which are relevant as a separate information model. Both models are extensible, but it is assumed that each may have unique rates of extension and modification, making their segregation useful.

Information items which have the identifier semantic should always appear at the beginning of an option template decalaration. These act to scope the details presented.

7.1 system