From majordomo@mil.doit.wisc.edu Thu May 1 11:28:44 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA17066 for ; Thu, 1 May 2003 11:28:44 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19BFPk-00012U-00 for ipfix-list@mil.doit.wisc.edu; Thu, 01 May 2003 09:52:00 -0500 Received: from [64.95.122.13] (helo=RS-SC-EXC4.rs.riverstonenet.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19BFPi-00012N-00 for ipfix@net.doit.wisc.edu; Thu, 01 May 2003 09:51:58 -0500 Received: from riverstonenet.com ([172.17.6.3]) by RS-SC-EXC4.rs.riverstonenet.com with Microsoft SMTPSVC(5.0.2195.4905); Thu, 1 May 2003 07:51:56 -0700 Message-ID: <3EB13485.3FA365D7@riverstonenet.com> Date: Thu, 01 May 2003 10:51:49 -0400 From: calato@riverstonenet.com X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.7-10 i686) X-Accept-Language: en MIME-Version: 1.0 To: "MEYER,JEFFREY D (HP-Cupertino,ex1)" CC: "'Mark Fullmer'" , ipfix@net.doit.wisc.edu Subject: Re: [ipfix] drawing the line between protocol and data model document References: <1D3D2C371FCBD947A7897FABBD3533A566BA84@xsun01.ptp.hp.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 01 May 2003 14:51:56.0749 (UTC) FILETIME=[363517D0:01C30FF1] Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit It sounds to me like the issue is at what level is the protocol aware of the data which is being passed (i.e 4 byte value vs IP address). Jeff mentioned that it would be handly to know the value of some unknown type is say an IP address. My question is what can you do with that since you don't know what is the IP address of? I'll add my 2 cents on where to draw the line between protocol and data model. The data model defines the semantics or meaning of the data along with the valid range of values. Where appropriate we should reference other standards. It also defines the meaing of multiple fields taken together (e.g. this set of fields is the flow key). It would be up to the protocol to define how the data is actaully represented. For example, the "Flow Octet Counter". The data model would specify how/when the counter is incrememted. And the data model would say it has has a valid range of 0 to 2^64. Yes, you could argue that a counter theoretically has no upper limit. The richness of the protocol encoding specification is another matter. The protocol would reference each element in the data model and define exaclty how that element is encoded. In the above example, the protocol is free to encode that as an unsigned byte if the value is < 256. As another example, lets take Source IP address. The data model would define that it is the value taken from the source address slot as defined in rfc 793. It may also define how NAT addresses are handled (example only, I'm not saying we do or don't handle NAT) The protocol would define how it is encoded. For example as a 4 byte unsigned value, or if it has the notion as an IP address value. It could also define it in more general terms as an address family, address length and address value. There are pros and cons in all three encodings. Note that the base protocol (aka netflow) ties semantics and value encoding together. The TYPE defines both the sematnics of the data and how it is encoded. Paul "MEYER,JEFFREY D (HP-Cupertino,ex1)" wrote: > > Mark, > > I'll try to clarify: > > Concept #1: The information model and the encoding are separate. > A well defined mapping determines how an information model is > to be encoded for a specific use (e.g. IPFIX). The encoding > may take whatever approach allows both ends to do something > useful with the data. As you point out there is some useful > stuff which can be done with the length alone. > > Concept #2: Derived typing. By defining a base set of types, > which must be understood, derived types can be defined in terms > of these base types. For instance Diameter does this. This > enables a core set of types to be defined and coded to, but > also enables richer types later on with an appropriate fallback > for systems which aren't aware of the extensions. > > The benefit I see of explicit typing is that instead of seeing > that a field is of length "4 bytes" something which will be seen > A LOT. Is that we can instead differentiate between IPAddr, Integer, > unsigned integer, and float. This seems handy to me. > > Your argument seems centered around the long term potential of > adding lots of fixed length fields whose base length is not 4,8 > or 16 bytes long, for instance MACAddr (although if we agree on > going the type route, I'd recommend adding MACAddr and then you > have 1,2,4,6,8 and 16 bytes covered. I guess I don't foresee > a lot more fixed length fields coming down the pipe in the IPFIX > domain. Hence my desire to make it more explicit for the types > which do occur all the time and are likely points of extension. > WHY? Because then a tool could at least present these in a meaningful > way. As opposed to incorrectly formatting a float type as an int. > > But hey, either way will work. If the goal is to limit the > amount of changes, I'll (grudgingly) go along. I haven't heard > any opinions from anyone but you in either direction. So, > at this point I don't see a quorom either way... > > Regards, > > Jeff Meyer > > > -----Original Message----- > > From: Mark Fullmer [mailto:maf@eng.oar.net] > > Sent: Tuesday, April 29, 2003 2:53 PM > > To: MEYER,JEFFREY D (HP-Cupertino,ex1) > > Cc: ipfix@net.doit.wisc.edu > > Subject: Re: [ipfix] drawing the line between protocol and data model > > docu ment > > > > > > IMHO crippling the protocol to serve the needs of a documentation tool > > is just silly. > > > > It would be possible to use both a Type name and a Length instead of > > the existing Length, which would allow a collector to process > > unknown Types. > > > > With the existing v9 protocol a collector can ignore an > > unknown attribute, > > or possibly just pass it on to post processing tools which > > might understand > > it. > > > > Your suggested changes no longer allow a collector to ignore unknown > > attributes if the Type is also unknown. Resorting to encoding all > > new Types after the initial protocol spec as octet strings is (IMHO) > > not acceptable. > > > > I'm still trying to understand what the benefits to explicit > > Types sent > > in templates are. > > > > mark > > > > On Tue, Apr 29, 2003 at 02:50:02PM -0400, MEYER,JEFFREY D > > (HP-Cupertino,ex1) wrote: > > > Juergen, > > > > > > I agree that the data model should have a well defined type > > > system. And that that typing system should reflect the > > requirements. > > > > > > This is in part why I was hoping that the protocol itself would > > > be more explicit here. (See my mail > > > http://ipfix.doit.wisc.edu/archive/1604.html) > > > > > > There is also a type list in that mail. However, I would suggest > > > using the names from XML-Schema part 2 Data types, as this would > > > facilitate creating a formal/normative definition, versus > > reinventing > > > names for the types. > > > > > > Here are the type names from XML-Schema/IPDR: > > > > > > boolean - true/false (may be encoded in byte) > > > byte > > > unsignedByte > > > short > > > unsignedShort > > > int - 32-bit signed > > > long - 64-bit signed > > > unsignedInt > > > unsignedLong > > > float > > > double > > > dateTime - 32-bit integer representing seconds since EPOCH > > > (1/1/1970 0:00 GMT) > > > string - UTF-8 formatted text > > > ipdr:dateTimeMsec - 64-bit integer representing msecs > > since EPOCH > > > ipdr:ipV4Addr > > > ipdr:ipV6Addr > > > ipdr:UUID > > > hexBinary - arbitrary sequence of octets > > > > > > If the protocol itself has a basic "octetString" (or from > > XML, hexBinary) > > > payload, > > > then this can be used to carry new classes of information > > in the future w/o > > > rejiggering the protocol itself. > > > > > > In addition other "derived" types may be created from the > > base types. For > > > instance in IPDR, the ipdr:dateTimeMsec is derived from long. > > > > > > Regards, > > > > > > Jeff Meyer > > > > > > > -----Original Message----- > > > > From: Juergen Quittek [mailto:quittek@ccrle.nec.de] > > > > Sent: Monday, April 28, 2003 2:06 PM > > > > To: ipfix@net.doit.wisc.edu > > > > Subject: [ipfix] drawing the line between protocol and data model > > > > document > > > > > > > > > > > > Hi all, > > > > > > > > When shaping the planned documents, we have to draw a line > > > > between the protocol and the data model. > > > > > > > > Definitely, the data model document defines all field types, > > > > but does it also define basic data types, such as '16 bit > > > > unsigned integer'? > > > > > > > > Maybe we should have a set of basic data types defined in > > > > the protocol document and restrict the data model doc to > > these types? > > > > This would further structure the data model. > > > > > > > > Any thoughts? > > > > > > > > Juergen > > > > > > > > -- > > > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > > > > in message body > > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > > "unsubscribe ipfix" in message body > > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > > > -- > > > Help mailto:majordomo@net.doit.wisc.edu and say > > "help" in message body > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > "unsubscribe ipfix" in message body > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu May 1 11:28:44 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA17065 for ; Thu, 1 May 2003 11:28:44 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19BFYU-0001Gj-00 for ipfix-list@mil.doit.wisc.edu; Thu, 01 May 2003 10:01:02 -0500 Received: from [64.95.122.13] (helo=RS-SC-EXC4.rs.riverstonenet.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19BFYS-0001GZ-00 for ipfix@net.doit.wisc.edu; Thu, 01 May 2003 10:01:01 -0500 Received: from riverstonenet.com ([172.17.6.3]) by RS-SC-EXC4.rs.riverstonenet.com with Microsoft SMTPSVC(5.0.2195.4905); Thu, 1 May 2003 08:00:58 -0700 Message-ID: <3EB136A8.B4C678EA@riverstonenet.com> Date: Thu, 01 May 2003 11:00:56 -0400 From: calato@riverstonenet.com X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.7-10 i686) X-Accept-Language: en MIME-Version: 1.0 To: Juergen Quittek CC: "MEYER,JEFFREY D (HP-Cupertino,ex1)" , "'Nevil Brownlee'" , ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Editors for IPFIX drafts References: <1D3D2C371FCBD947A7897FABBD3533A566BA6B@xsun01.ptp.hp.com> <16107501.1051570398@[10.1.1.26]> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 01 May 2003 15:00:59.0275 (UTC) FILETIME=[7993FDB0:01C30FF2] Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit I'm a little confused. Are we talking about XML descriptions as an appendix with prose as the main body or XML as the main body with tools translating it into a more readable form? Or XML as an appendix with tools translating it for use as sections mixed in with the prose? Paul Juergen Quittek wrote: > > Jeff, > > -- MEYER,JEFFREY D (HP-Cupertino,ex1) wrote on 28 April 2003 09:20 -0700: > > > > Juergen, > > > > Thanks for the response. > > > > I agree XML in and of itself is marginally human readable. > > > > I think that basic tools such as Xalan (http://xml.apache.org/), can > > be used to "transform" a base XML document into a variety of formats, > > including formatting consistent with RFC's. Note that all the IPDR > > RFC's were written in XML using Marshall Rose's xml2rfc tools > > (http://xml.resource.org/). > > I also like Marshall's tools. > If Paul agrees, we should use them for the data model document. > > > So the model I would picture is having the Normative XML data model > > as an appendix, and having generated information from this data model > > populate some of the body sections to aid human readers. > > This looks like a reasonable way to go. > > Juergen > > > Regards, > > > > Jeff Meyer > > > >> -----Original Message----- > >> From: Juergen Quittek [mailto:quittek@ccrle.nec.de] > >> Sent: Monday, April 28, 2003 8:24 AM > >> To: MEYER,JEFFREY D (HP-Cupertino,ex1); 'Nevil Brownlee'; > >> ipfix@net.doit.wisc.edu > >> Subject: RE: [ipfix] Editors for IPFIX drafts > >> > >> > >> Jeff, > >> > >> I do see a value in such a formal description. But there is a > >> trafe-off. > >> > >> I also see a degradation of human readability. You loose means of > >> structuring your document text when using XML, for example grouping > >> attributes that are related. (Of course you could do so in > >> XML, but it is > >> much harder to find the heading "Port-related Attributes" in > >> an XML document, > >> because headings cannot be well highlighted and numbering of > >> headings is > >> not well supported. > >> > >> Anyway, I like the idea of an XML encoding of the model, but > >> I we should > >> not go this way if we cannot find a good way of getting it > >> hunam readable > >> (maybe by applying formatting rules). > >> > >> An option would also be adding the XML code to the document > >> as an appendix. > >> > >> Juergen > >> > >> > >> -- MEYER,JEFFREY D (HP-Cupertino,ex1) wrote on 28 April 2003 > >> 10:57 -0400: > >> > >> > Hi, > >> > > >> > My particular interest on working on the information model would > >> > be on developing a more formal and standard way of describing the > >> > information elements which are used in IPFIX communication. > >> > > >> > The modelling of information elements should ideally be done > >> > using a formal description language, so that tools can be > >> constructed > >> > which do not require humans to transcribe this information. > >> > > >> > I would propose the use of XML-Schema to provide this formal > >> > description. Through the use of XSL processors, prose > >> based descriptions > >> > and tables can be produced if these are considered desireable. > >> > > >> > NOTE: describing the information model formally does NOT mean > >> > that encoding and transport for IPFIX need to change in any way. It > >> > simply means that rather than having as the base > >> description of information > >> > items some prose in a document, you instead have a machine readable > >> > representation. > >> > > >> > > >> > As an example of a formal (but currently incomplete) information > >> > model for IPFIX, see: > >> > http://www.ipdr.org/documents/ipfix/ipfixService-20020902.xml > >> > > >> > I would recommend making such a representation the normative form > >> > of the IPFIX information model, much like SNMP MIBs. > >> > > >> > > >> > Please let me know if people see value in this exercise. > >> > > >> > Regards, > >> > > >> > Jeff Meyer > >> > > >> >> -----Original Message----- > >> >> From: Nevil Brownlee [mailto:n.brownlee@auckland.ac.nz] > >> >> Sent: Sunday, April 27, 2003 8:44 PM > >> >> To: ipfix@net.doit.wisc.edu > >> >> Subject: [ipfix] Editors for IPFIX drafts > >> >> > >> >> > >> >> > >> >> Hello all: > >> >> > >> >> Following the IPFIX meeting at the San Francisco IETF we asked for > >> >> volunteers to edit the IPFIX drafts. The current list is: > >> >> > >> >> Applicability: Tanja Zseby, Reinaldo Penno > >> >> > >> >> Architecture: Ganesh Sadasivan, Nevil Brownlee > >> >> > >> >> Information Model: Paul Calato, Jeff Meyer, Juergen Quittek > >> >> > >> >> Protocol: Mark Fulmer, Paul Calato, Reinaldo Penno > >> >> > >> >> At this stage we already have old (expired) versions of the > >> >> Architecture (-02.txt) and Data Model (-01.txt) drafts on the IPFIX > >> >> web page; would the editors of all four drafts please email > >> >> me an initial > >> >> version of their new drafts to published as placeholders > >> >> within the next > >> >> few weeks. > >> >> > >> >> After that, the editors will work on developing better > >> versions of the > >> >> drafts which can be discussed on the IPFIX list and submitted > >> >> by mid-June, > >> >> i.e. in plenty of time for the Vienna IETF meeting in July. > >> >> > >> >> Cheers, Nevil > >> >> > >> >> PS: Good to see some real discussion on the list. Lets > >> concentrate on > >> >> producing text the draft editors can use! > >> >> > >> >> -------------------------------------------------------------- > >> >> --------- > >> >> Nevil Brownlee Director, Technology > >> Development > >> >> Phone: +64 9 373 7599 x88941 ITSS, The University > >> of Auckland > >> >> FAX: +64 9 373 7021 Private Bag 92019, Auckland, > >> New Zealand > >> >> > >> >> > >> >> ------------------------------------------------- > >> >> This mail sent through University of Auckland > >> >> http://www.auckland.ac.nz/ > >> >> > >> >> -- > >> >> Help mailto:majordomo@net.doit.wisc.edu and say "help" > >> >> in message body > >> >> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > >> >> "unsubscribe ipfix" in message body > >> >> Archive http://ipfix.doit.wisc.edu/archive/ > >> >> > >> > > >> > -- > >> > Help mailto:majordomo@net.doit.wisc.edu and say > >> "help" in message body > >> > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > >> > "unsubscribe ipfix" in message body > >> > Archive http://ipfix.doit.wisc.edu/archive/ > >> > >> > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu May 1 11:41:06 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA17964 for ; Thu, 1 May 2003 11:41:06 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19BFr3-0001hb-00 for ipfix-list@mil.doit.wisc.edu; Thu, 01 May 2003 10:20:13 -0500 Received: from tokyo.ccrle.nec.de ([195.37.70.2]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19BFr1-0001hS-00 for ipfix@net.doit.wisc.edu; Thu, 01 May 2003 10:20:11 -0500 Received: from venus.office (venus.office [10.1.1.11]) by tokyo.ccrle.nec.de (8.12.9/8.12.8) with ESMTP id h41FJhVI005860; Thu, 1 May 2003 17:19:44 +0200 (CEST) Received: from [10.1.1.26] (dial02.office [10.1.1.26]) by venus.office (Postfix on SuSE Linux eMail Server 3.0) with ESMTP id F19C8A0826; Thu, 1 May 2003 17:13:02 +0200 (CEST) Date: Thu, 01 May 2003 17:21:06 +0200 From: Juergen Quittek To: calato@riverstonenet.com Cc: "MEYER,JEFFREY D (HP-Cupertino,ex1)" , "'Nevil Brownlee'" , ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Editors for IPFIX drafts Message-ID: <10620801.1051809664@[10.1.1.26]> In-Reply-To: <3EB136A8.B4C678EA@riverstonenet.com> References: <3EB136A8.B4C678EA@riverstonenet.com> X-Mailer: Mulberry/2.1.2 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Paul, I agree to Jeff, that there is some value in having an XML representation of the specification. However, in the end this document will become an RFC and it must be well readable to human beings. Therefore, I think that the main body must be human readable, but we can provide an appendix, that contains an equivalent machine readable spec. Different to Jeff, I think that the hunam readable main body should be the normative part, while the XML appendix is just given as support for users of XML tools allowing them to genrate protocol stacks and/or tools for consistency checking. As Jeff suggested, we can work with an XML representation when developing the spec and generate human readable text automatically from the XML code. This will help us avoiding fomal mistakes in the human readable spec and we would always have consistency between human readable and XML part. Juergen -- calato@riverstonenet.com wrote on 01 May 2003 11:00 -0400: > > I'm a little confused. Are we talking about XML descriptions > as an appendix with prose as the main body or XML as the > main body with tools translating it into a more readable > form? Or XML as an appendix with tools translating it for > use as sections mixed in with the prose? > > > Paul > > Juergen Quittek wrote: >> >> Jeff, >> >> -- MEYER,JEFFREY D (HP-Cupertino,ex1) wrote on 28 April 2003 09:20 -0700: >> > >> > Juergen, >> > >> > Thanks for the response. >> > >> > I agree XML in and of itself is marginally human readable. >> > >> > I think that basic tools such as Xalan (http://xml.apache.org/), can >> > be used to "transform" a base XML document into a variety of formats, >> > including formatting consistent with RFC's. Note that all the IPDR >> > RFC's were written in XML using Marshall Rose's xml2rfc tools >> > (http://xml.resource.org/). >> >> I also like Marshall's tools. >> If Paul agrees, we should use them for the data model document. >> >> > So the model I would picture is having the Normative XML data model >> > as an appendix, and having generated information from this data model >> > populate some of the body sections to aid human readers. >> >> This looks like a reasonable way to go. >> >> Juergen >> >> > Regards, >> > >> > Jeff Meyer >> > >> >> -----Original Message----- >> >> From: Juergen Quittek [mailto:quittek@ccrle.nec.de] >> >> Sent: Monday, April 28, 2003 8:24 AM >> >> To: MEYER,JEFFREY D (HP-Cupertino,ex1); 'Nevil Brownlee'; >> >> ipfix@net.doit.wisc.edu >> >> Subject: RE: [ipfix] Editors for IPFIX drafts >> >> >> >> >> >> Jeff, >> >> >> >> I do see a value in such a formal description. But there is a >> >> trafe-off. >> >> >> >> I also see a degradation of human readability. You loose means of >> >> structuring your document text when using XML, for example grouping >> >> attributes that are related. (Of course you could do so in >> >> XML, but it is >> >> much harder to find the heading "Port-related Attributes" in >> >> an XML document, >> >> because headings cannot be well highlighted and numbering of >> >> headings is >> >> not well supported. >> >> >> >> Anyway, I like the idea of an XML encoding of the model, but >> >> I we should >> >> not go this way if we cannot find a good way of getting it >> >> hunam readable >> >> (maybe by applying formatting rules). >> >> >> >> An option would also be adding the XML code to the document >> >> as an appendix. >> >> >> >> Juergen >> >> >> >> >> >> -- MEYER,JEFFREY D (HP-Cupertino,ex1) wrote on 28 April 2003 >> >> 10:57 -0400: >> >> >> >> > Hi, >> >> > >> >> > My particular interest on working on the information model would >> >> > be on developing a more formal and standard way of describing the >> >> > information elements which are used in IPFIX communication. >> >> > >> >> > The modelling of information elements should ideally be done >> >> > using a formal description language, so that tools can be >> >> constructed >> >> > which do not require humans to transcribe this information. >> >> > >> >> > I would propose the use of XML-Schema to provide this formal >> >> > description. Through the use of XSL processors, prose >> >> based descriptions >> >> > and tables can be produced if these are considered desireable. >> >> > >> >> > NOTE: describing the information model formally does NOT mean >> >> > that encoding and transport for IPFIX need to change in any way. It >> >> > simply means that rather than having as the base >> >> description of information >> >> > items some prose in a document, you instead have a machine readable >> >> > representation. >> >> > >> >> > >> >> > As an example of a formal (but currently incomplete) information >> >> > model for IPFIX, see: >> >> > http://www.ipdr.org/documents/ipfix/ipfixService-20020902.xml >> >> > >> >> > I would recommend making such a representation the normative form >> >> > of the IPFIX information model, much like SNMP MIBs. >> >> > >> >> > >> >> > Please let me know if people see value in this exercise. >> >> > >> >> > Regards, >> >> > >> >> > Jeff Meyer >> >> > >> >> >> -----Original Message----- >> >> >> From: Nevil Brownlee [mailto:n.brownlee@auckland.ac.nz] >> >> >> Sent: Sunday, April 27, 2003 8:44 PM >> >> >> To: ipfix@net.doit.wisc.edu >> >> >> Subject: [ipfix] Editors for IPFIX drafts >> >> >> >> >> >> >> >> >> >> >> >> Hello all: >> >> >> >> >> >> Following the IPFIX meeting at the San Francisco IETF we asked for >> >> >> volunteers to edit the IPFIX drafts. The current list is: >> >> >> >> >> >> Applicability: Tanja Zseby, Reinaldo Penno >> >> >> >> >> >> Architecture: Ganesh Sadasivan, Nevil Brownlee >> >> >> >> >> >> Information Model: Paul Calato, Jeff Meyer, Juergen Quittek >> >> >> >> >> >> Protocol: Mark Fulmer, Paul Calato, Reinaldo Penno >> >> >> >> >> >> At this stage we already have old (expired) versions of the >> >> >> Architecture (-02.txt) and Data Model (-01.txt) drafts on the IPFIX >> >> >> web page; would the editors of all four drafts please email >> >> >> me an initial >> >> >> version of their new drafts to published as placeholders >> >> >> within the next >> >> >> few weeks. >> >> >> >> >> >> After that, the editors will work on developing better >> >> versions of the >> >> >> drafts which can be discussed on the IPFIX list and submitted >> >> >> by mid-June, >> >> >> i.e. in plenty of time for the Vienna IETF meeting in July. >> >> >> >> >> >> Cheers, Nevil >> >> >> >> >> >> PS: Good to see some real discussion on the list. Lets >> >> concentrate on >> >> >> producing text the draft editors can use! >> >> >> >> >> >> -------------------------------------------------------------- >> >> >> --------- >> >> >> Nevil Brownlee Director, Technology >> >> Development >> >> >> Phone: +64 9 373 7599 x88941 ITSS, The University >> >> of Auckland >> >> >> FAX: +64 9 373 7021 Private Bag 92019, Auckland, >> >> New Zealand >> >> >> >> >> >> >> >> >> ------------------------------------------------- >> >> >> This mail sent through University of Auckland >> >> >> http://www.auckland.ac.nz/ >> >> >> >> >> >> -- >> >> >> Help mailto:majordomo@net.doit.wisc.edu and say "help" >> >> >> in message body >> >> >> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >> >> >> "unsubscribe ipfix" in message body >> >> >> Archive http://ipfix.doit.wisc.edu/archive/ >> >> >> >> >> > >> >> > -- >> >> > Help mailto:majordomo@net.doit.wisc.edu and say >> >> "help" in message body >> >> > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >> >> > "unsubscribe ipfix" in message body >> >> > Archive http://ipfix.doit.wisc.edu/archive/ >> >> >> >> >> >> -- >> Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body >> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >> "unsubscribe ipfix" in message body >> Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu May 1 11:46:26 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA18253 for ; Thu, 1 May 2003 11:46:26 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19BFso-0001mT-00 for ipfix-list@mil.doit.wisc.edu; Thu, 01 May 2003 10:22:02 -0500 Received: from [64.95.122.13] (helo=RS-SC-EXC4.rs.riverstonenet.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19BFsm-0001m0-00 for ipfix@net.doit.wisc.edu; Thu, 01 May 2003 10:22:00 -0500 Received: from riverstonenet.com ([172.17.6.3]) by RS-SC-EXC4.rs.riverstonenet.com with Microsoft SMTPSVC(5.0.2195.4905); Thu, 1 May 2003 08:21:58 -0700 Message-ID: <3EB13B94.6B4DEBC9@riverstonenet.com> Date: Thu, 01 May 2003 11:21:56 -0400 From: calato@riverstonenet.com X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.7-10 i686) X-Accept-Language: en MIME-Version: 1.0 To: Reinaldo Penno CC: "'Maurizio Molina'" , ipfix@net.doit.wisc.edu Subject: Re: [ipfix] export packet length References: <0A11633F61BD9F40B43ABCC694004F93F18EEC@zsc3c026.us.nortel.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 01 May 2003 15:21:59.0221 (UTC) FILETIME=[68907E50:01C30FF5] Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Since the only thing mandated for the transport is congestion awareness it would seems to make sense to have a length field in the IPFIX protocol. I don't like the idea of having different formats based on the transport (someone mentioned that as an option). A 2 byte length field seems a small price to pay or better transport independance. Paul > Reinaldo Penno wrote: > > That's a good idea. > > Other protocols like HTTP and SIP make it mandatory to include the > Content-Length header (which tell the length of the payload) when > carried over TCP. It helps a lot when dealing with fragmentation. > > Regards, > > > -----Original Message----- > > From: Maurizio Molina [mailto:molina@ccrle.nec.de] > > Sent: Friday, April 25, 2003 11:50 AM > > To: ipfix@net.doit.wisc.edu > > Subject: [ipfix] export packet length > > > > > > Hi all, > > I hope not to rise an issue already discussed in the WG. If > > yes, I apologise in advance. If an IPFIX export packet is > > carried over TCP, it's not unusual that it gets split across > > multiple TCP segments. At the collector side, on the > > contrary, the implementation of the parsing is much simpler > > if only full export packet are parsed. If the information > > about the WHOLE export packet were contained in the header, > > it would be simple to wait until all the bytes composing an > > export packets are read from the TCP socket before beginning > > the parsing. Unfortunately, this information isn't currently > > contained in Netflow 9 packet header (I suppose, due to > > Netflow's 9 "bias" towards UDP...). Therefore, a parsing > > implementation having to deal with this "fragentation" > > problem without this information needs to be much more > > complicated (e.g. getting step by step the lenght of each > > flow set, see if it's all there, parse it and go on....). In > > summary: if IPFIX has to be carried over tcp, having the > > length of the export packet in the packet header would be > > beneficial. Regards, Maurizio > > > > > > -- > > Maurizio Molina > > Research Staff member > > Network Laboratories Heidelberg > > NEC Europe Ltd. > > Kurfuersten-Anlage 36, 69115 Heidelberg, Germany. > > Tel. (49)6221 90511-18 Fax: (49)6221 90511-55 > > e-mail: molina@ccrle.nec.de > > Web: www.ccrle.nec.de > > > > > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > > in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu May 1 11:49:01 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA18354 for ; Thu, 1 May 2003 11:49:01 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19BFzQ-0001uJ-00 for ipfix-list@mil.doit.wisc.edu; Thu, 01 May 2003 10:28:52 -0500 Received: from [64.95.122.13] (helo=RS-SC-EXC4.rs.riverstonenet.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19BFzP-0001uD-00 for ipfix@net.doit.wisc.edu; Thu, 01 May 2003 10:28:51 -0500 Received: from riverstonenet.com ([172.17.6.3]) by RS-SC-EXC4.rs.riverstonenet.com with Microsoft SMTPSVC(5.0.2195.4905); Thu, 1 May 2003 08:28:50 -0700 Message-ID: <3EB13D30.DFE75100@riverstonenet.com> Date: Thu, 01 May 2003 11:28:48 -0400 From: calato@riverstonenet.com X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.7-10 i686) X-Accept-Language: en MIME-Version: 1.0 To: ipfixx Subject: [ipfix] Issue tracking Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 01 May 2003 15:28:50.0744 (UTC) FILETIME=[5DD9EB80:01C30FF6] Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit One thing that has bugged me for a long time about this working group is there is no issue tracking and resolution mechanism. Things are discussed on the list with no final word. Since I'm working on the data model and protocol documents I'm proposing issue tracking and resolution for those areas. I'd like one document which tracks issues along with current status and final resolutions. I'll be happy to maintain it. If there is agreement, I'll start the process now before too many issues arise. Comments please. Paul -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu May 1 11:56:17 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA18761 for ; Thu, 1 May 2003 11:56:17 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19BG8p-00029W-00 for ipfix-list@mil.doit.wisc.edu; Thu, 01 May 2003 10:38:35 -0500 Received: from eng4.oar.net ([192.148.244.24]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 19BG8n-00029H-00 for ipfix@net.doit.wisc.edu; Thu, 01 May 2003 10:38:33 -0500 Received: (qmail 32239 invoked by uid 4454); 1 May 2003 15:38:32 -0000 Date: Thu, 1 May 2003 11:38:32 -0400 From: Mark Fullmer To: "MEYER,JEFFREY D \(HP-Cupertino,ex1\)" Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] drawing the line between protocol and data model docu ment Message-ID: <20030501113832.A32141@net.ohio-state.edu> References: <1D3D2C371FCBD947A7897FABBD3533A566BA84@xsun01.ptp.hp.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: <1D3D2C371FCBD947A7897FABBD3533A566BA84@xsun01.ptp.hp.com>; from jeff.meyer2@hp.com on Tue, Apr 29, 2003 at 06:25:11PM -0400 Precedence: bulk Sender: majordomo listserver So the difference boils down to a tool being able to display = 0x01020304 vs = 1.2.3.4 In this situation someone will either grab a new version of the software or an updated data definition to resolve the unknown attribute, which would also define it's type. mark On Tue, Apr 29, 2003 at 06:25:11PM -0400, MEYER,JEFFREY D (HP-Cupertino,ex1) wrote: > Mark, > > I'll try to clarify: > > Concept #1: The information model and the encoding are separate. > A well defined mapping determines how an information model is > to be encoded for a specific use (e.g. IPFIX). The encoding > may take whatever approach allows both ends to do something > useful with the data. As you point out there is some useful > stuff which can be done with the length alone. > > Concept #2: Derived typing. By defining a base set of types, > which must be understood, derived types can be defined in terms > of these base types. For instance Diameter does this. This > enables a core set of types to be defined and coded to, but > also enables richer types later on with an appropriate fallback > for systems which aren't aware of the extensions. > > The benefit I see of explicit typing is that instead of seeing > that a field is of length "4 bytes" something which will be seen > A LOT. Is that we can instead differentiate between IPAddr, Integer, > unsigned integer, and float. This seems handy to me. > > Your argument seems centered around the long term potential of > adding lots of fixed length fields whose base length is not 4,8 > or 16 bytes long, for instance MACAddr (although if we agree on > going the type route, I'd recommend adding MACAddr and then you > have 1,2,4,6,8 and 16 bytes covered. I guess I don't foresee > a lot more fixed length fields coming down the pipe in the IPFIX > domain. Hence my desire to make it more explicit for the types > which do occur all the time and are likely points of extension. > WHY? Because then a tool could at least present these in a meaningful > way. As opposed to incorrectly formatting a float type as an int. > > But hey, either way will work. If the goal is to limit the > amount of changes, I'll (grudgingly) go along. I haven't heard > any opinions from anyone but you in either direction. So, > at this point I don't see a quorom either way... > > Regards, > > Jeff Meyer > > > > -----Original Message----- > > From: Mark Fullmer [mailto:maf@eng.oar.net] > > Sent: Tuesday, April 29, 2003 2:53 PM > > To: MEYER,JEFFREY D (HP-Cupertino,ex1) > > Cc: ipfix@net.doit.wisc.edu > > Subject: Re: [ipfix] drawing the line between protocol and data model > > docu ment > > > > > > IMHO crippling the protocol to serve the needs of a documentation tool > > is just silly. > > > > It would be possible to use both a Type name and a Length instead of > > the existing Length, which would allow a collector to process > > unknown Types. > > > > With the existing v9 protocol a collector can ignore an > > unknown attribute, > > or possibly just pass it on to post processing tools which > > might understand > > it. > > > > Your suggested changes no longer allow a collector to ignore unknown > > attributes if the Type is also unknown. Resorting to encoding all > > new Types after the initial protocol spec as octet strings is (IMHO) > > not acceptable. > > > > I'm still trying to understand what the benefits to explicit > > Types sent > > in templates are. > > > > mark > > > > On Tue, Apr 29, 2003 at 02:50:02PM -0400, MEYER,JEFFREY D > > (HP-Cupertino,ex1) wrote: > > > Juergen, > > > > > > I agree that the data model should have a well defined type > > > system. And that that typing system should reflect the > > requirements. > > > > > > This is in part why I was hoping that the protocol itself would > > > be more explicit here. (See my mail > > > http://ipfix.doit.wisc.edu/archive/1604.html) > > > > > > There is also a type list in that mail. However, I would suggest > > > using the names from XML-Schema part 2 Data types, as this would > > > facilitate creating a formal/normative definition, versus > > reinventing > > > names for the types. > > > > > > Here are the type names from XML-Schema/IPDR: > > > > > > boolean - true/false (may be encoded in byte) > > > byte > > > unsignedByte > > > short > > > unsignedShort > > > int - 32-bit signed > > > long - 64-bit signed > > > unsignedInt > > > unsignedLong > > > float > > > double > > > dateTime - 32-bit integer representing seconds since EPOCH > > > (1/1/1970 0:00 GMT) > > > string - UTF-8 formatted text > > > ipdr:dateTimeMsec - 64-bit integer representing msecs > > since EPOCH > > > ipdr:ipV4Addr > > > ipdr:ipV6Addr > > > ipdr:UUID > > > hexBinary - arbitrary sequence of octets > > > > > > If the protocol itself has a basic "octetString" (or from > > XML, hexBinary) > > > payload, > > > then this can be used to carry new classes of information > > in the future w/o > > > rejiggering the protocol itself. > > > > > > In addition other "derived" types may be created from the > > base types. For > > > instance in IPDR, the ipdr:dateTimeMsec is derived from long. > > > > > > Regards, > > > > > > Jeff Meyer > > > > > > > -----Original Message----- > > > > From: Juergen Quittek [mailto:quittek@ccrle.nec.de] > > > > Sent: Monday, April 28, 2003 2:06 PM > > > > To: ipfix@net.doit.wisc.edu > > > > Subject: [ipfix] drawing the line between protocol and data model > > > > document > > > > > > > > > > > > Hi all, > > > > > > > > When shaping the planned documents, we have to draw a line > > > > between the protocol and the data model. > > > > > > > > Definitely, the data model document defines all field types, > > > > but does it also define basic data types, such as '16 bit > > > > unsigned integer'? > > > > > > > > Maybe we should have a set of basic data types defined in > > > > the protocol document and restrict the data model doc to > > these types? > > > > This would further structure the data model. > > > > > > > > Any thoughts? > > > > > > > > Juergen > > > > > > > > -- > > > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > > > > in message body > > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > > "unsubscribe ipfix" in message body > > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > > > -- > > > Help mailto:majordomo@net.doit.wisc.edu and say > > "help" in message body > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > "unsubscribe ipfix" in message body > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu May 1 11:56:44 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA18788 for ; Thu, 1 May 2003 11:56:44 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19BG1s-0001yd-00 for ipfix-list@mil.doit.wisc.edu; Thu, 01 May 2003 10:31:24 -0500 Received: from tokyo.ccrle.nec.de ([195.37.70.2]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19BG1q-0001yU-00 for ipfix@net.doit.wisc.edu; Thu, 01 May 2003 10:31:23 -0500 Received: from venus.office (venus.office [10.1.1.11]) by tokyo.ccrle.nec.de (8.12.9/8.12.8) with ESMTP id h41FVCVI005978; Thu, 1 May 2003 17:31:13 +0200 (CEST) Received: from [10.1.1.26] (dial02.office [10.1.1.26]) by venus.office (Postfix on SuSE Linux eMail Server 3.0) with ESMTP id A9CDF91EA1; Thu, 1 May 2003 17:24:32 +0200 (CEST) Date: Thu, 01 May 2003 17:32:35 +0200 From: Juergen Quittek To: calato@riverstonenet.com, "MEYER,JEFFREY D (HP-Cupertino,ex1)" Cc: "'Mark Fullmer'" , ipfix@net.doit.wisc.edu Subject: Re: [ipfix] drawing the line between protocol and data model document Message-ID: <11310473.1051810354@[10.1.1.26]> In-Reply-To: <3EB13485.3FA365D7@riverstonenet.com> References: <3EB13485.3FA365D7@riverstonenet.com> X-Mailer: Mulberry/2.1.2 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Paul, Do I understand you right, that you suggest, that for each flow attribute the info model doc defines the semantics and the protocol document defines the data encoding? What about the following: The protocol document defines a set of basic types (counter64, IP address, ...) including their binary representation (big endian, ...). And the info model document defines the the list of all attributes, defines their semantics and selects a basic type from the protocol doc for each attribute. This would have the advantage, that the protocol doc must not contain a list of all attributes. Also extending the attributes can be done by just updating the info model doc (assuming that the set of basic data types is sufficient for the extension). Juergen -- calato@riverstonenet.com wrote on 01 May 2003 10:51 -0400: > > It sounds to me like the issue is at what level is the protocol > aware of the data which is being passed (i.e 4 byte value > vs IP address). > > Jeff mentioned that it would be handly to know the value of some > unknown type is say an IP address. My question is what can you do with > that since you don't know what is the IP address of? > > > > I'll add my 2 cents on where to draw the line between protocol > and data model. > > The data model defines the semantics or meaning of the data along > with the valid range of values. Where appropriate we should reference > other standards. It also defines the meaing of multiple fields taken > together (e.g. this set of fields is the flow key). > > It would be up to the protocol to define how the data is actaully > represented. > > For example, the "Flow Octet Counter". The data model would specify > how/when the counter is incrememted. And the data model would say > it has has a valid range of 0 to 2^64. Yes, you could argue that a > counter theoretically has no upper limit. > > > The richness of the protocol encoding specification is another > matter. The protocol would reference each element in the data > model and define exaclty how that element is encoded. In the above > example, the protocol is free to encode that as an unsigned byte > if the value is < 256. > > As another example, lets take Source IP address. The data model > would define that it is the value taken from the source address > slot as defined in rfc 793. It may also define how NAT addresses > are handled (example only, I'm not saying we do or don't handle NAT) > > The protocol would define how it is encoded. For example > as a 4 byte unsigned value, or if it has the notion as an > IP address value. It could also define it in more general terms > as an address family, address length and address value. There are > pros and cons in all three encodings. > > Note that the base protocol (aka netflow) ties semantics and > value encoding together. The TYPE defines both the sematnics > of the data and how it is encoded. > > > Paul > > > "MEYER,JEFFREY D (HP-Cupertino,ex1)" wrote: >> >> Mark, >> >> I'll try to clarify: >> >> Concept #1: The information model and the encoding are separate. >> A well defined mapping determines how an information model is >> to be encoded for a specific use (e.g. IPFIX). The encoding >> may take whatever approach allows both ends to do something >> useful with the data. As you point out there is some useful >> stuff which can be done with the length alone. >> >> Concept #2: Derived typing. By defining a base set of types, >> which must be understood, derived types can be defined in terms >> of these base types. For instance Diameter does this. This >> enables a core set of types to be defined and coded to, but >> also enables richer types later on with an appropriate fallback >> for systems which aren't aware of the extensions. >> >> The benefit I see of explicit typing is that instead of seeing >> that a field is of length "4 bytes" something which will be seen >> A LOT. Is that we can instead differentiate between IPAddr, Integer, >> unsigned integer, and float. This seems handy to me. >> >> Your argument seems centered around the long term potential of >> adding lots of fixed length fields whose base length is not 4,8 >> or 16 bytes long, for instance MACAddr (although if we agree on >> going the type route, I'd recommend adding MACAddr and then you >> have 1,2,4,6,8 and 16 bytes covered. I guess I don't foresee >> a lot more fixed length fields coming down the pipe in the IPFIX >> domain. Hence my desire to make it more explicit for the types >> which do occur all the time and are likely points of extension. >> WHY? Because then a tool could at least present these in a meaningful >> way. As opposed to incorrectly formatting a float type as an int. >> >> But hey, either way will work. If the goal is to limit the >> amount of changes, I'll (grudgingly) go along. I haven't heard >> any opinions from anyone but you in either direction. So, >> at this point I don't see a quorom either way... >> >> Regards, >> >> Jeff Meyer >> >> > -----Original Message----- >> > From: Mark Fullmer [mailto:maf@eng.oar.net] >> > Sent: Tuesday, April 29, 2003 2:53 PM >> > To: MEYER,JEFFREY D (HP-Cupertino,ex1) >> > Cc: ipfix@net.doit.wisc.edu >> > Subject: Re: [ipfix] drawing the line between protocol and data model >> > docu ment >> > >> > >> > IMHO crippling the protocol to serve the needs of a documentation tool >> > is just silly. >> > >> > It would be possible to use both a Type name and a Length instead of >> > the existing Length, which would allow a collector to process >> > unknown Types. >> > >> > With the existing v9 protocol a collector can ignore an >> > unknown attribute, >> > or possibly just pass it on to post processing tools which >> > might understand >> > it. >> > >> > Your suggested changes no longer allow a collector to ignore unknown >> > attributes if the Type is also unknown. Resorting to encoding all >> > new Types after the initial protocol spec as octet strings is (IMHO) >> > not acceptable. >> > >> > I'm still trying to understand what the benefits to explicit >> > Types sent >> > in templates are. >> > >> > mark >> > >> > On Tue, Apr 29, 2003 at 02:50:02PM -0400, MEYER,JEFFREY D >> > (HP-Cupertino,ex1) wrote: >> > > Juergen, >> > > >> > > I agree that the data model should have a well defined type >> > > system. And that that typing system should reflect the >> > requirements. >> > > >> > > This is in part why I was hoping that the protocol itself would >> > > be more explicit here. (See my mail >> > > http://ipfix.doit.wisc.edu/archive/1604.html) >> > > >> > > There is also a type list in that mail. However, I would suggest >> > > using the names from XML-Schema part 2 Data types, as this would >> > > facilitate creating a formal/normative definition, versus >> > reinventing >> > > names for the types. >> > > >> > > Here are the type names from XML-Schema/IPDR: >> > > >> > > boolean - true/false (may be encoded in byte) >> > > byte >> > > unsignedByte >> > > short >> > > unsignedShort >> > > int - 32-bit signed >> > > long - 64-bit signed >> > > unsignedInt >> > > unsignedLong >> > > float >> > > double >> > > dateTime - 32-bit integer representing seconds since EPOCH >> > > (1/1/1970 0:00 GMT) >> > > string - UTF-8 formatted text >> > > ipdr:dateTimeMsec - 64-bit integer representing msecs >> > since EPOCH >> > > ipdr:ipV4Addr >> > > ipdr:ipV6Addr >> > > ipdr:UUID >> > > hexBinary - arbitrary sequence of octets >> > > >> > > If the protocol itself has a basic "octetString" (or from >> > XML, hexBinary) >> > > payload, >> > > then this can be used to carry new classes of information >> > in the future w/o >> > > rejiggering the protocol itself. >> > > >> > > In addition other "derived" types may be created from the >> > base types. For >> > > instance in IPDR, the ipdr:dateTimeMsec is derived from long. >> > > >> > > Regards, >> > > >> > > Jeff Meyer >> > > >> > > > -----Original Message----- >> > > > From: Juergen Quittek [mailto:quittek@ccrle.nec.de] >> > > > Sent: Monday, April 28, 2003 2:06 PM >> > > > To: ipfix@net.doit.wisc.edu >> > > > Subject: [ipfix] drawing the line between protocol and data model >> > > > document >> > > > >> > > > >> > > > Hi all, >> > > > >> > > > When shaping the planned documents, we have to draw a line >> > > > between the protocol and the data model. >> > > > >> > > > Definitely, the data model document defines all field types, >> > > > but does it also define basic data types, such as '16 bit >> > > > unsigned integer'? >> > > > >> > > > Maybe we should have a set of basic data types defined in >> > > > the protocol document and restrict the data model doc to >> > these types? >> > > > This would further structure the data model. >> > > > >> > > > Any thoughts? >> > > > >> > > > Juergen >> > > > >> > > > -- >> > > > Help mailto:majordomo@net.doit.wisc.edu and say "help" >> > > > in message body >> > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >> > > > "unsubscribe ipfix" in message body >> > > > Archive http://ipfix.doit.wisc.edu/archive/ >> > > > >> > > >> > > -- >> > > Help mailto:majordomo@net.doit.wisc.edu and say >> > "help" in message body >> > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >> > > "unsubscribe ipfix" in message body >> > > Archive http://ipfix.doit.wisc.edu/archive/ >> > >> >> -- >> Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body >> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >> "unsubscribe ipfix" in message body >> Archive http://ipfix.doit.wisc.edu/archive/ > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu May 1 12:03:54 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA19439 for ; Thu, 1 May 2003 12:03:54 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19BGNl-0002Tq-00 for ipfix-list@mil.doit.wisc.edu; Thu, 01 May 2003 10:54:01 -0500 Received: from [64.95.122.13] (helo=RS-SC-EXC4.rs.riverstonenet.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19BGNj-0002Tj-00 for ipfix@net.doit.wisc.edu; Thu, 01 May 2003 10:53:59 -0500 Received: from riverstonenet.com ([172.17.6.3]) by RS-SC-EXC4.rs.riverstonenet.com with Microsoft SMTPSVC(5.0.2195.4905); Thu, 1 May 2003 08:53:57 -0700 Message-ID: <3EB14313.81E37D40@riverstonenet.com> Date: Thu, 01 May 2003 11:53:55 -0400 From: calato@riverstonenet.com X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.7-10 i686) X-Accept-Language: en MIME-Version: 1.0 To: Mark Fullmer CC: "MEYER,JEFFREY D (HP-Cupertino,ex1)" , ipfix@net.doit.wisc.edu Subject: Re: [ipfix] drawing the line between protocol and data model docu ment References: <1D3D2C371FCBD947A7897FABBD3533A566BA84@xsun01.ptp.hp.com> <20030501113832.A32141@net.ohio-state.edu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 01 May 2003 15:53:58.0195 (UTC) FILETIME=[E05CB030:01C30FF9] Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Mark Fullmer wrote: > > So the difference boils down to a tool being able to display > > = 0x01020304 > vs > = 1.2.3.4 > > In this situation someone will either grab a new version of the > software or an updated data definition to resolve the unknown attribute, > which would also define it's type. > I don't see much value in that. Paul > mark > > On Tue, Apr 29, 2003 at 06:25:11PM -0400, MEYER,JEFFREY D (HP-Cupertino,ex1) wrote: > > Mark, > > > > I'll try to clarify: > > > > Concept #1: The information model and the encoding are separate. > > A well defined mapping determines how an information model is > > to be encoded for a specific use (e.g. IPFIX). The encoding > > may take whatever approach allows both ends to do something > > useful with the data. As you point out there is some useful > > stuff which can be done with the length alone. > > > > Concept #2: Derived typing. By defining a base set of types, > > which must be understood, derived types can be defined in terms > > of these base types. For instance Diameter does this. This > > enables a core set of types to be defined and coded to, but > > also enables richer types later on with an appropriate fallback > > for systems which aren't aware of the extensions. > > > > The benefit I see of explicit typing is that instead of seeing > > that a field is of length "4 bytes" something which will be seen > > A LOT. Is that we can instead differentiate between IPAddr, Integer, > > unsigned integer, and float. This seems handy to me. > > > > Your argument seems centered around the long term potential of > > adding lots of fixed length fields whose base length is not 4,8 > > or 16 bytes long, for instance MACAddr (although if we agree on > > going the type route, I'd recommend adding MACAddr and then you > > have 1,2,4,6,8 and 16 bytes covered. I guess I don't foresee > > a lot more fixed length fields coming down the pipe in the IPFIX > > domain. Hence my desire to make it more explicit for the types > > which do occur all the time and are likely points of extension. > > WHY? Because then a tool could at least present these in a meaningful > > way. As opposed to incorrectly formatting a float type as an int. > > > > But hey, either way will work. If the goal is to limit the > > amount of changes, I'll (grudgingly) go along. I haven't heard > > any opinions from anyone but you in either direction. So, > > at this point I don't see a quorom either way... > > > > Regards, > > > > Jeff Meyer > > > > > > > -----Original Message----- > > > From: Mark Fullmer [mailto:maf@eng.oar.net] > > > Sent: Tuesday, April 29, 2003 2:53 PM > > > To: MEYER,JEFFREY D (HP-Cupertino,ex1) > > > Cc: ipfix@net.doit.wisc.edu > > > Subject: Re: [ipfix] drawing the line between protocol and data model > > > docu ment > > > > > > > > > IMHO crippling the protocol to serve the needs of a documentation tool > > > is just silly. > > > > > > It would be possible to use both a Type name and a Length instead of > > > the existing Length, which would allow a collector to process > > > unknown Types. > > > > > > With the existing v9 protocol a collector can ignore an > > > unknown attribute, > > > or possibly just pass it on to post processing tools which > > > might understand > > > it. > > > > > > Your suggested changes no longer allow a collector to ignore unknown > > > attributes if the Type is also unknown. Resorting to encoding all > > > new Types after the initial protocol spec as octet strings is (IMHO) > > > not acceptable. > > > > > > I'm still trying to understand what the benefits to explicit > > > Types sent > > > in templates are. > > > > > > mark > > > > > > On Tue, Apr 29, 2003 at 02:50:02PM -0400, MEYER,JEFFREY D > > > (HP-Cupertino,ex1) wrote: > > > > Juergen, > > > > > > > > I agree that the data model should have a well defined type > > > > system. And that that typing system should reflect the > > > requirements. > > > > > > > > This is in part why I was hoping that the protocol itself would > > > > be more explicit here. (See my mail > > > > http://ipfix.doit.wisc.edu/archive/1604.html) > > > > > > > > There is also a type list in that mail. However, I would suggest > > > > using the names from XML-Schema part 2 Data types, as this would > > > > facilitate creating a formal/normative definition, versus > > > reinventing > > > > names for the types. > > > > > > > > Here are the type names from XML-Schema/IPDR: > > > > > > > > boolean - true/false (may be encoded in byte) > > > > byte > > > > unsignedByte > > > > short > > > > unsignedShort > > > > int - 32-bit signed > > > > long - 64-bit signed > > > > unsignedInt > > > > unsignedLong > > > > float > > > > double > > > > dateTime - 32-bit integer representing seconds since EPOCH > > > > (1/1/1970 0:00 GMT) > > > > string - UTF-8 formatted text > > > > ipdr:dateTimeMsec - 64-bit integer representing msecs > > > since EPOCH > > > > ipdr:ipV4Addr > > > > ipdr:ipV6Addr > > > > ipdr:UUID > > > > hexBinary - arbitrary sequence of octets > > > > > > > > If the protocol itself has a basic "octetString" (or from > > > XML, hexBinary) > > > > payload, > > > > then this can be used to carry new classes of information > > > in the future w/o > > > > rejiggering the protocol itself. > > > > > > > > In addition other "derived" types may be created from the > > > base types. For > > > > instance in IPDR, the ipdr:dateTimeMsec is derived from long. > > > > > > > > Regards, > > > > > > > > Jeff Meyer > > > > > > > > > -----Original Message----- > > > > > From: Juergen Quittek [mailto:quittek@ccrle.nec.de] > > > > > Sent: Monday, April 28, 2003 2:06 PM > > > > > To: ipfix@net.doit.wisc.edu > > > > > Subject: [ipfix] drawing the line between protocol and data model > > > > > document > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > When shaping the planned documents, we have to draw a line > > > > > between the protocol and the data model. > > > > > > > > > > Definitely, the data model document defines all field types, > > > > > but does it also define basic data types, such as '16 bit > > > > > unsigned integer'? > > > > > > > > > > Maybe we should have a set of basic data types defined in > > > > > the protocol document and restrict the data model doc to > > > these types? > > > > > This would further structure the data model. > > > > > > > > > > Any thoughts? > > > > > > > > > > Juergen > > > > > > > > > > -- > > > > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > > > > > in message body > > > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > > > "unsubscribe ipfix" in message body > > > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > > > > > > -- > > > > Help mailto:majordomo@net.doit.wisc.edu and say > > > "help" in message body > > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > > "unsubscribe ipfix" in message body > > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu May 1 12:11:58 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA19932 for ; Thu, 1 May 2003 12:11:58 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19BGai-0002q1-00 for ipfix-list@mil.doit.wisc.edu; Thu, 01 May 2003 11:07:24 -0500 Received: from zrc2s0jx.nortelnetworks.com ([47.103.122.112]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19BGag-0002pv-00 for ipfix@net.doit.wisc.edu; Thu, 01 May 2003 11:07:23 -0500 Received: from zsc3c028.us.nortel.com (zsc3c028.us.nortel.com [47.81.138.28]) by zrc2s0jx.nortelnetworks.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id h41G7EV13425; Thu, 1 May 2003 11:07:14 -0500 (CDT) Received: by zsc3c028.us.nortel.com with Internet Mail Service (5.5.2653.19) id <2R1CMFWF>; Thu, 1 May 2003 09:07:15 -0700 Message-ID: <0A11633F61BD9F40B43ABCC694004F93F18F1F@zsc3c026.us.nortel.com> From: "Reinaldo Penno" To: "'calato@riverstonenet.com'" , ipfixx Subject: RE: [ipfix] Issue tracking Date: Thu, 1 May 2003 09:07:11 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C30FFB.B90E8D94" Precedence: bulk Sender: majordomo listserver This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C30FFB.B90E8D94 Content-Type: text/plain; charset="ISO-8859-1" Good idea...This is something that has also bothered me. Sometimes we spend days on some threads and in the end they just fade away (with most likely people interpreting the outcome differently)... > -----Original Message----- > From: calato@riverstonenet.com [mailto:calato@riverstonenet.com] > Sent: Thursday, May 01, 2003 11:29 AM > To: ipfixx > Subject: [ipfix] Issue tracking > > > > > One thing that has bugged me for a long time about this > working group is there is no issue tracking and resolution > mechanism. Things are discussed on the list with no final word. > > Since I'm working on the data model and protocol documents > I'm proposing issue tracking and resolution for those > areas. > > I'd like one document which tracks issues along with current > status and final resolutions. I'll be happy to maintain it. > > If there is agreement, I'll start the process now before > too many issues arise. > > Comments please. > > Paul > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" > in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > ------_=_NextPart_001_01C30FFB.B90E8D94 Content-Type: text/html; charset="ISO-8859-1" Content-Transfer-Encoding: quoted-printable RE: [ipfix] Issue tracking

Good idea...This is something that has also bothered = me. Sometimes we spend days on some threads and in the end they just = fade away (with most likely people interpreting the outcome = differently)...

> -----Original Message-----
> From: calato@riverstonenet.com [mailto:calato@riverstonenet.com= ]
> Sent: Thursday, May 01, 2003 11:29 AM
> To: ipfixx
> Subject: [ipfix] Issue tracking
>
>
>
>
> One thing that has bugged me for a long time = about this
> working group is there is no issue tracking and = resolution
> mechanism. Things are discussed on the list = with no final word.
>
> Since I'm working on the data model and = protocol documents
> I'm proposing issue tracking and resolution for = those
> areas.
>
> I'd like one document which tracks issues along = with current
> status and final resolutions. I'll be happy to = maintain it.
>
> If there is agreement, I'll start the process = now before
> too many issues arise.
>
> Comments please.
>
> Paul
>
> --
> Help        = mailto:majordomo@net.doit.wi= sc.edu and say "help"
> in message body
> Unsubscribe mailto:majordomo@net.doit.wi= sc.edu and say
> "unsubscribe ipfix" in message = body
> Archive     http://ipfix.doit.wisc.edu/archive/
>

------_=_NextPart_001_01C30FFB.B90E8D94-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu May 1 13:24:57 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA23802 for ; Thu, 1 May 2003 13:24:56 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19BHeL-0004Nl-00 for ipfix-list@mil.doit.wisc.edu; Thu, 01 May 2003 12:15:13 -0500 Received: from atlrel6.hp.com ([156.153.255.205]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19BHeJ-0004Lt-00 for ipfix@net.doit.wisc.edu; Thu, 01 May 2003 12:15:11 -0500 Received: from xatlrelay1.atl.hp.com (xatlrelay1.atl.hp.com [15.45.89.190]) by atlrel6.hp.com (Postfix) with ESMTP id 29C121C01178; Thu, 1 May 2003 13:14:16 -0400 (EDT) Received: from xatlbh4.atl.hp.com (xatlbh4.atl.hp.com [15.45.89.189]) by xatlrelay1.atl.hp.com (Postfix) with ESMTP id D273B1C000BF; Thu, 1 May 2003 13:14:12 -0400 (EDT) Received: by xatlbh4.atl.hp.com with Internet Mail Service (5.5.2655.55) id ; Thu, 1 May 2003 13:14:12 -0400 Message-ID: <1D3D2C371FCBD947A7897FABBD3533A566BA93@xsun01.ptp.hp.com> From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" To: "'Reinaldo Penno'" , "'calato@riverstonenet.com'" , ipfixx Subject: RE: [ipfix] Issue tracking Date: Thu, 1 May 2003 13:14:05 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2655.55) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C31005.118BA070" Precedence: bulk Sender: majordomo listserver This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C31005.118BA070 Content-Type: text/plain; charset="iso-8859-1" Paul, Great idea and thanks for volunteering. I'm all for it. -- Jeff -----Original Message----- From: Reinaldo Penno [mailto:rpenno@nortelnetworks.com] Sent: Thursday, May 01, 2003 9:07 AM To: 'calato@riverstonenet.com'; ipfixx Subject: RE: [ipfix] Issue tracking Good idea...This is something that has also bothered me. Sometimes we spend days on some threads and in the end they just fade away (with most likely people interpreting the outcome differently)... > -----Original Message----- > From: calato@riverstonenet.com [ mailto:calato@riverstonenet.com ] > Sent: Thursday, May 01, 2003 11:29 AM > To: ipfixx > Subject: [ipfix] Issue tracking > > > > > One thing that has bugged me for a long time about this > working group is there is no issue tracking and resolution > mechanism. Things are discussed on the list with no final word. > > Since I'm working on the data model and protocol documents > I'm proposing issue tracking and resolution for those > areas. > > I'd like one document which tracks issues along with current > status and final resolutions. I'll be happy to maintain it. > > If there is agreement, I'll start the process now before > too many issues arise. > > Comments please. > > Paul > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" > in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > ------_=_NextPart_001_01C31005.118BA070 Content-Type: text/html; charset="iso-8859-1" RE: [ipfix] Issue tracking
Paul,
 
  Great idea and thanks for volunteering.  I'm all for it.
 
-- Jeff
-----Original Message-----
From: Reinaldo Penno [mailto:rpenno@nortelnetworks.com]
Sent: Thursday, May 01, 2003 9:07 AM
To: 'calato@riverstonenet.com'; ipfixx
Subject: RE: [ipfix] Issue tracking

Good idea...This is something that has also bothered me. Sometimes we spend days on some threads and in the end they just fade away (with most likely people interpreting the outcome differently)...

> -----Original Message-----
> From: calato@riverstonenet.com [mailto:calato@riverstonenet.com]
> Sent: Thursday, May 01, 2003 11:29 AM
> To: ipfixx
> Subject: [ipfix] Issue tracking
>
>
>
>
> One thing that has bugged me for a long time about this
> working group is there is no issue tracking and resolution
> mechanism. Things are discussed on the list with no final word.
>
> Since I'm working on the data model and protocol documents
> I'm proposing issue tracking and resolution for those
> areas.
>
> I'd like one document which tracks issues along with current
> status and final resolutions. I'll be happy to maintain it.
>
> If there is agreement, I'll start the process now before
> too many issues arise.
>
> Comments please.
>
> Paul
>
> --
> Help        mailto:majordomo@net.doit.wisc.edu and say "help"
> in message body
> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say
> "unsubscribe ipfix" in message body
> Archive     http://ipfix.doit.wisc.edu/archive/
>

------_=_NextPart_001_01C31005.118BA070-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu May 1 13:53:17 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA25135 for ; Thu, 1 May 2003 13:53:16 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19BI7i-00052j-00 for ipfix-list@mil.doit.wisc.edu; Thu, 01 May 2003 12:45:34 -0500 Received: from usmail.xacct.com ([204.253.100.12]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19BI7g-00052c-00 for ipfix@net.doit.wisc.edu; Thu, 01 May 2003 12:45:32 -0500 Received: from Givoly (inside.us.xacct.com [204.253.100.102]) by usmail.xacct.com (8.11.2/8.11.2) with SMTP id h41HkQi07084; Thu, 1 May 2003 10:46:43 -0700 From: "Tal Givoly" To: , "ipfixx" Subject: RE: [ipfix] Issue tracking Date: Thu, 1 May 2003 10:40:48 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-Reply-To: <3EB13D30.DFE75100@riverstonenet.com> Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Paul, Clearly a useful suggestion. The fact that no issues are more formally tracked and resolved has been slowing things down and making them more difficult to follow. Tal -----Original Message----- From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu]On Behalf Of calato@riverstonenet.com Sent: Thursday, May 01, 2003 8:29 AM To: ipfixx Subject: [ipfix] Issue tracking One thing that has bugged me for a long time about this working group is there is no issue tracking and resolution mechanism. Things are discussed on the list with no final word. Since I'm working on the data model and protocol documents I'm proposing issue tracking and resolution for those areas. I'd like one document which tracks issues along with current status and final resolutions. I'll be happy to maintain it. If there is agreement, I'll start the process now before too many issues arise. Comments please. Paul -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu May 1 13:48:27 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA24932 for ; Thu, 1 May 2003 13:48:27 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19BI1b-0004tZ-00 for ipfix-list@mil.doit.wisc.edu; Thu, 01 May 2003 12:39:15 -0500 Received: from palrel11.hp.com ([156.153.255.246]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19BI1Z-0004tR-00 for ipfix@net.doit.wisc.edu; Thu, 01 May 2003 12:39:13 -0500 Received: from xparelay2.ptp.hp.com (xparelay2.ptp.hp.com [15.1.28.65]) by palrel11.hp.com (Postfix) with ESMTP id 1F1801C02457; Thu, 1 May 2003 10:39:13 -0700 (PDT) Received: from xpabh2.ptp.hp.com (xpabh2.ptp.hp.com [15.1.28.61]) by xparelay2.ptp.hp.com (Postfix) with ESMTP id EA3801C00A6E; Thu, 1 May 2003 10:39:12 -0700 (PDT) Received: by xpabh2.ptp.hp.com with Internet Mail Service (5.5.2655.55) id ; Thu, 1 May 2003 10:39:12 -0700 Message-ID: <1D3D2C371FCBD947A7897FABBD3533A566BA94@xsun01.ptp.hp.com> From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" To: "'Mark Fullmer'" , "MEYER,JEFFREY D (HP-Cupertino,ex1)" Cc: ipfix@net.doit.wisc.edu Subject: RE: [ipfix] drawing the line between protocol and data model docu ment Date: Thu, 1 May 2003 10:39:10 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2655.55) Content-Type: text/plain; charset="iso-8859-1" Precedence: bulk Sender: majordomo listserver vs. = 16909060 (integer) vs. = whatever the heck 0x01020304 represents in IEEE Floating Point notation. (something around 4.06) So, yes that's all it buys you. Since both alternatives take the same amount of space, I'd prefer the one with more intrinsic information. But, I guess I'm in the minority :-( Regards, Jeff Meyer > -----Original Message----- > From: Mark Fullmer [mailto:maf@eng.oar.net] > Sent: Thursday, May 01, 2003 8:39 AM > To: MEYER,JEFFREY D (HP-Cupertino,ex1) > Cc: ipfix@net.doit.wisc.edu > Subject: Re: [ipfix] drawing the line between protocol and data model > docu ment > > > So the difference boils down to a tool being able to display > > = 0x01020304 > vs > = 1.2.3.4 > > In this situation someone will either grab a new version of the > software or an updated data definition to resolve the unknown > attribute, > which would also define it's type. > > mark > > On Tue, Apr 29, 2003 at 06:25:11PM -0400, MEYER,JEFFREY D > (HP-Cupertino,ex1) wrote: > > Mark, > > > > I'll try to clarify: > > > > Concept #1: The information model and the encoding are separate. > > A well defined mapping determines how an information model is > > to be encoded for a specific use (e.g. IPFIX). The encoding > > may take whatever approach allows both ends to do something > > useful with the data. As you point out there is some useful > > stuff which can be done with the length alone. > > > > Concept #2: Derived typing. By defining a base set of types, > > which must be understood, derived types can be defined in terms > > of these base types. For instance Diameter does this. This > > enables a core set of types to be defined and coded to, but > > also enables richer types later on with an appropriate fallback > > for systems which aren't aware of the extensions. > > > > The benefit I see of explicit typing is that instead of seeing > > that a field is of length "4 bytes" something which will be seen > > A LOT. Is that we can instead differentiate between > IPAddr, Integer, > > unsigned integer, and float. This seems handy to me. > > > > Your argument seems centered around the long term potential of > > adding lots of fixed length fields whose base length is not 4,8 > > or 16 bytes long, for instance MACAddr (although if we agree on > > going the type route, I'd recommend adding MACAddr and then you > > have 1,2,4,6,8 and 16 bytes covered. I guess I don't foresee > > a lot more fixed length fields coming down the pipe in the IPFIX > > domain. Hence my desire to make it more explicit for the types > > which do occur all the time and are likely points of extension. > > WHY? Because then a tool could at least present these in a > meaningful > > way. As opposed to incorrectly formatting a float type as an int. > > > > But hey, either way will work. If the goal is to limit the > > amount of changes, I'll (grudgingly) go along. I haven't heard > > any opinions from anyone but you in either direction. So, > > at this point I don't see a quorom either way... > > > > Regards, > > > > Jeff Meyer > > > > > > > -----Original Message----- > > > From: Mark Fullmer [mailto:maf@eng.oar.net] > > > Sent: Tuesday, April 29, 2003 2:53 PM > > > To: MEYER,JEFFREY D (HP-Cupertino,ex1) > > > Cc: ipfix@net.doit.wisc.edu > > > Subject: Re: [ipfix] drawing the line between protocol > and data model > > > docu ment > > > > > > > > > IMHO crippling the protocol to serve the needs of a > documentation tool > > > is just silly. > > > > > > It would be possible to use both a Type name and a Length > instead of > > > the existing Length, which would allow a collector to process > > > unknown Types. > > > > > > With the existing v9 protocol a collector can ignore an > > > unknown attribute, > > > or possibly just pass it on to post processing tools which > > > might understand > > > it. > > > > > > Your suggested changes no longer allow a collector to > ignore unknown > > > attributes if the Type is also unknown. Resorting to encoding all > > > new Types after the initial protocol spec as octet > strings is (IMHO) > > > not acceptable. > > > > > > I'm still trying to understand what the benefits to explicit > > > Types sent > > > in templates are. > > > > > > mark > > > > > > On Tue, Apr 29, 2003 at 02:50:02PM -0400, MEYER,JEFFREY D > > > (HP-Cupertino,ex1) wrote: > > > > Juergen, > > > > > > > > I agree that the data model should have a well defined type > > > > system. And that that typing system should reflect the > > > requirements. > > > > > > > > This is in part why I was hoping that the protocol > itself would > > > > be more explicit here. (See my mail > > > > http://ipfix.doit.wisc.edu/archive/1604.html) > > > > > > > > There is also a type list in that mail. However, I > would suggest > > > > using the names from XML-Schema part 2 Data types, as this would > > > > facilitate creating a formal/normative definition, versus > > > reinventing > > > > names for the types. > > > > > > > > Here are the type names from XML-Schema/IPDR: > > > > > > > > boolean - true/false (may be encoded in byte) > > > > byte > > > > unsignedByte > > > > short > > > > unsignedShort > > > > int - 32-bit signed > > > > long - 64-bit signed > > > > unsignedInt > > > > unsignedLong > > > > float > > > > double > > > > dateTime - 32-bit integer representing seconds > since EPOCH > > > > (1/1/1970 0:00 GMT) > > > > string - UTF-8 formatted text > > > > ipdr:dateTimeMsec - 64-bit integer representing msecs > > > since EPOCH > > > > ipdr:ipV4Addr > > > > ipdr:ipV6Addr > > > > ipdr:UUID > > > > hexBinary - arbitrary sequence of octets > > > > > > > > If the protocol itself has a basic "octetString" (or from > > > XML, hexBinary) > > > > payload, > > > > then this can be used to carry new classes of information > > > in the future w/o > > > > rejiggering the protocol itself. > > > > > > > > In addition other "derived" types may be created from the > > > base types. For > > > > instance in IPDR, the ipdr:dateTimeMsec is derived from long. > > > > > > > > Regards, > > > > > > > > Jeff Meyer > > > > > > > > > -----Original Message----- > > > > > From: Juergen Quittek [mailto:quittek@ccrle.nec.de] > > > > > Sent: Monday, April 28, 2003 2:06 PM > > > > > To: ipfix@net.doit.wisc.edu > > > > > Subject: [ipfix] drawing the line between protocol > and data model > > > > > document > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > When shaping the planned documents, we have to draw a line > > > > > between the protocol and the data model. > > > > > > > > > > Definitely, the data model document defines all field types, > > > > > but does it also define basic data types, such as '16 bit > > > > > unsigned integer'? > > > > > > > > > > Maybe we should have a set of basic data types defined in > > > > > the protocol document and restrict the data model doc to > > > these types? > > > > > This would further structure the data model. > > > > > > > > > > Any thoughts? > > > > > > > > > > Juergen > > > > > > > > > > -- > > > > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > > > > > in message body > > > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > > > "unsubscribe ipfix" in message body > > > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > > > > > > -- > > > > Help mailto:majordomo@net.doit.wisc.edu and say > > > "help" in message body > > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > > "unsubscribe ipfix" in message body > > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say > "help" in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu May 1 14:13:54 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA26401 for ; Thu, 1 May 2003 14:13:53 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19BILr-0005N4-00 for ipfix-list@mil.doit.wisc.edu; Thu, 01 May 2003 13:00:11 -0500 Received: from tokyo.ccrle.nec.de ([195.37.70.2]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19BILp-0005MU-00 for ipfix@net.doit.wisc.edu; Thu, 01 May 2003 13:00:10 -0500 Received: from venus.office (venus.office [10.1.1.11]) by tokyo.ccrle.nec.de (8.12.9/8.12.8) with ESMTP id h41I04VI007489; Thu, 1 May 2003 20:00:04 +0200 (CEST) Received: from [10.1.1.26] (dial02.office [10.1.1.26]) by venus.office (Postfix on SuSE Linux eMail Server 3.0) with ESMTP id 39847A1500; Thu, 1 May 2003 19:53:25 +0200 (CEST) Date: Thu, 01 May 2003 20:01:31 +0200 From: Juergen Quittek To: Tal Givoly , calato@riverstonenet.com, ipfixx Subject: RE: [ipfix] Issue tracking Message-ID: <20247023.1051819291@[10.1.1.26]> In-Reply-To: References: X-Mailer: Mulberry/2.1.2 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit There are tools for issue tracking. What about setting up bugzilla for this purpose? Juergen -- Tal Givoly wrote on 01 May 2003 10:40 -0700: > Paul, > > Clearly a useful suggestion. The fact that no issues are more formally > tracked and resolved has been slowing things down and making them more > difficult to follow. > > Tal > > -----Original Message----- > From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu]On Behalf > Of calato@riverstonenet.com > Sent: Thursday, May 01, 2003 8:29 AM > To: ipfixx > Subject: [ipfix] Issue tracking > > > > > One thing that has bugged me for a long time about this working > group is there is no issue tracking and resolution mechanism. > Things are discussed on the list with no final word. > > Since I'm working on the data model and protocol documents > I'm proposing issue tracking and resolution for those > areas. > > I'd like one document which tracks issues along with current > status and final resolutions. I'll be happy to maintain it. > > If there is agreement, I'll start the process now before > too many issues arise. > > Comments please. > > Paul > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message > body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu May 1 14:16:41 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA26689 for ; Thu, 1 May 2003 14:16:41 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19BIPP-0005TH-00 for ipfix-list@mil.doit.wisc.edu; Thu, 01 May 2003 13:03:51 -0500 Received: from tokyo.ccrle.nec.de ([195.37.70.2]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19BIPN-0005T9-00 for ipfix@net.doit.wisc.edu; Thu, 01 May 2003 13:03:49 -0500 Received: from venus.office (venus.office [10.1.1.11]) by tokyo.ccrle.nec.de (8.12.9/8.12.8) with ESMTP id h41I3mVI007516; Thu, 1 May 2003 20:03:48 +0200 (CEST) Received: from [10.1.1.26] (dial02.office [10.1.1.26]) by venus.office (Postfix on SuSE Linux eMail Server 3.0) with ESMTP id 0D499A14F5; Thu, 1 May 2003 19:57:08 +0200 (CEST) Date: Thu, 01 May 2003 20:05:14 +0200 From: Juergen Quittek To: calato@riverstonenet.com, Reinaldo Penno Cc: "'Maurizio Molina'" , ipfix@net.doit.wisc.edu Subject: Re: [ipfix] export packet length Message-ID: <20469864.1051819514@[10.1.1.26]> In-Reply-To: <3EB13B94.6B4DEBC9@riverstonenet.com> References: <3EB13B94.6B4DEBC9@riverstonenet.com> X-Mailer: Mulberry/2.1.2 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit I agree. These two byte appear to be well spent. Juergen -- calato@riverstonenet.com wrote on 01 May 2003 11:21 -0400: > > > Since the only thing mandated for the transport is congestion awareness > it would seems to make sense to have a length field in the IPFIX > protocol. > > I don't like the idea of having different formats based on the > transport (someone mentioned that as an option). A 2 byte length > field seems a small price to pay or better transport independance. > > Paul > > >> Reinaldo Penno wrote: >> >> That's a good idea. >> >> Other protocols like HTTP and SIP make it mandatory to include the >> Content-Length header (which tell the length of the payload) when >> carried over TCP. It helps a lot when dealing with fragmentation. >> >> Regards, >> >> > -----Original Message----- >> > From: Maurizio Molina [mailto:molina@ccrle.nec.de] >> > Sent: Friday, April 25, 2003 11:50 AM >> > To: ipfix@net.doit.wisc.edu >> > Subject: [ipfix] export packet length >> > >> > >> > Hi all, >> > I hope not to rise an issue already discussed in the WG. If >> > yes, I apologise in advance. If an IPFIX export packet is >> > carried over TCP, it's not unusual that it gets split across >> > multiple TCP segments. At the collector side, on the >> > contrary, the implementation of the parsing is much simpler >> > if only full export packet are parsed. If the information >> > about the WHOLE export packet were contained in the header, >> > it would be simple to wait until all the bytes composing an >> > export packets are read from the TCP socket before beginning >> > the parsing. Unfortunately, this information isn't currently >> > contained in Netflow 9 packet header (I suppose, due to >> > Netflow's 9 "bias" towards UDP...). Therefore, a parsing >> > implementation having to deal with this "fragentation" >> > problem without this information needs to be much more >> > complicated (e.g. getting step by step the lenght of each >> > flow set, see if it's all there, parse it and go on....). In >> > summary: if IPFIX has to be carried over tcp, having the >> > length of the export packet in the packet header would be >> > beneficial. Regards, Maurizio >> > >> > >> > -- >> > Maurizio Molina >> > Research Staff member >> > Network Laboratories Heidelberg >> > NEC Europe Ltd. >> > Kurfuersten-Anlage 36, 69115 Heidelberg, Germany. >> > Tel. (49)6221 90511-18 Fax: (49)6221 90511-55 >> > e-mail: molina@ccrle.nec.de >> > Web: www.ccrle.nec.de >> > >> > >> > >> > -- >> > Help mailto:majordomo@net.doit.wisc.edu and say "help" >> > in message body >> > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >> > "unsubscribe ipfix" in message body >> > Archive http://ipfix.doit.wisc.edu/archive/ >> > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu May 1 14:17:09 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA26728 for ; Thu, 1 May 2003 14:17:09 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19BIPR-0005TO-00 for ipfix-list@mil.doit.wisc.edu; Thu, 01 May 2003 13:03:53 -0500 Received: from atlrel7.hp.com ([156.153.255.213]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19BIPP-0005TF-00 for ipfix@net.doit.wisc.edu; Thu, 01 May 2003 13:03:51 -0500 Received: from xatlrelay1.atl.hp.com (xatlrelay1.atl.hp.com [15.45.89.190]) by atlrel7.hp.com (Postfix) with ESMTP id 72B181C0116F; Thu, 1 May 2003 14:03:50 -0400 (EDT) Received: from xatlbh3.atl.hp.com (xatlbh3.atl.hp.com [15.45.89.188]) by xatlrelay1.atl.hp.com (Postfix) with ESMTP id BCE071C009D2; Thu, 1 May 2003 14:03:49 -0400 (EDT) Received: by xatlbh3.atl.hp.com with Internet Mail Service (5.5.2655.55) id ; Thu, 1 May 2003 14:03:49 -0400 Message-ID: <1D3D2C371FCBD947A7897FABBD3533A566BA95@xsun01.ptp.hp.com> From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" To: "'Juergen Quittek'" , calato@riverstonenet.com Cc: "MEYER,JEFFREY D (HP-Cupertino,ex1)" , "'Nevil Brownlee'" , ipfix@net.doit.wisc.edu Subject: RE: [ipfix] Editors for IPFIX drafts Date: Thu, 1 May 2003 13:50:20 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2655.55) Content-Type: text/plain; charset="iso-8859-1" Precedence: bulk Sender: majordomo listserver Hi, If the human readable part is generated from the XML appendix, then I guess it doesn't matter which part you call normative, since they both originate from the same source. The development model I see is the following: Generate XML Info Model Run model through XSLT (xalan) translater to produce prose (consistent with RFC 2629) [I'll volunteer to write the style sheet translation) Incorporate the raw model as appendix and the generated prose description into the appropriate section (consistent w/ RFC 2629) along with other text of the Information Model Draft. Run Marshall's tool http://xml.resource.org/ to produce nroff style and HTML versions of document. Submit nroff style to RFC editor, post HTML style in addition to ipfix website (easier to navigate). For examples of the dual formatted output, see the IPDR submissions to IPDR located at: http://www.ipdr.org/documents/ipfix Regards, Jeff Meyer > -----Original Message----- > From: Juergen Quittek [mailto:quittek@ccrle.nec.de] > Sent: Thursday, May 01, 2003 8:21 AM > To: calato@riverstonenet.com > Cc: MEYER,JEFFREY D (HP-Cupertino,ex1); 'Nevil Brownlee'; > ipfix@net.doit.wisc.edu > Subject: Re: [ipfix] Editors for IPFIX drafts > > > Paul, > > I agree to Jeff, that there is some value in having an XML > representation > of the specification. However, in the end this document will > become an RFC > and it must be well readable to human beings. > > Therefore, I think that the main body must be human readable, > but we can > provide an appendix, that contains an equivalent machine > readable spec. > > Different to Jeff, I think that the hunam readable main body should be > the normative part, while the XML appendix is just given as > support for > users of XML tools allowing them to genrate protocol stacks > and/or tools > for consistency checking. > > As Jeff suggested, we can work with an XML representation > when developing > the spec and generate human readable text automatically from > the XML code. > This will help us avoiding fomal mistakes in the human > readable spec and we > would always have consistency between human readable and XML part. > > Juergen > > > -- calato@riverstonenet.com wrote on 01 May 2003 11:00 -0400: > > > > > I'm a little confused. Are we talking about XML descriptions > > as an appendix with prose as the main body or XML as the > > main body with tools translating it into a more readable > > form? Or XML as an appendix with tools translating it for > > use as sections mixed in with the prose? > > > > > > Paul > > > > Juergen Quittek wrote: > >> > >> Jeff, > >> > >> -- MEYER,JEFFREY D (HP-Cupertino,ex1) wrote on 28 April > 2003 09:20 -0700: > >> > > >> > Juergen, > >> > > >> > Thanks for the response. > >> > > >> > I agree XML in and of itself is marginally human readable. > >> > > >> > I think that basic tools such as Xalan > (http://xml.apache.org/), can > >> > be used to "transform" a base XML document into a > variety of formats, > >> > including formatting consistent with RFC's. Note that > all the IPDR > >> > RFC's were written in XML using Marshall Rose's xml2rfc tools > >> > (http://xml.resource.org/). > >> > >> I also like Marshall's tools. > >> If Paul agrees, we should use them for the data model document. > >> > >> > So the model I would picture is having the Normative > XML data model > >> > as an appendix, and having generated information from > this data model > >> > populate some of the body sections to aid human readers. > >> > >> This looks like a reasonable way to go. > >> > >> Juergen > >> > >> > Regards, > >> > > >> > Jeff Meyer > >> > > >> >> -----Original Message----- > >> >> From: Juergen Quittek [mailto:quittek@ccrle.nec.de] > >> >> Sent: Monday, April 28, 2003 8:24 AM > >> >> To: MEYER,JEFFREY D (HP-Cupertino,ex1); 'Nevil Brownlee'; > >> >> ipfix@net.doit.wisc.edu > >> >> Subject: RE: [ipfix] Editors for IPFIX drafts > >> >> > >> >> > >> >> Jeff, > >> >> > >> >> I do see a value in such a formal description. But there is a > >> >> trafe-off. > >> >> > >> >> I also see a degradation of human readability. You > loose means of > >> >> structuring your document text when using XML, for > example grouping > >> >> attributes that are related. (Of course you could do so in > >> >> XML, but it is > >> >> much harder to find the heading "Port-related Attributes" in > >> >> an XML document, > >> >> because headings cannot be well highlighted and numbering of > >> >> headings is > >> >> not well supported. > >> >> > >> >> Anyway, I like the idea of an XML encoding of the model, but > >> >> I we should > >> >> not go this way if we cannot find a good way of getting it > >> >> hunam readable > >> >> (maybe by applying formatting rules). > >> >> > >> >> An option would also be adding the XML code to the document > >> >> as an appendix. > >> >> > >> >> Juergen > >> >> > >> >> > >> >> -- MEYER,JEFFREY D (HP-Cupertino,ex1) wrote on 28 April 2003 > >> >> 10:57 -0400: > >> >> > >> >> > Hi, > >> >> > > >> >> > My particular interest on working on the > information model would > >> >> > be on developing a more formal and standard way of > describing the > >> >> > information elements which are used in IPFIX communication. > >> >> > > >> >> > The modelling of information elements should > ideally be done > >> >> > using a formal description language, so that tools can be > >> >> constructed > >> >> > which do not require humans to transcribe this information. > >> >> > > >> >> > I would propose the use of XML-Schema to provide > this formal > >> >> > description. Through the use of XSL processors, prose > >> >> based descriptions > >> >> > and tables can be produced if these are considered desireable. > >> >> > > >> >> > NOTE: describing the information model formally > does NOT mean > >> >> > that encoding and transport for IPFIX need to change > in any way. It > >> >> > simply means that rather than having as the base > >> >> description of information > >> >> > items some prose in a document, you instead have a > machine readable > >> >> > representation. > >> >> > > >> >> > > >> >> > As an example of a formal (but currently > incomplete) information > >> >> > model for IPFIX, see: > >> >> > http://www.ipdr.org/documents/ipfix/ipfixService-20020902.xml > >> >> > > >> >> > I would recommend making such a representation the > normative form > >> >> > of the IPFIX information model, much like SNMP MIBs. > >> >> > > >> >> > > >> >> > Please let me know if people see value in this exercise. > >> >> > > >> >> > Regards, > >> >> > > >> >> > Jeff Meyer > >> >> > > >> >> >> -----Original Message----- > >> >> >> From: Nevil Brownlee [mailto:n.brownlee@auckland.ac.nz] > >> >> >> Sent: Sunday, April 27, 2003 8:44 PM > >> >> >> To: ipfix@net.doit.wisc.edu > >> >> >> Subject: [ipfix] Editors for IPFIX drafts > >> >> >> > >> >> >> > >> >> >> > >> >> >> Hello all: > >> >> >> > >> >> >> Following the IPFIX meeting at the San Francisco > IETF we asked for > >> >> >> volunteers to edit the IPFIX drafts. The current list is: > >> >> >> > >> >> >> Applicability: Tanja Zseby, Reinaldo Penno > >> >> >> > >> >> >> Architecture: Ganesh Sadasivan, Nevil Brownlee > >> >> >> > >> >> >> Information Model: Paul Calato, Jeff Meyer, Juergen Quittek > >> >> >> > >> >> >> Protocol: Mark Fulmer, Paul Calato, Reinaldo Penno > >> >> >> > >> >> >> At this stage we already have old (expired) versions of the > >> >> >> Architecture (-02.txt) and Data Model (-01.txt) > drafts on the IPFIX > >> >> >> web page; would the editors of all four drafts please email > >> >> >> me an initial > >> >> >> version of their new drafts to published as placeholders > >> >> >> within the next > >> >> >> few weeks. > >> >> >> > >> >> >> After that, the editors will work on developing better > >> >> versions of the > >> >> >> drafts which can be discussed on the IPFIX list and submitted > >> >> >> by mid-June, > >> >> >> i.e. in plenty of time for the Vienna IETF meeting in July. > >> >> >> > >> >> >> Cheers, Nevil > >> >> >> > >> >> >> PS: Good to see some real discussion on the list. Lets > >> >> concentrate on > >> >> >> producing text the draft editors can use! > >> >> >> > >> >> >> > -------------------------------------------------------------- > >> >> >> --------- > >> >> >> Nevil Brownlee Director, Technology > >> >> Development > >> >> >> Phone: +64 9 373 7599 x88941 ITSS, The University > >> >> of Auckland > >> >> >> FAX: +64 9 373 7021 Private Bag 92019, Auckland, > >> >> New Zealand > >> >> >> > >> >> >> > >> >> >> ------------------------------------------------- > >> >> >> This mail sent through University of Auckland > >> >> >> http://www.auckland.ac.nz/ > >> >> >> > >> >> >> -- > >> >> >> Help mailto:majordomo@net.doit.wisc.edu and say "help" > >> >> >> in message body > >> >> >> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > >> >> >> "unsubscribe ipfix" in message body > >> >> >> Archive http://ipfix.doit.wisc.edu/archive/ > >> >> >> > >> >> > > >> >> > -- > >> >> > Help mailto:majordomo@net.doit.wisc.edu and say > >> >> "help" in message body > >> >> > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > >> >> > "unsubscribe ipfix" in message body > >> >> > Archive http://ipfix.doit.wisc.edu/archive/ > >> >> > >> >> > >> > >> -- > >> Help mailto:majordomo@net.doit.wisc.edu and say > "help" in message body > >> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > >> "unsubscribe ipfix" in message body > >> Archive http://ipfix.doit.wisc.edu/archive/ > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu May 1 14:53:05 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA29018 for ; Thu, 1 May 2003 14:53:05 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19BJ0C-0006IM-00 for ipfix-list@mil.doit.wisc.edu; Thu, 01 May 2003 13:41:52 -0500 Received: from atlrel9.hp.com ([156.153.255.214]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19BJ0B-0006IH-00 for ipfix@net.doit.wisc.edu; Thu, 01 May 2003 13:41:51 -0500 Received: from xatlrelay2.atl.hp.com (xatlrelay2.atl.hp.com [15.45.89.191]) by atlrel9.hp.com (Postfix) with ESMTP id 862F81C01409; Thu, 1 May 2003 14:41:50 -0400 (EDT) Received: from xatlbh4.atl.hp.com (xatlbh4.atl.hp.com [15.45.89.189]) by xatlrelay2.atl.hp.com (Postfix) with ESMTP id 0B4491C000A7; Thu, 1 May 2003 14:41:50 -0400 (EDT) Received: by xatlbh4.atl.hp.com with Internet Mail Service (5.5.2655.55) id ; Thu, 1 May 2003 14:41:49 -0400 Message-ID: <1D3D2C371FCBD947A7897FABBD3533A566BA99@xsun01.ptp.hp.com> From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" To: "'Juergen Quittek'" , Tal Givoly , calato@riverstonenet.com, ipfixx Subject: RE: [ipfix] Issue tracking Date: Thu, 1 May 2003 14:41:47 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2655.55) Content-Type: text/plain; charset="iso-8859-1" Precedence: bulk Sender: majordomo listserver Whatever Paul is willing to set up is fine w/ me... -- Jeff > -----Original Message----- > From: Juergen Quittek [mailto:quittek@ccrle.nec.de] > Sent: Thursday, May 01, 2003 11:02 AM > To: Tal Givoly; calato@riverstonenet.com; ipfixx > Subject: RE: [ipfix] Issue tracking > > > There are tools for issue tracking. > > What about setting up bugzilla for this purpose? > > Juergen > > > -- Tal Givoly wrote on 01 May 2003 10:40 -0700: > > > Paul, > > > > Clearly a useful suggestion. The fact that no issues are > more formally > > tracked and resolved has been slowing things down and > making them more > > difficult to follow. > > > > Tal > > > > -----Original Message----- > > From: majordomo listserver > [mailto:majordomo@mil.doit.wisc.edu]On Behalf > > Of calato@riverstonenet.com > > Sent: Thursday, May 01, 2003 8:29 AM > > To: ipfixx > > Subject: [ipfix] Issue tracking > > > > > > > > > > One thing that has bugged me for a long time about this working > > group is there is no issue tracking and resolution mechanism. > > Things are discussed on the list with no final word. > > > > Since I'm working on the data model and protocol documents > > I'm proposing issue tracking and resolution for those > > areas. > > > > I'd like one document which tracks issues along with current > > status and final resolutions. I'll be happy to maintain it. > > > > If there is agreement, I'll start the process now before > > too many issues arise. > > > > Comments please. > > > > Paul > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say > "help" in message > > body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say > "help" in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" > in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu May 1 17:18:33 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA11500 for ; Thu, 1 May 2003 17:18:32 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19BKYQ-0000to-00 for ipfix-list@mil.doit.wisc.edu; Thu, 01 May 2003 15:21:18 -0500 Received: from eng4.oar.net ([192.148.244.24]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 19BKYP-0000tj-00 for ipfix@net.doit.wisc.edu; Thu, 01 May 2003 15:21:17 -0500 Received: (qmail 33310 invoked by uid 4454); 1 May 2003 20:21:16 -0000 Date: Thu, 1 May 2003 16:21:16 -0400 From: Mark Fullmer To: calato@riverstonenet.com Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] drawing the line between protocol and data model document Message-ID: <20030501162116.B33173@net.ohio-state.edu> References: <1D3D2C371FCBD947A7897FABBD3533A566BA84@xsun01.ptp.hp.com> <3EB13485.3FA365D7@riverstonenet.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: <3EB13485.3FA365D7@riverstonenet.com>; from calato@riverstonenet.com on Thu, May 01, 2003 at 10:51:49AM -0400 Precedence: bulk Sender: majordomo listserver On Thu, May 01, 2003 at 10:51:49AM -0400, calato@riverstonenet.com wrote: > The richness of the protocol encoding specification is another > matter. The protocol would reference each element in the data > model and define exaclty how that element is encoded. In the above > example, the protocol is free to encode that as an unsigned byte > if the value is < 256. The ramifications of doing this with a template based system are a bit ugly. Consider a flow with just 2 counters -- octets, packets. Each could have a 1 to 8 byte encoding. That's a potential of 64 different templates for representing 1 type of flow. This (at first glance) looks like it adds considerable cycles to the encode / decode process to save a few bytes in the transport. The current v9 spec appears to allow this, but only for IN_BYTES and IN_PKTS. > counter with length > IN_BYTES 1 N N x 8 bits for bytes > associated with an IP Flow Is this true? Benoit? mark -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu May 1 17:19:55 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA11551 for ; Thu, 1 May 2003 17:19:55 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19BL8m-0001fc-00 for ipfix-list@mil.doit.wisc.edu; Thu, 01 May 2003 15:58:52 -0500 Received: from psg.com ([147.28.0.62]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19BL8k-0001fU-00 for ipfix@net.doit.wisc.edu; Thu, 01 May 2003 15:58:50 -0500 Received: from localhost ([127.0.0.1] helo=roam.psg.com) by psg.com with esmtp (Exim 3.36 #1) id 19BL8j-0000iR-02; Thu, 01 May 2003 20:58:49 +0000 Received: from localhost ([127.0.0.1] helo=roam.psg.com ident=randy) by roam.psg.com with esmtp (Exim 4.14) id 19BJ2h-000Az8-Q2; Thu, 01 May 2003 11:44:27 -0700 From: Randy Bush MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Date: Thu, 1 May 2003 11:44:27 -0700 To: calato@riverstonenet.com Cc: ipfixx Subject: Re: [ipfix] Issue tracking References: <3EB13D30.DFE75100@riverstonenet.com> Message-Id: Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit rob austein and i have set up an experimental ticket system for wgs' use in issue tracking. randy -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu May 1 17:21:18 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA11633 for ; Thu, 1 May 2003 17:21:17 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19BKLn-0000aE-00 for ipfix-list@mil.doit.wisc.edu; Thu, 01 May 2003 15:08:15 -0500 Received: from eng4.oar.net ([192.148.244.24]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 19BKLl-0000a8-00 for ipfix@net.doit.wisc.edu; Thu, 01 May 2003 15:08:13 -0500 Received: (qmail 33275 invoked by uid 4454); 1 May 2003 20:08:13 -0000 Date: Thu, 1 May 2003 16:08:13 -0400 From: Mark Fullmer To: "MEYER,JEFFREY D \(HP-Cupertino,ex1\)" Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] drawing the line between protocol and data model docu ment Message-ID: <20030501160813.A33173@net.ohio-state.edu> References: <1D3D2C371FCBD947A7897FABBD3533A566BA94@xsun01.ptp.hp.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: <1D3D2C371FCBD947A7897FABBD3533A566BA94@xsun01.ptp.hp.com>; from jeff.meyer2@hp.com on Thu, May 01, 2003 at 10:39:10AM -0700 Precedence: bulk Sender: majordomo listserver The problem I see is if we go down the road of having attribute semantics encoded in the protocol it's easy to go the next step of adding textual descriptions to the templates, so you get = 16909060, where 'SOMECOUNTER' is a text string learned dynamically from an options template or whatever. Then someone will want ifName and ifAlias mappings, then ifSpeed, etc. Others are welcome to chime in, but again IMHO the current model is what I prefer. If we have to go down the road of encoding attribute types it must be done in a way where new types can be added without breaking existing collectors, ie encode both a Type and a Length. mark On Thu, May 01, 2003 at 10:39:10AM -0700, MEYER,JEFFREY D (HP-Cupertino,ex1) wrote: > vs. > > = 16909060 (integer) > > vs. > > = whatever the heck 0x01020304 represents > in IEEE Floating Point notation. (something around 4.06) > > > So, yes that's all it buys you. Since both alternatives take > the same amount of space, I'd prefer the one with more > intrinsic information. But, I guess I'm in the minority :-( > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu May 1 18:06:58 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA14694 for ; Thu, 1 May 2003 18:06:58 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19BLxk-0002qc-00 for ipfix-list@mil.doit.wisc.edu; Thu, 01 May 2003 16:51:32 -0500 Received: from auds952.usa.alcatel.com ([143.209.238.7]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19BLxj-0002qV-00 for ipfix@net.doit.wisc.edu; Thu, 01 May 2003 16:51:31 -0500 Received: from alcatel.com (localhost [127.0.0.1]) by auds952.usa.alcatel.com (8.12.8p1/8.12.8) with ESMTP id h41LpRjB006041; Thu, 1 May 2003 16:51:27 -0500 (CDT) Message-ID: <3EB196DE.E3D22BD7@alcatel.com> Date: Thu, 01 May 2003 16:51:26 -0500 From: Alex Audu X-Mailer: Mozilla 4.72 [en] (WinNT; U) X-Accept-Language: en MIME-Version: 1.0 To: calato@riverstonenet.com CC: ipfixx Subject: Re: [ipfix] Issue tracking References: <3EB13D30.DFE75100@riverstonenet.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Great idea! I am for it! Cheers, Alex. calato@riverstonenet.com wrote: > One thing that has bugged me for a long time about this working > group is there is no issue tracking and resolution mechanism. > Things are discussed on the list with no final word. > > Since I'm working on the data model and protocol documents > I'm proposing issue tracking and resolution for those > areas. > > I'd like one document which tracks issues along with current > status and final resolutions. I'll be happy to maintain it. > > If there is agreement, I'll start the process now before > too many issues arise. > > Comments please. > > Paul > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu May 1 23:07:45 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA23347 for ; Thu, 1 May 2003 23:07:45 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19BQPN-0000xA-00 for ipfix-list@mil.doit.wisc.edu; Thu, 01 May 2003 21:36:21 -0500 Received: from atlrel8.hp.com ([156.153.255.206]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19BQPK-0000x4-00 for ipfix@net.doit.wisc.edu; Thu, 01 May 2003 21:36:18 -0500 Received: from xatlrelay2.atl.hp.com (xatlrelay2.atl.hp.com [15.45.89.191]) by atlrel8.hp.com (Postfix) with ESMTP id 706091C01D0E; Thu, 1 May 2003 22:36:18 -0400 (EDT) Received: from xatlbh3.atl.hp.com (xatlbh3.atl.hp.com [15.45.89.188]) by xatlrelay2.atl.hp.com (Postfix) with ESMTP id 3353D1C000BF; Thu, 1 May 2003 22:36:18 -0400 (EDT) Received: by xatlbh3.atl.hp.com with Internet Mail Service (5.5.2655.55) id ; Thu, 1 May 2003 22:36:18 -0400 Message-ID: <1D3D2C371FCBD947A7897FABBD3533A566BAA2@xsun01.ptp.hp.com> From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" To: "'Sebastian Zander'" , "MEYER,JEFFREY D (HP-Cupertino,ex1)" Cc: "'Mark Fullmer'" , ipfix@net.doit.wisc.edu Subject: RE: [ipfix] drawing the line between protocol and data model docu ment Date: Thu, 1 May 2003 22:36:17 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2655.55) Content-Type: text/plain; charset="iso-8859-1" Precedence: bulk Sender: majordomo listserver Sebastian, Thanks for speaking up! I agree with your restatement of the relationship between typing and semantics. Regarding types like UTF8 or OctetString, as I read the proposal from Benoit, the "Length Encoding" model would use a special value of 0xFFFF instead of length in the template. This implies that the encoding of the "variable length" field would be of the form (length) (value), versus fixed length types which can be encoded by just placing their (value) in the payload. (see http://ipfix.doit.wisc.edu/archive/1602.html) This same model would apply in a type based scheme, if the type were string or hexBinary (indicated by the appropriate integer identifier in the template), then the encoded value would still be of the form (length) (value) in the payload. Regardless of what template scheme is chosen, however, I think it is important that the protocol spec speak in terms of how types are encoded, not how each individual element declared in the information model is encoded. This makes the inevitable extension of the information model easier. Regards, Jeff Meyer > -----Original Message----- > From: Sebastian Zander [mailto:zander@fokus.fraunhofer.de] > Sent: Thursday, May 01, 2003 5:57 PM > To: MEYER,JEFFREY D (HP-Cupertino,ex1) > Cc: 'Mark Fullmer'; ipfix@net.doit.wisc.edu > Subject: Re: [ipfix] drawing the line between protocol and data model > docu ment > > > Jeff, > > i support the idea of typing. I think there is some > misunderstanding here > by some people who confuse types with semantics. A type does > not define the > semantics. In my opinion typing has the advantage that even > if i don't know > the semantics by the time of the decoding i still can > correctly decode the > value and present it. It also has the advantage that for a > well known type > the encoding is well defined. If that is not the case people > may encode > attributes of the same type differently just making > interoperability harder. > I would define these basic data types in the protocol document. > > I am not so sure if both alternatives take the same space > because there > are some types which which do not have an intrinsic length > definition. An > utf or octet string could have an arbitrary length. Without > typing everything > would be basically an octetstring of some length ;-). However > i though the > protocol is supossed to use templates and so it would only affect the > template exchange wouldn't it? > > Cheers, > > Sebastian > > MEYER,JEFFREY D (HP-Cupertino,ex1) wrote: > > vs. > > > > = 16909060 (integer) > > > > vs. > > > > = whatever the heck 0x01020304 represents > > in IEEE Floating Point notation. (something around 4.06) > > > > > > So, yes that's all it buys you. Since both alternatives take > > the same amount of space, I'd prefer the one with more > > intrinsic information. But, I guess I'm in the minority :-( > > > > > > Regards, > > > > Jeff Meyer > > > > > > > >>-----Original Message----- > >>From: Mark Fullmer [mailto:maf@eng.oar.net] > >>Sent: Thursday, May 01, 2003 8:39 AM > >>To: MEYER,JEFFREY D (HP-Cupertino,ex1) > >>Cc: ipfix@net.doit.wisc.edu > >>Subject: Re: [ipfix] drawing the line between protocol and > data model > >>docu ment > >> > >> > >>So the difference boils down to a tool being able to display > >> > >> = 0x01020304 > >> vs > >> = 1.2.3.4 > >> > >>In this situation someone will either grab a new version of the > >>software or an updated data definition to resolve the unknown > >>attribute, > >>which would also define it's type. > >> > >>mark > >> > >>On Tue, Apr 29, 2003 at 06:25:11PM -0400, MEYER,JEFFREY D > >>(HP-Cupertino,ex1) wrote: > >> > >>>Mark, > >>> > >>> I'll try to clarify: > >>> > >>> Concept #1: The information model and the encoding are separate. > >>> A well defined mapping determines how an information model is > >>> to be encoded for a specific use (e.g. IPFIX). The encoding > >>> may take whatever approach allows both ends to do something > >>> useful with the data. As you point out there is some useful > >>> stuff which can be done with the length alone. > >>> > >>> Concept #2: Derived typing. By defining a base set of types, > >>> which must be understood, derived types can be defined in terms > >>> of these base types. For instance Diameter does this. This > >>> enables a core set of types to be defined and coded to, but > >>> also enables richer types later on with an appropriate fallback > >>> for systems which aren't aware of the extensions. > >>> > >>> The benefit I see of explicit typing is that instead of seeing > >>>that a field is of length "4 bytes" something which will be seen > >>>A LOT. Is that we can instead differentiate between > >> > >>IPAddr, Integer, > >> > >>>unsigned integer, and float. This seems handy to me. > >>> > >>> Your argument seems centered around the long term potential of > >>>adding lots of fixed length fields whose base length is not 4,8 > >>>or 16 bytes long, for instance MACAddr (although if we agree on > >>>going the type route, I'd recommend adding MACAddr and then you > >>>have 1,2,4,6,8 and 16 bytes covered. I guess I don't foresee > >>>a lot more fixed length fields coming down the pipe in the IPFIX > >>>domain. Hence my desire to make it more explicit for the types > >>>which do occur all the time and are likely points of extension. > >>>WHY? Because then a tool could at least present these in a > >> > >>meaningful > >> > >>>way. As opposed to incorrectly formatting a float type as an int. > >>> > >>> But hey, either way will work. If the goal is to limit the > >>>amount of changes, I'll (grudgingly) go along. I haven't heard > >>>any opinions from anyone but you in either direction. So, > >>>at this point I don't see a quorom either way... > >>> > >>>Regards, > >>> > >>> Jeff Meyer > >>> > >>> > >>> > >>>>-----Original Message----- > >>>>From: Mark Fullmer [mailto:maf@eng.oar.net] > >>>>Sent: Tuesday, April 29, 2003 2:53 PM > >>>>To: MEYER,JEFFREY D (HP-Cupertino,ex1) > >>>>Cc: ipfix@net.doit.wisc.edu > >>>>Subject: Re: [ipfix] drawing the line between protocol > >>> > >>and data model > >> > >>>>docu ment > >>>> > >>>> > >>>>IMHO crippling the protocol to serve the needs of a > >>> > >>documentation tool > >> > >>>>is just silly. > >>>> > >>>>It would be possible to use both a Type name and a Length > >>> > >>instead of > >> > >>>>the existing Length, which would allow a collector to process > >>>>unknown Types. > >>>> > >>>>With the existing v9 protocol a collector can ignore an > >>>>unknown attribute, > >>>>or possibly just pass it on to post processing tools which > >>>>might understand > >>>>it. > >>>> > >>>>Your suggested changes no longer allow a collector to > >>> > >>ignore unknown > >> > >>>>attributes if the Type is also unknown. Resorting to encoding all > >>>>new Types after the initial protocol spec as octet > >>> > >>strings is (IMHO) > >> > >>>>not acceptable. > >>>> > >>>>I'm still trying to understand what the benefits to explicit > >>>>Types sent > >>>>in templates are. > >>>> > >>>>mark > >>>> > >>>>On Tue, Apr 29, 2003 at 02:50:02PM -0400, MEYER,JEFFREY D > >>>>(HP-Cupertino,ex1) wrote: > >>>> > >>>>>Juergen, > >>>>> > >>>>> I agree that the data model should have a well defined type > >>>>>system. And that that typing system should reflect the > >>>> > >>>>requirements. > >>>> > >>>>> This is in part why I was hoping that the protocol > >>>> > >>itself would > >> > >>>>>be more explicit here. (See my mail > >>>>>http://ipfix.doit.wisc.edu/archive/1604.html) > >>>>> > >>>>> There is also a type list in that mail. However, I > >>>> > >>would suggest > >> > >>>>>using the names from XML-Schema part 2 Data types, as this would > >>>>>facilitate creating a formal/normative definition, versus > >>>> > >>>>reinventing > >>>> > >>>>>names for the types. > >>>>> > >>>>> Here are the type names from XML-Schema/IPDR: > >>>>> > >>>>> boolean - true/false (may be encoded in byte) > >>>>> byte > >>>>> unsignedByte > >>>>> short > >>>>> unsignedShort > >>>>> int - 32-bit signed > >>>>> long - 64-bit signed > >>>>> unsignedInt > >>>>> unsignedLong > >>>>> float > >>>>> double > >>>>> dateTime - 32-bit integer representing seconds > >>>> > >>since EPOCH > >> > >>>>>(1/1/1970 0:00 GMT) > >>>>> string - UTF-8 formatted text > >>>>> ipdr:dateTimeMsec - 64-bit integer representing msecs > >>>> > >>>>since EPOCH > >>>> > >>>>> ipdr:ipV4Addr > >>>>> ipdr:ipV6Addr > >>>>> ipdr:UUID > >>>>> hexBinary - arbitrary sequence of octets > >>>>> > >>>>> If the protocol itself has a basic "octetString" (or from > >>>> > >>>>XML, hexBinary) > >>>> > >>>>>payload, > >>>>>then this can be used to carry new classes of information > >>>> > >>>>in the future w/o > >>>> > >>>>>rejiggering the protocol itself. > >>>>> > >>>>> In addition other "derived" types may be created from the > >>>> > >>>>base types. For > >>>> > >>>>>instance in IPDR, the ipdr:dateTimeMsec is derived from long. > >>>>> > >>>>>Regards, > >>>>> > >>>>> Jeff Meyer > >>>>> > >>>>> > >>>>>>-----Original Message----- > >>>>>>From: Juergen Quittek [mailto:quittek@ccrle.nec.de] > >>>>>>Sent: Monday, April 28, 2003 2:06 PM > >>>>>>To: ipfix@net.doit.wisc.edu > >>>>>>Subject: [ipfix] drawing the line between protocol > >>>>> > >>and data model > >> > >>>>>>document > >>>>>> > >>>>>> > >>>>>>Hi all, > >>>>>> > >>>>>>When shaping the planned documents, we have to draw a line > >>>>>>between the protocol and the data model. > >>>>>> > >>>>>>Definitely, the data model document defines all field types, > >>>>>>but does it also define basic data types, such as '16 bit > >>>>>>unsigned integer'? > >>>>>> > >>>>>>Maybe we should have a set of basic data types defined in > >>>>>>the protocol document and restrict the data model doc to > >>>>> > >>>>these types? > >>>> > >>>>>>This would further structure the data model. > >>>>>> > >>>>>>Any thoughts? > >>>>>> > >>>>>> Juergen > >>>>>> > >>>>>>-- > >>>>>>Help mailto:majordomo@net.doit.wisc.edu and say "help" > >>>>>>in message body > >>>>>>Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > >>>>>>"unsubscribe ipfix" in message body > >>>>>>Archive http://ipfix.doit.wisc.edu/archive/ > >>>>>> > >>>>> > >>>>>-- > >>>>>Help mailto:majordomo@net.doit.wisc.edu and say > >>>> > >>>>"help" in message body > >>>> > >>>>>Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > >>>>>"unsubscribe ipfix" in message body > >>>>>Archive http://ipfix.doit.wisc.edu/archive/ > >>>> > >>>-- > >>>Help mailto:majordomo@net.doit.wisc.edu and say > >> > >>"help" in message body > >> > >>>Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > >>>"unsubscribe ipfix" in message body > >>>Archive http://ipfix.doit.wisc.edu/archive/ > >> > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say > "help" in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > -- > Sebastian Zander E-mail: > zander@fokus.fraunhofer.de > Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 > Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 > D-10589 Berlin, Germany > www.fokus.fraunhofer.de/usr/sebastian.zander > > > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri May 2 04:57:23 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id EAA09532 for ; Fri, 2 May 2003 04:57:23 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19BW94-00013U-00 for ipfix-list@mil.doit.wisc.edu; Fri, 02 May 2003 03:43:54 -0500 Received: from sj-core-1.cisco.com ([171.71.177.237]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19BW92-00013N-00 for ipfix@net.doit.wisc.edu; Fri, 02 May 2003 03:43:52 -0500 Received: from cisco.com (mrwint.cisco.com [64.103.71.48]) by sj-core-1.cisco.com (8.12.6/8.12.6) with ESMTP id h428heLr004494; Fri, 2 May 2003 01:43:42 -0700 (PDT) Received: from cisco.com ([64.103.97.193]) by cisco.com (8.8.8/2.6/Cisco List Logging/8.8.8) with ESMTP id JAA25802; Fri, 2 May 2003 09:43:39 +0100 (BST) Message-ID: <3EB22F4D.8030207@cisco.com> Date: Fri, 02 May 2003 09:41:49 +0100 From: Stewart Bryant Reply-To: stbryant@cisco.com Organization: Cisco Systems, Inc. User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.2) Gecko/20030208 Netscape/7.02 X-Accept-Language: en-us, en MIME-Version: 1.0 To: calato@riverstonenet.com CC: Reinaldo Penno , "'Maurizio Molina'" , ipfix@net.doit.wisc.edu Subject: Re: [ipfix] export packet length References: <0A11633F61BD9F40B43ABCC694004F93F18EEC@zsc3c026.us.nortel.com> <3EB13B94.6B4DEBC9@riverstonenet.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit calato@riverstonenet.com wrote: > > Since the only thing mandated for the transport is congestion awareness > it would seems to make sense to have a length field in the IPFIX > protocol. > > I don't like the idea of having different formats based on the > transport (someone mentioned that as an option). A 2 byte length > field seems a small price to pay or better transport independance. > I agree with this. If we go with two types, we will always have to take into account the maximum possible size of the object, the error cases etc. The one byte overhead seems a small price for the reduction in complexity. Stewart > Paul > > > >>Reinaldo Penno wrote: >> >>That's a good idea. >> >>Other protocols like HTTP and SIP make it mandatory to include the >>Content-Length header (which tell the length of the payload) when >>carried over TCP. It helps a lot when dealing with fragmentation. >> >>Regards, >> >> >>>-----Original Message----- >>>From: Maurizio Molina [mailto:molina@ccrle.nec.de] >>>Sent: Friday, April 25, 2003 11:50 AM >>>To: ipfix@net.doit.wisc.edu >>>Subject: [ipfix] export packet length >>> >>> >>>Hi all, >>>I hope not to rise an issue already discussed in the WG. If >>>yes, I apologise in advance. If an IPFIX export packet is >>>carried over TCP, it's not unusual that it gets split across >>>multiple TCP segments. At the collector side, on the >>>contrary, the implementation of the parsing is much simpler >>>if only full export packet are parsed. If the information >>>about the WHOLE export packet were contained in the header, >>>it would be simple to wait until all the bytes composing an >>>export packets are read from the TCP socket before beginning >>>the parsing. Unfortunately, this information isn't currently >>>contained in Netflow 9 packet header (I suppose, due to >>>Netflow's 9 "bias" towards UDP...). Therefore, a parsing >>>implementation having to deal with this "fragentation" >>>problem without this information needs to be much more >>>complicated (e.g. getting step by step the lenght of each >>>flow set, see if it's all there, parse it and go on....). In >>>summary: if IPFIX has to be carried over tcp, having the >>>length of the export packet in the packet header would be >>>beneficial. Regards, Maurizio >>> >>> >>>-- >>>Maurizio Molina >>>Research Staff member >>>Network Laboratories Heidelberg >>>NEC Europe Ltd. >>>Kurfuersten-Anlage 36, 69115 Heidelberg, Germany. >>>Tel. (49)6221 90511-18 Fax: (49)6221 90511-55 >>>e-mail: molina@ccrle.nec.de >>>Web: www.ccrle.nec.de >>> >>> >>> >>>-- >>>Help mailto:majordomo@net.doit.wisc.edu and say "help" >>>in message body >>>Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >>>"unsubscribe ipfix" in message body >>>Archive http://ipfix.doit.wisc.edu/archive/ >>> >> > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue May 6 10:18:19 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA23661 for ; Tue, 6 May 2003 10:18:18 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19D2re-0005ZA-00 for ipfix-list@mil.doit.wisc.edu; Tue, 06 May 2003 08:52:14 -0500 Received: from [64.95.122.13] (helo=RS-SC-EXC4.rs.riverstonenet.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19D2rd-0005Z3-00 for ipfix@net.doit.wisc.edu; Tue, 06 May 2003 08:52:13 -0500 Received: from riverstonenet.com ([172.17.6.3]) by RS-SC-EXC4.rs.riverstonenet.com with Microsoft SMTPSVC(5.0.2195.4905); Tue, 6 May 2003 06:52:12 -0700 Message-ID: <3EB7BE0A.5A7DE183@riverstonenet.com> Date: Tue, 06 May 2003 09:52:10 -0400 From: calato@riverstonenet.com X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.7-10 i686) X-Accept-Language: en MIME-Version: 1.0 To: Randy Bush CC: ipfixx Subject: Re: [ipfix] Issue tracking References: <3EB13D30.DFE75100@riverstonenet.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 06 May 2003 13:52:13.0096 (UTC) FILETIME=[B23FCA80:01C313D6] Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Great! Can you provide a pointer to more info. Paul Randy Bush wrote: > > rob austein and i have set up an experimental ticket system for wgs' > use in issue tracking. > > randy -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue May 6 10:19:34 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA23697 for ; Tue, 6 May 2003 10:19:33 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19D2qj-0005Xc-00 for ipfix-list@mil.doit.wisc.edu; Tue, 06 May 2003 08:51:17 -0500 Received: from [64.95.122.13] (helo=RS-SC-EXC4.rs.riverstonenet.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19D2qh-0005XR-00 for ipfix@net.doit.wisc.edu; Tue, 06 May 2003 08:51:15 -0500 Received: from riverstonenet.com ([172.17.6.3]) by RS-SC-EXC4.rs.riverstonenet.com with Microsoft SMTPSVC(5.0.2195.4905); Tue, 6 May 2003 06:51:14 -0700 Message-ID: <3EB7BDCF.B835D8C2@riverstonenet.com> Date: Tue, 06 May 2003 09:51:11 -0400 From: calato@riverstonenet.com X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.7-10 i686) X-Accept-Language: en MIME-Version: 1.0 To: Mark Fullmer CC: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] drawing the line between protocol and data model document References: <1D3D2C371FCBD947A7897FABBD3533A566BA84@xsun01.ptp.hp.com> <3EB13485.3FA365D7@riverstonenet.com> <20030501162116.B33173@net.ohio-state.edu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 06 May 2003 13:51:14.0938 (UTC) FILETIME=[8F9595A0:01C313D6] Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Just a note on this. I'm not at this point advocating any encoding scheme. Merely saying that the data model is independent of the encoding scheme. Whether you encode it as 8 bytes or 1 byte doesn't matter to the data model as the semantics remain the same. However, that said, I still need to think more about this issue from a practical standpoint. Theory doesn't always line up nicely with reality :-) Paul Mark Fullmer wrote: > > On Thu, May 01, 2003 at 10:51:49AM -0400, calato@riverstonenet.com wrote: > > > > > The richness of the protocol encoding specification is another > > matter. The protocol would reference each element in the data > > model and define exaclty how that element is encoded. In the above > > example, the protocol is free to encode that as an unsigned byte > > if the value is < 256. > > The ramifications of doing this with a template based system are a bit > ugly. Consider a flow with just 2 counters -- octets, packets. > Each could have a 1 to 8 byte encoding. That's a potential of 64 > different templates for representing 1 type of flow. > > This (at first glance) looks like it adds considerable cycles to the > encode / decode process to save a few bytes in the transport. > > The current v9 spec appears to allow this, but only for IN_BYTES and > IN_PKTS. > > > counter with length > > IN_BYTES 1 N N x 8 bits for bytes > > associated with an IP Flow > > Is this true? Benoit? > > mark -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue May 6 11:14:51 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA25762 for ; Tue, 6 May 2003 11:14:51 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19D3wF-00070O-00 for ipfix-list@mil.doit.wisc.edu; Tue, 06 May 2003 10:01:03 -0500 Received: from [64.95.122.13] (helo=RS-SC-EXC4.rs.riverstonenet.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19D3wE-00070G-00 for ipfix@net.doit.wisc.edu; Tue, 06 May 2003 10:01:02 -0500 Received: from riverstonenet.com ([172.17.6.3]) by RS-SC-EXC4.rs.riverstonenet.com with Microsoft SMTPSVC(5.0.2195.4905); Tue, 6 May 2003 08:00:59 -0700 Message-ID: <3EB7CE29.DD91933F@riverstonenet.com> Date: Tue, 06 May 2003 11:00:57 -0400 From: calato@riverstonenet.com X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.7-10 i686) X-Accept-Language: en MIME-Version: 1.0 To: "MEYER,JEFFREY D (HP-Cupertino,ex1)" CC: "'Juergen Quittek'" , "'Nevil Brownlee'" , ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Editors for IPFIX drafts References: <1D3D2C371FCBD947A7897FABBD3533A566BA95@xsun01.ptp.hp.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 06 May 2003 15:00:59.0941 (UTC) FILETIME=[4E0A5D50:01C313E0] Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Seems like a reasonable model. I'd say lets head that way and see how it goes. Are there any MS word templates for creating the XML part? Paul "MEYER,JEFFREY D (HP-Cupertino,ex1)" wrote: > > Hi, > > If the human readable part is generated from the XML appendix, > then I guess it doesn't matter which part you call normative, > since they both originate from the same source. > > The development model I see is the following: > > Generate XML Info Model > Run model through XSLT (xalan) translater to produce prose > (consistent with RFC 2629) [I'll volunteer to write the > style sheet translation) > Incorporate the raw model as appendix and the generated prose > description into the appropriate section (consistent w/ > RFC 2629) along with other text of the Information Model Draft. > Run Marshall's tool http://xml.resource.org/ to produce nroff style > and HTML versions of document. Submit nroff style to RFC editor, > post HTML style in addition to ipfix website (easier to navigate). > > For examples of the dual formatted output, see the IPDR submissions > to IPDR located at: > > http://www.ipdr.org/documents/ipfix > > Regards, > > Jeff Meyer > > > -----Original Message----- > > From: Juergen Quittek [mailto:quittek@ccrle.nec.de] > > Sent: Thursday, May 01, 2003 8:21 AM > > To: calato@riverstonenet.com > > Cc: MEYER,JEFFREY D (HP-Cupertino,ex1); 'Nevil Brownlee'; > > ipfix@net.doit.wisc.edu > > Subject: Re: [ipfix] Editors for IPFIX drafts > > > > > > Paul, > > > > I agree to Jeff, that there is some value in having an XML > > representation > > of the specification. However, in the end this document will > > become an RFC > > and it must be well readable to human beings. > > > > Therefore, I think that the main body must be human readable, > > but we can > > provide an appendix, that contains an equivalent machine > > readable spec. > > > > Different to Jeff, I think that the hunam readable main body should be > > the normative part, while the XML appendix is just given as > > support for > > users of XML tools allowing them to genrate protocol stacks > > and/or tools > > for consistency checking. > > > > As Jeff suggested, we can work with an XML representation > > when developing > > the spec and generate human readable text automatically from > > the XML code. > > This will help us avoiding fomal mistakes in the human > > readable spec and we > > would always have consistency between human readable and XML part. > > > > Juergen > > > > > > -- calato@riverstonenet.com wrote on 01 May 2003 11:00 -0400: > > > > > > > > I'm a little confused. Are we talking about XML descriptions > > > as an appendix with prose as the main body or XML as the > > > main body with tools translating it into a more readable > > > form? Or XML as an appendix with tools translating it for > > > use as sections mixed in with the prose? > > > > > > > > > Paul > > > > > > Juergen Quittek wrote: > > >> > > >> Jeff, > > >> > > >> -- MEYER,JEFFREY D (HP-Cupertino,ex1) wrote on 28 April > > 2003 09:20 -0700: > > >> > > > >> > Juergen, > > >> > > > >> > Thanks for the response. > > >> > > > >> > I agree XML in and of itself is marginally human readable. > > >> > > > >> > I think that basic tools such as Xalan > > (http://xml.apache.org/), can > > >> > be used to "transform" a base XML document into a > > variety of formats, > > >> > including formatting consistent with RFC's. Note that > > all the IPDR > > >> > RFC's were written in XML using Marshall Rose's xml2rfc tools > > >> > (http://xml.resource.org/). > > >> > > >> I also like Marshall's tools. > > >> If Paul agrees, we should use them for the data model document. > > >> > > >> > So the model I would picture is having the Normative > > XML data model > > >> > as an appendix, and having generated information from > > this data model > > >> > populate some of the body sections to aid human readers. > > >> > > >> This looks like a reasonable way to go. > > >> > > >> Juergen > > >> > > >> > Regards, > > >> > > > >> > Jeff Meyer > > >> > > > >> >> -----Original Message----- > > >> >> From: Juergen Quittek [mailto:quittek@ccrle.nec.de] > > >> >> Sent: Monday, April 28, 2003 8:24 AM > > >> >> To: MEYER,JEFFREY D (HP-Cupertino,ex1); 'Nevil Brownlee'; > > >> >> ipfix@net.doit.wisc.edu > > >> >> Subject: RE: [ipfix] Editors for IPFIX drafts > > >> >> > > >> >> > > >> >> Jeff, > > >> >> > > >> >> I do see a value in such a formal description. But there is a > > >> >> trafe-off. > > >> >> > > >> >> I also see a degradation of human readability. You > > loose means of > > >> >> structuring your document text when using XML, for > > example grouping > > >> >> attributes that are related. (Of course you could do so in > > >> >> XML, but it is > > >> >> much harder to find the heading "Port-related Attributes" in > > >> >> an XML document, > > >> >> because headings cannot be well highlighted and numbering of > > >> >> headings is > > >> >> not well supported. > > >> >> > > >> >> Anyway, I like the idea of an XML encoding of the model, but > > >> >> I we should > > >> >> not go this way if we cannot find a good way of getting it > > >> >> hunam readable > > >> >> (maybe by applying formatting rules). > > >> >> > > >> >> An option would also be adding the XML code to the document > > >> >> as an appendix. > > >> >> > > >> >> Juergen > > >> >> > > >> >> > > >> >> -- MEYER,JEFFREY D (HP-Cupertino,ex1) wrote on 28 April 2003 > > >> >> 10:57 -0400: > > >> >> > > >> >> > Hi, > > >> >> > > > >> >> > My particular interest on working on the > > information model would > > >> >> > be on developing a more formal and standard way of > > describing the > > >> >> > information elements which are used in IPFIX communication. > > >> >> > > > >> >> > The modelling of information elements should > > ideally be done > > >> >> > using a formal description language, so that tools can be > > >> >> constructed > > >> >> > which do not require humans to transcribe this information. > > >> >> > > > >> >> > I would propose the use of XML-Schema to provide > > this formal > > >> >> > description. Through the use of XSL processors, prose > > >> >> based descriptions > > >> >> > and tables can be produced if these are considered desireable. > > >> >> > > > >> >> > NOTE: describing the information model formally > > does NOT mean > > >> >> > that encoding and transport for IPFIX need to change > > in any way. It > > >> >> > simply means that rather than having as the base > > >> >> description of information > > >> >> > items some prose in a document, you instead have a > > machine readable > > >> >> > representation. > > >> >> > > > >> >> > > > >> >> > As an example of a formal (but currently > > incomplete) information > > >> >> > model for IPFIX, see: > > >> >> > http://www.ipdr.org/documents/ipfix/ipfixService-20020902.xml > > >> >> > > > >> >> > I would recommend making such a representation the > > normative form > > >> >> > of the IPFIX information model, much like SNMP MIBs. > > >> >> > > > >> >> > > > >> >> > Please let me know if people see value in this exercise. > > >> >> > > > >> >> > Regards, > > >> >> > > > >> >> > Jeff Meyer > > >> >> > > > >> >> >> -----Original Message----- > > >> >> >> From: Nevil Brownlee [mailto:n.brownlee@auckland.ac.nz] > > >> >> >> Sent: Sunday, April 27, 2003 8:44 PM > > >> >> >> To: ipfix@net.doit.wisc.edu > > >> >> >> Subject: [ipfix] Editors for IPFIX drafts > > >> >> >> > > >> >> >> > > >> >> >> > > >> >> >> Hello all: > > >> >> >> > > >> >> >> Following the IPFIX meeting at the San Francisco > > IETF we asked for > > >> >> >> volunteers to edit the IPFIX drafts. The current list is: > > >> >> >> > > >> >> >> Applicability: Tanja Zseby, Reinaldo Penno > > >> >> >> > > >> >> >> Architecture: Ganesh Sadasivan, Nevil Brownlee > > >> >> >> > > >> >> >> Information Model: Paul Calato, Jeff Meyer, Juergen Quittek > > >> >> >> > > >> >> >> Protocol: Mark Fulmer, Paul Calato, Reinaldo Penno > > >> >> >> > > >> >> >> At this stage we already have old (expired) versions of the > > >> >> >> Architecture (-02.txt) and Data Model (-01.txt) > > drafts on the IPFIX > > >> >> >> web page; would the editors of all four drafts please email > > >> >> >> me an initial > > >> >> >> version of their new drafts to published as placeholders > > >> >> >> within the next > > >> >> >> few weeks. > > >> >> >> > > >> >> >> After that, the editors will work on developing better > > >> >> versions of the > > >> >> >> drafts which can be discussed on the IPFIX list and submitted > > >> >> >> by mid-June, > > >> >> >> i.e. in plenty of time for the Vienna IETF meeting in July. > > >> >> >> > > >> >> >> Cheers, Nevil > > >> >> >> > > >> >> >> PS: Good to see some real discussion on the list. Lets > > >> >> concentrate on > > >> >> >> producing text the draft editors can use! > > >> >> >> > > >> >> >> > > -------------------------------------------------------------- > > >> >> >> --------- > > >> >> >> Nevil Brownlee Director, Technology > > >> >> Development > > >> >> >> Phone: +64 9 373 7599 x88941 ITSS, The University > > >> >> of Auckland > > >> >> >> FAX: +64 9 373 7021 Private Bag 92019, Auckland, > > >> >> New Zealand > > >> >> >> > > >> >> >> > > >> >> >> ------------------------------------------------- > > >> >> >> This mail sent through University of Auckland > > >> >> >> http://www.auckland.ac.nz/ > > >> >> >> > > >> >> >> -- > > >> >> >> Help mailto:majordomo@net.doit.wisc.edu and say "help" > > >> >> >> in message body > > >> >> >> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > >> >> >> "unsubscribe ipfix" in message body > > >> >> >> Archive http://ipfix.doit.wisc.edu/archive/ > > >> >> >> > > >> >> > > > >> >> > -- > > >> >> > Help mailto:majordomo@net.doit.wisc.edu and say > > >> >> "help" in message body > > >> >> > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > >> >> > "unsubscribe ipfix" in message body > > >> >> > Archive http://ipfix.doit.wisc.edu/archive/ > > >> >> > > >> >> > > >> > > >> -- > > >> Help mailto:majordomo@net.doit.wisc.edu and say > > "help" in message body > > >> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > >> "unsubscribe ipfix" in message body > > >> Archive http://ipfix.doit.wisc.edu/archive/ > > > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue May 6 12:02:38 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA27259 for ; Tue, 6 May 2003 12:02:37 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19D4cX-00004w-00 for ipfix-list@mil.doit.wisc.edu; Tue, 06 May 2003 10:44:45 -0500 Received: from [64.95.122.13] (helo=RS-SC-EXC4.rs.riverstonenet.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19D4cV-00004p-00 for ipfix@net.doit.wisc.edu; Tue, 06 May 2003 10:44:43 -0500 Received: from riverstonenet.com ([172.17.6.3]) by RS-SC-EXC4.rs.riverstonenet.com with Microsoft SMTPSVC(5.0.2195.4905); Tue, 6 May 2003 08:44:41 -0700 Message-ID: <3EB7D865.292A6190@riverstonenet.com> Date: Tue, 06 May 2003 11:44:37 -0400 From: calato@riverstonenet.com X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.7-10 i686) X-Accept-Language: en MIME-Version: 1.0 To: "MEYER,JEFFREY D (HP-Cupertino,ex1)" CC: "'Sebastian Zander'" , "'Mark Fullmer'" , ipfix@net.doit.wisc.edu Subject: Re: [ipfix] drawing the line between protocol and data model document References: <1D3D2C371FCBD947A7897FABBD3533A566BAA2@xsun01.ptp.hp.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 06 May 2003 15:44:41.0897 (UTC) FILETIME=[68D91990:01C313E6] Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit OK. I'm confused. Let me run through an example and people can correct me where I'm wrong. Lets take "Source IP v4 address". Lets also hold off on variable length fields for now. All options need some kind of type value to indicate what the value semantically means. Lets pick 57 == Source IP v4 address. So what we need next is info on the value part so we can decode it and it should be backward compatible (i.e. older versions can at least skip unknown elements. Option I: Element and Length 57, 4 Any software understanding element 57 knows it is the Source IP address and can display/handle it correctly. Any one else would skip over the value in the flow set. Option II: Element and data type Lets say data type 22 == IP V4 Address 57, 22 Any software understanding element 57 knows it is the Source IP address. But what if it doesn't know what data type 22 is? How can I skip over it? Option III: Element and data type and length 57, 22, 4 Any software which understands 57 and 22 can handle it correclty. If it only understands 22, it can ONLY display the value in some rational way, nothing else can be done. If I don't understand 22 I can at least skip over it by using the length. Correct away! Paul "MEYER,JEFFREY D (HP-Cupertino,ex1)" wrote: > > Sebastian, > > Thanks for speaking up! I agree with your restatement of the > relationship between typing and semantics. > > Regarding types like UTF8 or OctetString, as I read the proposal > from Benoit, the "Length Encoding" model would use a special value > of 0xFFFF instead of length in the template. This implies that > the encoding of the "variable length" field would be of the form > (length) (value), versus fixed length types which can be encoded by > just placing their (value) in the payload. > > (see http://ipfix.doit.wisc.edu/archive/1602.html) > > This same model would apply in a type based scheme, if the type > were string or hexBinary (indicated by the appropriate integer > identifier in the template), then the encoded value would still be > of the form (length) (value) in the payload. > > Regardless of what template scheme is chosen, however, I think it > is important that the protocol spec speak in terms of how types are > encoded, not how each individual element declared in the information model > is encoded. This makes the inevitable extension of the information > model easier. > > Regards, > > Jeff Meyer > > > -----Original Message----- > > From: Sebastian Zander [mailto:zander@fokus.fraunhofer.de] > > Sent: Thursday, May 01, 2003 5:57 PM > > To: MEYER,JEFFREY D (HP-Cupertino,ex1) > > Cc: 'Mark Fullmer'; ipfix@net.doit.wisc.edu > > Subject: Re: [ipfix] drawing the line between protocol and data model > > docu ment > > > > > > Jeff, > > > > i support the idea of typing. I think there is some > > misunderstanding here > > by some people who confuse types with semantics. A type does > > not define the > > semantics. In my opinion typing has the advantage that even > > if i don't know > > the semantics by the time of the decoding i still can > > correctly decode the > > value and present it. It also has the advantage that for a > > well known type > > the encoding is well defined. If that is not the case people > > may encode > > attributes of the same type differently just making > > interoperability harder. > > I would define these basic data types in the protocol document. > > > > I am not so sure if both alternatives take the same space > > because there > > are some types which which do not have an intrinsic length > > definition. An > > utf or octet string could have an arbitrary length. Without > > typing everything > > would be basically an octetstring of some length ;-). However > > i though the > > protocol is supossed to use templates and so it would only affect the > > template exchange wouldn't it? > > > > Cheers, > > > > Sebastian > > > > MEYER,JEFFREY D (HP-Cupertino,ex1) wrote: > > > vs. > > > > > > = 16909060 (integer) > > > > > > vs. > > > > > > = whatever the heck 0x01020304 represents > > > in IEEE Floating Point notation. (something around 4.06) > > > > > > > > > So, yes that's all it buys you. Since both alternatives take > > > the same amount of space, I'd prefer the one with more > > > intrinsic information. But, I guess I'm in the minority :-( > > > > > > > > > Regards, > > > > > > Jeff Meyer > > > > > > > > > > > >>-----Original Message----- > > >>From: Mark Fullmer [mailto:maf@eng.oar.net] > > >>Sent: Thursday, May 01, 2003 8:39 AM > > >>To: MEYER,JEFFREY D (HP-Cupertino,ex1) > > >>Cc: ipfix@net.doit.wisc.edu > > >>Subject: Re: [ipfix] drawing the line between protocol and > > data model > > >>docu ment > > >> > > >> > > >>So the difference boils down to a tool being able to display > > >> > > >> = 0x01020304 > > >> vs > > >> = 1.2.3.4 > > >> > > >>In this situation someone will either grab a new version of the > > >>software or an updated data definition to resolve the unknown > > >>attribute, > > >>which would also define it's type. > > >> > > >>mark > > >> > > >>On Tue, Apr 29, 2003 at 06:25:11PM -0400, MEYER,JEFFREY D > > >>(HP-Cupertino,ex1) wrote: > > >> > > >>>Mark, > > >>> > > >>> I'll try to clarify: > > >>> > > >>> Concept #1: The information model and the encoding are separate. > > >>> A well defined mapping determines how an information model is > > >>> to be encoded for a specific use (e.g. IPFIX). The encoding > > >>> may take whatever approach allows both ends to do something > > >>> useful with the data. As you point out there is some useful > > >>> stuff which can be done with the length alone. > > >>> > > >>> Concept #2: Derived typing. By defining a base set of types, > > >>> which must be understood, derived types can be defined in terms > > >>> of these base types. For instance Diameter does this. This > > >>> enables a core set of types to be defined and coded to, but > > >>> also enables richer types later on with an appropriate fallback > > >>> for systems which aren't aware of the extensions. > > >>> > > >>> The benefit I see of explicit typing is that instead of seeing > > >>>that a field is of length "4 bytes" something which will be seen > > >>>A LOT. Is that we can instead differentiate between > > >> > > >>IPAddr, Integer, > > >> > > >>>unsigned integer, and float. This seems handy to me. > > >>> > > >>> Your argument seems centered around the long term potential of > > >>>adding lots of fixed length fields whose base length is not 4,8 > > >>>or 16 bytes long, for instance MACAddr (although if we agree on > > >>>going the type route, I'd recommend adding MACAddr and then you > > >>>have 1,2,4,6,8 and 16 bytes covered. I guess I don't foresee > > >>>a lot more fixed length fields coming down the pipe in the IPFIX > > >>>domain. Hence my desire to make it more explicit for the types > > >>>which do occur all the time and are likely points of extension. > > >>>WHY? Because then a tool could at least present these in a > > >> > > >>meaningful > > >> > > >>>way. As opposed to incorrectly formatting a float type as an int. > > >>> > > >>> But hey, either way will work. If the goal is to limit the > > >>>amount of changes, I'll (grudgingly) go along. I haven't heard > > >>>any opinions from anyone but you in either direction. So, > > >>>at this point I don't see a quorom either way... > > >>> > > >>>Regards, > > >>> > > >>> Jeff Meyer > > >>> > > >>> > > >>> > > >>>>-----Original Message----- > > >>>>From: Mark Fullmer [mailto:maf@eng.oar.net] > > >>>>Sent: Tuesday, April 29, 2003 2:53 PM > > >>>>To: MEYER,JEFFREY D (HP-Cupertino,ex1) > > >>>>Cc: ipfix@net.doit.wisc.edu > > >>>>Subject: Re: [ipfix] drawing the line between protocol > > >>> > > >>and data model > > >> > > >>>>docu ment > > >>>> > > >>>> > > >>>>IMHO crippling the protocol to serve the needs of a > > >>> > > >>documentation tool > > >> > > >>>>is just silly. > > >>>> > > >>>>It would be possible to use both a Type name and a Length > > >>> > > >>instead of > > >> > > >>>>the existing Length, which would allow a collector to process > > >>>>unknown Types. > > >>>> > > >>>>With the existing v9 protocol a collector can ignore an > > >>>>unknown attribute, > > >>>>or possibly just pass it on to post processing tools which > > >>>>might understand > > >>>>it. > > >>>> > > >>>>Your suggested changes no longer allow a collector to > > >>> > > >>ignore unknown > > >> > > >>>>attributes if the Type is also unknown. Resorting to encoding all > > >>>>new Types after the initial protocol spec as octet > > >>> > > >>strings is (IMHO) > > >> > > >>>>not acceptable. > > >>>> > > >>>>I'm still trying to understand what the benefits to explicit > > >>>>Types sent > > >>>>in templates are. > > >>>> > > >>>>mark > > >>>> > > >>>>On Tue, Apr 29, 2003 at 02:50:02PM -0400, MEYER,JEFFREY D > > >>>>(HP-Cupertino,ex1) wrote: > > >>>> > > >>>>>Juergen, > > >>>>> > > >>>>> I agree that the data model should have a well defined type > > >>>>>system. And that that typing system should reflect the > > >>>> > > >>>>requirements. > > >>>> > > >>>>> This is in part why I was hoping that the protocol > > >>>> > > >>itself would > > >> > > >>>>>be more explicit here. (See my mail > > >>>>>http://ipfix.doit.wisc.edu/archive/1604.html) > > >>>>> > > >>>>> There is also a type list in that mail. However, I > > >>>> > > >>would suggest > > >> > > >>>>>using the names from XML-Schema part 2 Data types, as this would > > >>>>>facilitate creating a formal/normative definition, versus > > >>>> > > >>>>reinventing > > >>>> > > >>>>>names for the types. > > >>>>> > > >>>>> Here are the type names from XML-Schema/IPDR: > > >>>>> > > >>>>> boolean - true/false (may be encoded in byte) > > >>>>> byte > > >>>>> unsignedByte > > >>>>> short > > >>>>> unsignedShort > > >>>>> int - 32-bit signed > > >>>>> long - 64-bit signed > > >>>>> unsignedInt > > >>>>> unsignedLong > > >>>>> float > > >>>>> double > > >>>>> dateTime - 32-bit integer representing seconds > > >>>> > > >>since EPOCH > > >> > > >>>>>(1/1/1970 0:00 GMT) > > >>>>> string - UTF-8 formatted text > > >>>>> ipdr:dateTimeMsec - 64-bit integer representing msecs > > >>>> > > >>>>since EPOCH > > >>>> > > >>>>> ipdr:ipV4Addr > > >>>>> ipdr:ipV6Addr > > >>>>> ipdr:UUID > > >>>>> hexBinary - arbitrary sequence of octets > > >>>>> > > >>>>> If the protocol itself has a basic "octetString" (or from > > >>>> > > >>>>XML, hexBinary) > > >>>> > > >>>>>payload, > > >>>>>then this can be used to carry new classes of information > > >>>> > > >>>>in the future w/o > > >>>> > > >>>>>rejiggering the protocol itself. > > >>>>> > > >>>>> In addition other "derived" types may be created from the > > >>>> > > >>>>base types. For > > >>>> > > >>>>>instance in IPDR, the ipdr:dateTimeMsec is derived from long. > > >>>>> > > >>>>>Regards, > > >>>>> > > >>>>> Jeff Meyer > > >>>>> > > >>>>> > > >>>>>>-----Original Message----- > > >>>>>>From: Juergen Quittek [mailto:quittek@ccrle.nec.de] > > >>>>>>Sent: Monday, April 28, 2003 2:06 PM > > >>>>>>To: ipfix@net.doit.wisc.edu > > >>>>>>Subject: [ipfix] drawing the line between protocol > > >>>>> > > >>and data model > > >> > > >>>>>>document > > >>>>>> > > >>>>>> > > >>>>>>Hi all, > > >>>>>> > > >>>>>>When shaping the planned documents, we have to draw a line > > >>>>>>between the protocol and the data model. > > >>>>>> > > >>>>>>Definitely, the data model document defines all field types, > > >>>>>>but does it also define basic data types, such as '16 bit > > >>>>>>unsigned integer'? > > >>>>>> > > >>>>>>Maybe we should have a set of basic data types defined in > > >>>>>>the protocol document and restrict the data model doc to > > >>>>> > > >>>>these types? > > >>>> > > >>>>>>This would further structure the data model. > > >>>>>> > > >>>>>>Any thoughts? > > >>>>>> > > >>>>>> Juergen > > >>>>>> > > >>>>>>-- > > >>>>>>Help mailto:majordomo@net.doit.wisc.edu and say "help" > > >>>>>>in message body > > >>>>>>Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > >>>>>>"unsubscribe ipfix" in message body > > >>>>>>Archive http://ipfix.doit.wisc.edu/archive/ > > >>>>>> > > >>>>> > > >>>>>-- > > >>>>>Help mailto:majordomo@net.doit.wisc.edu and say > > >>>> > > >>>>"help" in message body > > >>>> > > >>>>>Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > >>>>>"unsubscribe ipfix" in message body > > >>>>>Archive http://ipfix.doit.wisc.edu/archive/ > > >>>> > > >>>-- > > >>>Help mailto:majordomo@net.doit.wisc.edu and say > > >> > > >>"help" in message body > > >> > > >>>Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > >>>"unsubscribe ipfix" in message body > > >>>Archive http://ipfix.doit.wisc.edu/archive/ > > >> > > > > > > -- > > > Help mailto:majordomo@net.doit.wisc.edu and say > > "help" in message body > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > "unsubscribe ipfix" in message body > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > > -- > > Sebastian Zander E-mail: > > zander@fokus.fraunhofer.de > > Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 > > Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 > > D-10589 Berlin, Germany > > www.fokus.fraunhofer.de/usr/sebastian.zander > > > > > > > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue May 6 12:07:38 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA27454 for ; Tue, 6 May 2003 12:07:38 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19D4nb-0000NT-00 for ipfix-list@mil.doit.wisc.edu; Tue, 06 May 2003 10:56:11 -0500 Received: from [64.95.122.13] (helo=RS-SC-EXC4.rs.riverstonenet.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19D4nZ-0000NK-00 for ipfix@net.doit.wisc.edu; Tue, 06 May 2003 10:56:09 -0500 Received: from riverstonenet.com ([172.17.6.3]) by RS-SC-EXC4.rs.riverstonenet.com with Microsoft SMTPSVC(5.0.2195.4905); Tue, 6 May 2003 08:56:08 -0700 Message-ID: <3EB7DB16.1886F07C@riverstonenet.com> Date: Tue, 06 May 2003 11:56:06 -0400 From: calato@riverstonenet.com X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.7-10 i686) X-Accept-Language: en MIME-Version: 1.0 To: Maurizio Molina CC: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] export packet length - LAST CALL References: <3EA95941.57BFEE44@ccrle.nec.de> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 06 May 2003 15:56:09.0114 (UTC) FILETIME=[027617A0:01C313E8] Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit There seems to be a consensus that having a length field in the packet header is a good idea. I would like to bring this issue to last call. Anyone opposed please state your case one more time. If this doesn't seem like the right process to close out issues, suggestions are welcome. Paul Maurizio Molina wrote: > > Hi all, > I hope not to rise an issue already discussed in the WG. If yes, I > apologise in advance. > If an IPFIX export packet is carried over TCP, it's not unusual that it > gets split across multiple TCP segments. > At the collector side, on the contrary, the implementation of the > parsing is much simpler if only full export packet are parsed. > If the information about the WHOLE export packet were contained in the > header, it would be simple to wait until all the bytes composing an > export packets are read from the TCP socket before beginning the > parsing. > Unfortunately, this information isn't currently contained in Netflow 9 > packet header (I suppose, due to Netflow's 9 "bias" towards UDP...). > Therefore, a parsing implementation having to deal with this > "fragentation" problem without this information needs to be much more > complicated (e.g. getting step by step the lenght of each flow set, see > if it's all there, parse it and go on....). > In summary: if IPFIX has to be carried over tcp, having the length of > the export packet in the packet header would be beneficial. > Regards, > Maurizio > > -- > Maurizio Molina > Research Staff member > Network Laboratories Heidelberg > NEC Europe Ltd. > Kurfuersten-Anlage 36, 69115 Heidelberg, Germany. > Tel. (49)6221 90511-18 Fax: (49)6221 90511-55 > e-mail: molina@ccrle.nec.de > Web: www.ccrle.nec.de > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue May 6 13:19:45 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA00129 for ; Tue, 6 May 2003 13:19:45 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19D5yq-0001wp-00 for ipfix-list@mil.doit.wisc.edu; Tue, 06 May 2003 12:11:52 -0500 Received: from usmail.xacct.com ([204.253.100.12]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19D5yo-0001wg-00 for ipfix@net.doit.wisc.edu; Tue, 06 May 2003 12:11:50 -0500 Received: from Givoly (000-017-426.area1.spcsdns.net [68.24.68.155]) by usmail.xacct.com (8.11.2/8.11.2) with SMTP id h46HHxi22118; Tue, 6 May 2003 10:18:06 -0700 From: "Tal Givoly" To: , "Maurizio Molina" Cc: Subject: RE: [ipfix] export packet length - LAST CALL Date: Tue, 6 May 2003 10:11:17 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-Reply-To: <3EB7DB16.1886F07C@riverstonenet.com> Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit I support this and prefer that any length field be encoded in a simple form (either 2 bytes or 4 bytes, but not "extendable"). Tal -----Original Message----- From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu]On Behalf Of calato@riverstonenet.com Sent: Tuesday, May 06, 2003 8:56 AM To: Maurizio Molina Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] export packet length - LAST CALL There seems to be a consensus that having a length field in the packet header is a good idea. I would like to bring this issue to last call. Anyone opposed please state your case one more time. If this doesn't seem like the right process to close out issues, suggestions are welcome. Paul Maurizio Molina wrote: > > Hi all, > I hope not to rise an issue already discussed in the WG. If yes, I > apologise in advance. > If an IPFIX export packet is carried over TCP, it's not unusual that it > gets split across multiple TCP segments. > At the collector side, on the contrary, the implementation of the > parsing is much simpler if only full export packet are parsed. > If the information about the WHOLE export packet were contained in the > header, it would be simple to wait until all the bytes composing an > export packets are read from the TCP socket before beginning the > parsing. > Unfortunately, this information isn't currently contained in Netflow 9 > packet header (I suppose, due to Netflow's 9 "bias" towards UDP...). > Therefore, a parsing implementation having to deal with this > "fragentation" problem without this information needs to be much more > complicated (e.g. getting step by step the lenght of each flow set, see > if it's all there, parse it and go on....). > In summary: if IPFIX has to be carried over tcp, having the length of > the export packet in the packet header would be beneficial. > Regards, > Maurizio > > -- > Maurizio Molina > Research Staff member > Network Laboratories Heidelberg > NEC Europe Ltd. > Kurfuersten-Anlage 36, 69115 Heidelberg, Germany. > Tel. (49)6221 90511-18 Fax: (49)6221 90511-55 > e-mail: molina@ccrle.nec.de > Web: www.ccrle.nec.de > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue May 6 16:44:06 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA07909 for ; Tue, 6 May 2003 16:44:05 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19D9AZ-00062G-00 for ipfix-list@mil.doit.wisc.edu; Tue, 06 May 2003 15:36:11 -0500 Received: from eng4.oar.net ([192.148.244.24]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 19D9AY-00062A-00 for ipfix@net.doit.wisc.edu; Tue, 06 May 2003 15:36:10 -0500 Received: (qmail 58267 invoked by uid 4454); 6 May 2003 20:36:09 -0000 Date: Tue, 6 May 2003 16:36:09 -0400 From: Mark Fullmer To: calato@riverstonenet.com Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] export packet length - LAST CALL Message-ID: <20030506163609.A58211@net.ohio-state.edu> References: <3EA95941.57BFEE44@ccrle.nec.de> <3EB7DB16.1886F07C@riverstonenet.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: <3EB7DB16.1886F07C@riverstonenet.com>; from calato@riverstonenet.com on Tue, May 06, 2003 at 11:56:06AM -0400 Precedence: bulk Sender: majordomo listserver To close this issue I would like to see the length specified (2 or 4 byte fixed header field). If 2 bytes then is replacing the existing count field okay. And yes, I think closing some of the recently discussed open issues with last call's is the right way to get things moving. mark On Tue, May 06, 2003 at 11:56:06AM -0400, calato@riverstonenet.com wrote: > > There seems to be a consensus that having a length field > in the packet header is a good idea. > > I would like to bring this issue to last call. Anyone > opposed please state your case one more time. > > If this doesn't seem like the right process to close > out issues, suggestions are welcome. > > Paul > > > Maurizio Molina wrote: > > > > Hi all, > > I hope not to rise an issue already discussed in the WG. If yes, I > > apologise in advance. > > If an IPFIX export packet is carried over TCP, it's not unusual that it > > gets split across multiple TCP segments. > > At the collector side, on the contrary, the implementation of the > > parsing is much simpler if only full export packet are parsed. > > If the information about the WHOLE export packet were contained in the > > header, it would be simple to wait until all the bytes composing an > > export packets are read from the TCP socket before beginning the > > parsing. > > Unfortunately, this information isn't currently contained in Netflow 9 > > packet header (I suppose, due to Netflow's 9 "bias" towards UDP...). > > Therefore, a parsing implementation having to deal with this > > "fragentation" problem without this information needs to be much more > > complicated (e.g. getting step by step the lenght of each flow set, see > > if it's all there, parse it and go on....). > > In summary: if IPFIX has to be carried over tcp, having the length of > > the export packet in the packet header would be beneficial. > > Regards, > > Maurizio > > > > -- > > Maurizio Molina > > Research Staff member > > Network Laboratories Heidelberg > > NEC Europe Ltd. > > Kurfuersten-Anlage 36, 69115 Heidelberg, Germany. > > Tel. (49)6221 90511-18 Fax: (49)6221 90511-55 > > e-mail: molina@ccrle.nec.de > > Web: www.ccrle.nec.de > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue May 6 20:00:16 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA14617 for ; Tue, 6 May 2003 20:00:16 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19DBy7-0001uy-00 for ipfix-list@mil.doit.wisc.edu; Tue, 06 May 2003 18:35:31 -0500 Received: from sj-core-2.cisco.com ([171.71.177.254]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19DBy5-0001us-00 for ipfix@net.doit.wisc.edu; Tue, 06 May 2003 18:35:29 -0500 Received: from mira-sjc5-f.cisco.com (IDENT:mirapoint@mira-sjc5-f.cisco.com [171.71.163.13]) by sj-core-2.cisco.com (8.12.6/8.12.6) with ESMTP id h46NZAxI003579; Tue, 6 May 2003 16:35:11 -0700 (PDT) Received: from cisco.com (dhcp-171-71-204-182.cisco.com [171.71.204.182]) by mira-sjc5-f.cisco.com (Mirapoint Messaging Server MOS 3.3.3-GR) with ESMTP id AFY20670; Tue, 6 May 2003 16:42:20 -0700 (PDT) Message-ID: <3EB846AD.EEF9595C@cisco.com> Date: Tue, 06 May 2003 16:35:09 -0700 From: Ganesh Sadasivan Organization: Cisco Systems Inc X-Mailer: Mozilla 4.76 [en]C-CCK-MCD (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: calato@riverstonenet.com CC: "MEYER,JEFFREY D (HP-Cupertino,ex1)" , "'Sebastian Zander'" , "'Mark Fullmer'" , ipfix@net.doit.wisc.edu Subject: Re: [ipfix] drawing the line between protocol and data model document References: <1D3D2C371FCBD947A7897FABBD3533A566BAA2@xsun01.ptp.hp.com> <3EB7D865.292A6190@riverstonenet.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit calato@riverstonenet.com wrote: > OK. I'm confused. Let me run through an example and people can correct > me where I'm wrong. > > Lets take "Source IP v4 address". Lets also hold off on variable > length fields for now. > > All options need some kind of type value to indicate what the value > semantically means. Lets pick 57 == Source IP v4 address. So what we > need next is info on the value part so we can decode it and it should > be backward compatible (i.e. older versions can at least skip unknown > elements. > > Option I: Element and Length > > 57, 4 > > Any software understanding element 57 knows it is the Source > IP address and can display/handle it correctly. Any one else would > skip over the value in the flow set. > > Option II: Element and data type > > Lets say data type 22 == IP V4 Address > > 57, 22 > > Any software understanding element 57 knows it is the Source > IP address. But what if it doesn't know what data type 22 is? > How can I skip over it? > > Option III: Element and data type and length > > 57, 22, 4 > > Any software which understands 57 and 22 can handle it correclty. > If it only understands 22, it can ONLY display the value in > some rational way, nothing else can be done. If I don't understand 22 > I can at least skip over it by using the length. If protocol were to do the encoding, does it not depend on the encoding (regardless of the type) the way a field is intrepreted/displayed by the collector software ? Eg. One could encode IP address as a 15 byte string or a 4 byte unsigned. On the otherhand if 57 always maps to 22, is there a need to specify the type to decode this in s/w (agreed that this has to be specified in the document)? Is display the only advantage behind defining the data (encoding) type? What other benefits do we get out of option III? Ganesh > > > Correct away! > > Paul > > "MEYER,JEFFREY D (HP-Cupertino,ex1)" wrote: > > > > Sebastian, > > > > Thanks for speaking up! I agree with your restatement of the > > relationship between typing and semantics. > > > > Regarding types like UTF8 or OctetString, as I read the proposal > > from Benoit, the "Length Encoding" model would use a special value > > of 0xFFFF instead of length in the template. This implies that > > the encoding of the "variable length" field would be of the form > > (length) (value), versus fixed length types which can be encoded by > > just placing their (value) in the payload. > > > > (see http://ipfix.doit.wisc.edu/archive/1602.html) > > > > This same model would apply in a type based scheme, if the type > > were string or hexBinary (indicated by the appropriate integer > > identifier in the template), then the encoded value would still be > > of the form (length) (value) in the payload. > > > > Regardless of what template scheme is chosen, however, I think it > > is important that the protocol spec speak in terms of how types are > > encoded, not how each individual element declared in the information model > > is encoded. This makes the inevitable extension of the information > > model easier. > > > > Regards, > > > > Jeff Meyer > > > > > -----Original Message----- > > > From: Sebastian Zander [mailto:zander@fokus.fraunhofer.de] > > > Sent: Thursday, May 01, 2003 5:57 PM > > > To: MEYER,JEFFREY D (HP-Cupertino,ex1) > > > Cc: 'Mark Fullmer'; ipfix@net.doit.wisc.edu > > > Subject: Re: [ipfix] drawing the line between protocol and data model > > > docu ment > > > > > > > > > Jeff, > > > > > > i support the idea of typing. I think there is some > > > misunderstanding here > > > by some people who confuse types with semantics. A type does > > > not define the > > > semantics. In my opinion typing has the advantage that even > > > if i don't know > > > the semantics by the time of the decoding i still can > > > correctly decode the > > > value and present it. It also has the advantage that for a > > > well known type > > > the encoding is well defined. If that is not the case people > > > may encode > > > attributes of the same type differently just making > > > interoperability harder. > > > I would define these basic data types in the protocol document. > > > > > > I am not so sure if both alternatives take the same space > > > because there > > > are some types which which do not have an intrinsic length > > > definition. An > > > utf or octet string could have an arbitrary length. Without > > > typing everything > > > would be basically an octetstring of some length ;-). However > > > i though the > > > protocol is supossed to use templates and so it would only affect the > > > template exchange wouldn't it? > > > > > > Cheers, > > > > > > Sebastian > > > > > > MEYER,JEFFREY D (HP-Cupertino,ex1) wrote: > > > > vs. > > > > > > > > = 16909060 (integer) > > > > > > > > vs. > > > > > > > > = whatever the heck 0x01020304 represents > > > > in IEEE Floating Point notation. (something around 4.06) > > > > > > > > > > > > So, yes that's all it buys you. Since both alternatives take > > > > the same amount of space, I'd prefer the one with more > > > > intrinsic information. But, I guess I'm in the minority :-( > > > > > > > > > > > > Regards, > > > > > > > > Jeff Meyer > > > > > > > > > > > > > > > >>-----Original Message----- > > > >>From: Mark Fullmer [mailto:maf@eng.oar.net] > > > >>Sent: Thursday, May 01, 2003 8:39 AM > > > >>To: MEYER,JEFFREY D (HP-Cupertino,ex1) > > > >>Cc: ipfix@net.doit.wisc.edu > > > >>Subject: Re: [ipfix] drawing the line between protocol and > > > data model > > > >>docu ment > > > >> > > > >> > > > >>So the difference boils down to a tool being able to display > > > >> > > > >> = 0x01020304 > > > >> vs > > > >> = 1.2.3.4 > > > >> > > > >>In this situation someone will either grab a new version of the > > > >>software or an updated data definition to resolve the unknown > > > >>attribute, > > > >>which would also define it's type. > > > >> > > > >>mark > > > >> > > > >>On Tue, Apr 29, 2003 at 06:25:11PM -0400, MEYER,JEFFREY D > > > >>(HP-Cupertino,ex1) wrote: > > > >> > > > >>>Mark, > > > >>> > > > >>> I'll try to clarify: > > > >>> > > > >>> Concept #1: The information model and the encoding are separate. > > > >>> A well defined mapping determines how an information model is > > > >>> to be encoded for a specific use (e.g. IPFIX). The encoding > > > >>> may take whatever approach allows both ends to do something > > > >>> useful with the data. As you point out there is some useful > > > >>> stuff which can be done with the length alone. > > > >>> > > > >>> Concept #2: Derived typing. By defining a base set of types, > > > >>> which must be understood, derived types can be defined in terms > > > >>> of these base types. For instance Diameter does this. This > > > >>> enables a core set of types to be defined and coded to, but > > > >>> also enables richer types later on with an appropriate fallback > > > >>> for systems which aren't aware of the extensions. > > > >>> > > > >>> The benefit I see of explicit typing is that instead of seeing > > > >>>that a field is of length "4 bytes" something which will be seen > > > >>>A LOT. Is that we can instead differentiate between > > > >> > > > >>IPAddr, Integer, > > > >> > > > >>>unsigned integer, and float. This seems handy to me. > > > >>> > > > >>> Your argument seems centered around the long term potential of > > > >>>adding lots of fixed length fields whose base length is not 4,8 > > > >>>or 16 bytes long, for instance MACAddr (although if we agree on > > > >>>going the type route, I'd recommend adding MACAddr and then you > > > >>>have 1,2,4,6,8 and 16 bytes covered. I guess I don't foresee > > > >>>a lot more fixed length fields coming down the pipe in the IPFIX > > > >>>domain. Hence my desire to make it more explicit for the types > > > >>>which do occur all the time and are likely points of extension. > > > >>>WHY? Because then a tool could at least present these in a > > > >> > > > >>meaningful > > > >> > > > >>>way. As opposed to incorrectly formatting a float type as an int. > > > >>> > > > >>> But hey, either way will work. If the goal is to limit the > > > >>>amount of changes, I'll (grudgingly) go along. I haven't heard > > > >>>any opinions from anyone but you in either direction. So, > > > >>>at this point I don't see a quorom either way... > > > >>> > > > >>>Regards, > > > >>> > > > >>> Jeff Meyer > > > >>> > > > >>> > > > >>> > > > >>>>-----Original Message----- > > > >>>>From: Mark Fullmer [mailto:maf@eng.oar.net] > > > >>>>Sent: Tuesday, April 29, 2003 2:53 PM > > > >>>>To: MEYER,JEFFREY D (HP-Cupertino,ex1) > > > >>>>Cc: ipfix@net.doit.wisc.edu > > > >>>>Subject: Re: [ipfix] drawing the line between protocol > > > >>> > > > >>and data model > > > >> > > > >>>>docu ment > > > >>>> > > > >>>> > > > >>>>IMHO crippling the protocol to serve the needs of a > > > >>> > > > >>documentation tool > > > >> > > > >>>>is just silly. > > > >>>> > > > >>>>It would be possible to use both a Type name and a Length > > > >>> > > > >>instead of > > > >> > > > >>>>the existing Length, which would allow a collector to process > > > >>>>unknown Types. > > > >>>> > > > >>>>With the existing v9 protocol a collector can ignore an > > > >>>>unknown attribute, > > > >>>>or possibly just pass it on to post processing tools which > > > >>>>might understand > > > >>>>it. > > > >>>> > > > >>>>Your suggested changes no longer allow a collector to > > > >>> > > > >>ignore unknown > > > >> > > > >>>>attributes if the Type is also unknown. Resorting to encoding all > > > >>>>new Types after the initial protocol spec as octet > > > >>> > > > >>strings is (IMHO) > > > >> > > > >>>>not acceptable. > > > >>>> > > > >>>>I'm still trying to understand what the benefits to explicit > > > >>>>Types sent > > > >>>>in templates are. > > > >>>> > > > >>>>mark > > > >>>> > > > >>>>On Tue, Apr 29, 2003 at 02:50:02PM -0400, MEYER,JEFFREY D > > > >>>>(HP-Cupertino,ex1) wrote: > > > >>>> > > > >>>>>Juergen, > > > >>>>> > > > >>>>> I agree that the data model should have a well defined type > > > >>>>>system. And that that typing system should reflect the > > > >>>> > > > >>>>requirements. > > > >>>> > > > >>>>> This is in part why I was hoping that the protocol > > > >>>> > > > >>itself would > > > >> > > > >>>>>be more explicit here. (See my mail > > > >>>>>http://ipfix.doit.wisc.edu/archive/1604.html) > > > >>>>> > > > >>>>> There is also a type list in that mail. However, I > > > >>>> > > > >>would suggest > > > >> > > > >>>>>using the names from XML-Schema part 2 Data types, as this would > > > >>>>>facilitate creating a formal/normative definition, versus > > > >>>> > > > >>>>reinventing > > > >>>> > > > >>>>>names for the types. > > > >>>>> > > > >>>>> Here are the type names from XML-Schema/IPDR: > > > >>>>> > > > >>>>> boolean - true/false (may be encoded in byte) > > > >>>>> byte > > > >>>>> unsignedByte > > > >>>>> short > > > >>>>> unsignedShort > > > >>>>> int - 32-bit signed > > > >>>>> long - 64-bit signed > > > >>>>> unsignedInt > > > >>>>> unsignedLong > > > >>>>> float > > > >>>>> double > > > >>>>> dateTime - 32-bit integer representing seconds > > > >>>> > > > >>since EPOCH > > > >> > > > >>>>>(1/1/1970 0:00 GMT) > > > >>>>> string - UTF-8 formatted text > > > >>>>> ipdr:dateTimeMsec - 64-bit integer representing msecs > > > >>>> > > > >>>>since EPOCH > > > >>>> > > > >>>>> ipdr:ipV4Addr > > > >>>>> ipdr:ipV6Addr > > > >>>>> ipdr:UUID > > > >>>>> hexBinary - arbitrary sequence of octets > > > >>>>> > > > >>>>> If the protocol itself has a basic "octetString" (or from > > > >>>> > > > >>>>XML, hexBinary) > > > >>>> > > > >>>>>payload, > > > >>>>>then this can be used to carry new classes of information > > > >>>> > > > >>>>in the future w/o > > > >>>> > > > >>>>>rejiggering the protocol itself. > > > >>>>> > > > >>>>> In addition other "derived" types may be created from the > > > >>>> > > > >>>>base types. For > > > >>>> > > > >>>>>instance in IPDR, the ipdr:dateTimeMsec is derived from long. > > > >>>>> > > > >>>>>Regards, > > > >>>>> > > > >>>>> Jeff Meyer > > > >>>>> > > > >>>>> > > > >>>>>>-----Original Message----- > > > >>>>>>From: Juergen Quittek [mailto:quittek@ccrle.nec.de] > > > >>>>>>Sent: Monday, April 28, 2003 2:06 PM > > > >>>>>>To: ipfix@net.doit.wisc.edu > > > >>>>>>Subject: [ipfix] drawing the line between protocol > > > >>>>> > > > >>and data model > > > >> > > > >>>>>>document > > > >>>>>> > > > >>>>>> > > > >>>>>>Hi all, > > > >>>>>> > > > >>>>>>When shaping the planned documents, we have to draw a line > > > >>>>>>between the protocol and the data model. > > > >>>>>> > > > >>>>>>Definitely, the data model document defines all field types, > > > >>>>>>but does it also define basic data types, such as '16 bit > > > >>>>>>unsigned integer'? > > > >>>>>> > > > >>>>>>Maybe we should have a set of basic data types defined in > > > >>>>>>the protocol document and restrict the data model doc to > > > >>>>> > > > >>>>these types? > > > >>>> > > > >>>>>>This would further structure the data model. > > > >>>>>> > > > >>>>>>Any thoughts? > > > >>>>>> > > > >>>>>> Juergen > > > >>>>>> > > > >>>>>>-- > > > >>>>>>Help mailto:majordomo@net.doit.wisc.edu and say "help" > > > >>>>>>in message body > > > >>>>>>Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > >>>>>>"unsubscribe ipfix" in message body > > > >>>>>>Archive http://ipfix.doit.wisc.edu/archive/ > > > >>>>>> > > > >>>>> > > > >>>>>-- > > > >>>>>Help mailto:majordomo@net.doit.wisc.edu and say > > > >>>> > > > >>>>"help" in message body > > > >>>> > > > >>>>>Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > >>>>>"unsubscribe ipfix" in message body > > > >>>>>Archive http://ipfix.doit.wisc.edu/archive/ > > > >>>> > > > >>>-- > > > >>>Help mailto:majordomo@net.doit.wisc.edu and say > > > >> > > > >>"help" in message body > > > >> > > > >>>Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > >>>"unsubscribe ipfix" in message body > > > >>>Archive http://ipfix.doit.wisc.edu/archive/ > > > >> > > > > > > > > -- > > > > Help mailto:majordomo@net.doit.wisc.edu and say > > > "help" in message body > > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > > "unsubscribe ipfix" in message body > > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > > > > > > -- > > > Sebastian Zander E-mail: > > > zander@fokus.fraunhofer.de > > > Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 > > > Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 > > > D-10589 Berlin, Germany > > > www.fokus.fraunhofer.de/usr/sebastian.zander > > > > > > > > > > > > > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue May 6 20:09:13 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA14809 for ; Tue, 6 May 2003 20:09:13 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19DCDn-0002Df-00 for ipfix-list@mil.doit.wisc.edu; Tue, 06 May 2003 18:51:43 -0500 Received: from sj-core-5.cisco.com ([171.71.177.238]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19DCDl-0002DZ-00 for ipfix@net.doit.wisc.edu; Tue, 06 May 2003 18:51:41 -0500 Received: from mira-sjc5-f.cisco.com (IDENT:mirapoint@mira-sjc5-f.cisco.com [171.71.163.13]) by sj-core-5.cisco.com (8.12.6/8.12.6) with ESMTP id h46NpVua000888; Tue, 6 May 2003 16:51:32 -0700 (PDT) Received: from cisco.com (dhcp-171-71-204-182.cisco.com [171.71.204.182]) by mira-sjc5-f.cisco.com (Mirapoint Messaging Server MOS 3.3.3-GR) with ESMTP id AFY22542; Tue, 6 May 2003 16:58:40 -0700 (PDT) Message-ID: <3EB84A81.84DD73DE@cisco.com> Date: Tue, 06 May 2003 16:51:29 -0700 From: Ganesh Sadasivan Organization: Cisco Systems Inc X-Mailer: Mozilla 4.76 [en]C-CCK-MCD (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: calato@riverstonenet.com CC: Maurizio Molina , ipfix@net.doit.wisc.edu Subject: Re: [ipfix] export packet length - LAST CALL References: <3EA95941.57BFEE44@ccrle.nec.de> <3EB7DB16.1886F07C@riverstonenet.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit calato@riverstonenet.com wrote: > There seems to be a consensus that having a length field > in the packet header is a good idea. Fully agreed. -ganesh > > > I would like to bring this issue to last call. Anyone > opposed please state your case one more time. > > If this doesn't seem like the right process to close > out issues, suggestions are welcome. > > Paul > > Maurizio Molina wrote: > > > > Hi all, > > I hope not to rise an issue already discussed in the WG. If yes, I > > apologise in advance. > > If an IPFIX export packet is carried over TCP, it's not unusual that it > > gets split across multiple TCP segments. > > At the collector side, on the contrary, the implementation of the > > parsing is much simpler if only full export packet are parsed. > > If the information about the WHOLE export packet were contained in the > > header, it would be simple to wait until all the bytes composing an > > export packets are read from the TCP socket before beginning the > > parsing. > > Unfortunately, this information isn't currently contained in Netflow 9 > > packet header (I suppose, due to Netflow's 9 "bias" towards UDP...). > > Therefore, a parsing implementation having to deal with this > > "fragentation" problem without this information needs to be much more > > complicated (e.g. getting step by step the lenght of each flow set, see > > if it's all there, parse it and go on....). > > In summary: if IPFIX has to be carried over tcp, having the length of > > the export packet in the packet header would be beneficial. > > Regards, > > Maurizio > > > > -- > > Maurizio Molina > > Research Staff member > > Network Laboratories Heidelberg > > NEC Europe Ltd. > > Kurfuersten-Anlage 36, 69115 Heidelberg, Germany. > > Tel. (49)6221 90511-18 Fax: (49)6221 90511-55 > > e-mail: molina@ccrle.nec.de > > Web: www.ccrle.nec.de > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue May 6 20:40:21 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA15641 for ; Tue, 6 May 2003 20:40:21 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19DCpf-00032Q-00 for ipfix-list@mil.doit.wisc.edu; Tue, 06 May 2003 19:30:51 -0500 Received: from sj-core-5.cisco.com ([171.71.177.238]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19DCpd-00032F-00 for ipfix@net.doit.wisc.edu; Tue, 06 May 2003 19:30:49 -0500 Received: from mira-sjc5-f.cisco.com (IDENT:mirapoint@mira-sjc5-f.cisco.com [171.71.163.13]) by sj-core-5.cisco.com (8.12.6/8.12.6) with ESMTP id h470UYua016778; Tue, 6 May 2003 17:30:34 -0700 (PDT) Received: from cisco.com (dhcp-171-71-204-182.cisco.com [171.71.204.182]) by mira-sjc5-f.cisco.com (Mirapoint Messaging Server MOS 3.3.3-GR) with ESMTP id AFY27039; Tue, 6 May 2003 17:37:44 -0700 (PDT) Message-ID: <3EB853A9.2B5159C@cisco.com> Date: Tue, 06 May 2003 17:30:34 -0700 From: Ganesh Sadasivan Organization: Cisco Systems Inc X-Mailer: Mozilla 4.76 [en]C-CCK-MCD (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: "MEYER,JEFFREY D (HP-Cupertino,ex1)" CC: "'Sebastian Zander'" , "'Mark Fullmer'" , ipfix@net.doit.wisc.edu Subject: Re: [ipfix] drawing the line between protocol and data model document References: <1D3D2C371FCBD947A7897FABBD3533A566BAA2@xsun01.ptp.hp.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit I agree to the statement that typing and semantics are indepenedent of each other and need to be documented. Also encoding for the types should be in a standard manner. The benefits that I understood from the e-mail thread is a better presentation. Is that alone a very strong argument go with this approach in an implementation (.i.e specify the data types also in the templates)? I am not sure how much this would be useful if the semantics is not known. Thanks Gansh "MEYER,JEFFREY D (HP-Cupertino,ex1)" wrote: > Sebastian, > > Thanks for speaking up! I agree with your restatement of the > relationship between typing and semantics. > > Regarding types like UTF8 or OctetString, as I read the proposal > from Benoit, the "Length Encoding" model would use a special value > of 0xFFFF instead of length in the template. This implies that > the encoding of the "variable length" field would be of the form > (length) (value), versus fixed length types which can be encoded by > just placing their (value) in the payload. > > (see http://ipfix.doit.wisc.edu/archive/1602.html) > > This same model would apply in a type based scheme, if the type > were string or hexBinary (indicated by the appropriate integer > identifier in the template), then the encoded value would still be > of the form (length) (value) in the payload. > > Regardless of what template scheme is chosen, however, I think it > is important that the protocol spec speak in terms of how types are > encoded, not how each individual element declared in the information model > is encoded. This makes the inevitable extension of the information > model easier. > > Regards, > > Jeff Meyer > > > -----Original Message----- > > From: Sebastian Zander [mailto:zander@fokus.fraunhofer.de] > > Sent: Thursday, May 01, 2003 5:57 PM > > To: MEYER,JEFFREY D (HP-Cupertino,ex1) > > Cc: 'Mark Fullmer'; ipfix@net.doit.wisc.edu > > Subject: Re: [ipfix] drawing the line between protocol and data model > > docu ment > > > > > > Jeff, > > > > i support the idea of typing. I think there is some > > misunderstanding here > > by some people who confuse types with semantics. A type does > > not define the > > semantics. In my opinion typing has the advantage that even > > if i don't know > > the semantics by the time of the decoding i still can > > correctly decode the > > value and present it. It also has the advantage that for a > > well known type > > the encoding is well defined. If that is not the case people > > may encode > > attributes of the same type differently just making > > interoperability harder. > > I would define these basic data types in the protocol document. > > > > I am not so sure if both alternatives take the same space > > because there > > are some types which which do not have an intrinsic length > > definition. An > > utf or octet string could have an arbitrary length. Without > > typing everything > > would be basically an octetstring of some length ;-). However > > i though the > > protocol is supossed to use templates and so it would only affect the > > template exchange wouldn't it? > > > > Cheers, > > > > Sebastian > > > > MEYER,JEFFREY D (HP-Cupertino,ex1) wrote: > > > vs. > > > > > > = 16909060 (integer) > > > > > > vs. > > > > > > = whatever the heck 0x01020304 represents > > > in IEEE Floating Point notation. (something around 4.06) > > > > > > > > > So, yes that's all it buys you. Since both alternatives take > > > the same amount of space, I'd prefer the one with more > > > intrinsic information. But, I guess I'm in the minority :-( > > > > > > > > > Regards, > > > > > > Jeff Meyer > > > > > > > > > > > >>-----Original Message----- > > >>From: Mark Fullmer [mailto:maf@eng.oar.net] > > >>Sent: Thursday, May 01, 2003 8:39 AM > > >>To: MEYER,JEFFREY D (HP-Cupertino,ex1) > > >>Cc: ipfix@net.doit.wisc.edu > > >>Subject: Re: [ipfix] drawing the line between protocol and > > data model > > >>docu ment > > >> > > >> > > >>So the difference boils down to a tool being able to display > > >> > > >> = 0x01020304 > > >> vs > > >> = 1.2.3.4 > > >> > > >>In this situation someone will either grab a new version of the > > >>software or an updated data definition to resolve the unknown > > >>attribute, > > >>which would also define it's type. > > >> > > >>mark > > >> > > >>On Tue, Apr 29, 2003 at 06:25:11PM -0400, MEYER,JEFFREY D > > >>(HP-Cupertino,ex1) wrote: > > >> > > >>>Mark, > > >>> > > >>> I'll try to clarify: > > >>> > > >>> Concept #1: The information model and the encoding are separate. > > >>> A well defined mapping determines how an information model is > > >>> to be encoded for a specific use (e.g. IPFIX). The encoding > > >>> may take whatever approach allows both ends to do something > > >>> useful with the data. As you point out there is some useful > > >>> stuff which can be done with the length alone. > > >>> > > >>> Concept #2: Derived typing. By defining a base set of types, > > >>> which must be understood, derived types can be defined in terms > > >>> of these base types. For instance Diameter does this. This > > >>> enables a core set of types to be defined and coded to, but > > >>> also enables richer types later on with an appropriate fallback > > >>> for systems which aren't aware of the extensions. > > >>> > > >>> The benefit I see of explicit typing is that instead of seeing > > >>>that a field is of length "4 bytes" something which will be seen > > >>>A LOT. Is that we can instead differentiate between > > >> > > >>IPAddr, Integer, > > >> > > >>>unsigned integer, and float. This seems handy to me. > > >>> > > >>> Your argument seems centered around the long term potential of > > >>>adding lots of fixed length fields whose base length is not 4,8 > > >>>or 16 bytes long, for instance MACAddr (although if we agree on > > >>>going the type route, I'd recommend adding MACAddr and then you > > >>>have 1,2,4,6,8 and 16 bytes covered. I guess I don't foresee > > >>>a lot more fixed length fields coming down the pipe in the IPFIX > > >>>domain. Hence my desire to make it more explicit for the types > > >>>which do occur all the time and are likely points of extension. > > >>>WHY? Because then a tool could at least present these in a > > >> > > >>meaningful > > >> > > >>>way. As opposed to incorrectly formatting a float type as an int. > > >>> > > >>> But hey, either way will work. If the goal is to limit the > > >>>amount of changes, I'll (grudgingly) go along. I haven't heard > > >>>any opinions from anyone but you in either direction. So, > > >>>at this point I don't see a quorom either way... > > >>> > > >>>Regards, > > >>> > > >>> Jeff Meyer > > >>> > > >>> > > >>> > > >>>>-----Original Message----- > > >>>>From: Mark Fullmer [mailto:maf@eng.oar.net] > > >>>>Sent: Tuesday, April 29, 2003 2:53 PM > > >>>>To: MEYER,JEFFREY D (HP-Cupertino,ex1) > > >>>>Cc: ipfix@net.doit.wisc.edu > > >>>>Subject: Re: [ipfix] drawing the line between protocol > > >>> > > >>and data model > > >> > > >>>>docu ment > > >>>> > > >>>> > > >>>>IMHO crippling the protocol to serve the needs of a > > >>> > > >>documentation tool > > >> > > >>>>is just silly. > > >>>> > > >>>>It would be possible to use both a Type name and a Length > > >>> > > >>instead of > > >> > > >>>>the existing Length, which would allow a collector to process > > >>>>unknown Types. > > >>>> > > >>>>With the existing v9 protocol a collector can ignore an > > >>>>unknown attribute, > > >>>>or possibly just pass it on to post processing tools which > > >>>>might understand > > >>>>it. > > >>>> > > >>>>Your suggested changes no longer allow a collector to > > >>> > > >>ignore unknown > > >> > > >>>>attributes if the Type is also unknown. Resorting to encoding all > > >>>>new Types after the initial protocol spec as octet > > >>> > > >>strings is (IMHO) > > >> > > >>>>not acceptable. > > >>>> > > >>>>I'm still trying to understand what the benefits to explicit > > >>>>Types sent > > >>>>in templates are. > > >>>> > > >>>>mark > > >>>> > > >>>>On Tue, Apr 29, 2003 at 02:50:02PM -0400, MEYER,JEFFREY D > > >>>>(HP-Cupertino,ex1) wrote: > > >>>> > > >>>>>Juergen, > > >>>>> > > >>>>> I agree that the data model should have a well defined type > > >>>>>system. And that that typing system should reflect the > > >>>> > > >>>>requirements. > > >>>> > > >>>>> This is in part why I was hoping that the protocol > > >>>> > > >>itself would > > >> > > >>>>>be more explicit here. (See my mail > > >>>>>http://ipfix.doit.wisc.edu/archive/1604.html) > > >>>>> > > >>>>> There is also a type list in that mail. However, I > > >>>> > > >>would suggest > > >> > > >>>>>using the names from XML-Schema part 2 Data types, as this would > > >>>>>facilitate creating a formal/normative definition, versus > > >>>> > > >>>>reinventing > > >>>> > > >>>>>names for the types. > > >>>>> > > >>>>> Here are the type names from XML-Schema/IPDR: > > >>>>> > > >>>>> boolean - true/false (may be encoded in byte) > > >>>>> byte > > >>>>> unsignedByte > > >>>>> short > > >>>>> unsignedShort > > >>>>> int - 32-bit signed > > >>>>> long - 64-bit signed > > >>>>> unsignedInt > > >>>>> unsignedLong > > >>>>> float > > >>>>> double > > >>>>> dateTime - 32-bit integer representing seconds > > >>>> > > >>since EPOCH > > >> > > >>>>>(1/1/1970 0:00 GMT) > > >>>>> string - UTF-8 formatted text > > >>>>> ipdr:dateTimeMsec - 64-bit integer representing msecs > > >>>> > > >>>>since EPOCH > > >>>> > > >>>>> ipdr:ipV4Addr > > >>>>> ipdr:ipV6Addr > > >>>>> ipdr:UUID > > >>>>> hexBinary - arbitrary sequence of octets > > >>>>> > > >>>>> If the protocol itself has a basic "octetString" (or from > > >>>> > > >>>>XML, hexBinary) > > >>>> > > >>>>>payload, > > >>>>>then this can be used to carry new classes of information > > >>>> > > >>>>in the future w/o > > >>>> > > >>>>>rejiggering the protocol itself. > > >>>>> > > >>>>> In addition other "derived" types may be created from the > > >>>> > > >>>>base types. For > > >>>> > > >>>>>instance in IPDR, the ipdr:dateTimeMsec is derived from long. > > >>>>> > > >>>>>Regards, > > >>>>> > > >>>>> Jeff Meyer > > >>>>> > > >>>>> > > >>>>>>-----Original Message----- > > >>>>>>From: Juergen Quittek [mailto:quittek@ccrle.nec.de] > > >>>>>>Sent: Monday, April 28, 2003 2:06 PM > > >>>>>>To: ipfix@net.doit.wisc.edu > > >>>>>>Subject: [ipfix] drawing the line between protocol > > >>>>> > > >>and data model > > >> > > >>>>>>document > > >>>>>> > > >>>>>> > > >>>>>>Hi all, > > >>>>>> > > >>>>>>When shaping the planned documents, we have to draw a line > > >>>>>>between the protocol and the data model. > > >>>>>> > > >>>>>>Definitely, the data model document defines all field types, > > >>>>>>but does it also define basic data types, such as '16 bit > > >>>>>>unsigned integer'? > > >>>>>> > > >>>>>>Maybe we should have a set of basic data types defined in > > >>>>>>the protocol document and restrict the data model doc to > > >>>>> > > >>>>these types? > > >>>> > > >>>>>>This would further structure the data model. > > >>>>>> > > >>>>>>Any thoughts? > > >>>>>> > > >>>>>> Juergen > > >>>>>> > > >>>>>>-- > > >>>>>>Help mailto:majordomo@net.doit.wisc.edu and say "help" > > >>>>>>in message body > > >>>>>>Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > >>>>>>"unsubscribe ipfix" in message body > > >>>>>>Archive http://ipfix.doit.wisc.edu/archive/ > > >>>>>> > > >>>>> > > >>>>>-- > > >>>>>Help mailto:majordomo@net.doit.wisc.edu and say > > >>>> > > >>>>"help" in message body > > >>>> > > >>>>>Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > >>>>>"unsubscribe ipfix" in message body > > >>>>>Archive http://ipfix.doit.wisc.edu/archive/ > > >>>> > > >>>-- > > >>>Help mailto:majordomo@net.doit.wisc.edu and say > > >> > > >>"help" in message body > > >> > > >>>Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > >>>"unsubscribe ipfix" in message body > > >>>Archive http://ipfix.doit.wisc.edu/archive/ > > >> > > > > > > -- > > > Help mailto:majordomo@net.doit.wisc.edu and say > > "help" in message body > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > "unsubscribe ipfix" in message body > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > > -- > > Sebastian Zander E-mail: > > zander@fokus.fraunhofer.de > > Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 > > Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 > > D-10589 Berlin, Germany > > www.fokus.fraunhofer.de/usr/sebastian.zander > > > > > > > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue May 6 21:03:35 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA16068 for ; Tue, 6 May 2003 21:03:34 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19DDDU-0003Wx-00 for ipfix-list@mil.doit.wisc.edu; Tue, 06 May 2003 19:55:28 -0500 Received: from c3p0.cc.swin.edu.au ([136.186.1.10] helo=net.cc.swin.edu.au) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19DDDS-0003Wn-00 for ipfix@net.doit.wisc.edu; Tue, 06 May 2003 19:55:26 -0500 Received: from fokus.fraunhofer.de (szander-laptop.caia.swin.edu.au [136.186.229.90]) by net.cc.swin.edu.au (8.9.3p2/8.9.3) with ESMTP id KAA24375; Wed, 7 May 2003 10:54:50 +1000 (AEST) Message-ID: <3EB857F3.50001@fokus.fraunhofer.de> Date: Wed, 07 May 2003 02:48:51 +0200 From: Sebastian Zander Organization: Fraunhofer FOKUS User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.2) Gecko/20030208 Netscape/7.02 X-Accept-Language: en-us, en MIME-Version: 1.0 To: calato@riverstonenet.com CC: jeff.meyer2@hp.com, maf@eng.oar.net, ipfix@net.doit.wisc.edu Subject: Re: [ipfix] drawing the line between protocol and data model document References: <1D3D2C371FCBD947A7897FABBD3533A566BAA2@xsun01.ptp.hp.com> <3EB7D865.292A6190@riverstonenet.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit calato@riverstonenet.com wrote: > OK. I'm confused. Let me run through an example and people can correct > me where I'm wrong. > > Lets take "Source IP v4 address". Lets also hold off on variable > length fields for now. > > All options need some kind of type value to indicate what the value > semantically means. Lets pick 57 == Source IP v4 address. So what we > need next is info on the value part so we can decode it and it should > be backward compatible (i.e. older versions can at least skip unknown > elements. > > Option I: Element and Length > > 57, 4 > > Any software understanding element 57 knows it is the Source > IP address and can display/handle it correctly. Any one else would No it can't necessarily handle it correctly because you have not specified how it is encoded. > skip over the value in the flow set. > > > Option II: Element and data type > > Lets say data type 22 == IP V4 Address > > 57, 22 > > Any software understanding element 57 knows it is the Source > IP address. But what if it doesn't know what data type 22 is? > How can I skip over it? I don't thing this option is a good idea > Option III: Element and data type and length > > 57, 22, 4 > > Any software which understands 57 and 22 can handle it correclty. > If it only understands 22, it can ONLY display the value in > some rational way, nothing else can be done. If I don't understand 22 > I can at least skip over it by using the length. In my opinion the ability to display a value correctly even if i do not know the semantics is quite nice. I can learn the semantics later without having to store the attribute in its binary form. I even might be able to learn the semantics just by looking at the value. What's the disadvantage? templates become bigger but ideally they are not exchanged too often anyway... Also everyone who knows 22 knows exactly knows how to encode 57. In my opinion the later is mandatory for the protocol. So even if the type is not send to the collector to save bandwidth we must define the encoding. Instead of defining the encoding for each attribute/field again and again i propose to define some basic types and then map the attributes to that basic types. Having a specification like the following will only cause problems because some people might not know how to encode ip adresses or might mess up host and network byte order etc. > IP_SRC_ADDR 8 N Source IP Address > IPv4 have N=4 > IPv6 have N=16 Just my 0.2 cents, Sebastian > Correct away! > > Paul > > "MEYER,JEFFREY D (HP-Cupertino,ex1)" wrote: > >>Sebastian, >> >> Thanks for speaking up! I agree with your restatement of the >>relationship between typing and semantics. >> >> Regarding types like UTF8 or OctetString, as I read the proposal >>from Benoit, the "Length Encoding" model would use a special value >>of 0xFFFF instead of length in the template. This implies that >>the encoding of the "variable length" field would be of the form >>(length) (value), versus fixed length types which can be encoded by >>just placing their (value) in the payload. >> >> (see http://ipfix.doit.wisc.edu/archive/1602.html) >> >> This same model would apply in a type based scheme, if the type >>were string or hexBinary (indicated by the appropriate integer >>identifier in the template), then the encoded value would still be >>of the form (length) (value) in the payload. >> >> Regardless of what template scheme is chosen, however, I think it >>is important that the protocol spec speak in terms of how types are >>encoded, not how each individual element declared in the information model >>is encoded. This makes the inevitable extension of the information >>model easier. >> >>Regards, >> >> Jeff Meyer >> >> >>>-----Original Message----- >>>From: Sebastian Zander [mailto:zander@fokus.fraunhofer.de] >>>Sent: Thursday, May 01, 2003 5:57 PM >>>To: MEYER,JEFFREY D (HP-Cupertino,ex1) >>>Cc: 'Mark Fullmer'; ipfix@net.doit.wisc.edu >>>Subject: Re: [ipfix] drawing the line between protocol and data model >>>docu ment >>> >>> >>>Jeff, >>> >>>i support the idea of typing. I think there is some >>>misunderstanding here >>>by some people who confuse types with semantics. A type does >>>not define the >>>semantics. In my opinion typing has the advantage that even >>>if i don't know >>>the semantics by the time of the decoding i still can >>>correctly decode the >>>value and present it. It also has the advantage that for a >>>well known type >>>the encoding is well defined. If that is not the case people >>>may encode >>>attributes of the same type differently just making >>>interoperability harder. >>>I would define these basic data types in the protocol document. >>> >>>I am not so sure if both alternatives take the same space >>>because there >>>are some types which which do not have an intrinsic length >>>definition. An >>>utf or octet string could have an arbitrary length. Without >>>typing everything >>>would be basically an octetstring of some length ;-). However >>>i though the >>>protocol is supossed to use templates and so it would only affect the >>>template exchange wouldn't it? >>> >>>Cheers, >>> >>>Sebastian >>> >>>MEYER,JEFFREY D (HP-Cupertino,ex1) wrote: >>> >>>>vs. >>>> >>>> = 16909060 (integer) >>>> >>>>vs. >>>> >>>> = whatever the heck 0x01020304 represents >>>> in IEEE Floating Point notation. (something around 4.06) >>>> >>>> >>>>So, yes that's all it buys you. Since both alternatives take >>>>the same amount of space, I'd prefer the one with more >>>>intrinsic information. But, I guess I'm in the minority :-( >>>> >>>> >>>>Regards, >>>> >>>> Jeff Meyer >>>> >>>> >>>> >>>> >>>>>-----Original Message----- >>>>>From: Mark Fullmer [mailto:maf@eng.oar.net] >>>>>Sent: Thursday, May 01, 2003 8:39 AM >>>>>To: MEYER,JEFFREY D (HP-Cupertino,ex1) >>>>>Cc: ipfix@net.doit.wisc.edu >>>>>Subject: Re: [ipfix] drawing the line between protocol and >>>> >>>data model >>> >>>>>docu ment >>>>> >>>>> >>>>>So the difference boils down to a tool being able to display >>>>> >>>>> = 0x01020304 >>>>> vs >>>>> = 1.2.3.4 >>>>> >>>>>In this situation someone will either grab a new version of the >>>>>software or an updated data definition to resolve the unknown >>>>>attribute, >>>>>which would also define it's type. >>>>> >>>>>mark >>>>> >>>>>On Tue, Apr 29, 2003 at 06:25:11PM -0400, MEYER,JEFFREY D >>>>>(HP-Cupertino,ex1) wrote: >>>>> >>>>> >>>>>>Mark, >>>>>> >>>>>> I'll try to clarify: >>>>>> >>>>>> Concept #1: The information model and the encoding are separate. >>>>>> A well defined mapping determines how an information model is >>>>>> to be encoded for a specific use (e.g. IPFIX). The encoding >>>>>> may take whatever approach allows both ends to do something >>>>>> useful with the data. As you point out there is some useful >>>>>> stuff which can be done with the length alone. >>>>>> >>>>>> Concept #2: Derived typing. By defining a base set of types, >>>>>> which must be understood, derived types can be defined in terms >>>>>> of these base types. For instance Diameter does this. This >>>>>> enables a core set of types to be defined and coded to, but >>>>>> also enables richer types later on with an appropriate fallback >>>>>> for systems which aren't aware of the extensions. >>>>>> >>>>>> The benefit I see of explicit typing is that instead of seeing >>>>>>that a field is of length "4 bytes" something which will be seen >>>>>>A LOT. Is that we can instead differentiate between >>>>> >>>>>IPAddr, Integer, >>>>> >>>>> >>>>>>unsigned integer, and float. This seems handy to me. >>>>>> >>>>>> Your argument seems centered around the long term potential of >>>>>>adding lots of fixed length fields whose base length is not 4,8 >>>>>>or 16 bytes long, for instance MACAddr (although if we agree on >>>>>>going the type route, I'd recommend adding MACAddr and then you >>>>>>have 1,2,4,6,8 and 16 bytes covered. I guess I don't foresee >>>>>>a lot more fixed length fields coming down the pipe in the IPFIX >>>>>>domain. Hence my desire to make it more explicit for the types >>>>>>which do occur all the time and are likely points of extension. >>>>>>WHY? Because then a tool could at least present these in a >>>>> >>>>>meaningful >>>>> >>>>> >>>>>>way. As opposed to incorrectly formatting a float type as an int. >>>>>> >>>>>> But hey, either way will work. If the goal is to limit the >>>>>>amount of changes, I'll (grudgingly) go along. I haven't heard >>>>>>any opinions from anyone but you in either direction. So, >>>>>>at this point I don't see a quorom either way... >>>>>> >>>>>>Regards, >>>>>> >>>>>> Jeff Meyer >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>>-----Original Message----- >>>>>>>From: Mark Fullmer [mailto:maf@eng.oar.net] >>>>>>>Sent: Tuesday, April 29, 2003 2:53 PM >>>>>>>To: MEYER,JEFFREY D (HP-Cupertino,ex1) >>>>>>>Cc: ipfix@net.doit.wisc.edu >>>>>>>Subject: Re: [ipfix] drawing the line between protocol >>>>>> >>>>>and data model >>>>> >>>>> >>>>>>>docu ment >>>>>>> >>>>>>> >>>>>>>IMHO crippling the protocol to serve the needs of a >>>>>> >>>>>documentation tool >>>>> >>>>> >>>>>>>is just silly. >>>>>>> >>>>>>>It would be possible to use both a Type name and a Length >>>>>> >>>>>instead of >>>>> >>>>> >>>>>>>the existing Length, which would allow a collector to process >>>>>>>unknown Types. >>>>>>> >>>>>>>With the existing v9 protocol a collector can ignore an >>>>>>>unknown attribute, >>>>>>>or possibly just pass it on to post processing tools which >>>>>>>might understand >>>>>>>it. >>>>>>> >>>>>>>Your suggested changes no longer allow a collector to >>>>>> >>>>>ignore unknown >>>>> >>>>> >>>>>>>attributes if the Type is also unknown. Resorting to encoding all >>>>>>>new Types after the initial protocol spec as octet >>>>>> >>>>>strings is (IMHO) >>>>> >>>>> >>>>>>>not acceptable. >>>>>>> >>>>>>>I'm still trying to understand what the benefits to explicit >>>>>>>Types sent >>>>>>>in templates are. >>>>>>> >>>>>>>mark >>>>>>> >>>>>>>On Tue, Apr 29, 2003 at 02:50:02PM -0400, MEYER,JEFFREY D >>>>>>>(HP-Cupertino,ex1) wrote: >>>>>>> >>>>>>> >>>>>>>>Juergen, >>>>>>>> >>>>>>>> I agree that the data model should have a well defined type >>>>>>>>system. And that that typing system should reflect the >>>>>>> >>>>>>>requirements. >>>>>>> >>>>>>> >>>>>>>> This is in part why I was hoping that the protocol >>>>>>> >>>>>itself would >>>>> >>>>> >>>>>>>>be more explicit here. (See my mail >>>>>>>>http://ipfix.doit.wisc.edu/archive/1604.html) >>>>>>>> >>>>>>>> There is also a type list in that mail. However, I >>>>>>> >>>>>would suggest >>>>> >>>>> >>>>>>>>using the names from XML-Schema part 2 Data types, as this would >>>>>>>>facilitate creating a formal/normative definition, versus >>>>>>> >>>>>>>reinventing >>>>>>> >>>>>>> >>>>>>>>names for the types. >>>>>>>> >>>>>>>> Here are the type names from XML-Schema/IPDR: >>>>>>>> >>>>>>>> boolean - true/false (may be encoded in byte) >>>>>>>> byte >>>>>>>> unsignedByte >>>>>>>> short >>>>>>>> unsignedShort >>>>>>>> int - 32-bit signed >>>>>>>> long - 64-bit signed >>>>>>>> unsignedInt >>>>>>>> unsignedLong >>>>>>>> float >>>>>>>> double >>>>>>>> dateTime - 32-bit integer representing seconds >>>>>>> >>>>>since EPOCH >>>>> >>>>> >>>>>>>>(1/1/1970 0:00 GMT) >>>>>>>> string - UTF-8 formatted text >>>>>>>> ipdr:dateTimeMsec - 64-bit integer representing msecs >>>>>>> >>>>>>>since EPOCH >>>>>>> >>>>>>> >>>>>>>> ipdr:ipV4Addr >>>>>>>> ipdr:ipV6Addr >>>>>>>> ipdr:UUID >>>>>>>> hexBinary - arbitrary sequence of octets >>>>>>>> >>>>>>>> If the protocol itself has a basic "octetString" (or from >>>>>>> >>>>>>>XML, hexBinary) >>>>>>> >>>>>>> >>>>>>>>payload, >>>>>>>>then this can be used to carry new classes of information >>>>>>> >>>>>>>in the future w/o >>>>>>> >>>>>>> >>>>>>>>rejiggering the protocol itself. >>>>>>>> >>>>>>>> In addition other "derived" types may be created from the >>>>>>> >>>>>>>base types. For >>>>>>> >>>>>>> >>>>>>>>instance in IPDR, the ipdr:dateTimeMsec is derived from long. >>>>>>>> >>>>>>>>Regards, >>>>>>>> >>>>>>>> Jeff Meyer >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>>-----Original Message----- >>>>>>>>>From: Juergen Quittek [mailto:quittek@ccrle.nec.de] >>>>>>>>>Sent: Monday, April 28, 2003 2:06 PM >>>>>>>>>To: ipfix@net.doit.wisc.edu >>>>>>>>>Subject: [ipfix] drawing the line between protocol >>>>>>>> >>>>>and data model >>>>> >>>>> >>>>>>>>>document >>>>>>>>> >>>>>>>>> >>>>>>>>>Hi all, >>>>>>>>> >>>>>>>>>When shaping the planned documents, we have to draw a line >>>>>>>>>between the protocol and the data model. >>>>>>>>> >>>>>>>>>Definitely, the data model document defines all field types, >>>>>>>>>but does it also define basic data types, such as '16 bit >>>>>>>>>unsigned integer'? >>>>>>>>> >>>>>>>>>Maybe we should have a set of basic data types defined in >>>>>>>>>the protocol document and restrict the data model doc to >>>>>>>> >>>>>>>these types? >>>>>>> >>>>>>> >>>>>>>>>This would further structure the data model. >>>>>>>>> >>>>>>>>>Any thoughts? >>>>>>>>> >>>>>>>>> Juergen >>>>>>>>> >>>>>>>>>-- >>>>>>>>>Help mailto:majordomo@net.doit.wisc.edu and say "help" >>>>>>>>>in message body >>>>>>>>>Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >>>>>>>>>"unsubscribe ipfix" in message body >>>>>>>>>Archive http://ipfix.doit.wisc.edu/archive/ >>>>>>>>> >>>>>>>> >>>>>>>>-- >>>>>>>>Help mailto:majordomo@net.doit.wisc.edu and say >>>>>>> >>>>>>>"help" in message body >>>>>>> >>>>>>> >>>>>>>>Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >>>>>>>>"unsubscribe ipfix" in message body >>>>>>>>Archive http://ipfix.doit.wisc.edu/archive/ >>>>>>> >>>>>>-- >>>>>>Help mailto:majordomo@net.doit.wisc.edu and say >>>>> >>>>>"help" in message body >>>>> >>>>> >>>>>>Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >>>>>>"unsubscribe ipfix" in message body >>>>>>Archive http://ipfix.doit.wisc.edu/archive/ >>>>> >>>>-- >>>>Help mailto:majordomo@net.doit.wisc.edu and say >>> >>>"help" in message body >>> >>>>Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >>>>"unsubscribe ipfix" in message body >>>>Archive http://ipfix.doit.wisc.edu/archive/ >>>> >>> >>> >>>-- >>>Sebastian Zander E-mail: >>>zander@fokus.fraunhofer.de >>>Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 >>>Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 >>>D-10589 Berlin, Germany >>>www.fokus.fraunhofer.de/usr/sebastian.zander >>> >>> >>> >>> >> >>-- >>Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body >>Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >>"unsubscribe ipfix" in message body >>Archive http://ipfix.doit.wisc.edu/archive/ > > -- Sebastian Zander E-mail: zander@fokus.fraunhofer.de Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 D-10589 Berlin, Germany www.fokus.fraunhofer.de/usr/sebastian.zander -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue May 6 22:13:17 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA17677 for ; Tue, 6 May 2003 22:13:16 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19DEFG-0004vO-00 for ipfix-list@mil.doit.wisc.edu; Tue, 06 May 2003 21:01:22 -0500 Received: from c3p0.cc.swin.edu.au ([136.186.1.10] helo=net.cc.swin.edu.au) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19DEFE-0004vF-00 for ipfix@net.doit.wisc.edu; Tue, 06 May 2003 21:01:20 -0500 Received: from fokus.fraunhofer.de (szander-laptop.caia.swin.edu.au [136.186.229.90]) by net.cc.swin.edu.au (8.9.3p2/8.9.3) with ESMTP id MAA02409 for ; Wed, 7 May 2003 12:01:17 +1000 (AEST) Message-ID: <3EB86787.1040800@fokus.fraunhofer.de> Date: Wed, 07 May 2003 03:55:19 +0200 From: Sebastian Zander Organization: Fraunhofer FOKUS User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.2) Gecko/20030208 Netscape/7.02 X-Accept-Language: en-us, en MIME-Version: 1.0 To: ipfix@net.doit.wisc.edu Subject: [ipfix] extensibility of the ipfix protocol Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hi, although extensibility is mandatory by the requirements there hasn't been a lot discussion on how to do it and the chosen candidate protocol doesn't say much about extensibility. So here are my thoughts about extensibility. The protocol specification must define how the protocol can be extended. I see the following (but there may be more) ways of extending the protocol: 1. Extending the fixed packet header This would allow to add new header fields without changing the protocol specification. A bit (X) could be stolen from version to indicate the presence of extension header(s). Extension header(s) follow the fixed header if X=1. Extension header(s) consists of type, length and value. 2. Extending the attribute/field space This is certainly needed. It could be done at least in two ways: a) A flat number space where numbers are specified in extension RFCs and registered by IANA. A nice thing would a dynamic space where numbers are not defined by IANA but could be negotiated out of band (see payload type in the RTP specification). Such a space could also be used as safe playground. b) A number space which is defined in extension RFCs (IANA) and the possibility of using vendor specific attributes by having an optional vendor ID field in the template specification. The vendor ID would be IANA assigned but the vendor then could use its private field ID space (see the Diameter Base protocol specification). Adds more complexity and overhead to the protocol but could be very useful when the protocol becomes widely adopted and a lot of extensions are done. 3. Extending the type space (if there are types at all) Probably there won't be much need to extent this because the protocol specification could do a good job of specifying all the basic types. If this really needs to be extended it would be probably done with a revised protocol specification. Comments? Opinions? Cheers, Sebastian -- Sebastian Zander E-mail: zander@fokus.fraunhofer.de Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 D-10589 Berlin, Germany www.fokus.fraunhofer.de/usr/sebastian.zander -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue May 6 23:53:57 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA19559 for ; Tue, 6 May 2003 23:53:57 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19DFsz-00071I-00 for ipfix-list@mil.doit.wisc.edu; Tue, 06 May 2003 22:46:29 -0500 Received: from eng4.oar.net ([192.148.244.24]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 19DFsx-00071C-00 for ipfix@net.doit.wisc.edu; Tue, 06 May 2003 22:46:27 -0500 Received: (qmail 60050 invoked by uid 4454); 7 May 2003 03:46:27 -0000 Date: Tue, 6 May 2003 23:46:27 -0400 From: Mark Fullmer To: Sebastian Zander Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] extensibility of the ipfix protocol Message-ID: <20030506234627.A60008@net.ohio-state.edu> References: <3EB86787.1040800@fokus.fraunhofer.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: <3EB86787.1040800@fokus.fraunhofer.de>; from zander@fokus.fraunhofer.de on Wed, May 07, 2003 at 03:55:19AM +0200 Precedence: bulk Sender: majordomo listserver One way to look at NetFlow v9 is a fixed header + TLV's. Type Definition ----------------------------- 0 Flowset 1 Options flowset 2..255 undefined 256 .. 65535 data templates defined by 0 or 1 This should address (1), although the protocol document needs verbage. You could also just use the existing options template mechanism. I would like to see the number space used a little differently so a bit (high bit, low bit, whatever) can be set that informs the collector what to do if it can't decode the new option -- ignore it and move on or give up. (2) I don't see where the flat space of 65534 attributes is going to be a problem. This would need to be partitioned into a well known, reserved for future use, and reserved for vendor playground. NetFlow prior to v9 had 31 unique attributes. (3) Still up for debate if this shows up in the protocol. mark On Wed, May 07, 2003 at 03:55:19AM +0200, Sebastian Zander wrote: > Hi, > > although extensibility is mandatory by the requirements there hasn't > been a lot discussion on how to do it and the chosen candidate protocol > doesn't say much about extensibility. So here are my thoughts about > extensibility. > > The protocol specification must define how the protocol can be extended. > > I see the following (but there may be more) ways of extending the > protocol: > > 1. Extending the fixed packet header > This would allow to add new header fields without changing the > protocol specification. > > A bit (X) could be stolen from version to indicate the presence of > extension header(s). Extension header(s) follow the fixed header if > X=1. Extension header(s) consists of type, length and value. > > 2. Extending the attribute/field space > This is certainly needed. It could be done at least in two ways: > a) A flat number space where numbers are specified in extension > RFCs and registered by IANA. A nice thing would a dynamic > space where numbers are not defined by IANA but could be negotiated > out of band (see payload type in the RTP specification). Such a > space could also be used as safe playground. > b) A number space which is defined in extension RFCs (IANA) and the > possibility of using vendor specific attributes by having an optional > vendor ID field in the template specification. The vendor ID would > be IANA assigned but the vendor then could use its private field ID > space (see the Diameter Base protocol specification). Adds more > complexity and overhead to the protocol but could be very useful > when the protocol becomes widely adopted and a lot of extensions > are done. > > 3. Extending the type space (if there are types at all) > Probably there won't be much need to extent this because the protocol > specification could do a good job of specifying all the basic types. > If this really needs to be extended it would be probably done with a > revised protocol specification. > > Comments? Opinions? > > Cheers, > > Sebastian > > -- > Sebastian Zander E-mail: zander@fokus.fraunhofer.de > Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 > Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 > D-10589 Berlin, Germany www.fokus.fraunhofer.de/usr/sebastian.zander > > > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed May 7 00:17:33 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA19930 for ; Wed, 7 May 2003 00:17:32 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19DG6E-0007LS-00 for ipfix-list@mil.doit.wisc.edu; Tue, 06 May 2003 23:00:10 -0500 Received: from eng4.oar.net ([192.148.244.24]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 19DG6D-0007LE-00 for ipfix@net.doit.wisc.edu; Tue, 06 May 2003 23:00:09 -0500 Received: (qmail 60116 invoked by uid 4454); 7 May 2003 04:00:08 -0000 Date: Wed, 7 May 2003 00:00:08 -0400 From: Mark Fullmer To: calato@riverstonenet.com Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] drawing the line between protocol and data model document Message-ID: <20030507000008.B60008@net.ohio-state.edu> References: <1D3D2C371FCBD947A7897FABBD3533A566BAA2@xsun01.ptp.hp.com> <3EB7D865.292A6190@riverstonenet.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: <3EB7D865.292A6190@riverstonenet.com>; from calato@riverstonenet.com on Tue, May 06, 2003 at 11:44:37AM -0400 Precedence: bulk Sender: majordomo listserver This is a good summary. After thinking about this for a few days one other alternative would be to allow the exporter to upload a Attribute -> Type mapping using the options template mechanism. Either way, now you have the ability to display (for an unknown Attribute) <99 UNKNOWN> = 1.2.3.4 When to be really useful you want = 1.2.3.4 And this is the type of information that could be put in a MIB, along with similar tables such as ifName, ifDescr, ifSpeed which can be just as useful when post processing flows. I'm going to stand by my opinion that this doesn't belong in the IPFIX protocol because it's not something that's needed to glue the exporter and collector together. mark On Tue, May 06, 2003 at 11:44:37AM -0400, calato@riverstonenet.com wrote: > > OK. I'm confused. Let me run through an example and people can correct > me where I'm wrong. > > Lets take "Source IP v4 address". Lets also hold off on variable > length fields for now. > > All options need some kind of type value to indicate what the value > semantically means. Lets pick 57 == Source IP v4 address. So what we > need next is info on the value part so we can decode it and it should > be backward compatible (i.e. older versions can at least skip unknown > elements. > > Option I: Element and Length > > 57, 4 > > Any software understanding element 57 knows it is the Source > IP address and can display/handle it correctly. Any one else would > skip over the value in the flow set. > > > Option II: Element and data type > > Lets say data type 22 == IP V4 Address > > 57, 22 > > Any software understanding element 57 knows it is the Source > IP address. But what if it doesn't know what data type 22 is? > How can I skip over it? > > Option III: Element and data type and length > > 57, 22, 4 > > Any software which understands 57 and 22 can handle it correclty. > If it only understands 22, it can ONLY display the value in > some rational way, nothing else can be done. If I don't understand 22 > I can at least skip over it by using the length. > > Correct away! > > Paul > > "MEYER,JEFFREY D (HP-Cupertino,ex1)" wrote: > > > > Sebastian, > > > > Thanks for speaking up! I agree with your restatement of the > > relationship between typing and semantics. > > > > Regarding types like UTF8 or OctetString, as I read the proposal > > from Benoit, the "Length Encoding" model would use a special value > > of 0xFFFF instead of length in the template. This implies that > > the encoding of the "variable length" field would be of the form > > (length) (value), versus fixed length types which can be encoded by > > just placing their (value) in the payload. > > > > (see http://ipfix.doit.wisc.edu/archive/1602.html) > > > > This same model would apply in a type based scheme, if the type > > were string or hexBinary (indicated by the appropriate integer > > identifier in the template), then the encoded value would still be > > of the form (length) (value) in the payload. > > > > Regardless of what template scheme is chosen, however, I think it > > is important that the protocol spec speak in terms of how types are > > encoded, not how each individual element declared in the information model > > is encoded. This makes the inevitable extension of the information > > model easier. > > > > Regards, > > > > Jeff Meyer > > > > > -----Original Message----- > > > From: Sebastian Zander [mailto:zander@fokus.fraunhofer.de] > > > Sent: Thursday, May 01, 2003 5:57 PM > > > To: MEYER,JEFFREY D (HP-Cupertino,ex1) > > > Cc: 'Mark Fullmer'; ipfix@net.doit.wisc.edu > > > Subject: Re: [ipfix] drawing the line between protocol and data model > > > docu ment > > > > > > > > > Jeff, > > > > > > i support the idea of typing. I think there is some > > > misunderstanding here > > > by some people who confuse types with semantics. A type does > > > not define the > > > semantics. In my opinion typing has the advantage that even > > > if i don't know > > > the semantics by the time of the decoding i still can > > > correctly decode the > > > value and present it. It also has the advantage that for a > > > well known type > > > the encoding is well defined. If that is not the case people > > > may encode > > > attributes of the same type differently just making > > > interoperability harder. > > > I would define these basic data types in the protocol document. > > > > > > I am not so sure if both alternatives take the same space > > > because there > > > are some types which which do not have an intrinsic length > > > definition. An > > > utf or octet string could have an arbitrary length. Without > > > typing everything > > > would be basically an octetstring of some length ;-). However > > > i though the > > > protocol is supossed to use templates and so it would only affect the > > > template exchange wouldn't it? > > > > > > Cheers, > > > > > > Sebastian > > > > > > MEYER,JEFFREY D (HP-Cupertino,ex1) wrote: > > > > vs. > > > > > > > > = 16909060 (integer) > > > > > > > > vs. > > > > > > > > = whatever the heck 0x01020304 represents > > > > in IEEE Floating Point notation. (something around 4.06) > > > > > > > > > > > > So, yes that's all it buys you. Since both alternatives take > > > > the same amount of space, I'd prefer the one with more > > > > intrinsic information. But, I guess I'm in the minority :-( > > > > > > > > > > > > Regards, > > > > > > > > Jeff Meyer > > > > > > > > > > > > > > > >>-----Original Message----- > > > >>From: Mark Fullmer [mailto:maf@eng.oar.net] > > > >>Sent: Thursday, May 01, 2003 8:39 AM > > > >>To: MEYER,JEFFREY D (HP-Cupertino,ex1) > > > >>Cc: ipfix@net.doit.wisc.edu > > > >>Subject: Re: [ipfix] drawing the line between protocol and > > > data model > > > >>docu ment > > > >> > > > >> > > > >>So the difference boils down to a tool being able to display > > > >> > > > >> = 0x01020304 > > > >> vs > > > >> = 1.2.3.4 > > > >> > > > >>In this situation someone will either grab a new version of the > > > >>software or an updated data definition to resolve the unknown > > > >>attribute, > > > >>which would also define it's type. > > > >> > > > >>mark > > > >> > > > >>On Tue, Apr 29, 2003 at 06:25:11PM -0400, MEYER,JEFFREY D > > > >>(HP-Cupertino,ex1) wrote: > > > >> > > > >>>Mark, > > > >>> > > > >>> I'll try to clarify: > > > >>> > > > >>> Concept #1: The information model and the encoding are separate. > > > >>> A well defined mapping determines how an information model is > > > >>> to be encoded for a specific use (e.g. IPFIX). The encoding > > > >>> may take whatever approach allows both ends to do something > > > >>> useful with the data. As you point out there is some useful > > > >>> stuff which can be done with the length alone. > > > >>> > > > >>> Concept #2: Derived typing. By defining a base set of types, > > > >>> which must be understood, derived types can be defined in terms > > > >>> of these base types. For instance Diameter does this. This > > > >>> enables a core set of types to be defined and coded to, but > > > >>> also enables richer types later on with an appropriate fallback > > > >>> for systems which aren't aware of the extensions. > > > >>> > > > >>> The benefit I see of explicit typing is that instead of seeing > > > >>>that a field is of length "4 bytes" something which will be seen > > > >>>A LOT. Is that we can instead differentiate between > > > >> > > > >>IPAddr, Integer, > > > >> > > > >>>unsigned integer, and float. This seems handy to me. > > > >>> > > > >>> Your argument seems centered around the long term potential of > > > >>>adding lots of fixed length fields whose base length is not 4,8 > > > >>>or 16 bytes long, for instance MACAddr (although if we agree on > > > >>>going the type route, I'd recommend adding MACAddr and then you > > > >>>have 1,2,4,6,8 and 16 bytes covered. I guess I don't foresee > > > >>>a lot more fixed length fields coming down the pipe in the IPFIX > > > >>>domain. Hence my desire to make it more explicit for the types > > > >>>which do occur all the time and are likely points of extension. > > > >>>WHY? Because then a tool could at least present these in a > > > >> > > > >>meaningful > > > >> > > > >>>way. As opposed to incorrectly formatting a float type as an int. > > > >>> > > > >>> But hey, either way will work. If the goal is to limit the > > > >>>amount of changes, I'll (grudgingly) go along. I haven't heard > > > >>>any opinions from anyone but you in either direction. So, > > > >>>at this point I don't see a quorom either way... > > > >>> > > > >>>Regards, > > > >>> > > > >>> Jeff Meyer > > > >>> > > > >>> > > > >>> > > > >>>>-----Original Message----- > > > >>>>From: Mark Fullmer [mailto:maf@eng.oar.net] > > > >>>>Sent: Tuesday, April 29, 2003 2:53 PM > > > >>>>To: MEYER,JEFFREY D (HP-Cupertino,ex1) > > > >>>>Cc: ipfix@net.doit.wisc.edu > > > >>>>Subject: Re: [ipfix] drawing the line between protocol > > > >>> > > > >>and data model > > > >> > > > >>>>docu ment > > > >>>> > > > >>>> > > > >>>>IMHO crippling the protocol to serve the needs of a > > > >>> > > > >>documentation tool > > > >> > > > >>>>is just silly. > > > >>>> > > > >>>>It would be possible to use both a Type name and a Length > > > >>> > > > >>instead of > > > >> > > > >>>>the existing Length, which would allow a collector to process > > > >>>>unknown Types. > > > >>>> > > > >>>>With the existing v9 protocol a collector can ignore an > > > >>>>unknown attribute, > > > >>>>or possibly just pass it on to post processing tools which > > > >>>>might understand > > > >>>>it. > > > >>>> > > > >>>>Your suggested changes no longer allow a collector to > > > >>> > > > >>ignore unknown > > > >> > > > >>>>attributes if the Type is also unknown. Resorting to encoding all > > > >>>>new Types after the initial protocol spec as octet > > > >>> > > > >>strings is (IMHO) > > > >> > > > >>>>not acceptable. > > > >>>> > > > >>>>I'm still trying to understand what the benefits to explicit > > > >>>>Types sent > > > >>>>in templates are. > > > >>>> > > > >>>>mark > > > >>>> > > > >>>>On Tue, Apr 29, 2003 at 02:50:02PM -0400, MEYER,JEFFREY D > > > >>>>(HP-Cupertino,ex1) wrote: > > > >>>> > > > >>>>>Juergen, > > > >>>>> > > > >>>>> I agree that the data model should have a well defined type > > > >>>>>system. And that that typing system should reflect the > > > >>>> > > > >>>>requirements. > > > >>>> > > > >>>>> This is in part why I was hoping that the protocol > > > >>>> > > > >>itself would > > > >> > > > >>>>>be more explicit here. (See my mail > > > >>>>>http://ipfix.doit.wisc.edu/archive/1604.html) > > > >>>>> > > > >>>>> There is also a type list in that mail. However, I > > > >>>> > > > >>would suggest > > > >> > > > >>>>>using the names from XML-Schema part 2 Data types, as this would > > > >>>>>facilitate creating a formal/normative definition, versus > > > >>>> > > > >>>>reinventing > > > >>>> > > > >>>>>names for the types. > > > >>>>> > > > >>>>> Here are the type names from XML-Schema/IPDR: > > > >>>>> > > > >>>>> boolean - true/false (may be encoded in byte) > > > >>>>> byte > > > >>>>> unsignedByte > > > >>>>> short > > > >>>>> unsignedShort > > > >>>>> int - 32-bit signed > > > >>>>> long - 64-bit signed > > > >>>>> unsignedInt > > > >>>>> unsignedLong > > > >>>>> float > > > >>>>> double > > > >>>>> dateTime - 32-bit integer representing seconds > > > >>>> > > > >>since EPOCH > > > >> > > > >>>>>(1/1/1970 0:00 GMT) > > > >>>>> string - UTF-8 formatted text > > > >>>>> ipdr:dateTimeMsec - 64-bit integer representing msecs > > > >>>> > > > >>>>since EPOCH > > > >>>> > > > >>>>> ipdr:ipV4Addr > > > >>>>> ipdr:ipV6Addr > > > >>>>> ipdr:UUID > > > >>>>> hexBinary - arbitrary sequence of octets > > > >>>>> > > > >>>>> If the protocol itself has a basic "octetString" (or from > > > >>>> > > > >>>>XML, hexBinary) > > > >>>> > > > >>>>>payload, > > > >>>>>then this can be used to carry new classes of information > > > >>>> > > > >>>>in the future w/o > > > >>>> > > > >>>>>rejiggering the protocol itself. > > > >>>>> > > > >>>>> In addition other "derived" types may be created from the > > > >>>> > > > >>>>base types. For > > > >>>> > > > >>>>>instance in IPDR, the ipdr:dateTimeMsec is derived from long. > > > >>>>> > > > >>>>>Regards, > > > >>>>> > > > >>>>> Jeff Meyer > > > >>>>> > > > >>>>> > > > >>>>>>-----Original Message----- > > > >>>>>>From: Juergen Quittek [mailto:quittek@ccrle.nec.de] > > > >>>>>>Sent: Monday, April 28, 2003 2:06 PM > > > >>>>>>To: ipfix@net.doit.wisc.edu > > > >>>>>>Subject: [ipfix] drawing the line between protocol > > > >>>>> > > > >>and data model > > > >> > > > >>>>>>document > > > >>>>>> > > > >>>>>> > > > >>>>>>Hi all, > > > >>>>>> > > > >>>>>>When shaping the planned documents, we have to draw a line > > > >>>>>>between the protocol and the data model. > > > >>>>>> > > > >>>>>>Definitely, the data model document defines all field types, > > > >>>>>>but does it also define basic data types, such as '16 bit > > > >>>>>>unsigned integer'? > > > >>>>>> > > > >>>>>>Maybe we should have a set of basic data types defined in > > > >>>>>>the protocol document and restrict the data model doc to > > > >>>>> > > > >>>>these types? > > > >>>> > > > >>>>>>This would further structure the data model. > > > >>>>>> > > > >>>>>>Any thoughts? > > > >>>>>> > > > >>>>>> Juergen > > > >>>>>> > > > >>>>>>-- > > > >>>>>>Help mailto:majordomo@net.doit.wisc.edu and say "help" > > > >>>>>>in message body > > > >>>>>>Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > >>>>>>"unsubscribe ipfix" in message body > > > >>>>>>Archive http://ipfix.doit.wisc.edu/archive/ > > > >>>>>> > > > >>>>> > > > >>>>>-- > > > >>>>>Help mailto:majordomo@net.doit.wisc.edu and say > > > >>>> > > > >>>>"help" in message body > > > >>>> > > > >>>>>Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > >>>>>"unsubscribe ipfix" in message body > > > >>>>>Archive http://ipfix.doit.wisc.edu/archive/ > > > >>>> > > > >>>-- > > > >>>Help mailto:majordomo@net.doit.wisc.edu and say > > > >> > > > >>"help" in message body > > > >> > > > >>>Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > >>>"unsubscribe ipfix" in message body > > > >>>Archive http://ipfix.doit.wisc.edu/archive/ > > > >> > > > > > > > > -- > > > > Help mailto:majordomo@net.doit.wisc.edu and say > > > "help" in message body > > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > > "unsubscribe ipfix" in message body > > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > > > > > > -- > > > Sebastian Zander E-mail: > > > zander@fokus.fraunhofer.de > > > Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 > > > Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 > > > D-10589 Berlin, Germany > > > www.fokus.fraunhofer.de/usr/sebastian.zander > > > > > > > > > > > > > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed May 7 00:59:22 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA20792 for ; Wed, 7 May 2003 00:59:21 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19DGuV-0000X2-00 for ipfix-list@mil.doit.wisc.edu; Tue, 06 May 2003 23:52:07 -0500 Received: from h65s138a81n47.user.nortelnetworks.com ([47.81.138.65] helo=zsc3s004.nortelnetworks.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19DGuR-0000Wt-00 for ipfix@net.doit.wisc.edu; Tue, 06 May 2003 23:52:03 -0500 Received: from zsc3c028.us.nortel.com (zsc3c028.us.nortel.com [47.81.138.28]) by zsc3s004.nortelnetworks.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id h474poU23926; Tue, 6 May 2003 21:51:51 -0700 (PDT) Received: from zsc3c026.us.nortel.com ([47.81.138.26]) by zsc3c028.us.nortel.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id 2R1CVZQ9; Tue, 6 May 2003 21:51:21 -0700 Received: from private2xsth6c (artpt5q8.us.nortel.com [47.140.52.126]) by zsc3c026.us.nortel.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id J4HALQHS; Tue, 6 May 2003 21:51:20 -0700 Message-ID: <000901c31454$4e43a410$7e348c2f@private2xsth6c> X-Sybari-Space: 00000000 00000000 00000000 From: "Reinaldo Penno" To: "Mark Fullmer" , "Sebastian Zander" Cc: References: <3EB86787.1040800@fokus.fraunhofer.de> <20030506234627.A60008@net.ohio-state.edu> Subject: Re: [ipfix] extensibility of the ipfix protocol Date: Wed, 7 May 2003 00:51:20 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1106 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit One of the reasons behind extensibility was to make the protocol more reliable (or open to reliability extensions...). I guess we need to explain how having more types will help that. ----- Original Message ----- From: "Mark Fullmer" To: "Sebastian Zander" Cc: Sent: Tuesday, May 06, 2003 11:46 PM Subject: Re: [ipfix] extensibility of the ipfix protocol > One way to look at NetFlow v9 is a fixed header + TLV's. > > Type Definition > ----------------------------- > 0 Flowset > 1 Options flowset > 2..255 undefined > 256 .. 65535 data templates defined by 0 or 1 > > This should address (1), although the protocol document needs verbage. > > You could also just use the existing options template mechanism. > > I would like to see the number space used a little differently so a bit > (high bit, low bit, whatever) can be set that informs the collector what > to do if it can't decode the new option -- ignore it and move on or > give up. > > (2) > I don't see where the flat space of 65534 attributes is going to be > a problem. This would need to be partitioned into a well known, > reserved for future use, and reserved for vendor playground. NetFlow > prior to v9 had 31 unique attributes. > > (3) > Still up for debate if this shows up in the protocol. > > > mark > > On Wed, May 07, 2003 at 03:55:19AM +0200, Sebastian Zander wrote: > > Hi, > > > > although extensibility is mandatory by the requirements there hasn't > > been a lot discussion on how to do it and the chosen candidate protocol > > doesn't say much about extensibility. So here are my thoughts about > > extensibility. > > > > The protocol specification must define how the protocol can be extended. > > > > I see the following (but there may be more) ways of extending the > > protocol: > > > > 1. Extending the fixed packet header > > This would allow to add new header fields without changing the > > protocol specification. > > > > A bit (X) could be stolen from version to indicate the presence of > > extension header(s). Extension header(s) follow the fixed header if > > X=1. Extension header(s) consists of type, length and value. > > > > 2. Extending the attribute/field space > > This is certainly needed. It could be done at least in two ways: > > a) A flat number space where numbers are specified in extension > > RFCs and registered by IANA. A nice thing would a dynamic > > space where numbers are not defined by IANA but could be negotiated > > out of band (see payload type in the RTP specification). Such a > > space could also be used as safe playground. > > b) A number space which is defined in extension RFCs (IANA) and the > > possibility of using vendor specific attributes by having an optional > > vendor ID field in the template specification. The vendor ID would > > be IANA assigned but the vendor then could use its private field ID > > space (see the Diameter Base protocol specification). Adds more > > complexity and overhead to the protocol but could be very useful > > when the protocol becomes widely adopted and a lot of extensions > > are done. > > > > 3. Extending the type space (if there are types at all) > > Probably there won't be much need to extent this because the protocol > > specification could do a good job of specifying all the basic types. > > If this really needs to be extended it would be probably done with a > > revised protocol specification. > > > > Comments? Opinions? > > > > Cheers, > > > > Sebastian > > > > -- > > Sebastian Zander E-mail: zander@fokus.fraunhofer.de > > Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 > > Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 > > D-10589 Berlin, Germany www.fokus.fraunhofer.de/usr/sebastian.zander > > > > > > > > > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed May 7 01:57:32 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id BAA21545 for ; Wed, 7 May 2003 01:57:32 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19DHoJ-0001aa-00 for ipfix-list@mil.doit.wisc.edu; Wed, 07 May 2003 00:49:47 -0500 Received: from [130.216.191.4] (helo=mailhost2.auckland.ac.nz) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19DHoG-0001aR-00 for ipfix@net.doit.wisc.edu; Wed, 07 May 2003 00:49:45 -0500 Received: from mailhost.auckland.ac.nz (IDENT:mirapoint@mailhost [130.216.191.61]) by mailhost2.auckland.ac.nz (8.12.9/8.12.9/8.12.9-ua) with ESMTP id h475nfcT009336; Wed, 7 May 2003 17:49:41 +1200 (NZST) Received: from hotlava.auckland.ac.nz (hotlava.auckland.ac.nz [130.216.191.123]) by mailhost.auckland.ac.nz (Mirapoint Messaging Server MOS 3.3.2-CR) with ESMTP id APQ34174; Wed, 7 May 2003 17:49:40 +1200 (NZST) Received: (from apache@localhost) by hotlava.auckland.ac.nz (8.11.6/8.11.6) id h475ndR28331; Wed, 7 May 2003 17:49:39 +1200 X-Authentication-Warning: hotlava.auckland.ac.nz: apache set sender to n.brownlee@auckland.ac.nz using -f Received: from nebbiolo.itss.auckland.ac.nz (nebbiolo.itss.auckland.ac.nz [130.216.4.167]) by hotlava.auckland.ac.nz (Horde) with HTTP for ; Wed, 7 May 2003 17:49:39 +1200 Message-ID: <1052286579.b18b38645d8fe@hotlava.auckland.ac.nz> Date: Wed, 7 May 2003 17:49:39 +1200 From: Nevil Brownlee To: calato@riverstonenet.com Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] export packet length - LAST CALL References: <3EA95941.57BFEE44@ccrle.nec.de> <3EB7DB16.1886F07C@riverstonenet.com> In-Reply-To: <3EB7DB16.1886F07C@riverstonenet.com> MIME-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Internet Messaging Program (IMP) 4.0-cvs X-Originating-IP: 130.216.4.167 Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hi all: Following up on the latest round of discusisons: 1) Using Randy & Rob's WG issues tracking system sounds fine. 2) Running Last Calls after mailing list discussion is a good way to reach consensus (and thereby close) issues. 3) On 'extensibility' of theIPFIX protocol. My take on this is that - We want an information model which can be extended by adding new attributes, and a protocol which can handle such new attributes, so we need to put some design effort in now to achieve that. - For reliability we agreed that the initial version of IPFIX has to work with only one-way transport, but that we need sufficient hooks to add reliability later on. Both these goals are clearly being worked on, that's great. But we need to be careful to balance our development efforts so as to produce a good, simple protocol now, which can be extended as needed later - without unduly delaying our efforts to get the initial version done fairly quickly. "By the end of this year" to reach near-final drafts sounds about right to me ... Cheers, Nevil ----------------------------------------------------------------------- Nevil Brownlee Director, Technology Development Phone: +64 9 373 7599 x88941 ITSS, The University of Auckland FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand ------------------------------------------------- This mail sent through University of Auckland http://www.auckland.ac.nz/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed May 7 05:12:06 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id FAA21698 for ; Wed, 7 May 2003 05:12:05 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19DKbq-0005vM-00 for ipfix-list@mil.doit.wisc.edu; Wed, 07 May 2003 03:49:06 -0500 Received: from tokyo.ccrle.nec.de ([195.37.70.2]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19DKbn-0005v8-00 for ipfix@net.doit.wisc.edu; Wed, 07 May 2003 03:49:04 -0500 Received: from venus.office (venus.office [10.1.1.11]) by tokyo.ccrle.nec.de (8.12.9/8.12.8) with ESMTP id h478n0VI019288 for ; Wed, 7 May 2003 10:49:00 +0200 (CEST) Received: from ccrle.nec.de (molina.office [10.1.1.126]) by venus.office (Postfix on SuSE Linux eMail Server 3.0) with ESMTP id 61F38BE6D for ; Wed, 7 May 2003 10:41:28 +0200 (CEST) Message-ID: <3EB8C7E9.BC41382C@ccrle.nec.de> Date: Wed, 07 May 2003 10:46:33 +0200 From: Maurizio Molina X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] export packet length - LAST CALL References: <3EA95941.57BFEE44@ccrle.nec.de> <3EB7DB16.1886F07C@riverstonenet.com> <20030506163609.A58211@net.ohio-state.edu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit I also support the encoding of the packet length in the header as a non-extensible, fixed header field. IMO it could be done in one of the 3 following ways, which I list in my order of preference: 1) 2 bytes, additional to the current header 2) 2 bytes, replacing the 2 bytes Count header 3) 4 bytes, additional to the curent header Maurizio Mark Fullmer wrote: > To close this issue I would like to see the length specified > (2 or 4 byte fixed header field). If 2 bytes then is replacing the existing > count field okay. > > And yes, I think closing some of the recently discussed open issues with last > call's is the right way to get things moving. > > mark > > On Tue, May 06, 2003 at 11:56:06AM -0400, calato@riverstonenet.com wrote: > > > > There seems to be a consensus that having a length field > > in the packet header is a good idea. > > > > I would like to bring this issue to last call. Anyone > > opposed please state your case one more time. > > > > If this doesn't seem like the right process to close > > out issues, suggestions are welcome. > > > > Paul > > > > > > Maurizio Molina wrote: > > > > > > Hi all, > > > I hope not to rise an issue already discussed in the WG. If yes, I > > > apologise in advance. > > > If an IPFIX export packet is carried over TCP, it's not unusual that it > > > gets split across multiple TCP segments. > > > At the collector side, on the contrary, the implementation of the > > > parsing is much simpler if only full export packet are parsed. > > > If the information about the WHOLE export packet were contained in the > > > header, it would be simple to wait until all the bytes composing an > > > export packets are read from the TCP socket before beginning the > > > parsing. > > > Unfortunately, this information isn't currently contained in Netflow 9 > > > packet header (I suppose, due to Netflow's 9 "bias" towards UDP...). > > > Therefore, a parsing implementation having to deal with this > > > "fragentation" problem without this information needs to be much more > > > complicated (e.g. getting step by step the lenght of each flow set, see > > > if it's all there, parse it and go on....). > > > In summary: if IPFIX has to be carried over tcp, having the length of > > > the export packet in the packet header would be beneficial. > > > Regards, > > > Maurizio > > > > > > -- > > > Maurizio Molina > > > Research Staff member > > > Network Laboratories Heidelberg > > > NEC Europe Ltd. > > > Kurfuersten-Anlage 36, 69115 Heidelberg, Germany. > > > Tel. (49)6221 90511-18 Fax: (49)6221 90511-55 > > > e-mail: molina@ccrle.nec.de > > > Web: www.ccrle.nec.de > > > > > > -- > > > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > "unsubscribe ipfix" in message body > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed May 7 11:42:11 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA06168 for ; Wed, 7 May 2003 11:42:11 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19DQnR-0007Dx-00 for ipfix-list@mil.doit.wisc.edu; Wed, 07 May 2003 10:25:29 -0500 Received: from auds952.usa.alcatel.com ([143.209.238.7]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19DQnQ-0007Dr-00 for ipfix@net.doit.wisc.edu; Wed, 07 May 2003 10:25:28 -0500 Received: from alcatel.com (localhost [127.0.0.1]) by auds952.usa.alcatel.com (8.12.8p1/8.12.8) with ESMTP id h47FPHjB016022; Wed, 7 May 2003 10:25:17 -0500 (CDT) Message-ID: <3EB9255A.770B9748@alcatel.com> Date: Wed, 07 May 2003 10:25:14 -0500 From: Alex Audu X-Mailer: Mozilla 4.72 [en] (WinNT; U) X-Accept-Language: en MIME-Version: 1.0 To: Sebastian Zander CC: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] extensibility of the ipfix protocol References: <3EB86787.1040800@fokus.fraunhofer.de> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Extensions should also allow a vendor to implement unique features not ordinarily supported by the protocol specifications. This is useful for differentiation. One way to achieve this in the case of NetFlow v9 is to add an optional Extension FlowSet to the generic Export Packet structure. All the type definitions necessary to decode extension flow data will be specified here. It may even be in plain TLV format, or any other formats preferred by the vendor. Regards, Alex.. Sebastian Zander wrote: > Hi, > > although extensibility is mandatory by the requirements there hasn't > been a lot discussion on how to do it and the chosen candidate protocol > doesn't say much about extensibility. So here are my thoughts about > extensibility. > > The protocol specification must define how the protocol can be extended. > > I see the following (but there may be more) ways of extending the > protocol: > > 1. Extending the fixed packet header > This would allow to add new header fields without changing the > protocol specification. > > A bit (X) could be stolen from version to indicate the presence of > extension header(s). Extension header(s) follow the fixed header if > X=1. Extension header(s) consists of type, length and value. > > 2. Extending the attribute/field space > This is certainly needed. It could be done at least in two ways: > a) A flat number space where numbers are specified in extension > RFCs and registered by IANA. A nice thing would a dynamic > space where numbers are not defined by IANA but could be negotiated > out of band (see payload type in the RTP specification). Such a > space could also be used as safe playground. > b) A number space which is defined in extension RFCs (IANA) and the > possibility of using vendor specific attributes by having an optional > vendor ID field in the template specification. The vendor ID would > be IANA assigned but the vendor then could use its private field ID > space (see the Diameter Base protocol specification). Adds more > complexity and overhead to the protocol but could be very useful > when the protocol becomes widely adopted and a lot of extensions > are done. > > 3. Extending the type space (if there are types at all) > Probably there won't be much need to extent this because the protocol > specification could do a good job of specifying all the basic types. > If this really needs to be extended it would be probably done with a > revised protocol specification. > > Comments? Opinions? > > Cheers, > > Sebastian > > -- > Sebastian Zander E-mail: zander@fokus.fraunhofer.de > Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 > Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 > D-10589 Berlin, Germany www.fokus.fraunhofer.de/usr/sebastian.zander > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu May 8 20:29:09 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA17675 for ; Thu, 8 May 2003 20:29:09 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19DvId-00030X-00 for ipfix-list@mil.doit.wisc.edu; Thu, 08 May 2003 18:59:43 -0500 Received: from palrel13.hp.com ([156.153.255.238]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19DvIa-00030Q-00 for ipfix@net.doit.wisc.edu; Thu, 08 May 2003 18:59:40 -0500 Received: from xparelay2.ptp.hp.com (xparelay2.ptp.hp.com [15.1.28.65]) by palrel13.hp.com (Postfix) with ESMTP id E2B3D1C011D3 for ; Thu, 8 May 2003 16:59:39 -0700 (PDT) Received: from xpabh2.ptp.hp.com (xpabh2.ptp.hp.com [15.1.28.61]) by xparelay2.ptp.hp.com (Postfix) with ESMTP id B71D81C00A77 for ; Thu, 8 May 2003 16:59:39 -0700 (PDT) Received: by xpabh2.ptp.hp.com with Internet Mail Service (5.5.2655.55) id ; Thu, 8 May 2003 16:59:39 -0700 Message-ID: <1D3D2C371FCBD947A7897FABBD3533A566BABA@xsun01.ptp.hp.com> From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" To: "'ipfix@net.doit.wisc.edu'" Subject: RE: [ipfix] export packet length - LAST CALL Date: Thu, 8 May 2003 16:59:33 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2655.55) Content-Type: text/plain; charset="iso-8859-1" Precedence: bulk Sender: majordomo listserver Nevil, Thanks for summarizing the current status and seeking quick and rough consensus. Regarding the consensus aspect, do you have any ground rules, i.e. lack of disagreement on mailing list w/in n days, or number of pro/con e-mails from unique authors w/in n days, or some anything else? Perhaps there are already documented or undocumented precedents from other WG's we'd wish to follow? I must admit that I feel I'm missing something here. The hummmm process at the face to face meetings I understand and can accept, the pre-meeting methodology puzzles me. Regards, Jeff Meyer -----Original Message----- From: Nevil Brownlee [mailto:n.brownlee@auckland.ac.nz] Sent: Tuesday, May 06, 2003 10:50 PM To: calato@riverstonenet.com Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] export packet length - LAST CALL Hi all: Following up on the latest round of discusisons: 1) Using Randy & Rob's WG issues tracking system sounds fine. 2) Running Last Calls after mailing list discussion is a good way to reach consensus (and thereby close) issues. 3) On 'extensibility' of theIPFIX protocol. My take on this is that - We want an information model which can be extended by adding new attributes, and a protocol which can handle such new attributes, so we need to put some design effort in now to achieve that. - For reliability we agreed that the initial version of IPFIX has to work with only one-way transport, but that we need sufficient hooks to add reliability later on. Both these goals are clearly being worked on, that's great. But we need to be careful to balance our development efforts so as to produce a good, simple protocol now, which can be extended as needed later - without unduly delaying our efforts to get the initial version done fairly quickly. "By the end of this year" to reach near-final drafts sounds about right to me ... Cheers, Nevil ----------------------------------------------------------------------- Nevil Brownlee Director, Technology Development Phone: +64 9 373 7599 x88941 ITSS, The University of Auckland FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand ------------------------------------------------- This mail sent through University of Auckland http://www.auckland.ac.nz/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri May 9 14:30:30 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA26297 for ; Fri, 9 May 2003 14:30:30 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19ECHM-0003CB-00 for ipfix-list@mil.doit.wisc.edu; Fri, 09 May 2003 13:07:32 -0500 Received: from h65s138a81n47.user.nortelnetworks.com ([47.81.138.65] helo=zsc3s004.nortelnetworks.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19ECHL-0003C3-00 for ipfix@net.doit.wisc.edu; Fri, 09 May 2003 13:07:31 -0500 Received: from zsc3c028.us.nortel.com (zsc3c028.us.nortel.com [47.81.138.28]) by zsc3s004.nortelnetworks.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id h49I7Rr07387; Fri, 9 May 2003 11:07:28 -0700 (PDT) Received: by zsc3c028.us.nortel.com with Internet Mail Service (5.5.2653.19) id ; Fri, 9 May 2003 11:07:12 -0700 Message-ID: <0A11633F61BD9F40B43ABCC694004F93F18F5D@zsc3c026.us.nortel.com> From: "Reinaldo Penno" To: "'ipfix@net.doit.wisc.edu'" Cc: "'Mark Fullmer'" , calato@riverstonenet.com Subject: [ipfix] Lost Flow Records - Synch Message Date: Fri, 9 May 2003 11:07:12 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C31655.D08A80F8" Precedence: bulk Sender: majordomo listserver This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C31655.D08A80F8 Content-Type: text/plain; charset="ISO-8859-1" The editors of the protocol draft had a conf call and we came up with the main list of issues we need to sort out at this point. The first issue we need to get closure is the method to account for lost flow records. The proposal is in line with the "do not make assumption that the transport will be TCP". This does not preclude and/or is intended to fix the higher reliability extensions requirement. Mark will send an email with a proposal on this subject The proposal on the table is to send a IPFix Sync message periodically (periodicity is configurable), with the following information (aside the standard IPFix header) * Number of flow records sent (for each template?) * Packets and bytes sent (for each template?) Please advise if this is okay/not okay, etc, etc Regards, Reinaldo ------_=_NextPart_001_01C31655.D08A80F8 Content-Type: text/html; charset="ISO-8859-1" Content-Transfer-Encoding: quoted-printable [ipfix] Lost Flow Records - Synch Message

The editors of the protocol draft had a conf call and = we came up with the main list of issues we need to sort out at this = point.

The first issue we need to get closure is the method = to account for lost flow records. The proposal is in line with the = "do not  make assumption that the transport will be = TCP". This does not preclude and/or is intended to fix the higher = reliability extensions requirement. Mark will send an email with a = proposal on this subject

The proposal on the table is to send a IPFix Sync = message periodically (periodicity is configurable), with the following = information (aside the standard IPFix header) 

* Number of flow records sent (for each = template?)
* Packets and bytes sent (for each template?)

Please advise if this is okay/not okay, etc, = etc

Regards,

Reinaldo

------_=_NextPart_001_01C31655.D08A80F8-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri May 9 15:09:08 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA28170 for ; Fri, 9 May 2003 15:09:08 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19ED3U-0004BM-00 for ipfix-list@mil.doit.wisc.edu; Fri, 09 May 2003 13:57:16 -0500 Received: from auds952.usa.alcatel.com ([143.209.238.7]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19ED3T-0004BG-00 for ipfix@net.doit.wisc.edu; Fri, 09 May 2003 13:57:15 -0500 Received: from alcatel.com (localhost [127.0.0.1]) by auds952.usa.alcatel.com (8.12.8p1/8.12.8) with ESMTP id h49IvAjB016110; Fri, 9 May 2003 13:57:10 -0500 (CDT) Message-ID: <3EBBFA05.E20A5A20@alcatel.com> Date: Fri, 09 May 2003 13:57:10 -0500 From: Alex Audu X-Mailer: Mozilla 4.72 [en] (WinNT; U) X-Accept-Language: en MIME-Version: 1.0 To: Reinaldo Penno CC: "'ipfix@net.doit.wisc.edu'" , "'Mark Fullmer'" , calato@riverstonenet.com Subject: Re: [ipfix] Lost Flow Records - Synch Message References: <0A11633F61BD9F40B43ABCC694004F93F18F5D@zsc3c026.us.nortel.com> Content-Type: multipart/alternative; boundary="------------E7393BB96DA45AA1966EB3B3" Precedence: bulk Sender: majordomo listserver --------------E7393BB96DA45AA1966EB3B3 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Is the sequence number in the header packet not enough for tracking lost flow records? Alex. Reinaldo Penno wrote: > > > The editors of the protocol draft had a conf call and we came up with > the main list of issues we need to sort out at this point. > > The first issue we need to get closure is the method to account for > lost flow records. The proposal is in line with the "do not make > assumption that the transport will be TCP". This does not preclude > and/or is intended to fix the higher reliability extensions > requirement. Mark will send an email with a proposal on this subject > > The proposal on the table is to send a IPFix Sync message periodically > (periodicity is configurable), with the following information (aside > the standard IPFix header) > > * Number of flow records sent (for each template?) > * Packets and bytes sent (for each template?) > > Please advise if this is okay/not okay, etc, etc > > Regards, > > Reinaldo --------------E7393BB96DA45AA1966EB3B3 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit Is the sequence number in the header packet  not enough for tracking lost flow records?

Alex.

Reinaldo Penno wrote:

 

The editors of the protocol draft had a conf call and we came up with the main list of issues we need to sort out at this point.

The first issue we need to get closure is the method to account for lost flow records. The proposal is in line with the "do not  make assumption that the transport will be TCP". This does not preclude and/or is intended to fix the higher reliability extensions requirement. Mark will send an email with a proposal on this subject

The proposal on the table is to send a IPFix Sync message periodically (periodicity is configurable), with the following information (aside the standard IPFix header)

* Number of flow records sent (for each template?)
* Packets and bytes sent (for each template?)

Please advise if this is okay/not okay, etc, etc

Regards,

Reinaldo

--------------E7393BB96DA45AA1966EB3B3-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Sat May 10 00:42:04 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA11895 for ; Sat, 10 May 2003 00:42:04 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19ELhI-0007Un-00 for ipfix-list@mil.doit.wisc.edu; Fri, 09 May 2003 23:10:56 -0500 Received: from atlrel8.hp.com ([156.153.255.206]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19ELhF-0007Uf-00 for ipfix@net.doit.wisc.edu; Fri, 09 May 2003 23:10:53 -0500 Received: from xatlrelay1.atl.hp.com (xatlrelay1.atl.hp.com [15.45.89.190]) by atlrel8.hp.com (Postfix) with ESMTP id 16EA31C019D9; Sat, 10 May 2003 00:10:52 -0400 (EDT) Received: from xatlbh1.atl.hp.com (xatlbh1.atl.hp.com [15.45.89.186]) by xatlrelay1.atl.hp.com (Postfix) with ESMTP id A612F1C00A09; Sat, 10 May 2003 00:10:51 -0400 (EDT) Received: by xatlbh1.atl.hp.com with Internet Mail Service (5.5.2655.55) id ; Sat, 10 May 2003 00:10:51 -0400 Message-ID: <1D3D2C371FCBD947A7897FABBD3533A566BACF@xsun01.ptp.hp.com> From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" To: "'calato@riverstonenet.com'" Cc: "'Sebastian Zander'" , "'Mark Fullmer'" , ipfix@net.doit.wisc.edu Subject: RE: [ipfix] drawing the line between protocol and data model docu ment Date: Sat, 10 May 2003 00:10:46 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2655.55) Content-Type: text/plain; charset="iso-8859-1" Precedence: bulk Sender: majordomo listserver Paul, Thanks for putting forward a good example to talk around. I'd like to clarify the concept of typing of attributes. In particular the concept of "base types" and "derived types". Your dismissal of option II, implies to me that you are not making the distinction between these two types of types. If we assume that extending the typing system is a good thing, then there still exists the possibility of defining a set of "base types" which are defined ONCE in the base protocol and then defining a set of "derived types" which have additional semantics, but are encoded in the format of one of the base types. This concept is not new. Look at any programming language or database schema. Or look at the Diameter base protocol sections 4.2 and 4.3. http://www.diameter.org/drafts/latest/draft-ietf-aaa-diameter-17.txt Section 4.2 is titled. "Basic AVP Data Formats". Section 4.3 is titled "Derived AVP Data Formats" Section 4.2 defines the following set of base data types: OctetString Integer32 Integer64 Unsigned32 Unsigned64 Float32 Float64 In addition, Diameter defines "Grouped" which is structured type, which is not relevant to IPFIX, in my opinion. My proposal had a few additional "base types" indicated by "*": *boolean - true/false (may be encoded in byte) *byte *unsignedByte *short *unsignedShort int - 32-bit signed long - 64-bit signed unsignedInt unsignedLong float double *dateTime - 32-bit integer representing seconds *string - UTF-8 formatted text *ipdr:dateTimeMsec - 64-bit integer representing msecs *ipdr:ipV4Addr *ipdr:ipV6Addr *ipdr:UUID hexBinary - arbitrary sequence of octets Derived types have a relationship to a base type. I.e. when defining a derived type you reference the unique base type from which it is derived. So based on the discussion and the notation you introduced there are 4 pieces of information a collector should know, either explicitly or indirectly: - The attribute id - The derived attribute type - The base attribute type - The length of the encoded attribute More details on each: - The "attribute id" identifies a unique attribute which may appear in an IPFIX record. The attribute ids are assigned as part of the information model. Extension of the information model, implies adding new attributes, with new attribute ids. This implies some sort of managed namespace (or namespaces). Managing the namespace implies that a registry of defined attribute ids is maintained, and that additional attribute ids can be incorporated through some process at a later date. - The "attribute type" indicates the type of information held in the attribute. E.g. is it an integer, a string, a floating point value, etc. For practical reasons, types may reflect some encoding or machine representation information. E.g. for a human we could simply indicate whether a particular attribute is numeric, but for practical reasons, distinguishing between shorts, longs, floats, etc. is also included. To enable extension, new attribute types may be introduced later. However, these new types should be considered as derived from some base type, to enable legacy implementations to do some basic parsing of the field. The base type should be capable of completely holding the data represented by the derived type, so that no information loss occurs. Since one of the base types is hexBinary (same as OctetString in Diameter), any new type can always be completely held by this base type. - The length is the number of bytes occupied in the the encoded IPFIX record for a given attribute. If an implementation was guaranteed access to the information model(s) used by a given IPFIX expoter, then (in theory) a collector would ONLY need the attribute ID in a template. Since all other information could be inferred from the information model (assuming it was machine readable). For practical reasons, it is best to assume that the information model may not be available. And to assume the collector may not be in synch with any recent changes to the information model, including the introduction of new derived types. In this case the collector can completely parse the record with either: a. (id, encoded length) [Paul's option I] b. (id, base type). It will NOT work if we provide the collector with ONLY the following information (id, derived type) [Paul's option II], because the collector may not be able to determine how many bytes to skip. One might chose to provide additional information in the template such as: c. (id, derived type, length) [Paul's option III] d. (id, base type, length) e. (id, derived type, base type) f. (id, derived type, base type, length) Since compactness is a concern, and since there is clearly confusion around the use of base types vs. derived types in the encoding, I'm inclined to throw in the towel and say let's continue with option a, which is what NFv9 does today. However, I hope is also clear to folks now, that by using the concept of base types, length does NOT need to be in the template to enable parsing of unknown attributes. Regards, Jeff Meyer -----Original Message----- From: calato@riverstonenet.com [mailto:calato@riverstonenet.com] Sent: Tuesday, May 06, 2003 8:45 AM To: MEYER,JEFFREY D (HP-Cupertino,ex1) Cc: 'Sebastian Zander'; 'Mark Fullmer'; ipfix@net.doit.wisc.edu Subject: Re: [ipfix] drawing the line between protocol and data model document OK. I'm confused. Let me run through an example and people can correct me where I'm wrong. Lets take "Source IP v4 address". Lets also hold off on variable length fields for now. All options need some kind of type value to indicate what the value semantically means. Lets pick 57 == Source IP v4 address. So what we need next is info on the value part so we can decode it and it should be backward compatible (i.e. older versions can at least skip unknown elements. Option I: Element and Length 57, 4 Any software understanding element 57 knows it is the Source IP address and can display/handle it correctly. Any one else would skip over the value in the flow set. Option II: Element and data type Lets say data type 22 == IP V4 Address 57, 22 Any software understanding element 57 knows it is the Source IP address. But what if it doesn't know what data type 22 is? How can I skip over it? Option III: Element and data type and length 57, 22, 4 Any software which understands 57 and 22 can handle it correclty. If it only understands 22, it can ONLY display the value in some rational way, nothing else can be done. If I don't understand 22 I can at least skip over it by using the length. Correct away! Paul "MEYER,JEFFREY D (HP-Cupertino,ex1)" wrote: > > Sebastian, > > Thanks for speaking up! I agree with your restatement of the > relationship between typing and semantics. > > Regarding types like UTF8 or OctetString, as I read the proposal > from Benoit, the "Length Encoding" model would use a special value > of 0xFFFF instead of length in the template. This implies that > the encoding of the "variable length" field would be of the form > (length) (value), versus fixed length types which can be encoded by > just placing their (value) in the payload. > > (see http://ipfix.doit.wisc.edu/archive/1602.html) > > This same model would apply in a type based scheme, if the type > were string or hexBinary (indicated by the appropriate integer > identifier in the template), then the encoded value would still be > of the form (length) (value) in the payload. > > Regardless of what template scheme is chosen, however, I think it > is important that the protocol spec speak in terms of how types are > encoded, not how each individual element declared in the information model > is encoded. This makes the inevitable extension of the information > model easier. > > Regards, > > Jeff Meyer > > > -----Original Message----- > > From: Sebastian Zander [mailto:zander@fokus.fraunhofer.de] > > Sent: Thursday, May 01, 2003 5:57 PM > > To: MEYER,JEFFREY D (HP-Cupertino,ex1) > > Cc: 'Mark Fullmer'; ipfix@net.doit.wisc.edu > > Subject: Re: [ipfix] drawing the line between protocol and data model > > docu ment > > > > > > Jeff, > > > > i support the idea of typing. I think there is some > > misunderstanding here > > by some people who confuse types with semantics. A type does > > not define the > > semantics. In my opinion typing has the advantage that even > > if i don't know > > the semantics by the time of the decoding i still can > > correctly decode the > > value and present it. It also has the advantage that for a > > well known type > > the encoding is well defined. If that is not the case people > > may encode > > attributes of the same type differently just making > > interoperability harder. > > I would define these basic data types in the protocol document. > > > > I am not so sure if both alternatives take the same space > > because there > > are some types which which do not have an intrinsic length > > definition. An > > utf or octet string could have an arbitrary length. Without > > typing everything > > would be basically an octetstring of some length ;-). However > > i though the > > protocol is supossed to use templates and so it would only affect the > > template exchange wouldn't it? > > > > Cheers, > > > > Sebastian > > > > MEYER,JEFFREY D (HP-Cupertino,ex1) wrote: > > > vs. > > > > > > = 16909060 (integer) > > > > > > vs. > > > > > > = whatever the heck 0x01020304 represents > > > in IEEE Floating Point notation. (something around 4.06) > > > > > > > > > So, yes that's all it buys you. Since both alternatives take > > > the same amount of space, I'd prefer the one with more > > > intrinsic information. But, I guess I'm in the minority :-( > > > > > > > > > Regards, > > > > > > Jeff Meyer > > > > > > > > > > > >>-----Original Message----- > > >>From: Mark Fullmer [mailto:maf@eng.oar.net] > > >>Sent: Thursday, May 01, 2003 8:39 AM > > >>To: MEYER,JEFFREY D (HP-Cupertino,ex1) > > >>Cc: ipfix@net.doit.wisc.edu > > >>Subject: Re: [ipfix] drawing the line between protocol and > > data model > > >>docu ment > > >> > > >> > > >>So the difference boils down to a tool being able to display > > >> > > >> = 0x01020304 > > >> vs > > >> = 1.2.3.4 > > >> > > >>In this situation someone will either grab a new version of the > > >>software or an updated data definition to resolve the unknown > > >>attribute, > > >>which would also define it's type. > > >> > > >>mark > > >> > > >>On Tue, Apr 29, 2003 at 06:25:11PM -0400, MEYER,JEFFREY D > > >>(HP-Cupertino,ex1) wrote: > > >> > > >>>Mark, > > >>> > > >>> I'll try to clarify: > > >>> > > >>> Concept #1: The information model and the encoding are separate. > > >>> A well defined mapping determines how an information model is > > >>> to be encoded for a specific use (e.g. IPFIX). The encoding > > >>> may take whatever approach allows both ends to do something > > >>> useful with the data. As you point out there is some useful > > >>> stuff which can be done with the length alone. > > >>> > > >>> Concept #2: Derived typing. By defining a base set of types, > > >>> which must be understood, derived types can be defined in terms > > >>> of these base types. For instance Diameter does this. This > > >>> enables a core set of types to be defined and coded to, but > > >>> also enables richer types later on with an appropriate fallback > > >>> for systems which aren't aware of the extensions. > > >>> > > >>> The benefit I see of explicit typing is that instead of seeing > > >>>that a field is of length "4 bytes" something which will be seen > > >>>A LOT. Is that we can instead differentiate between > > >> > > >>IPAddr, Integer, > > >> > > >>>unsigned integer, and float. This seems handy to me. > > >>> > > >>> Your argument seems centered around the long term potential of > > >>>adding lots of fixed length fields whose base length is not 4,8 > > >>>or 16 bytes long, for instance MACAddr (although if we agree on > > >>>going the type route, I'd recommend adding MACAddr and then you > > >>>have 1,2,4,6,8 and 16 bytes covered. I guess I don't foresee > > >>>a lot more fixed length fields coming down the pipe in the IPFIX > > >>>domain. Hence my desire to make it more explicit for the types > > >>>which do occur all the time and are likely points of extension. > > >>>WHY? Because then a tool could at least present these in a > > >> > > >>meaningful > > >> > > >>>way. As opposed to incorrectly formatting a float type as an int. > > >>> > > >>> But hey, either way will work. If the goal is to limit the > > >>>amount of changes, I'll (grudgingly) go along. I haven't heard > > >>>any opinions from anyone but you in either direction. So, > > >>>at this point I don't see a quorom either way... > > >>> > > >>>Regards, > > >>> > > >>> Jeff Meyer > > >>> > > >>> > > >>> > > >>>>-----Original Message----- > > >>>>From: Mark Fullmer [mailto:maf@eng.oar.net] > > >>>>Sent: Tuesday, April 29, 2003 2:53 PM > > >>>>To: MEYER,JEFFREY D (HP-Cupertino,ex1) > > >>>>Cc: ipfix@net.doit.wisc.edu > > >>>>Subject: Re: [ipfix] drawing the line between protocol > > >>> > > >>and data model > > >> > > >>>>docu ment > > >>>> > > >>>> > > >>>>IMHO crippling the protocol to serve the needs of a > > >>> > > >>documentation tool > > >> > > >>>>is just silly. > > >>>> > > >>>>It would be possible to use both a Type name and a Length > > >>> > > >>instead of > > >> > > >>>>the existing Length, which would allow a collector to process > > >>>>unknown Types. > > >>>> > > >>>>With the existing v9 protocol a collector can ignore an > > >>>>unknown attribute, > > >>>>or possibly just pass it on to post processing tools which > > >>>>might understand > > >>>>it. > > >>>> > > >>>>Your suggested changes no longer allow a collector to > > >>> > > >>ignore unknown > > >> > > >>>>attributes if the Type is also unknown. Resorting to encoding all > > >>>>new Types after the initial protocol spec as octet > > >>> > > >>strings is (IMHO) > > >> > > >>>>not acceptable. > > >>>> > > >>>>I'm still trying to understand what the benefits to explicit > > >>>>Types sent > > >>>>in templates are. > > >>>> > > >>>>mark > > >>>> > > >>>>On Tue, Apr 29, 2003 at 02:50:02PM -0400, MEYER,JEFFREY D > > >>>>(HP-Cupertino,ex1) wrote: > > >>>> > > >>>>>Juergen, > > >>>>> > > >>>>> I agree that the data model should have a well defined type > > >>>>>system. And that that typing system should reflect the > > >>>> > > >>>>requirements. > > >>>> > > >>>>> This is in part why I was hoping that the protocol > > >>>> > > >>itself would > > >> > > >>>>>be more explicit here. (See my mail > > >>>>>http://ipfix.doit.wisc.edu/archive/1604.html) > > >>>>> > > >>>>> There is also a type list in that mail. However, I > > >>>> > > >>would suggest > > >> > > >>>>>using the names from XML-Schema part 2 Data types, as this would > > >>>>>facilitate creating a formal/normative definition, versus > > >>>> > > >>>>reinventing > > >>>> > > >>>>>names for the types. > > >>>>> > > >>>>> Here are the type names from XML-Schema/IPDR: > > >>>>> > > >>>>> boolean - true/false (may be encoded in byte) > > >>>>> byte > > >>>>> unsignedByte > > >>>>> short > > >>>>> unsignedShort > > >>>>> int - 32-bit signed > > >>>>> long - 64-bit signed > > >>>>> unsignedInt > > >>>>> unsignedLong > > >>>>> float > > >>>>> double > > >>>>> dateTime - 32-bit integer representing seconds > > >>>> > > >>since EPOCH > > >> > > >>>>>(1/1/1970 0:00 GMT) > > >>>>> string - UTF-8 formatted text > > >>>>> ipdr:dateTimeMsec - 64-bit integer representing msecs > > >>>> > > >>>>since EPOCH > > >>>> > > >>>>> ipdr:ipV4Addr > > >>>>> ipdr:ipV6Addr > > >>>>> ipdr:UUID > > >>>>> hexBinary - arbitrary sequence of octets > > >>>>> > > >>>>> If the protocol itself has a basic "octetString" (or from > > >>>> > > >>>>XML, hexBinary) > > >>>> > > >>>>>payload, > > >>>>>then this can be used to carry new classes of information > > >>>> > > >>>>in the future w/o > > >>>> > > >>>>>rejiggering the protocol itself. > > >>>>> > > >>>>> In addition other "derived" types may be created from the > > >>>> > > >>>>base types. For > > >>>> > > >>>>>instance in IPDR, the ipdr:dateTimeMsec is derived from long. > > >>>>> > > >>>>>Regards, > > >>>>> > > >>>>> Jeff Meyer > > >>>>> > > >>>>> > > >>>>>>-----Original Message----- > > >>>>>>From: Juergen Quittek [mailto:quittek@ccrle.nec.de] > > >>>>>>Sent: Monday, April 28, 2003 2:06 PM > > >>>>>>To: ipfix@net.doit.wisc.edu > > >>>>>>Subject: [ipfix] drawing the line between protocol > > >>>>> > > >>and data model > > >> > > >>>>>>document > > >>>>>> > > >>>>>> > > >>>>>>Hi all, > > >>>>>> > > >>>>>>When shaping the planned documents, we have to draw a line > > >>>>>>between the protocol and the data model. > > >>>>>> > > >>>>>>Definitely, the data model document defines all field types, > > >>>>>>but does it also define basic data types, such as '16 bit > > >>>>>>unsigned integer'? > > >>>>>> > > >>>>>>Maybe we should have a set of basic data types defined in > > >>>>>>the protocol document and restrict the data model doc to > > >>>>> > > >>>>these types? > > >>>> > > >>>>>>This would further structure the data model. > > >>>>>> > > >>>>>>Any thoughts? > > >>>>>> > > >>>>>> Juergen > > >>>>>> > > >>>>>>-- > > >>>>>>Help mailto:majordomo@net.doit.wisc.edu and say "help" > > >>>>>>in message body > > >>>>>>Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > >>>>>>"unsubscribe ipfix" in message body > > >>>>>>Archive http://ipfix.doit.wisc.edu/archive/ > > >>>>>> > > >>>>> > > >>>>>-- > > >>>>>Help mailto:majordomo@net.doit.wisc.edu and say > > >>>> > > >>>>"help" in message body > > >>>> > > >>>>>Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > >>>>>"unsubscribe ipfix" in message body > > >>>>>Archive http://ipfix.doit.wisc.edu/archive/ > > >>>> > > >>>-- > > >>>Help mailto:majordomo@net.doit.wisc.edu and say > > >> > > >>"help" in message body > > >> > > >>>Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > >>>"unsubscribe ipfix" in message body > > >>>Archive http://ipfix.doit.wisc.edu/archive/ > > >> > > > > > > -- > > > Help mailto:majordomo@net.doit.wisc.edu and say > > "help" in message body > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > "unsubscribe ipfix" in message body > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > > -- > > Sebastian Zander E-mail: > > zander@fokus.fraunhofer.de > > Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 > > Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 > > D-10589 Berlin, Germany > > www.fokus.fraunhofer.de/usr/sebastian.zander > > > > > > > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Sun May 11 20:55:33 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA16527 for ; Sun, 11 May 2003 20:55:32 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19F1AD-0007hj-00 for ipfix-list@mil.doit.wisc.edu; Sun, 11 May 2003 19:27:33 -0500 Received: from ds20-1.cc.swin.edu.au ([136.186.1.150] helo=swin.edu.au) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19F1A9-0007ha-00 for ipfix@net.doit.wisc.edu; Sun, 11 May 2003 19:27:30 -0500 Received: from fokus.fraunhofer.de (szander-laptop.caia.swin.edu.au [136.186.229.90]) by swin.edu.au (8.9.3p2/8.9.3) with ESMTP id KAA2066893; Mon, 12 May 2003 10:27:08 +1000 (EST) Message-ID: <3EBEE8F0.3020600@fokus.fraunhofer.de> Date: Mon, 12 May 2003 02:21:04 +0200 From: Sebastian Zander Organization: Fraunhofer FOKUS User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.2) Gecko/20030208 Netscape/7.02 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Alex Audu CC: Reinaldo Penno , "'ipfix@net.doit.wisc.edu'" , "'Mark Fullmer'" , calato@riverstonenet.com Subject: Re: [ipfix] Lost Flow Records - Synch Message References: <0A11633F61BD9F40B43ABCC694004F93F18F5D@zsc3c026.us.nortel.com> <3EBBFA05.E20A5A20@alcatel.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Alex Audu wrote: > Is the sequence number in the header packet not enough for tracking > lost flow records? Having only the sequence number the collector only would know the number of packets lost. It would not know the number of flow records lost nor which flow records had been lost. > Alex. > > Reinaldo Penno wrote: > >> >> >> The editors of the protocol draft had a conf call and we came up with >> the main list of issues we need to sort out at this point. >> >> The first issue we need to get closure is the method to account for >> lost flow records. The proposal is in line with the "do not make >> assumption that the transport will be TCP". This does not preclude >> and/or is intended to fix the higher reliability extensions >> requirement. Mark will send an email with a proposal on this subject >> >> The proposal on the table is to send a IPFix Sync message periodically >> (periodicity is configurable), with the following information (aside >> the standard IPFix header) >> >> * Number of flow records sent (for each template?) >> * Packets and bytes sent (for each template?) >> >> Please advise if this is okay/not okay, etc, etc >> >> Regards, >> >> Reinaldo >> -- Sebastian Zander E-mail: zander@fokus.fraunhofer.de Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 D-10589 Berlin, Germany www.fokus.fraunhofer.de/usr/sebastian.zander -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Sun May 11 21:35:34 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA16890 for ; Sun, 11 May 2003 21:35:34 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19F1uZ-0000s0-00 for ipfix-list@mil.doit.wisc.edu; Sun, 11 May 2003 20:15:27 -0500 Received: from ds20-1.cc.swin.edu.au ([136.186.1.150] helo=swin.edu.au) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19F1uW-0000rt-00 for ipfix@net.doit.wisc.edu; Sun, 11 May 2003 20:15:24 -0500 Received: from fokus.fraunhofer.de (szander-laptop.caia.swin.edu.au [136.186.229.90]) by swin.edu.au (8.9.3p2/8.9.3) with ESMTP id LAA2072325; Mon, 12 May 2003 11:15:07 +1000 (EST) Message-ID: <3EBEF430.6060908@fokus.fraunhofer.de> Date: Mon, 12 May 2003 03:09:04 +0200 From: Sebastian Zander Organization: Fraunhofer FOKUS User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.2) Gecko/20030208 Netscape/7.02 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Reinaldo Penno CC: "'ipfix@net.doit.wisc.edu'" , "'Mark Fullmer'" , calato@riverstonenet.com Subject: Re: [ipfix] Lost Flow Records - Synch Message References: <0A11633F61BD9F40B43ABCC694004F93F18F5D@zsc3c026.us.nortel.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hi Reinaldo, having flow record counters per template is a very nice thing in my opinion. But if we have this information why would we need the packets and bytes sent per template information in addition? What about information about lost template definitions? (A loss of a template def could potentially cause flow records to be "lost" at the collector...) Cheers, Sebastian Reinaldo Penno wrote: > The editors of the protocol draft had a conf call and we came up with > the main list of issues we need to sort out at this point. > > The first issue we need to get closure is the method to account for lost > flow records. The proposal is in line with the "do not make assumption > that the transport will be TCP". This does not preclude and/or is > intended to fix the higher reliability extensions requirement. Mark will > send an email with a proposal on this subject > > The proposal on the table is to send a IPFix Sync message periodically > (periodicity is configurable), with the following information (aside the > standard IPFix header) > > * Number of flow records sent (for each template?) > * Packets and bytes sent (for each template?) > > Please advise if this is okay/not okay, etc, etc > > Regards, > > Reinaldo > -- Sebastian Zander E-mail: zander@fokus.fraunhofer.de Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 D-10589 Berlin, Germany www.fokus.fraunhofer.de/usr/sebastian.zander -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Sun May 11 22:24:29 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA17548 for ; Sun, 11 May 2003 22:24:28 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19F2p4-00021L-00 for ipfix-list@mil.doit.wisc.edu; Sun, 11 May 2003 21:13:50 -0500 Received: from eng4.oar.net ([192.148.244.24]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 19F2p2-00021G-00 for ipfix@net.doit.wisc.edu; Sun, 11 May 2003 21:13:48 -0500 Received: (qmail 87510 invoked by uid 4454); 12 May 2003 02:13:48 -0000 Date: Sun, 11 May 2003 22:13:48 -0400 From: Mark Fullmer To: Sebastian Zander Cc: "'ipfix@net.doit.wisc.edu'" Subject: Re: [ipfix] Lost Flow Records - Synch Message Message-ID: <20030511221348.A87500@net.ohio-state.edu> References: <0A11633F61BD9F40B43ABCC694004F93F18F5D@zsc3c026.us.nortel.com> <3EBEF430.6060908@fokus.fraunhofer.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: <3EBEF430.6060908@fokus.fraunhofer.de>; from zander@fokus.fraunhofer.de on Mon, May 12, 2003 at 03:09:04AM +0200 Precedence: bulk Sender: majordomo listserver How lost templates are handled depends on the transport. UDP - templates are periodically retransmitted so a collector should buffer data until the template arrives or resource exhaustion. TCP - Templates shouldn't be lost. There is a small window for a template to be lost in a primary/secondary failover situation. The failover details have not been finalized yet. SCPT - Hopefully the templates can be done over a reliable channel. Still have failover issues. mark On Mon, May 12, 2003 at 03:09:04AM +0200, Sebastian Zander wrote: > Hi Reinaldo, > > having flow record counters per template is a very nice thing in my > opinion. But if we have this information why would we need the packets > and bytes sent per template information in addition? > > What about information about lost template definitions? (A loss of a > template def could potentially cause flow records to be "lost" at the > collector...) > > Cheers, > > Sebastian > > Reinaldo Penno wrote: > > The editors of the protocol draft had a conf call and we came up with > > the main list of issues we need to sort out at this point. > > > > The first issue we need to get closure is the method to account for lost > > flow records. The proposal is in line with the "do not make assumption > > that the transport will be TCP". This does not preclude and/or is > > intended to fix the higher reliability extensions requirement. Mark will > > send an email with a proposal on this subject > > > > The proposal on the table is to send a IPFix Sync message periodically > > (periodicity is configurable), with the following information (aside the > > standard IPFix header) > > > > * Number of flow records sent (for each template?) > > * Packets and bytes sent (for each template?) > > > > Please advise if this is okay/not okay, etc, etc > > > > Regards, > > > > Reinaldo > > > > > -- > Sebastian Zander E-mail: zander@fokus.fraunhofer.de > Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 > Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 > D-10589 Berlin, Germany www.fokus.fraunhofer.de/usr/sebastian.zander > > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon May 12 04:23:17 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id EAA05120 for ; Mon, 12 May 2003 04:23:17 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19F8UE-0001aZ-00 for ipfix-list@mil.doit.wisc.edu; Mon, 12 May 2003 03:16:42 -0500 Received: from [130.216.191.4] (helo=mailhost2.auckland.ac.nz) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19F8UB-0001aN-00 for ipfix@net.doit.wisc.edu; Mon, 12 May 2003 03:16:40 -0500 Received: from mailhost.auckland.ac.nz (IDENT:mirapoint@mailhost [130.216.191.61]) by mailhost2.auckland.ac.nz (8.12.9/8.12.9/8.12.9-ua) with ESMTP id h4C8GLcT027773; Mon, 12 May 2003 20:16:22 +1200 (NZST) Received: from hotlava.auckland.ac.nz (hotlava.auckland.ac.nz [130.216.191.123]) by mailhost.auckland.ac.nz (Mirapoint Messaging Server MOS 3.3.2-CR) with ESMTP id APV65714; Mon, 12 May 2003 20:16:20 +1200 (NZST) Received: (from apache@localhost) by hotlava.auckland.ac.nz (8.11.6/8.11.6) id h4C8GJ314376; Mon, 12 May 2003 20:16:19 +1200 X-Authentication-Warning: hotlava.auckland.ac.nz: apache set sender to n.brownlee@auckland.ac.nz using -f Received: from dhcp-9-229.ripemtg.ripe.net (dhcp-9-229.ripemtg.ripe.net [193.0.9.229]) by hotlava.auckland.ac.nz (Horde) with HTTP for ; Mon, 12 May 2003 20:16:19 +1200 Message-ID: <1052727379.ae30d050022c1@hotlava.auckland.ac.nz> Date: Mon, 12 May 2003 20:16:19 +1200 From: Nevil Brownlee To: "MEYER,JEFFREY D (HP-Cupertino,ex1)" Cc: "'ipfix@net.doit.wisc.edu'" Subject: RE: [ipfix] export packet length - LAST CALL References: <1D3D2C371FCBD947A7897FABBD3533A566BABA@xsun01.ptp.hp.com> In-Reply-To: <1D3D2C371FCBD947A7897FABBD3533A566BABA@xsun01.ptp.hp.com> MIME-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Internet Messaging Program (IMP) 4.0-cvs X-Originating-IP: 193.0.9.229 Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hi Jeff: > Thanks for summarizing the current status and seeking quick and rough > consensus. > > Regarding the consensus aspect, do you have any ground rules, i.e. lack of > disagreement on mailing list w/in n days, or number of pro/con e-mails from > unique authors w/in n days, or some anything else? Perhaps there are > already documented or undocumented precedents from other WG's we'd wish to > follow? I'm a bit loth to set out rules, but how about this to start with: * Issue is raised on list, either directly or as comment on a posted document * At least one (or two?) other people post an opinion for or against the issue within say 5 days - now it's recognised as an issue * The document editor starts a last call for the issue: it's resolved if there are some 'for' responses, and no 'against' responses within 7 days. (Note that complete documents are subject to 2-week last calls). The point about rules is that our goal is to produce documents that are technically sound, and don't want that soundness compromised by voting rules being interpretted too literally. > I must admit that I feel I'm missing something here. The hummmm process > at the face to face meetings I understand and can accept, the pre-meeting > methodology puzzles me. Up until now we've worked by spelling out what was to happen, e.g. documents produced by .., discussion on list by .., discussion at IETF meetings, confirmed by discussion on IPFIX list. The hardest thing about getting a WG to actually work is to get enough participation and discussion to achieve real progress. Now that we have end goals visible, I trust that everyone will put the effort in to get the inital version of IPFIX completed real soon now. Cheers, Nevil ----------------------------------------------------------------------- Nevil Brownlee Director, Technology Development Phone: +64 9 373 7599 x88941 ITSS, The University of Auckland FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand ------------------------------------------------- This mail sent through University of Auckland http://www.auckland.ac.nz/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon May 12 07:13:12 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id HAA08128 for ; Mon, 12 May 2003 07:13:12 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19FAup-00068o-00 for ipfix-list@mil.doit.wisc.edu; Mon, 12 May 2003 05:52:19 -0500 Received: from tokyo.ccrle.nec.de ([195.37.70.2]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19FAun-00068i-00 for ipfix@net.doit.wisc.edu; Mon, 12 May 2003 05:52:17 -0500 Received: from venus.office (venus.office [10.1.1.11]) by tokyo.ccrle.nec.de (8.12.9/8.12.8) with ESMTP id h4CAplVI025275; Mon, 12 May 2003 12:51:48 +0200 (CEST) Received: from [10.1.1.128] (n-quittek.office [10.1.1.128]) by venus.office (Postfix on SuSE Linux eMail Server 3.0) with ESMTP id 4C929A3B3F; Mon, 12 May 2003 12:43:26 +0200 (CEST) Date: Mon, 12 May 2003 12:53:29 +0200 From: Juergen Quittek To: calato@riverstonenet.com, "MEYER,JEFFREY D (HP-Cupertino,ex1)" Cc: "'Nevil Brownlee'" , ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Editors for IPFIX drafts Message-ID: <14874308.1052744009@[10.1.1.128]> In-Reply-To: <3EB7CE29.DD91933F@riverstonenet.com> References: <3EB7CE29.DD91933F@riverstonenet.com> X-Mailer: Mulberry/2.1.2 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Paul, From my experience, it does not make sense to write a largish XML specification without an XML editor, that automates typing of 80% of the lengthy XML DTD or schema syntax. Just using a Word or ASCII editor would be a significant waste of time. I'm using a commercial editor (TurboXML) but there are certainly also free ones. Does anyone know a free tool? Jeff? Cheers, Juergen -- calato@riverstonenet.com wrote on 06 May 2003 11:00 -0400: > > Seems like a reasonable model. I'd say lets head that way > and see how it goes. > > Are there any MS word templates for creating the XML part? > > > Paul > > "MEYER,JEFFREY D (HP-Cupertino,ex1)" wrote: >> >> Hi, >> >> If the human readable part is generated from the XML appendix, >> then I guess it doesn't matter which part you call normative, >> since they both originate from the same source. >> >> The development model I see is the following: >> >> Generate XML Info Model >> Run model through XSLT (xalan) translater to produce prose >> (consistent with RFC 2629) [I'll volunteer to write the >> style sheet translation) >> Incorporate the raw model as appendix and the generated prose >> description into the appropriate section (consistent w/ >> RFC 2629) along with other text of the Information Model Draft. >> Run Marshall's tool http://xml.resource.org/ to produce nroff style >> and HTML versions of document. Submit nroff style to RFC editor, >> post HTML style in addition to ipfix website (easier to navigate). >> >> For examples of the dual formatted output, see the IPDR submissions >> to IPDR located at: >> >> http://www.ipdr.org/documents/ipfix >> >> Regards, >> >> Jeff Meyer >> >> > -----Original Message----- >> > From: Juergen Quittek [mailto:quittek@ccrle.nec.de] >> > Sent: Thursday, May 01, 2003 8:21 AM >> > To: calato@riverstonenet.com >> > Cc: MEYER,JEFFREY D (HP-Cupertino,ex1); 'Nevil Brownlee'; >> > ipfix@net.doit.wisc.edu >> > Subject: Re: [ipfix] Editors for IPFIX drafts >> > >> > >> > Paul, >> > >> > I agree to Jeff, that there is some value in having an XML >> > representation >> > of the specification. However, in the end this document will >> > become an RFC >> > and it must be well readable to human beings. >> > >> > Therefore, I think that the main body must be human readable, >> > but we can >> > provide an appendix, that contains an equivalent machine >> > readable spec. >> > >> > Different to Jeff, I think that the hunam readable main body should be >> > the normative part, while the XML appendix is just given as >> > support for >> > users of XML tools allowing them to genrate protocol stacks >> > and/or tools >> > for consistency checking. >> > >> > As Jeff suggested, we can work with an XML representation >> > when developing >> > the spec and generate human readable text automatically from >> > the XML code. >> > This will help us avoiding fomal mistakes in the human >> > readable spec and we >> > would always have consistency between human readable and XML part. >> > >> > Juergen >> > >> > >> > -- calato@riverstonenet.com wrote on 01 May 2003 11:00 -0400: >> > >> > > >> > > I'm a little confused. Are we talking about XML descriptions >> > > as an appendix with prose as the main body or XML as the >> > > main body with tools translating it into a more readable >> > > form? Or XML as an appendix with tools translating it for >> > > use as sections mixed in with the prose? >> > > >> > > >> > > Paul >> > > >> > > Juergen Quittek wrote: >> > >> >> > >> Jeff, >> > >> >> > >> -- MEYER,JEFFREY D (HP-Cupertino,ex1) wrote on 28 April >> > 2003 09:20 -0700: >> > >> > >> > >> > Juergen, >> > >> > >> > >> > Thanks for the response. >> > >> > >> > >> > I agree XML in and of itself is marginally human readable. >> > >> > >> > >> > I think that basic tools such as Xalan >> > (http://xml.apache.org/), can >> > >> > be used to "transform" a base XML document into a >> > variety of formats, >> > >> > including formatting consistent with RFC's. Note that >> > all the IPDR >> > >> > RFC's were written in XML using Marshall Rose's xml2rfc tools >> > >> > (http://xml.resource.org/). >> > >> >> > >> I also like Marshall's tools. >> > >> If Paul agrees, we should use them for the data model document. >> > >> >> > >> > So the model I would picture is having the Normative >> > XML data model >> > >> > as an appendix, and having generated information from >> > this data model >> > >> > populate some of the body sections to aid human readers. >> > >> >> > >> This looks like a reasonable way to go. >> > >> >> > >> Juergen >> > >> >> > >> > Regards, >> > >> > >> > >> > Jeff Meyer >> > >> > >> > >> >> -----Original Message----- >> > >> >> From: Juergen Quittek [mailto:quittek@ccrle.nec.de] >> > >> >> Sent: Monday, April 28, 2003 8:24 AM >> > >> >> To: MEYER,JEFFREY D (HP-Cupertino,ex1); 'Nevil Brownlee'; >> > >> >> ipfix@net.doit.wisc.edu >> > >> >> Subject: RE: [ipfix] Editors for IPFIX drafts >> > >> >> >> > >> >> >> > >> >> Jeff, >> > >> >> >> > >> >> I do see a value in such a formal description. But there is a >> > >> >> trafe-off. >> > >> >> >> > >> >> I also see a degradation of human readability. You >> > loose means of >> > >> >> structuring your document text when using XML, for >> > example grouping >> > >> >> attributes that are related. (Of course you could do so in >> > >> >> XML, but it is >> > >> >> much harder to find the heading "Port-related Attributes" in >> > >> >> an XML document, >> > >> >> because headings cannot be well highlighted and numbering of >> > >> >> headings is >> > >> >> not well supported. >> > >> >> >> > >> >> Anyway, I like the idea of an XML encoding of the model, but >> > >> >> I we should >> > >> >> not go this way if we cannot find a good way of getting it >> > >> >> hunam readable >> > >> >> (maybe by applying formatting rules). >> > >> >> >> > >> >> An option would also be adding the XML code to the document >> > >> >> as an appendix. >> > >> >> >> > >> >> Juergen >> > >> >> >> > >> >> >> > >> >> -- MEYER,JEFFREY D (HP-Cupertino,ex1) wrote on 28 April 2003 >> > >> >> 10:57 -0400: >> > >> >> >> > >> >> > Hi, >> > >> >> > >> > >> >> > My particular interest on working on the >> > information model would >> > >> >> > be on developing a more formal and standard way of >> > describing the >> > >> >> > information elements which are used in IPFIX communication. >> > >> >> > >> > >> >> > The modelling of information elements should >> > ideally be done >> > >> >> > using a formal description language, so that tools can be >> > >> >> constructed >> > >> >> > which do not require humans to transcribe this information. >> > >> >> > >> > >> >> > I would propose the use of XML-Schema to provide >> > this formal >> > >> >> > description. Through the use of XSL processors, prose >> > >> >> based descriptions >> > >> >> > and tables can be produced if these are considered desireable. >> > >> >> > >> > >> >> > NOTE: describing the information model formally >> > does NOT mean >> > >> >> > that encoding and transport for IPFIX need to change >> > in any way. It >> > >> >> > simply means that rather than having as the base >> > >> >> description of information >> > >> >> > items some prose in a document, you instead have a >> > machine readable >> > >> >> > representation. >> > >> >> > >> > >> >> > >> > >> >> > As an example of a formal (but currently >> > incomplete) information >> > >> >> > model for IPFIX, see: >> > >> >> > http://www.ipdr.org/documents/ipfix/ipfixService-20020902.xml >> > >> >> > >> > >> >> > I would recommend making such a representation the >> > normative form >> > >> >> > of the IPFIX information model, much like SNMP MIBs. >> > >> >> > >> > >> >> > >> > >> >> > Please let me know if people see value in this exercise. >> > >> >> > >> > >> >> > Regards, >> > >> >> > >> > >> >> > Jeff Meyer >> > >> >> > >> > >> >> >> -----Original Message----- >> > >> >> >> From: Nevil Brownlee [mailto:n.brownlee@auckland.ac.nz] >> > >> >> >> Sent: Sunday, April 27, 2003 8:44 PM >> > >> >> >> To: ipfix@net.doit.wisc.edu >> > >> >> >> Subject: [ipfix] Editors for IPFIX drafts >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> Hello all: >> > >> >> >> >> > >> >> >> Following the IPFIX meeting at the San Francisco >> > IETF we asked for >> > >> >> >> volunteers to edit the IPFIX drafts. The current list is: >> > >> >> >> >> > >> >> >> Applicability: Tanja Zseby, Reinaldo Penno >> > >> >> >> >> > >> >> >> Architecture: Ganesh Sadasivan, Nevil Brownlee >> > >> >> >> >> > >> >> >> Information Model: Paul Calato, Jeff Meyer, Juergen Quittek >> > >> >> >> >> > >> >> >> Protocol: Mark Fulmer, Paul Calato, Reinaldo Penno >> > >> >> >> >> > >> >> >> At this stage we already have old (expired) versions of the >> > >> >> >> Architecture (-02.txt) and Data Model (-01.txt) >> > drafts on the IPFIX >> > >> >> >> web page; would the editors of all four drafts please email >> > >> >> >> me an initial >> > >> >> >> version of their new drafts to published as placeholders >> > >> >> >> within the next >> > >> >> >> few weeks. >> > >> >> >> >> > >> >> >> After that, the editors will work on developing better >> > >> >> versions of the >> > >> >> >> drafts which can be discussed on the IPFIX list and submitted >> > >> >> >> by mid-June, >> > >> >> >> i.e. in plenty of time for the Vienna IETF meeting in July. >> > >> >> >> >> > >> >> >> Cheers, Nevil >> > >> >> >> >> > >> >> >> PS: Good to see some real discussion on the list. Lets >> > >> >> concentrate on >> > >> >> >> producing text the draft editors can use! >> > >> >> >> >> > >> >> >> >> > -------------------------------------------------------------- >> > >> >> >> --------- >> > >> >> >> Nevil Brownlee Director, Technology >> > >> >> Development >> > >> >> >> Phone: +64 9 373 7599 x88941 ITSS, The University >> > >> >> of Auckland >> > >> >> >> FAX: +64 9 373 7021 Private Bag 92019, Auckland, >> > >> >> New Zealand >> > >> >> >> >> > >> >> >> >> > >> >> >> ------------------------------------------------- >> > >> >> >> This mail sent through University of Auckland >> > >> >> >> http://www.auckland.ac.nz/ >> > >> >> >> >> > >> >> >> -- >> > >> >> >> Help mailto:majordomo@net.doit.wisc.edu and say "help" >> > >> >> >> in message body >> > >> >> >> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >> > >> >> >> "unsubscribe ipfix" in message body >> > >> >> >> Archive http://ipfix.doit.wisc.edu/archive/ >> > >> >> >> >> > >> >> > >> > >> >> > -- >> > >> >> > Help mailto:majordomo@net.doit.wisc.edu and say >> > >> >> "help" in message body >> > >> >> > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >> > >> >> > "unsubscribe ipfix" in message body >> > >> >> > Archive http://ipfix.doit.wisc.edu/archive/ >> > >> >> >> > >> >> >> > >> >> > >> -- >> > >> Help mailto:majordomo@net.doit.wisc.edu and say >> > "help" in message body >> > >> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >> > >> "unsubscribe ipfix" in message body >> > >> Archive http://ipfix.doit.wisc.edu/archive/ >> > >> > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon May 12 07:16:06 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id HAA08146 for ; Mon, 12 May 2003 07:16:06 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19FAph-00060U-00 for ipfix-list@mil.doit.wisc.edu; Mon, 12 May 2003 05:47:01 -0500 Received: from tokyo.ccrle.nec.de ([195.37.70.2]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19FApf-00060A-00 for ipfix@net.doit.wisc.edu; Mon, 12 May 2003 05:46:59 -0500 Received: from venus.office (venus.office [10.1.1.11]) by tokyo.ccrle.nec.de (8.12.9/8.12.8) with ESMTP id h4CAkqVI025123; Mon, 12 May 2003 12:46:53 +0200 (CEST) Received: from ccrle.nec.de (molina.office [10.1.1.126]) by venus.office (Postfix on SuSE Linux eMail Server 3.0) with ESMTP id BCEAB1AAB4; Mon, 12 May 2003 12:38:31 +0200 (CEST) Message-ID: <3EBF7B9E.60CCB3CB@ccrle.nec.de> Date: Mon, 12 May 2003 12:46:54 +0200 From: Maurizio Molina X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Reinaldo Penno Cc: "'ipfix@net.doit.wisc.edu'" , "'Mark Fullmer'" , calato@riverstonenet.com Subject: Re: [ipfix] Lost Flow Records - Synch Message References: <0A11633F61BD9F40B43ABCC694004F93F18F5D@zsc3c026.us.nortel.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Reinaldo Penno wrote: > > > The editors of the protocol draft had a conf call and we came up with > the main list of issues we need to sort out at this point. > > The first issue we need to get closure is the method to account for > lost flow records. The proposal is in line with the "do not make > assumption that the transport will be TCP". This does not preclude > and/or is intended to fix the higher reliability extensions > requirement. Mark will send an email with a proposal on this subject I just wonder whether it makes sense discussing the details listed below before the above mentioned proposal is circulated. Otherwise, we'd risk to open a discussion with all the participants havig their (own) idea about how the listed counters may be used by a protocol to achieve reliability. Regards, Maurizio > > > The proposal on the table is to send a IPFix Sync message periodically > (periodicity is configurable), with the following information (aside > the standard IPFix header) > > * Number of flow records sent (for each template?) > * Packets and bytes sent (for each template?) > > Please advise if this is okay/not okay, etc, etc > > Regards, > > Reinaldo -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon May 12 08:48:46 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA10759 for ; Mon, 12 May 2003 08:48:46 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19FCaS-0000W8-00 for ipfix-list@mil.doit.wisc.edu; Mon, 12 May 2003 07:39:24 -0500 Received: from tokyo.ccrle.nec.de ([195.37.70.2]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19FCaP-0000W1-00 for ipfix@net.doit.wisc.edu; Mon, 12 May 2003 07:39:21 -0500 Received: from venus.office (venus.office [10.1.1.11]) by tokyo.ccrle.nec.de (8.12.9/8.12.8) with ESMTP id h4CCd2VI033570; Mon, 12 May 2003 14:39:02 +0200 (CEST) Received: from ccrle.nec.de (molina.office [10.1.1.126]) by venus.office (Postfix on SuSE Linux eMail Server 3.0) with ESMTP id 7F1ECF835; Mon, 12 May 2003 14:30:40 +0200 (CEST) Message-ID: <3EBF95E8.40174AA2@ccrle.nec.de> Date: Mon, 12 May 2003 14:39:04 +0200 From: Maurizio Molina X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Sebastian Zander Cc: Alex Audu , "'ipfix@net.doit.wisc.edu'" Subject: Re: [ipfix] Lost Flow Records - Synch Message References: <0A11633F61BD9F40B43ABCC694004F93F18F5D@zsc3c026.us.nortel.com> <3EBBFA05.E20A5A20@alcatel.com> <3EBEE8F0.3020600@fokus.fraunhofer.de> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Sebastian Zander wrote: > Alex Audu wrote: > > Is the sequence number in the header packet not enough for tracking > > lost flow records? > > Having only the sequence number the collector only would know the number > of packets lost. It would not know the number of flow records lost nor > which flow records had been lost. I think Alex's comment shows that there are different issues and discussing before the _usage_ of the information could be beneficial, as I've asked in a previous mail. Here my 2 cents: To build a reliable information transfer over an unreliable transport (which however in the current requirement draft is NOT mandated) the sequence number in the packet header is probably enough. The number of flow records sent would be on th econtrary useful for the collector to judge on the reliability of the received information (draft-ietf-ipfix-reqs-09.txt, sec. 6.3.2, list points 3-5 (loss of flow records due to data transfer, mainteneance)). In this respect, I'm noting that the mentioned list points 1 and 2 (loss of flow records due to metering or exporting process limitation) would be difficult to quantitatively report. A meter/exporter unable to create a flow record (e.g. because of memory limitations) could be at most able to report the number of packets and bytes that it had to drop, but not the number of _not even created_ flow records. Regards, Maurizio > > > > Alex. > > > > Reinaldo Penno wrote: > > > >> > >> > >> The editors of the protocol draft had a conf call and we came up with > >> the main list of issues we need to sort out at this point. > >> > >> The first issue we need to get closure is the method to account for > >> lost flow records. The proposal is in line with the "do not make > >> assumption that the transport will be TCP". This does not preclude > >> and/or is intended to fix the higher reliability extensions > >> requirement. Mark will send an email with a proposal on this subject > >> > >> The proposal on the table is to send a IPFix Sync message periodically > >> (periodicity is configurable), with the following information (aside > >> the standard IPFix header) > >> > >> * Number of flow records sent (for each template?) > >> * Packets and bytes sent (for each template?) > >> > >> Please advise if this is okay/not okay, etc, etc > >> > >> Regards, > >> > >> Reinaldo > >> > > -- > Sebastian Zander E-mail: zander@fokus.fraunhofer.de > Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 > Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 > D-10589 Berlin, Germany www.fokus.fraunhofer.de/usr/sebastian.zander > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon May 12 10:17:44 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA14669 for ; Mon, 12 May 2003 10:17:43 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19FE2B-0002Yp-00 for ipfix-list@mil.doit.wisc.edu; Mon, 12 May 2003 09:12:07 -0500 Received: from [143.209.238.79] (helo=auds955.usa.alcatel.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19FE29-0002Yi-00 for ipfix@net.doit.wisc.edu; Mon, 12 May 2003 09:12:05 -0500 Received: from alcatel.com (localhost [127.0.0.1]) by auds955.usa.alcatel.com (8.12.9/8.12.9) with ESMTP id h4CEBtHI009904; Mon, 12 May 2003 09:11:55 -0500 (CDT) Message-ID: <3EBFABA7.16C6A071@alcatel.com> Date: Mon, 12 May 2003 09:11:51 -0500 From: Alex Audu X-Mailer: Mozilla 4.72 [en] (WinNT; U) X-Accept-Language: en MIME-Version: 1.0 To: Sebastian Zander CC: Reinaldo Penno , "'ipfix@net.doit.wisc.edu'" Subject: Re: [ipfix] Lost Flow Records - Synch Message References: <0A11633F61BD9F40B43ABCC694004F93F18F5D@zsc3c026.us.nortel.com> <3EBBFA05.E20A5A20@alcatel.com> <3EBEE8F0.3020600@fokus.fraunhofer.de> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit My concern was sending periodic sync messages may take away from bandwidth resources that could be put to better use exporting data. However, the sync message scheme would have minimum impact if the collector initiated it ONLY after detecting a gap in the sequence numbers of packets received. I think this may be a good compromise. Regards, Alex. Sebastian Zander wrote: > Alex Audu wrote: > > Is the sequence number in the header packet not enough for tracking > > lost flow records? > > Having only the sequence number the collector only would know the number > of packets lost. It would not know the number of flow records lost nor > which flow records had been lost. > > > Alex. > > > > Reinaldo Penno wrote: > > > >> > >> > >> The editors of the protocol draft had a conf call and we came up with > >> the main list of issues we need to sort out at this point. > >> > >> The first issue we need to get closure is the method to account for > >> lost flow records. The proposal is in line with the "do not make > >> assumption that the transport will be TCP". This does not preclude > >> and/or is intended to fix the higher reliability extensions > >> requirement. Mark will send an email with a proposal on this subject > >> > >> The proposal on the table is to send a IPFix Sync message periodically > >> (periodicity is configurable), with the following information (aside > >> the standard IPFix header) > >> > >> * Number of flow records sent (for each template?) > >> * Packets and bytes sent (for each template?) > >> > >> Please advise if this is okay/not okay, etc, etc > >> > >> Regards, > >> > >> Reinaldo > >> > > -- > Sebastian Zander E-mail: zander@fokus.fraunhofer.de > Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 > Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 > D-10589 Berlin, Germany www.fokus.fraunhofer.de/usr/sebastian.zander -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon May 12 11:00:02 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA17699 for ; Mon, 12 May 2003 11:00:02 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19FEiA-0003Pm-00 for ipfix-list@mil.doit.wisc.edu; Mon, 12 May 2003 09:55:30 -0500 Received: from usmail.xacct.com ([204.253.100.12]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19FEi8-0003PZ-00 for ipfix@net.doit.wisc.edu; Mon, 12 May 2003 09:55:28 -0500 Received: from Givoly (000-023-769.area1.spcsdns.net [68.24.93.148]) by usmail.xacct.com (8.11.2/8.11.2) with SMTP id h4CF1gi16401; Mon, 12 May 2003 08:01:43 -0700 From: "Tal Givoly" To: "Mark Fullmer" , "Sebastian Zander" Cc: Subject: RE: [ipfix] Lost Flow Records - Synch Message Date: Mon, 12 May 2003 07:54:54 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-Reply-To: <20030511221348.A87500@net.ohio-state.edu> Importance: Normal Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hi, Wasn't the suggestion of the evaluation committee and the WG chairs to begin with a TCP version of the protocol and later, perhaps, add SCTP (or SCTP-PR)? If that is the case, why are we discussing the behavior of a UDP version of it as that should not be part of the IPFIX standard at all. Regarding the TCP below, why wouldn't the templates be transmitted upon re-establishing a connection or fail-over to avoid the unnecessary complexities in the collector of receiving data for which it has not information regarding its template? = Tal -----Original Message----- From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu]On Behalf Of Mark Fullmer Sent: Sunday, May 11, 2003 7:14 PM To: Sebastian Zander Cc: 'ipfix@net.doit.wisc.edu' Subject: Re: [ipfix] Lost Flow Records - Synch Message How lost templates are handled depends on the transport. UDP - templates are periodically retransmitted so a collector should buffer data until the template arrives or resource exhaustion. TCP - Templates shouldn't be lost. There is a small window for a template to be lost in a primary/secondary failover situation. The failover details have not been finalized yet. SCPT - Hopefully the templates can be done over a reliable channel. Still have failover issues. mark On Mon, May 12, 2003 at 03:09:04AM +0200, Sebastian Zander wrote: > Hi Reinaldo, > > having flow record counters per template is a very nice thing in my > opinion. But if we have this information why would we need the packets > and bytes sent per template information in addition? > > What about information about lost template definitions? (A loss of a > template def could potentially cause flow records to be "lost" at the > collector...) > > Cheers, > > Sebastian > > Reinaldo Penno wrote: > > The editors of the protocol draft had a conf call and we came up with > > the main list of issues we need to sort out at this point. > > > > The first issue we need to get closure is the method to account for lost > > flow records. The proposal is in line with the "do not make assumption > > that the transport will be TCP". This does not preclude and/or is > > intended to fix the higher reliability extensions requirement. Mark will > > send an email with a proposal on this subject > > > > The proposal on the table is to send a IPFix Sync message periodically > > (periodicity is configurable), with the following information (aside the > > standard IPFix header) > > > > * Number of flow records sent (for each template?) > > * Packets and bytes sent (for each template?) > > > > Please advise if this is okay/not okay, etc, etc > > > > Regards, > > > > Reinaldo > > > > > -- > Sebastian Zander E-mail: zander@fokus.fraunhofer.de > Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 > Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 > D-10589 Berlin, Germany www.fokus.fraunhofer.de/usr/sebastian.zander > > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon May 12 11:25:02 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA19537 for ; Mon, 12 May 2003 11:25:01 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19FF6Y-0003zx-00 for ipfix-list@mil.doit.wisc.edu; Mon, 12 May 2003 10:20:42 -0500 Received: from eng4.oar.net ([192.148.244.24]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 19FF6X-0003zr-00 for ipfix@net.doit.wisc.edu; Mon, 12 May 2003 10:20:41 -0500 Received: (qmail 90539 invoked by uid 4454); 12 May 2003 15:20:40 -0000 Date: Mon, 12 May 2003 11:20:40 -0400 From: Mark Fullmer To: Tal Givoly Cc: Sebastian Zander , ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Lost Flow Records - Synch Message Message-ID: <20030512112040.A90471@net.ohio-state.edu> References: <20030511221348.A87500@net.ohio-state.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: ; from givoly@xacct.com on Mon, May 12, 2003 at 07:54:54AM -0700 Precedence: bulk Sender: majordomo listserver I'm only using UDP as an example of a lossy transport. We're supposed to be transport agnostic. The templates could be retransmitted on failover, and that's probably the easiest thing to do. Like I said the details have not been worked out yet. Things like when to fail over, how to fail back, how to ensure stability of the failover mechanism (prevent oscillations), do the collectors know about each other, does the exporter open connections to all the potential collectors and keep them primed with template info, do we support load balancing among collectors, etc. mark On Mon, May 12, 2003 at 07:54:54AM -0700, Tal Givoly wrote: > Hi, > > Wasn't the suggestion of the evaluation committee and the WG chairs to begin > with a TCP version of the protocol and later, perhaps, add SCTP (or > SCTP-PR)? If that is the case, why are we discussing the behavior of a UDP > version of it as that should not be part of the IPFIX standard at all. > > Regarding the TCP below, why wouldn't the templates be transmitted upon > re-establishing a connection or fail-over to avoid the unnecessary > complexities in the collector of receiving data for which it has not > information regarding its template? > = > > Tal > > -----Original Message----- > From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu]On Behalf > Of Mark Fullmer > Sent: Sunday, May 11, 2003 7:14 PM > To: Sebastian Zander > Cc: 'ipfix@net.doit.wisc.edu' > Subject: Re: [ipfix] Lost Flow Records - Synch Message > > > How lost templates are handled depends on the transport. > > UDP - templates are periodically retransmitted so a collector should > buffer data until the template arrives or resource exhaustion. > > TCP - Templates shouldn't be lost. There is a small window for > a template to be lost in a primary/secondary failover > situation. The failover details have not been finalized > yet. > > SCPT - Hopefully the templates can be done over a reliable > channel. Still have failover issues. > > mark > > On Mon, May 12, 2003 at 03:09:04AM +0200, Sebastian Zander wrote: > > Hi Reinaldo, > > > > having flow record counters per template is a very nice thing in my > > opinion. But if we have this information why would we need the packets > > and bytes sent per template information in addition? > > > > What about information about lost template definitions? (A loss of a > > template def could potentially cause flow records to be "lost" at the > > collector...) > > > > Cheers, > > > > Sebastian > > > > Reinaldo Penno wrote: > > > The editors of the protocol draft had a conf call and we came up with > > > the main list of issues we need to sort out at this point. > > > > > > The first issue we need to get closure is the method to account for lost > > > flow records. The proposal is in line with the "do not make assumption > > > that the transport will be TCP". This does not preclude and/or is > > > intended to fix the higher reliability extensions requirement. Mark will > > > send an email with a proposal on this subject > > > > > > The proposal on the table is to send a IPFix Sync message periodically > > > (periodicity is configurable), with the following information (aside the > > > standard IPFix header) > > > > > > * Number of flow records sent (for each template?) > > > * Packets and bytes sent (for each template?) > > > > > > Please advise if this is okay/not okay, etc, etc > > > > > > Regards, > > > > > > Reinaldo > > > > > > > > > -- > > Sebastian Zander E-mail: > zander@fokus.fraunhofer.de > > Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 > > Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 > > D-10589 Berlin, Germany > www.fokus.fraunhofer.de/usr/sebastian.zander > > > > > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message > body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon May 12 11:25:53 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA19607 for ; Mon, 12 May 2003 11:25:53 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19FF5f-0003yU-00 for ipfix-list@mil.doit.wisc.edu; Mon, 12 May 2003 10:19:47 -0500 Received: from kanfw1.ottawa.alcatel.ca ([192.75.23.69] helo=kanmx2.ca.alcatel.com) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 19FF5d-0003yO-00 for ipfix@net.doit.wisc.edu; Mon, 12 May 2003 10:19:45 -0500 Received: (qmail 16570 invoked from network); 12 May 2003 15:22:13 -0000 Received: from unknown (HELO alcatel.com) (138.120.51.110) by kanmx2.ca.alcatel.com with SMTP; 12 May 2003 15:22:13 -0000 Message-ID: <3EBFBB75.87B6E78C@alcatel.com> Date: Mon, 12 May 2003 11:19:17 -0400 From: Mark Thibodeau X-Mailer: Mozilla 4.78 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Tal Givoly CC: Mark Fullmer , Sebastian Zander , ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Lost Flow Records - Synch Message References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit I agree with Tal. With TCP, we do not have to worry about losing templates. For the lost flow statistics, a sequence number and exported flow count should be adequate. The collector can use the sequence number to detect that a packet has been lost. The total number of flows exported can be used to determine how many flows have been lost. It could be that this is redundant information though. With just the total flow count, the collector should be able to figure everything out. But, the sequence numbers make things a bit easier. I vote that we keep both mechanisms. Cheers, Mark Tal Givoly wrote: > Hi, > > Wasn't the suggestion of the evaluation committee and the WG chairs to begin > with a TCP version of the protocol and later, perhaps, add SCTP (or > SCTP-PR)? If that is the case, why are we discussing the behavior of a UDP > version of it as that should not be part of the IPFIX standard at all. > > Regarding the TCP below, why wouldn't the templates be transmitted upon > re-establishing a connection or fail-over to avoid the unnecessary > complexities in the collector of receiving data for which it has not > information regarding its template? > = > > Tal > > -----Original Message----- > From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu]On Behalf > Of Mark Fullmer > Sent: Sunday, May 11, 2003 7:14 PM > To: Sebastian Zander > Cc: 'ipfix@net.doit.wisc.edu' > Subject: Re: [ipfix] Lost Flow Records - Synch Message > > How lost templates are handled depends on the transport. > > UDP - templates are periodically retransmitted so a collector should > buffer data until the template arrives or resource exhaustion. > > TCP - Templates shouldn't be lost. There is a small window for > a template to be lost in a primary/secondary failover > situation. The failover details have not been finalized > yet. > > SCPT - Hopefully the templates can be done over a reliable > channel. Still have failover issues. > > mark > > On Mon, May 12, 2003 at 03:09:04AM +0200, Sebastian Zander wrote: > > Hi Reinaldo, > > > > having flow record counters per template is a very nice thing in my > > opinion. But if we have this information why would we need the packets > > and bytes sent per template information in addition? > > > > What about information about lost template definitions? (A loss of a > > template def could potentially cause flow records to be "lost" at the > > collector...) > > > > Cheers, > > > > Sebastian > > > > Reinaldo Penno wrote: > > > The editors of the protocol draft had a conf call and we came up with > > > the main list of issues we need to sort out at this point. > > > > > > The first issue we need to get closure is the method to account for lost > > > flow records. The proposal is in line with the "do not make assumption > > > that the transport will be TCP". This does not preclude and/or is > > > intended to fix the higher reliability extensions requirement. Mark will > > > send an email with a proposal on this subject > > > > > > The proposal on the table is to send a IPFix Sync message periodically > > > (periodicity is configurable), with the following information (aside the > > > standard IPFix header) > > > > > > * Number of flow records sent (for each template?) > > > * Packets and bytes sent (for each template?) > > > > > > Please advise if this is okay/not okay, etc, etc > > > > > > Regards, > > > > > > Reinaldo > > > > > > > > > -- > > Sebastian Zander E-mail: > zander@fokus.fraunhofer.de > > Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 > > Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 > > D-10589 Berlin, Germany > www.fokus.fraunhofer.de/usr/sebastian.zander > > > > > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message > body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- **************************************************** * Mark Thibodeau, P Eng. * 670 RSP Development - Firmware Designer * Alcatel CID - (613) 784-5375 * Fax - (613) 599-3696 * mark.thibodeau@alcatel.com **************************************************** -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon May 12 11:32:47 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA20756 for ; Mon, 12 May 2003 11:32:46 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19FFCf-00049g-00 for ipfix-list@mil.doit.wisc.edu; Mon, 12 May 2003 10:27:01 -0500 Received: from eng4.oar.net ([192.148.244.24]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 19FFCd-00049L-00 for ipfix@net.doit.wisc.edu; Mon, 12 May 2003 10:26:59 -0500 Received: (qmail 90572 invoked by uid 4454); 12 May 2003 15:26:59 -0000 Date: Mon, 12 May 2003 11:26:59 -0400 From: Mark Fullmer To: Alex Audu Cc: Sebastian Zander , Reinaldo Penno , "'ipfix@net.doit.wisc.edu'" Subject: Re: [ipfix] Lost Flow Records - Synch Message Message-ID: <20030512112659.B90471@net.ohio-state.edu> References: <0A11633F61BD9F40B43ABCC694004F93F18F5D@zsc3c026.us.nortel.com> <3EBBFA05.E20A5A20@alcatel.com> <3EBEE8F0.3020600@fokus.fraunhofer.de> <3EBFABA7.16C6A071@alcatel.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: <3EBFABA7.16C6A071@alcatel.com>; from Alex.Audu@alcatel.com on Mon, May 12, 2003 at 09:11:51AM -0500 Precedence: bulk Sender: majordomo listserver The synch message could be sent with a user configurable period. I was thinking the default would be O(minutes). I generate graphs from 5 minute updates right now (v5 deployments) which seems to work well. mark On Mon, May 12, 2003 at 09:11:51AM -0500, Alex Audu wrote: > My concern was sending periodic sync messages may take away from bandwidth resources > that could be put to better use exporting data. However, the sync message scheme would > have minimum impact if the collector initiated it ONLY after detecting a gap in the > sequence > numbers of packets received. I think this may be a good compromise. > > Regards, > Alex. > > Sebastian Zander wrote: > > > Alex Audu wrote: > > > Is the sequence number in the header packet not enough for tracking > > > lost flow records? > > > > Having only the sequence number the collector only would know the number > > of packets lost. It would not know the number of flow records lost nor > > which flow records had been lost. > > > > > Alex. > > > > > > Reinaldo Penno wrote: > > > > > >> > > >> > > >> The editors of the protocol draft had a conf call and we came up with > > >> the main list of issues we need to sort out at this point. > > >> > > >> The first issue we need to get closure is the method to account for > > >> lost flow records. The proposal is in line with the "do not make > > >> assumption that the transport will be TCP". This does not preclude > > >> and/or is intended to fix the higher reliability extensions > > >> requirement. Mark will send an email with a proposal on this subject > > >> > > >> The proposal on the table is to send a IPFix Sync message periodically > > >> (periodicity is configurable), with the following information (aside > > >> the standard IPFix header) > > >> > > >> * Number of flow records sent (for each template?) > > >> * Packets and bytes sent (for each template?) > > >> > > >> Please advise if this is okay/not okay, etc, etc > > >> > > >> Regards, > > >> > > >> Reinaldo > > >> > > > > -- > > Sebastian Zander E-mail: zander@fokus.fraunhofer.de > > Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 > > Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 > > D-10589 Berlin, Germany www.fokus.fraunhofer.de/usr/sebastian.zander > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon May 12 12:04:49 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA22303 for ; Mon, 12 May 2003 12:04:49 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19FFaS-0004ii-00 for ipfix-list@mil.doit.wisc.edu; Mon, 12 May 2003 10:51:36 -0500 Received: from atlrel6.hp.com ([156.153.255.205]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19FFaR-0004ib-00 for ipfix@net.doit.wisc.edu; Mon, 12 May 2003 10:51:35 -0500 Received: from xatlrelay1.atl.hp.com (xatlrelay1.atl.hp.com [15.45.89.190]) by atlrel6.hp.com (Postfix) with ESMTP id 7E4A61C00C7B; Mon, 12 May 2003 11:51:34 -0400 (EDT) Received: from xatlbh4.atl.hp.com (xatlbh4.atl.hp.com [15.45.89.189]) by xatlrelay1.atl.hp.com (Postfix) with ESMTP id 06A141C000A0; Mon, 12 May 2003 11:51:34 -0400 (EDT) Received: by xatlbh4.atl.hp.com with Internet Mail Service (5.5.2655.55) id ; Mon, 12 May 2003 11:51:33 -0400 Message-ID: <1D3D2C371FCBD947A7897FABBD3533A566BAD3@xsun01.ptp.hp.com> From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" To: "'Juergen Quittek'" , calato@riverstonenet.com, "MEYER,JEFFREY D (HP-Cupertino,ex1)" Cc: "'Nevil Brownlee'" , ipfix@net.doit.wisc.edu Subject: RE: [ipfix] Editors for IPFIX drafts Date: Mon, 12 May 2003 11:51:23 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2655.55) Content-Type: text/plain; charset="iso-8859-1" Precedence: bulk Sender: majordomo listserver Juergen, I'm using a basic text editor. Since there are typical bounding markup tags I typically cut/paste blocks of tag and fill in the meaningful detail. Not fancy, but I've found it workable. -- Jeff > -----Original Message----- > From: Juergen Quittek [mailto:quittek@ccrle.nec.de] > Sent: Monday, May 12, 2003 3:53 AM > To: calato@riverstonenet.com; MEYER,JEFFREY D (HP-Cupertino,ex1) > Cc: 'Nevil Brownlee'; ipfix@net.doit.wisc.edu > Subject: Re: [ipfix] Editors for IPFIX drafts > > > Paul, > > From my experience, it does not make sense to write a largish > XML specification without an XML editor, that automates > typing of 80% of the lengthy XML DTD or schema syntax. > > Just using a Word or ASCII editor would be a significant > waste of time. > > I'm using a commercial editor (TurboXML) but there are certainly > also free ones. Does anyone know a free tool? Jeff? > > Cheers, > > Juergen > > -- calato@riverstonenet.com wrote on 06 May 2003 11:00 -0400: > > > > > Seems like a reasonable model. I'd say lets head that way > > and see how it goes. > > > > Are there any MS word templates for creating the XML part? > > > > > > Paul > > > > "MEYER,JEFFREY D (HP-Cupertino,ex1)" wrote: > >> > >> Hi, > >> > >> If the human readable part is generated from the XML appendix, > >> then I guess it doesn't matter which part you call normative, > >> since they both originate from the same source. > >> > >> The development model I see is the following: > >> > >> Generate XML Info Model > >> Run model through XSLT (xalan) translater to produce prose > >> (consistent with RFC 2629) [I'll volunteer to write the > >> style sheet translation) > >> Incorporate the raw model as appendix and the generated prose > >> description into the appropriate section (consistent w/ > >> RFC 2629) along with other text of the Information > Model Draft. > >> Run Marshall's tool http://xml.resource.org/ to > produce nroff style > >> and HTML versions of document. Submit nroff style to > RFC editor, > >> post HTML style in addition to ipfix website (easier > to navigate). > >> > >> For examples of the dual formatted output, see the IPDR > submissions > >> to IPDR located at: > >> > >> http://www.ipdr.org/documents/ipfix > >> > >> Regards, > >> > >> Jeff Meyer > >> > >> > -----Original Message----- > >> > From: Juergen Quittek [mailto:quittek@ccrle.nec.de] > >> > Sent: Thursday, May 01, 2003 8:21 AM > >> > To: calato@riverstonenet.com > >> > Cc: MEYER,JEFFREY D (HP-Cupertino,ex1); 'Nevil Brownlee'; > >> > ipfix@net.doit.wisc.edu > >> > Subject: Re: [ipfix] Editors for IPFIX drafts > >> > > >> > > >> > Paul, > >> > > >> > I agree to Jeff, that there is some value in having an XML > >> > representation > >> > of the specification. However, in the end this document will > >> > become an RFC > >> > and it must be well readable to human beings. > >> > > >> > Therefore, I think that the main body must be human readable, > >> > but we can > >> > provide an appendix, that contains an equivalent machine > >> > readable spec. > >> > > >> > Different to Jeff, I think that the hunam readable main > body should be > >> > the normative part, while the XML appendix is just given as > >> > support for > >> > users of XML tools allowing them to genrate protocol stacks > >> > and/or tools > >> > for consistency checking. > >> > > >> > As Jeff suggested, we can work with an XML representation > >> > when developing > >> > the spec and generate human readable text automatically from > >> > the XML code. > >> > This will help us avoiding fomal mistakes in the human > >> > readable spec and we > >> > would always have consistency between human readable and > XML part. > >> > > >> > Juergen > >> > > >> > > >> > -- calato@riverstonenet.com wrote on 01 May 2003 11:00 -0400: > >> > > >> > > > >> > > I'm a little confused. Are we talking about XML descriptions > >> > > as an appendix with prose as the main body or XML as the > >> > > main body with tools translating it into a more readable > >> > > form? Or XML as an appendix with tools translating it for > >> > > use as sections mixed in with the prose? > >> > > > >> > > > >> > > Paul > >> > > > >> > > Juergen Quittek wrote: > >> > >> > >> > >> Jeff, > >> > >> > >> > >> -- MEYER,JEFFREY D (HP-Cupertino,ex1) wrote on 28 April > >> > 2003 09:20 -0700: > >> > >> > > >> > >> > Juergen, > >> > >> > > >> > >> > Thanks for the response. > >> > >> > > >> > >> > I agree XML in and of itself is marginally human readable. > >> > >> > > >> > >> > I think that basic tools such as Xalan > >> > (http://xml.apache.org/), can > >> > >> > be used to "transform" a base XML document into a > >> > variety of formats, > >> > >> > including formatting consistent with RFC's. Note that > >> > all the IPDR > >> > >> > RFC's were written in XML using Marshall Rose's > xml2rfc tools > >> > >> > (http://xml.resource.org/). > >> > >> > >> > >> I also like Marshall's tools. > >> > >> If Paul agrees, we should use them for the data model > document. > >> > >> > >> > >> > So the model I would picture is having the Normative > >> > XML data model > >> > >> > as an appendix, and having generated information from > >> > this data model > >> > >> > populate some of the body sections to aid human readers. > >> > >> > >> > >> This looks like a reasonable way to go. > >> > >> > >> > >> Juergen > >> > >> > >> > >> > Regards, > >> > >> > > >> > >> > Jeff Meyer > >> > >> > > >> > >> >> -----Original Message----- > >> > >> >> From: Juergen Quittek [mailto:quittek@ccrle.nec.de] > >> > >> >> Sent: Monday, April 28, 2003 8:24 AM > >> > >> >> To: MEYER,JEFFREY D (HP-Cupertino,ex1); 'Nevil Brownlee'; > >> > >> >> ipfix@net.doit.wisc.edu > >> > >> >> Subject: RE: [ipfix] Editors for IPFIX drafts > >> > >> >> > >> > >> >> > >> > >> >> Jeff, > >> > >> >> > >> > >> >> I do see a value in such a formal description. But > there is a > >> > >> >> trafe-off. > >> > >> >> > >> > >> >> I also see a degradation of human readability. You > >> > loose means of > >> > >> >> structuring your document text when using XML, for > >> > example grouping > >> > >> >> attributes that are related. (Of course you could do so in > >> > >> >> XML, but it is > >> > >> >> much harder to find the heading "Port-related > Attributes" in > >> > >> >> an XML document, > >> > >> >> because headings cannot be well highlighted and > numbering of > >> > >> >> headings is > >> > >> >> not well supported. > >> > >> >> > >> > >> >> Anyway, I like the idea of an XML encoding of the > model, but > >> > >> >> I we should > >> > >> >> not go this way if we cannot find a good way of getting it > >> > >> >> hunam readable > >> > >> >> (maybe by applying formatting rules). > >> > >> >> > >> > >> >> An option would also be adding the XML code to the document > >> > >> >> as an appendix. > >> > >> >> > >> > >> >> Juergen > >> > >> >> > >> > >> >> > >> > >> >> -- MEYER,JEFFREY D (HP-Cupertino,ex1) wrote on 28 > April 2003 > >> > >> >> 10:57 -0400: > >> > >> >> > >> > >> >> > Hi, > >> > >> >> > > >> > >> >> > My particular interest on working on the > >> > information model would > >> > >> >> > be on developing a more formal and standard way of > >> > describing the > >> > >> >> > information elements which are used in IPFIX > communication. > >> > >> >> > > >> > >> >> > The modelling of information elements should > >> > ideally be done > >> > >> >> > using a formal description language, so that tools can be > >> > >> >> constructed > >> > >> >> > which do not require humans to transcribe this > information. > >> > >> >> > > >> > >> >> > I would propose the use of XML-Schema to provide > >> > this formal > >> > >> >> > description. Through the use of XSL processors, prose > >> > >> >> based descriptions > >> > >> >> > and tables can be produced if these are > considered desireable. > >> > >> >> > > >> > >> >> > NOTE: describing the information model formally > >> > does NOT mean > >> > >> >> > that encoding and transport for IPFIX need to change > >> > in any way. It > >> > >> >> > simply means that rather than having as the base > >> > >> >> description of information > >> > >> >> > items some prose in a document, you instead have a > >> > machine readable > >> > >> >> > representation. > >> > >> >> > > >> > >> >> > > >> > >> >> > As an example of a formal (but currently > >> > incomplete) information > >> > >> >> > model for IPFIX, see: > >> > >> >> > > http://www.ipdr.org/documents/ipfix/ipfixService-20020902.xml > >> > >> >> > > >> > >> >> > I would recommend making such a representation the > >> > normative form > >> > >> >> > of the IPFIX information model, much like SNMP MIBs. > >> > >> >> > > >> > >> >> > > >> > >> >> > Please let me know if people see value in this > exercise. > >> > >> >> > > >> > >> >> > Regards, > >> > >> >> > > >> > >> >> > Jeff Meyer > >> > >> >> > > >> > >> >> >> -----Original Message----- > >> > >> >> >> From: Nevil Brownlee [mailto:n.brownlee@auckland.ac.nz] > >> > >> >> >> Sent: Sunday, April 27, 2003 8:44 PM > >> > >> >> >> To: ipfix@net.doit.wisc.edu > >> > >> >> >> Subject: [ipfix] Editors for IPFIX drafts > >> > >> >> >> > >> > >> >> >> > >> > >> >> >> > >> > >> >> >> Hello all: > >> > >> >> >> > >> > >> >> >> Following the IPFIX meeting at the San Francisco > >> > IETF we asked for > >> > >> >> >> volunteers to edit the IPFIX drafts. The > current list is: > >> > >> >> >> > >> > >> >> >> Applicability: Tanja Zseby, Reinaldo Penno > >> > >> >> >> > >> > >> >> >> Architecture: Ganesh Sadasivan, Nevil Brownlee > >> > >> >> >> > >> > >> >> >> Information Model: Paul Calato, Jeff Meyer, > Juergen Quittek > >> > >> >> >> > >> > >> >> >> Protocol: Mark Fulmer, Paul Calato, Reinaldo Penno > >> > >> >> >> > >> > >> >> >> At this stage we already have old (expired) > versions of the > >> > >> >> >> Architecture (-02.txt) and Data Model (-01.txt) > >> > drafts on the IPFIX > >> > >> >> >> web page; would the editors of all four drafts > please email > >> > >> >> >> me an initial > >> > >> >> >> version of their new drafts to published as placeholders > >> > >> >> >> within the next > >> > >> >> >> few weeks. > >> > >> >> >> > >> > >> >> >> After that, the editors will work on developing better > >> > >> >> versions of the > >> > >> >> >> drafts which can be discussed on the IPFIX list > and submitted > >> > >> >> >> by mid-June, > >> > >> >> >> i.e. in plenty of time for the Vienna IETF > meeting in July. > >> > >> >> >> > >> > >> >> >> Cheers, Nevil > >> > >> >> >> > >> > >> >> >> PS: Good to see some real discussion on the list. Lets > >> > >> >> concentrate on > >> > >> >> >> producing text the draft editors can use! > >> > >> >> >> > >> > >> >> >> > >> > -------------------------------------------------------------- > >> > >> >> >> --------- > >> > >> >> >> Nevil Brownlee Director, Technology > >> > >> >> Development > >> > >> >> >> Phone: +64 9 373 7599 x88941 ITSS, The University > >> > >> >> of Auckland > >> > >> >> >> FAX: +64 9 373 7021 Private Bag 92019, Auckland, > >> > >> >> New Zealand > >> > >> >> >> > >> > >> >> >> > >> > >> >> >> ------------------------------------------------- > >> > >> >> >> This mail sent through University of Auckland > >> > >> >> >> http://www.auckland.ac.nz/ > >> > >> >> >> > >> > >> >> >> -- > >> > >> >> >> Help mailto:majordomo@net.doit.wisc.edu > and say "help" > >> > >> >> >> in message body > >> > >> >> >> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > >> > >> >> >> "unsubscribe ipfix" in message body > >> > >> >> >> Archive http://ipfix.doit.wisc.edu/archive/ > >> > >> >> >> > >> > >> >> > > >> > >> >> > -- > >> > >> >> > Help mailto:majordomo@net.doit.wisc.edu and say > >> > >> >> "help" in message body > >> > >> >> > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > >> > >> >> > "unsubscribe ipfix" in message body > >> > >> >> > Archive http://ipfix.doit.wisc.edu/archive/ > >> > >> >> > >> > >> >> > >> > >> > >> > >> -- > >> > >> Help mailto:majordomo@net.doit.wisc.edu and say > >> > "help" in message body > >> > >> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > >> > >> "unsubscribe ipfix" in message body > >> > >> Archive http://ipfix.doit.wisc.edu/archive/ > >> > > >> > > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon May 12 12:06:37 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA22441 for ; Mon, 12 May 2003 12:06:37 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19FFkN-0004wd-00 for ipfix-list@mil.doit.wisc.edu; Mon, 12 May 2003 11:01:51 -0500 Received: from usmail.xacct.com ([204.253.100.12]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19FFkM-0004wW-00 for ipfix@net.doit.wisc.edu; Mon, 12 May 2003 11:01:50 -0500 Received: from Givoly (000-023-769.area1.spcsdns.net [68.24.93.148]) by usmail.xacct.com (8.11.2/8.11.2) with SMTP id h4CG83i17294; Mon, 12 May 2003 09:08:04 -0700 From: "Tal Givoly" To: "Mark Fullmer" Cc: "Sebastian Zander" , Subject: RE: [ipfix] Lost Flow Records - Synch Message Date: Mon, 12 May 2003 09:01:15 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-Reply-To: <20030512112040.A90471@net.ohio-state.edu> Importance: Normal Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Mark, But even if we do use SCTP-PR, the "control" information (such as templates, indication of synch messages, etc.) is most likely to be transported over the reliable SCTP link rather than the partially reliable one. This was suggested by Benoit in the IETF meeting. In that case, should we introduce undue complexity by supportinging arbitrary theoretical protocols? Tal -----Original Message----- From: Mark Fullmer [mailto:maf@eng.oar.net] Sent: Monday, May 12, 2003 8:21 AM To: Tal Givoly Cc: Sebastian Zander; ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Lost Flow Records - Synch Message I'm only using UDP as an example of a lossy transport. We're supposed to be transport agnostic. The templates could be retransmitted on failover, and that's probably the easiest thing to do. Like I said the details have not been worked out yet. Things like when to fail over, how to fail back, how to ensure stability of the failover mechanism (prevent oscillations), do the collectors know about each other, does the exporter open connections to all the potential collectors and keep them primed with template info, do we support load balancing among collectors, etc. mark On Mon, May 12, 2003 at 07:54:54AM -0700, Tal Givoly wrote: > Hi, > > Wasn't the suggestion of the evaluation committee and the WG chairs to begin > with a TCP version of the protocol and later, perhaps, add SCTP (or > SCTP-PR)? If that is the case, why are we discussing the behavior of a UDP > version of it as that should not be part of the IPFIX standard at all. > > Regarding the TCP below, why wouldn't the templates be transmitted upon > re-establishing a connection or fail-over to avoid the unnecessary > complexities in the collector of receiving data for which it has not > information regarding its template? > = > > Tal > > -----Original Message----- > From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu]On Behalf > Of Mark Fullmer > Sent: Sunday, May 11, 2003 7:14 PM > To: Sebastian Zander > Cc: 'ipfix@net.doit.wisc.edu' > Subject: Re: [ipfix] Lost Flow Records - Synch Message > > > How lost templates are handled depends on the transport. > > UDP - templates are periodically retransmitted so a collector should > buffer data until the template arrives or resource exhaustion. > > TCP - Templates shouldn't be lost. There is a small window for > a template to be lost in a primary/secondary failover > situation. The failover details have not been finalized > yet. > > SCPT - Hopefully the templates can be done over a reliable > channel. Still have failover issues. > > mark > > On Mon, May 12, 2003 at 03:09:04AM +0200, Sebastian Zander wrote: > > Hi Reinaldo, > > > > having flow record counters per template is a very nice thing in my > > opinion. But if we have this information why would we need the packets > > and bytes sent per template information in addition? > > > > What about information about lost template definitions? (A loss of a > > template def could potentially cause flow records to be "lost" at the > > collector...) > > > > Cheers, > > > > Sebastian > > > > Reinaldo Penno wrote: > > > The editors of the protocol draft had a conf call and we came up with > > > the main list of issues we need to sort out at this point. > > > > > > The first issue we need to get closure is the method to account for lost > > > flow records. The proposal is in line with the "do not make assumption > > > that the transport will be TCP". This does not preclude and/or is > > > intended to fix the higher reliability extensions requirement. Mark will > > > send an email with a proposal on this subject > > > > > > The proposal on the table is to send a IPFix Sync message periodically > > > (periodicity is configurable), with the following information (aside the > > > standard IPFix header) > > > > > > * Number of flow records sent (for each template?) > > > * Packets and bytes sent (for each template?) > > > > > > Please advise if this is okay/not okay, etc, etc > > > > > > Regards, > > > > > > Reinaldo > > > > > > > > > -- > > Sebastian Zander E-mail: > zander@fokus.fraunhofer.de > > Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 > > Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 > > D-10589 Berlin, Germany > www.fokus.fraunhofer.de/usr/sebastian.zander > > > > > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message > body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon May 12 12:07:59 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA22477 for ; Mon, 12 May 2003 12:07:58 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19FFkU-0004wr-00 for ipfix-list@mil.doit.wisc.edu; Mon, 12 May 2003 11:01:58 -0500 Received: from usmail.xacct.com ([204.253.100.12]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19FFkT-0004wm-00 for ipfix@net.doit.wisc.edu; Mon, 12 May 2003 11:01:57 -0500 Received: from Givoly (000-023-769.area1.spcsdns.net [68.24.93.148]) by usmail.xacct.com (8.11.2/8.11.2) with SMTP id h4CG8Di17299; Mon, 12 May 2003 09:08:14 -0700 From: "Tal Givoly" To: "Mark Fullmer" Cc: "Sebastian Zander" , Subject: RE: [ipfix] Lost Flow Records - Synch Message Date: Mon, 12 May 2003 09:01:24 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-Reply-To: <20030512112040.A90471@net.ohio-state.edu> Importance: Normal Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Mark, All those items you mention below regarding the fail-over are very valid items to be discussed. Perhaps we should create a set of items for each of them and discuss them independently rather than all in one lump? It seems to me that the mandatory requirement that the IPFIX protcol be extensible enough to support reliability extensions basically implies that at least one such extension needs to be demonstrated. Therefore, it makes sense to make an attempt to articulate just such an extension (if not standardize it). - when to fail over a) when the window of outbound messages has been reached and an acknowledgement has not been received from the collector. b) when a timeout on an acknowledged message exceeds a pre-determined length of time. c) when the collector sends a message indicating it is incapable of handling more traffic. d) transport protocol disconnection from the collector (TCP link fails). e) keep alives aren't received - if they are added, this would allow a low-throughput link to detect the effective disconnection from the collector before the transport protocol disconnection. In either of these cases, establish connectivity to a back-up collector (optionally as the connection could be already established - see below), retransmit all unacknowledged data records to a stand-by collector; - how to fail back a) periodic retry mechanism to re-establish connection with "primary" collector (introducing notion of priority of collectors is useful as the "secondary" or "back-up" collector could be more regionally co-located and shared by multiple exporters and may need to return to the role of back-up asap. b) failure of the secondary/back-up collector in the same way described above. c) collector establishes connection to the exporter to "take over" the burdon of collecting the data. - how to ensure stability of the failover mechanism (prevent oscillations) On one hand, neither of these fail-back modes causes short-term oscillations (immediate). On the other hand, if the machines are working under load conditions, and there is no way to communicate the load to the exporter, there is no way to avoid oscillations by re-establishing export to a collector that is already under load. Therefore, it is probably left as an implementation issue to make sure that a collector assume the collection process if it cannot handle the load. - do the collectors know about each other No. This introduces undue complexity. This could be left for developers of collectors to figure out whether this is of utility. Standardizing the methods of interactions between collectors is especially complex. - does the exporter open connections to all the potential collectors and keep them primed with template info. This would be ideal as it implies minimum needed footprint on device generating the information (information could be sent once conditions require a fail-over. However, this could be optional in implementation as from the collector-exporter interaction, there could be no implications and exporters could have options to do either (maintain multiple connections to collectors or maintain one active and create another upon failover). The collectors can be unaffected as well as the protocol itself. - do we support load balancing among collectors Nice extension option, however, it doesn't seem that the requirements document indicate this is a mandatory extension. If one can find a nice way of doing this technically as a bi-product of reliability extensions, I would support it. However, it is usually not merely a reliability extension. I recall someone (Paul :-) volunteering to track issues - I believe all these issues could be considered sub-items under the category of "reliability extensions" and may need to be discussed on separate threads. Tal -----Original Message----- From: Mark Fullmer [mailto:maf@eng.oar.net] Sent: Monday, May 12, 2003 8:21 AM To: Tal Givoly Cc: Sebastian Zander; ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Lost Flow Records - Synch Message I'm only using UDP as an example of a lossy transport. We're supposed to be transport agnostic. The templates could be retransmitted on failover, and that's probably the easiest thing to do. Like I said the details have not been worked out yet. Things like when to fail over, how to fail back, how to ensure stability of the failover mechanism (prevent oscillations), do the collectors know about each other, does the exporter open connections to all the potential collectors and keep them primed with template info, do we support load balancing among collectors, etc. mark On Mon, May 12, 2003 at 07:54:54AM -0700, Tal Givoly wrote: > Hi, > > Wasn't the suggestion of the evaluation committee and the WG chairs to begin > with a TCP version of the protocol and later, perhaps, add SCTP (or > SCTP-PR)? If that is the case, why are we discussing the behavior of a UDP > version of it as that should not be part of the IPFIX standard at all. > > Regarding the TCP below, why wouldn't the templates be transmitted upon > re-establishing a connection or fail-over to avoid the unnecessary > complexities in the collector of receiving data for which it has not > information regarding its template? > = > > Tal > > -----Original Message----- > From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu]On Behalf > Of Mark Fullmer > Sent: Sunday, May 11, 2003 7:14 PM > To: Sebastian Zander > Cc: 'ipfix@net.doit.wisc.edu' > Subject: Re: [ipfix] Lost Flow Records - Synch Message > > > How lost templates are handled depends on the transport. > > UDP - templates are periodically retransmitted so a collector should > buffer data until the template arrives or resource exhaustion. > > TCP - Templates shouldn't be lost. There is a small window for > a template to be lost in a primary/secondary failover > situation. The failover details have not been finalized > yet. > > SCPT - Hopefully the templates can be done over a reliable > channel. Still have failover issues. > > mark > > On Mon, May 12, 2003 at 03:09:04AM +0200, Sebastian Zander wrote: > > Hi Reinaldo, > > > > having flow record counters per template is a very nice thing in my > > opinion. But if we have this information why would we need the packets > > and bytes sent per template information in addition? > > > > What about information about lost template definitions? (A loss of a > > template def could potentially cause flow records to be "lost" at the > > collector...) > > > > Cheers, > > > > Sebastian > > > > Reinaldo Penno wrote: > > > The editors of the protocol draft had a conf call and we came up with > > > the main list of issues we need to sort out at this point. > > > > > > The first issue we need to get closure is the method to account for lost > > > flow records. The proposal is in line with the "do not make assumption > > > that the transport will be TCP". This does not preclude and/or is > > > intended to fix the higher reliability extensions requirement. Mark will > > > send an email with a proposal on this subject > > > > > > The proposal on the table is to send a IPFix Sync message periodically > > > (periodicity is configurable), with the following information (aside the > > > standard IPFix header) > > > > > > * Number of flow records sent (for each template?) > > > * Packets and bytes sent (for each template?) > > > > > > Please advise if this is okay/not okay, etc, etc > > > > > > Regards, > > > > > > Reinaldo > > > > > > > > > -- > > Sebastian Zander E-mail: > zander@fokus.fraunhofer.de > > Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 > > Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 > > D-10589 Berlin, Germany > www.fokus.fraunhofer.de/usr/sebastian.zander > > > > > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message > body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon May 12 22:01:45 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA11291 for ; Mon, 12 May 2003 22:01:45 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19FOgW-0000qc-00 for ipfix-list@mil.doit.wisc.edu; Mon, 12 May 2003 20:34:28 -0500 Received: from ds20-1.cc.swin.edu.au ([136.186.1.150] helo=swin.edu.au) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19FOgT-0000qT-00 for ipfix@net.doit.wisc.edu; Mon, 12 May 2003 20:34:25 -0500 Received: from fokus.fraunhofer.de (szander-laptop.caia.swin.edu.au [136.186.229.90]) by swin.edu.au (8.9.3p2/8.9.3) with ESMTP id LAA1672911; Tue, 13 May 2003 11:34:08 +1000 (EST) Message-ID: <3EC04A24.8010705@fokus.fraunhofer.de> Date: Tue, 13 May 2003 03:28:04 +0200 From: Sebastian Zander Organization: Fraunhofer FOKUS User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.2) Gecko/20030208 Netscape/7.02 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Maurizio Molina CC: Alex Audu , "'ipfix@net.doit.wisc.edu'" Subject: Re: [ipfix] Lost Flow Records - Synch Message References: <0A11633F61BD9F40B43ABCC694004F93F18F5D@zsc3c026.us.nortel.com> <3EBBFA05.E20A5A20@alcatel.com> <3EBEE8F0.3020600@fokus.fraunhofer.de> <3EBF95E8.40174AA2@ccrle.nec.de> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Maurizio Molina wrote: > > Sebastian Zander wrote: > > >>Alex Audu wrote: >> >>>Is the sequence number in the header packet not enough for tracking >>>lost flow records? >> >>Having only the sequence number the collector only would know the number >>of packets lost. It would not know the number of flow records lost nor >>which flow records had been lost. > > > I think Alex's comment shows that there are different issues and discussing before the > _usage_ of the information could be beneficial, as I've asked in a previous mail. Here my > 2 cents: > To build a reliable information transfer over an unreliable transport (which however in > the current requirement draft is NOT mandated) the sequence number in the packet header > is probably enough. Probably not. It only enables the collector to find out which packets have been lost. Actually unless you don't know the first sequence number the collector can't even find out about packets lost before the first arrives... > The number of flow records sent would be on th econtrary useful for the collector to > judge on the reliability of the received information (draft-ietf-ipfix-reqs-09.txt, sec. > 6.3.2, list points 3-5 (loss of flow records due to data transfer, mainteneance)). > In this respect, I'm noting that the mentioned list points 1 and 2 (loss of flow records > due to metering or exporting process limitation) would be difficult to quantitatively > report. A meter/exporter unable to create a flow record (e.g. because of memory > limitations) could be at most able to report the number of packets and bytes that it had > to drop, but not the number of _not even created_ flow records. I think that in case of point 2 you may be able to gauge the loss of flow records because after the metering process you have flow records. I totally agree that you may not know anything about lost flow records in case of point 1. That was and is my one of my issues with section 6.3.2, see old post http://ipfix.doit.wisc.edu/archive/1532.html. Cheers, Sebastian > Regards, > Maurizio > > >> >>>Alex. >>> >>>Reinaldo Penno wrote: >>> >>> >>>> >>>>The editors of the protocol draft had a conf call and we came up with >>>>the main list of issues we need to sort out at this point. >>>> >>>>The first issue we need to get closure is the method to account for >>>>lost flow records. The proposal is in line with the "do not make >>>>assumption that the transport will be TCP". This does not preclude >>>>and/or is intended to fix the higher reliability extensions >>>>requirement. Mark will send an email with a proposal on this subject >>>> >>>>The proposal on the table is to send a IPFix Sync message periodically >>>>(periodicity is configurable), with the following information (aside >>>>the standard IPFix header) >>>> >>>>* Number of flow records sent (for each template?) >>>>* Packets and bytes sent (for each template?) >>>> >>>>Please advise if this is okay/not okay, etc, etc >>>> >>>>Regards, >>>> >>>>Reinaldo >>>> >>> >>-- >>Sebastian Zander E-mail: zander@fokus.fraunhofer.de >>Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 >>Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 >>D-10589 Berlin, Germany www.fokus.fraunhofer.de/usr/sebastian.zander >> >>-- >>Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body >>Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >>"unsubscribe ipfix" in message body >>Archive http://ipfix.doit.wisc.edu/archive/ > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > -- Sebastian Zander E-mail: zander@fokus.fraunhofer.de Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 D-10589 Berlin, Germany www.fokus.fraunhofer.de/usr/sebastian.zander -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon May 12 22:08:58 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA11398 for ; Mon, 12 May 2003 22:08:58 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19FOte-00018O-00 for ipfix-list@mil.doit.wisc.edu; Mon, 12 May 2003 20:48:02 -0500 Received: from ds20-1.cc.swin.edu.au ([136.186.1.150] helo=swin.edu.au) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19FOtc-00018A-00 for ipfix@net.doit.wisc.edu; Mon, 12 May 2003 20:48:00 -0500 Received: from fokus.fraunhofer.de (szander-laptop.caia.swin.edu.au [136.186.229.90]) by swin.edu.au (8.9.3p2/8.9.3) with ESMTP id LAA1674452; Tue, 13 May 2003 11:47:35 +1000 (EST) Message-ID: <3EC04D4C.5020400@fokus.fraunhofer.de> Date: Tue, 13 May 2003 03:41:32 +0200 From: Sebastian Zander Organization: Fraunhofer FOKUS User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.2) Gecko/20030208 Netscape/7.02 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Mark Thibodeau CC: Tal Givoly , Mark Fullmer , ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Lost Flow Records - Synch Message References: <3EBFBB75.87B6E78C@alcatel.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Mark Thibodeau wrote: > I agree with Tal. With TCP, we do not have to worry about losing templates. > > For the lost flow statistics, a sequence number and exported flow count should be > adequate. The collector can use the sequence number to detect that a packet has > been lost. > > The total number of flows exported can be used to determine how many flows have > been lost. > > It could be that this is redundant information though. With just the total flow > count, the collector should be able to figure everything out. But, the sequence > numbers make things a bit easier. > > I vote that we keep both mechanisms. The sequence number would also be usefull for duplicate detection. There is a timestamp but it has only ms accuracy... > Cheers, > Mark > > Tal Givoly wrote: > > >>Hi, >> >>Wasn't the suggestion of the evaluation committee and the WG chairs to begin >>with a TCP version of the protocol and later, perhaps, add SCTP (or >>SCTP-PR)? If that is the case, why are we discussing the behavior of a UDP >>version of it as that should not be part of the IPFIX standard at all. >> >>Regarding the TCP below, why wouldn't the templates be transmitted upon >>re-establishing a connection or fail-over to avoid the unnecessary >>complexities in the collector of receiving data for which it has not >>information regarding its template? >>= >> >>Tal >> >>-----Original Message----- >>From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu]On Behalf >>Of Mark Fullmer >>Sent: Sunday, May 11, 2003 7:14 PM >>To: Sebastian Zander >>Cc: 'ipfix@net.doit.wisc.edu' >>Subject: Re: [ipfix] Lost Flow Records - Synch Message >> >>How lost templates are handled depends on the transport. >> >> UDP - templates are periodically retransmitted so a collector should >> buffer data until the template arrives or resource exhaustion. >> >> TCP - Templates shouldn't be lost. There is a small window for >> a template to be lost in a primary/secondary failover >> situation. The failover details have not been finalized >> yet. >> >> SCPT - Hopefully the templates can be done over a reliable >> channel. Still have failover issues. >> >>mark >> >>On Mon, May 12, 2003 at 03:09:04AM +0200, Sebastian Zander wrote: >> >>>Hi Reinaldo, >>> >>>having flow record counters per template is a very nice thing in my >>>opinion. But if we have this information why would we need the packets >>>and bytes sent per template information in addition? >>> >>>What about information about lost template definitions? (A loss of a >>>template def could potentially cause flow records to be "lost" at the >>>collector...) >>> >>>Cheers, >>> >>>Sebastian >>> >>>Reinaldo Penno wrote: >>> >>>>The editors of the protocol draft had a conf call and we came up with >>>>the main list of issues we need to sort out at this point. >>>> >>>>The first issue we need to get closure is the method to account for lost >>>>flow records. The proposal is in line with the "do not make assumption >>>>that the transport will be TCP". This does not preclude and/or is >>>>intended to fix the higher reliability extensions requirement. Mark will >>>>send an email with a proposal on this subject >>>> >>>>The proposal on the table is to send a IPFix Sync message periodically >>>>(periodicity is configurable), with the following information (aside the >>>>standard IPFix header) >>>> >>>>* Number of flow records sent (for each template?) >>>>* Packets and bytes sent (for each template?) >>>> >>>>Please advise if this is okay/not okay, etc, etc >>>> >>>>Regards, >>>> >>>>Reinaldo >>>> >>> >>> >>>-- >>>Sebastian Zander E-mail: >> >>zander@fokus.fraunhofer.de >> >>>Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 >>>Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 >>>D-10589 Berlin, Germany >> >>www.fokus.fraunhofer.de/usr/sebastian.zander >> >>> >>> >>-- >>Help mailto:majordomo@net.doit.wisc.edu and say "help" in message >>body >>Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >>"unsubscribe ipfix" in message body >>Archive http://ipfix.doit.wisc.edu/archive/ >> >>-- >>Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body >>Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >>"unsubscribe ipfix" in message body >>Archive http://ipfix.doit.wisc.edu/archive/ > > > -- > > **************************************************** > * Mark Thibodeau, P Eng. > * 670 RSP Development - Firmware Designer > * Alcatel CID - (613) 784-5375 > * Fax - (613) 599-3696 > * mark.thibodeau@alcatel.com > **************************************************** > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > -- Sebastian Zander E-mail: zander@fokus.fraunhofer.de Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 D-10589 Berlin, Germany www.fokus.fraunhofer.de/usr/sebastian.zander -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue May 13 00:51:57 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA14762 for ; Tue, 13 May 2003 00:51:57 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19FRUe-0004PM-00 for ipfix-list@mil.doit.wisc.edu; Mon, 12 May 2003 23:34:24 -0500 Received: from usmail.xacct.com ([204.253.100.12]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19FRUc-0004PD-00 for ipfix@net.doit.wisc.edu; Mon, 12 May 2003 23:34:22 -0500 Received: from Givoly ([192.168.0.3]) by usmail.xacct.com (8.11.2/8.11.2) with SMTP id h4D4eUi28222; Mon, 12 May 2003 21:40:36 -0700 From: "Tal Givoly" To: "Sebastian Zander" , "Mark Thibodeau" Cc: "Mark Fullmer" , Subject: RE: [ipfix] Lost Flow Records - Synch Message Date: Mon, 12 May 2003 21:33:41 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-Reply-To: <3EC04D4C.5020400@fokus.fraunhofer.de> Importance: Normal Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hi, For duplicate detection, each flow must have sequence numbers unique enough to disambiguate it from any other flow. Therefore, it usually makes sense to assign them per template and perhaps per startup time of device. For most devices it doesn't make sense to make them "globally unique" as context of transmitting device is known. But it is necessary to disambiguate flows from the same device after resets of device. It would greatly assist duplicate elimination if any flow that has been retransmitted is also marked as duplicate with some "duplicate bit". This is a hint to the collector so that it only needs to check those flow records marked as duplicate against other flows it has received. This has a benefit of optimizing duplicate elimination at the collector while introducing no overhead and minimum complexity to the exporter - as it already should know whether it is sending a flow record for the first time or for any subsequent retransmission. Tal -----Original Message----- From: Sebastian Zander [mailto:zander@fokus.fraunhofer.de] Sent: Monday, May 12, 2003 6:42 PM To: Mark Thibodeau Cc: Tal Givoly; Mark Fullmer; ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Lost Flow Records - Synch Message Mark Thibodeau wrote: > I agree with Tal. With TCP, we do not have to worry about losing templates. > > For the lost flow statistics, a sequence number and exported flow count should be > adequate. The collector can use the sequence number to detect that a packet has > been lost. > > The total number of flows exported can be used to determine how many flows have > been lost. > > It could be that this is redundant information though. With just the total flow > count, the collector should be able to figure everything out. But, the sequence > numbers make things a bit easier. > > I vote that we keep both mechanisms. The sequence number would also be usefull for duplicate detection. There is a timestamp but it has only ms accuracy... > Cheers, > Mark > > Tal Givoly wrote: > > >>Hi, >> >>Wasn't the suggestion of the evaluation committee and the WG chairs to begin >>with a TCP version of the protocol and later, perhaps, add SCTP (or >>SCTP-PR)? If that is the case, why are we discussing the behavior of a UDP >>version of it as that should not be part of the IPFIX standard at all. >> >>Regarding the TCP below, why wouldn't the templates be transmitted upon >>re-establishing a connection or fail-over to avoid the unnecessary >>complexities in the collector of receiving data for which it has not >>information regarding its template? >>= >> >>Tal >> >>-----Original Message----- >>From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu]On Behalf >>Of Mark Fullmer >>Sent: Sunday, May 11, 2003 7:14 PM >>To: Sebastian Zander >>Cc: 'ipfix@net.doit.wisc.edu' >>Subject: Re: [ipfix] Lost Flow Records - Synch Message >> >>How lost templates are handled depends on the transport. >> >> UDP - templates are periodically retransmitted so a collector should >> buffer data until the template arrives or resource exhaustion. >> >> TCP - Templates shouldn't be lost. There is a small window for >> a template to be lost in a primary/secondary failover >> situation. The failover details have not been finalized >> yet. >> >> SCPT - Hopefully the templates can be done over a reliable >> channel. Still have failover issues. >> >>mark >> >>On Mon, May 12, 2003 at 03:09:04AM +0200, Sebastian Zander wrote: >> >>>Hi Reinaldo, >>> >>>having flow record counters per template is a very nice thing in my >>>opinion. But if we have this information why would we need the packets >>>and bytes sent per template information in addition? >>> >>>What about information about lost template definitions? (A loss of a >>>template def could potentially cause flow records to be "lost" at the >>>collector...) >>> >>>Cheers, >>> >>>Sebastian >>> >>>Reinaldo Penno wrote: >>> >>>>The editors of the protocol draft had a conf call and we came up with >>>>the main list of issues we need to sort out at this point. >>>> >>>>The first issue we need to get closure is the method to account for lost >>>>flow records. The proposal is in line with the "do not make assumption >>>>that the transport will be TCP". This does not preclude and/or is >>>>intended to fix the higher reliability extensions requirement. Mark will >>>>send an email with a proposal on this subject >>>> >>>>The proposal on the table is to send a IPFix Sync message periodically >>>>(periodicity is configurable), with the following information (aside the >>>>standard IPFix header) >>>> >>>>* Number of flow records sent (for each template?) >>>>* Packets and bytes sent (for each template?) >>>> >>>>Please advise if this is okay/not okay, etc, etc >>>> >>>>Regards, >>>> >>>>Reinaldo >>>> >>> >>> >>>-- >>>Sebastian Zander E-mail: >> >>zander@fokus.fraunhofer.de >> >>>Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 >>>Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 >>>D-10589 Berlin, Germany >> >>www.fokus.fraunhofer.de/usr/sebastian.zander >> >>> >>> >>-- >>Help mailto:majordomo@net.doit.wisc.edu and say "help" in message >>body >>Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >>"unsubscribe ipfix" in message body >>Archive http://ipfix.doit.wisc.edu/archive/ >> >>-- >>Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body >>Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >>"unsubscribe ipfix" in message body >>Archive http://ipfix.doit.wisc.edu/archive/ > > > -- > > **************************************************** > * Mark Thibodeau, P Eng. > * 670 RSP Development - Firmware Designer > * Alcatel CID - (613) 784-5375 > * Fax - (613) 599-3696 > * mark.thibodeau@alcatel.com > **************************************************** > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > -- Sebastian Zander E-mail: zander@fokus.fraunhofer.de Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 D-10589 Berlin, Germany www.fokus.fraunhofer.de/usr/sebastian.zander -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed May 14 12:09:07 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA23105 for ; Wed, 14 May 2003 12:09:07 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19FyOn-00038q-00 for ipfix-list@mil.doit.wisc.edu; Wed, 14 May 2003 10:42:33 -0500 Received: from [64.95.122.13] (helo=RS-SC-EXC4.rs.riverstonenet.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19FyOl-00038j-00 for ipfix@net.doit.wisc.edu; Wed, 14 May 2003 10:42:31 -0500 Received: from riverstonenet.com ([172.17.6.3]) by RS-SC-EXC4.rs.riverstonenet.com with Microsoft SMTPSVC(5.0.2195.4905); Wed, 14 May 2003 08:42:30 -0700 Message-ID: <3EC227FD.5D7E416D@riverstonenet.com> Date: Wed, 14 May 2003 07:26:53 -0400 From: calato@riverstonenet.com X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.7-10 i686) X-Accept-Language: en MIME-Version: 1.0 To: "MEYER,JEFFREY D (HP-Cupertino,ex1)" CC: "'ipfix@net.doit.wisc.edu'" Subject: Re: [ipfix] export packet length - LAST CALL References: <1D3D2C371FCBD947A7897FABBD3533A566BABA@xsun01.ptp.hp.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 14 May 2003 15:42:31.0246 (UTC) FILETIME=[6E4756E0:01C31A2F] Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit "MEYER,JEFFREY D (HP-Cupertino,ex1)" wrote: > > Nevil, > > Thanks for summarizing the current status and seeking quick and rough > consensus. > > Regarding the consensus aspect, do you have any ground rules, i.e. lack of > disagreement on mailing list w/in n days, or number of pro/con e-mails from > unique authors w/in n days, or some anything else? Perhaps there are > already documented or undocumented precedents from other WG's we'd wish to > follow? Nothing so formal yet. I'll issue last call with the status as I see it from the mailing list. Then wait at least a week or 2 for mailing list responses. We'll need an exception process but what that is I'm not sure yet. If you find a documented process for this, kindly let me know. Paul > > I must admit that I feel I'm missing something here. The hummmm process > at the face to face meetings I understand and can accept, the pre-meeting > methodology puzzles me. > > Regards, > > Jeff Meyer > > -----Original Message----- > From: Nevil Brownlee [mailto:n.brownlee@auckland.ac.nz] > Sent: Tuesday, May 06, 2003 10:50 PM > To: calato@riverstonenet.com > Cc: ipfix@net.doit.wisc.edu > Subject: Re: [ipfix] export packet length - LAST CALL > > Hi all: > > Following up on the latest round of discusisons: > > 1) Using Randy & Rob's WG issues tracking system sounds fine. > > 2) Running Last Calls after mailing list discussion is a good way to > reach consensus (and thereby close) issues. > > 3) On 'extensibility' of theIPFIX protocol. My take on this is that > - We want an information model which can be extended by adding new > attributes, and a protocol which can handle such new attributes, > so we need to put some design effort in now to achieve that. > - For reliability we agreed that the initial version of IPFIX has > to work with only one-way transport, but that we need sufficient > hooks to add reliability later on. > Both these goals are clearly being worked on, that's great. But > we need to be careful to balance our development efforts so as > to produce a good, simple protocol now, which can be extended as > needed later - without unduly delaying our efforts to get the > initial version done fairly quickly. "By the end of this year" to > reach near-final drafts sounds about right to me ... > > Cheers, Nevil > > ----------------------------------------------------------------------- > Nevil Brownlee Director, Technology Development > Phone: +64 9 373 7599 x88941 ITSS, The University of Auckland > FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand > > ------------------------------------------------- > This mail sent through University of Auckland > http://www.auckland.ac.nz/ > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message > body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed May 14 12:46:02 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA27163 for ; Wed, 14 May 2003 12:46:02 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19Fz9d-0004Fs-00 for ipfix-list@mil.doit.wisc.edu; Wed, 14 May 2003 11:30:57 -0500 Received: from [64.95.122.13] (helo=RS-SC-EXC4.rs.riverstonenet.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19Fz9b-0004Fl-00 for ipfix@net.doit.wisc.edu; Wed, 14 May 2003 11:30:55 -0500 Received: from riverstonenet.com ([172.17.6.3]) by RS-SC-EXC4.rs.riverstonenet.com with Microsoft SMTPSVC(5.0.2195.4905); Wed, 14 May 2003 09:30:55 -0700 Message-ID: <3EC23339.102B65D0@riverstonenet.com> Date: Wed, 14 May 2003 08:14:49 -0400 From: calato@riverstonenet.com X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.7-10 i686) X-Accept-Language: en MIME-Version: 1.0 To: ipfixx Subject: [ipfix] Count Field in Header Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 14 May 2003 16:30:55.0396 (UTC) FILETIME=[31497640:01C31A36] Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit I would like to raise a new issue. Is the count field needed in the message header? What would it be used for? Paul -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed May 14 12:50:23 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA27277 for ; Wed, 14 May 2003 12:50:22 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19Fz8C-0004Dn-00 for ipfix-list@mil.doit.wisc.edu; Wed, 14 May 2003 11:29:28 -0500 Received: from [64.95.122.13] (helo=RS-SC-EXC4.rs.riverstonenet.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19Fz8A-0004Dg-00 for ipfix@net.doit.wisc.edu; Wed, 14 May 2003 11:29:26 -0500 Received: from riverstonenet.com ([172.17.6.3]) by RS-SC-EXC4.rs.riverstonenet.com with Microsoft SMTPSVC(5.0.2195.4905); Wed, 14 May 2003 09:29:25 -0700 Message-ID: <3EC232DF.1F5B914F@riverstonenet.com> Date: Wed, 14 May 2003 08:13:19 -0400 From: calato@riverstonenet.com X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.7-10 i686) X-Accept-Language: en MIME-Version: 1.0 To: Maurizio Molina CC: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] export packet length - LAST CALL References: <3EA95941.57BFEE44@ccrle.nec.de> <3EB7DB16.1886F07C@riverstonenet.com> <20030506163609.A58211@net.ohio-state.edu> <3EB8C7E9.BC41382C@ccrle.nec.de> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 14 May 2003 16:29:25.0675 (UTC) FILETIME=[FBCF1FB0:01C31A35] Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit We have consensus that the length field is needed. Now we need the details. I would like to see the first 4 bytes of the message look like this... 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Version | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Keeping or not keeping the counter field is, I think, a separate issue. Paul Maurizio Molina wrote: > > I also support the encoding of the packet length in the header as a non-extensible, > fixed header field. IMO it could be done in one of the 3 following ways, > which I list in my order of preference: > > 1) 2 bytes, additional to the current header > 2) 2 bytes, replacing the 2 bytes Count header > 3) 4 bytes, additional to the curent header > > Maurizio > > Mark Fullmer wrote: > > > To close this issue I would like to see the length specified > > (2 or 4 byte fixed header field). If 2 bytes then is replacing the existing > > count field okay. > > > > And yes, I think closing some of the recently discussed open issues with last > > call's is the right way to get things moving. > > > > mark > > > > On Tue, May 06, 2003 at 11:56:06AM -0400, calato@riverstonenet.com wrote: > > > > > > There seems to be a consensus that having a length field > > > in the packet header is a good idea. > > > > > > I would like to bring this issue to last call. Anyone > > > opposed please state your 0 1 2 > > > > > > If this doesn't seem like the right process to close > > > out issues, suggestions are welcome. > > > > > > Paul > > > > > > > > > Maurizio Molina wrote: > > > > > > > > Hi all, > > > > I hope not to rise an issue already discussed in the WG. If yes, I > > > > apologise in advance. > > > > If an IPFIX export packet is carried over TCP, it's not unusual that it > > > > gets split across multiple TCP segments. > > > > At the collector side, on the contrary, the implementation of the > > > > parsing is much simpler if only full export packet are parsed. > > > > If the information about the WHOLE export packet were contained in the > > > > header, it would be simple to wait until all the bytes composing an > > > > export packets are read from the TCP socket before beginning the > > > > parsing. > > > > Unfortunately, this information isn't currently contained in Netflow 9 > > > > packet header (I suppose, due to Netflow's 9 "bias" towards UDP...). > > > > Therefore, a parsing implementation having to deal with this > > > > "fragentation" problem without this information needs to be much more > > > > complicated (e.g. getting step by step the lenght of each flow set, see > > > > if it's all there, parse it and go on....). > > > > In summary: if IPFIX has to be carried over tcp, having the length of > > > > the export packet in the packet header would be beneficial. > > > > Regards, > > > > Maurizio > > > > > > > > -- > > > > Maurizio Molina > > > > Research Staff member > > > > Network Laboratories Heidelberg > > > > NEC Europe Ltd. > > > > Kurfuersten-Anlage 36, 69115 Heidelberg, Germany. > > > > Tel. (49)6221 90511-18 Fax: (49)6221 90511-55 > > > > e-mail: molina@ccrle.nec.de > > > > Web: www.ccrle.nec.de > > > > > > > > -- > > > > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > > "unsubscribe ipfix" in message body > > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > -- > > > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > "unsubscribe ipfix" in message body > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed May 14 12:51:51 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA27321 for ; Wed, 14 May 2003 12:51:51 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19FzBt-0004KD-00 for ipfix-list@mil.doit.wisc.edu; Wed, 14 May 2003 11:33:17 -0500 Received: from [64.95.122.13] (helo=RS-SC-EXC4.rs.riverstonenet.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19FzBs-0004K6-00 for ipfix@net.doit.wisc.edu; Wed, 14 May 2003 11:33:16 -0500 Received: from riverstonenet.com ([172.17.6.3]) by RS-SC-EXC4.rs.riverstonenet.com with Microsoft SMTPSVC(5.0.2195.4905); Wed, 14 May 2003 09:33:16 -0700 Message-ID: <3EC233C4.DE641F57@riverstonenet.com> Date: Wed, 14 May 2003 08:17:08 -0400 From: calato@riverstonenet.com X-Mailer: Mozilla 4.78 [en] (X11; U; Linux 2.4.7-10 i686) X-Accept-Language: en MIME-Version: 1.0 To: ipfixx Subject: [ipfix] Version number - major/minor Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 14 May 2003 16:33:16.0352 (UTC) FILETIME=[854DA400:01C31A36] Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Should we have the version field split into major and minor? Something like 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Major Ver | Minor ver | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ This would allow us to bump the minor when backward compatible changes are made, such as new field elements. Paul -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu May 15 09:04:11 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA07780 for ; Thu, 15 May 2003 09:04:11 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19GHwU-0006g1-00 for ipfix-list@mil.doit.wisc.edu; Thu, 15 May 2003 07:34:38 -0500 Received: from tokyo.ccrle.nec.de ([195.37.70.2]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19GHwP-0006fq-00 for ipfix@net.doit.wisc.edu; Thu, 15 May 2003 07:34:33 -0500 Received: from venus.office (venus.office [10.1.1.11]) by tokyo.ccrle.nec.de (8.12.9/8.12.8) with ESMTP id h4FCYBVI081821; Thu, 15 May 2003 14:34:28 +0200 (CEST) Received: from ccrle.nec.de (molina.office [10.1.1.126]) by venus.office (Postfix on SuSE Linux eMail Server 3.0) with ESMTP id 7A15BA556E; Thu, 15 May 2003 14:25:20 +0200 (CEST) Message-ID: <3EC38944.12867C79@ccrle.nec.de> Date: Thu, 15 May 2003 14:34:12 +0200 From: Maurizio Molina X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: calato@riverstonenet.com Cc: ipfixx Subject: Re: [ipfix] Count Field in Header References: <3EC23339.102B65D0@riverstonenet.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit It's true it isn't strictly necessary. However, knowing in advance how many flow set records are contained in the export packet may give the implementor more flexibility in writing the packet parsing. Maurizio calato@riverstonenet.com wrote: > I would like to raise a new issue. > > Is the count field needed in the message header? What would > it be used for? > > Paul > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu May 15 09:36:27 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA08575 for ; Thu, 15 May 2003 09:36:27 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19GIVN-0007Zl-00 for ipfix-list@mil.doit.wisc.edu; Thu, 15 May 2003 08:10:41 -0500 Received: from h65s138a81n47.user.nortelnetworks.com ([47.81.138.65] helo=zsc3s004.nortelnetworks.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19GIVL-0007Zf-00 for ipfix@net.doit.wisc.edu; Thu, 15 May 2003 08:10:39 -0500 Received: from zsc3c028.us.nortel.com (zsc3c028.us.nortel.com [47.81.138.28]) by zsc3s004.nortelnetworks.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id h4FDAZm01222; Thu, 15 May 2003 06:10:35 -0700 (PDT) Received: by zsc3c028.us.nortel.com with Internet Mail Service (5.5.2653.19) id ; Thu, 15 May 2003 06:10:21 -0700 Message-ID: <0A11633F61BD9F40B43ABCC694004F930218EF04@zsc3c026.us.nortel.com> From: "Reinaldo Penno" To: "'ipfix@net.doit.wisc.edu'" Cc: "'Mark Fullmer'" , calato@riverstonenet.com Subject: [ipfix] Lost Flow Records - Synch Message - Summary up to now Date: Thu, 15 May 2003 06:10:20 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C31AE3.56304C10" Precedence: bulk Sender: majordomo listserver This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C31AE3.56304C10 Content-Type: text/plain; charset="ISO-8859-1" I read the whole thread again and got the following points that I would like to flesh out. *Collector vs exporter initiated. If Exporter initiated the collector will know how many flow records (overall or per template) were lost associated with the number of lost packets based on the sequence number. Collector send sync message every O(minutes) If collector initiated, the collector would let the exporter know how many packets it missed. This might impact the failover process since the exporter wouldn't be sure of which last sequence number was actually received. Collector initiated whenever it detect gaps in the sequence numbers, export answers with a synch message. * Contents of the sync message *Flow record counters per template or number of flow records is enough? The total number of flow records *Lost of template definitions. ------_=_NextPart_001_01C31AE3.56304C10 Content-Type: text/html; charset="ISO-8859-1" Content-Transfer-Encoding: quoted-printable [ipfix] Lost Flow Records - Synch Message - Summary up to = now

I read the whole thread again and got the following = points that I would like to flesh out.

*Collector vs exporter initiated.

If Exporter initiated the collector will know how = many flow records (overall or per template) were lost associated with = the number of lost packets based on the sequence number.

        Collector = send sync message every O(minutes)

If collector initiated, the collector would let the = exporter know how many packets it missed.
        This = might impact the failover process since the exporter wouldn't be sure = of which last sequence number was    =         actually = received.

Collector initiated whenever it detect gaps in the = sequence numbers, export answers with a synch message.


* Contents of the sync message

        *Flow = record counters per template or number of flow records is enough? The = total number of flow records

        *Lost of = template definitions.



------_=_NextPart_001_01C31AE3.56304C10-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu May 15 10:14:06 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA10194 for ; Thu, 15 May 2003 10:14:06 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19GJHH-0000qU-00 for ipfix-list@mil.doit.wisc.edu; Thu, 15 May 2003 09:00:11 -0500 Received: from kanfw1.ottawa.alcatel.ca ([192.75.23.69] helo=kanmx2.ca.alcatel.com) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 19GJHF-0000qL-00 for ipfix@net.doit.wisc.edu; Thu, 15 May 2003 09:00:09 -0500 Received: (qmail 5563 invoked from network); 15 May 2003 14:02:37 -0000 Received: from unknown (HELO alcatel.com) (138.120.51.110) by kanmx2.ca.alcatel.com with SMTP; 15 May 2003 14:02:37 -0000 Message-ID: <3EC39D49.770E896E@alcatel.com> Date: Thu, 15 May 2003 09:59:37 -0400 From: Mark Thibodeau X-Mailer: Mozilla 4.78 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Reinaldo Penno CC: "'ipfix@net.doit.wisc.edu'" Subject: Re: [ipfix] Lost Flow Records - Synch Message - Summary up to now References: <0A11633F61BD9F40B43ABCC694004F930218EF04@zsc3c026.us.nortel.com> Content-Type: multipart/alternative; boundary="------------7CD1C767D3FE6109337E6685" Precedence: bulk Sender: majordomo listserver --------------7CD1C767D3FE6109337E6685 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Exporter initiated synch messages make sense. If the collector initiates the synch, there can be ambiguity in the state of the collector as you mentioned below. NetflowV9 already has the concept of "Option" templates and data flows. These option messages can be used to send the total number of flow records exported. The Option data can be sent at some configurable interval. For content, the number of flow records exported per template may be required. Some templates may be aggregated. Losing an aggregated flow record can be worse than losing a micro-flow record. Using only a global counter does not allow you to figure out how many aggregated flows have been lost. As for lost template definitions, this should not happen. Since IPFIX will be using TCP, the exporter will know whether or not the template was received successfully. /Mark Reinaldo Penno wrote: > > > I read the whole thread again and got the following points that I > would like to flesh out. > > *Collector vs exporter initiated. > > If Exporter initiated the collector will know how many flow records > (overall or per template) were lost associated with the number of lost > packets based on the sequence number. > > Collector send sync message every O(minutes) > > If collector initiated, the collector would let the exporter know how > many packets it missed. > This might impact the failover process since the exporter > wouldn't be sure of which last sequence number was actually > received. > > Collector initiated whenever it detect gaps in the sequence numbers, > export answers with a synch message. > > * Contents of the sync message > > *Flow record counters per template or number of flow records > is enough? The total number of flow records > > *Lost of template definitions. > -- **************************************************** * Mark Thibodeau, P Eng. * 670 RSP Development - Firmware Designer * Alcatel CID - (613) 784-5375 * Fax - (613) 599-3696 * mark.thibodeau@alcatel.com **************************************************** --------------7CD1C767D3FE6109337E6685 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit Exporter initiated synch messages make sense.  If the collector initiates the synch, there can be ambiguity in the state of the collector as you mentioned below.

NetflowV9 already has the concept of "Option" templates and data flows.  These option messages can be used to send the total number of flow records exported.  The Option data can be sent at some configurable interval.

For content, the number of flow records exported per template may be required.  Some templates may be aggregated.  Losing an aggregated flow record can be worse than losing a micro-flow record.  Using only a global counter does not allow you to figure out how many aggregated flows have been lost.

As for lost template definitions, this should not happen.  Since IPFIX will be using TCP, the exporter will know whether or not the template was received successfully.

/Mark

Reinaldo Penno wrote:

 

I read the whole thread again and got the following points that I would like to flesh out.

*Collector vs exporter initiated.

If Exporter initiated the collector will know how many flow records (overall or per template) were lost associated with the number of lost packets based on the sequence number.

        Collector send sync message every O(minutes)

If collector initiated, the collector would let the exporter know how many packets it missed.
        This might impact the failover process since the exporter wouldn't be sure of which last sequence number was            actually received.

Collector initiated whenever it detect gaps in the sequence numbers, export answers with a synch message.

* Contents of the sync message

        *Flow record counters per template or number of flow records is enough? The total number of flow records

        *Lost of template definitions.
 

--

****************************************************
* Mark Thibodeau, P Eng.
* 670 RSP Development - Firmware Designer
* Alcatel CID - (613) 784-5375
* Fax - (613) 599-3696
* mark.thibodeau@alcatel.com
****************************************************
  --------------7CD1C767D3FE6109337E6685-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu May 15 11:25:45 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA12211 for ; Thu, 15 May 2003 11:25:45 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19GKOQ-0002KH-00 for ipfix-list@mil.doit.wisc.edu; Thu, 15 May 2003 10:11:38 -0500 Received: from [130.216.191.4] (helo=mailhost2.auckland.ac.nz) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19GKOO-0002K7-00 for ipfix@net.doit.wisc.edu; Thu, 15 May 2003 10:11:36 -0500 Received: from mailhost.auckland.ac.nz (IDENT:mirapoint@mailhost [130.216.191.61]) by mailhost2.auckland.ac.nz (8.12.9/8.12.9/8.12.9-ua) with ESMTP id h4FFBMcT020968; Fri, 16 May 2003 03:11:22 +1200 (NZST) Received: from hotlava.auckland.ac.nz (hotlava.auckland.ac.nz [130.216.191.123]) by mailhost.auckland.ac.nz (Mirapoint Messaging Server MOS 3.3.5-GR) with ESMTP id APZ98106; Fri, 16 May 2003 03:11:21 +1200 (NZST) Received: (from apache@localhost) by hotlava.auckland.ac.nz (8.11.6/8.11.6) id h4FFBLZ25786; Fri, 16 May 2003 03:11:21 +1200 X-Authentication-Warning: hotlava.auckland.ac.nz: apache set sender to n.brownlee@auckland.ac.nz using -f Received: from dhcp-9-229.ripemtg.ripe.net (dhcp-9-229.ripemtg.ripe.net [193.0.9.229]) by hotlava.auckland.ac.nz (Horde) with HTTP for ; Fri, 16 May 2003 03:11:21 +1200 Message-ID: <1053011481.2d22d7efff88b@hotlava.auckland.ac.nz> Date: Fri, 16 May 2003 03:11:21 +1200 From: Nevil Brownlee To: Mark Thibodeau Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Summary up to now References: <0A11633F61BD9F40B43ABCC694004F930218EF04@zsc3c026.us.nortel.com> <3EC39D49.770E896E@alcatel.com> In-Reply-To: <3EC39D49.770E896E@alcatel.com> MIME-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Internet Messaging Program (IMP) 4.0-cvs X-Originating-IP: 193.0.9.229 Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Quoting Mark Thibodeau : > Exporter initiated synch messages make sense. If the collector > initiates the synch, there can be ambiguity in the state of the > collector as you mentioned below. I agree, exporter sync makes sense to me. > NetflowV9 already has the concept of "Option" templates and data flows. > These option messages can be used to send the total number of flow > records exported. The Option data can be sent at some configurable > interval. Ditto. > For content, the number of flow records exported per template may be > required. Some templates may be aggregated. Losing an aggregated flow > record can be worse than losing a micro-flow record. Using only a > global counter does not allow you to figure out how many aggregated > flows have been lost. > > As for lost template definitions, this should not happen. Since IPFIX > will be using TCP, the exporter will know whether or not the template > was received successfully. There's been some apparent confusion on the list about transport for IPFIX. Consensus from the last IPFIX meeting was that the 1.0 version of IPFIX could use TCP transport [so as to make it easier for developers to implement and test], but that version 1.0 would NOT rely on two-way messages between exporter and collector [so that it would be usable in a minimal environment]. So Mark's statement above is not a sufficient answer. Cheers, Nevil ----------------------------------------------------------------------- Nevil Brownlee Director, Technology Development Phone: +64 9 373 7599 x88941 ITSS, The University of Auckland FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand ------------------------------------------------- This mail sent through University of Auckland http://www.auckland.ac.nz/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu May 15 12:34:57 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA14509 for ; Thu, 15 May 2003 12:34:57 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19GLWu-0003sB-00 for ipfix-list@mil.doit.wisc.edu; Thu, 15 May 2003 11:24:28 -0500 Received: from login.caida.org ([192.172.226.78]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19GLWt-0003s3-00 for ipfix@net.doit.wisc.edu; Thu, 15 May 2003 11:24:27 -0500 Received: from login.caida.org (localhost [127.0.0.1]) by login.caida.org (8.12.9/8.12.9) with ESMTP id h4FGOM2e081129 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO); Thu, 15 May 2003 09:24:22 -0700 (PDT) Received: (from dmoore@localhost) by login.caida.org (8.12.9/8.12.9/Submit) id h4FGOMx8081128; Thu, 15 May 2003 09:24:22 -0700 (PDT) Date: Thu, 15 May 2003 09:24:22 -0700 From: David Moore To: Nevil Brownlee Cc: Mark Thibodeau , ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Summary up to now Message-ID: <20030515092422.Y46386@login.caida.org> References: <0A11633F61BD9F40B43ABCC694004F930218EF04@zsc3c026.us.nortel.com> <3EC39D49.770E896E@alcatel.com> <1053011481.2d22d7efff88b@hotlava.auckland.ac.nz> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1053011481.2d22d7efff88b@hotlava.auckland.ac.nz> User-Agent: Mutt/1.3.23i Precedence: bulk Sender: majordomo listserver On Fri, May 16, 2003 at 03:11:21AM +1200, Nevil Brownlee wrote: > Quoting Mark Thibodeau : > > As for lost template definitions, this should not happen. Since IPFIX > > will be using TCP, the exporter will know whether or not the template > > was received successfully. > > There's been some apparent confusion on the list about transport for IPFIX. > Consensus from the last IPFIX meeting was that the 1.0 version of IPFIX > could use TCP transport [so as to make it easier for developers to implement > and test], but that version 1.0 would NOT rely on two-way messages between > exporter and collector [so that it would be usable in a minimal environment]. > So Mark's statement above is not a sufficient answer. I'm probably missing something, but how does just using TCP ensure that the exporter knows that it was received successfully? (especially in environments where you can't reach into the guts of TCP). If the process hands a large buffer to the kernel/OS, it will take it and let the process continue. Some or all of that data may be successfully transmitted via TCP to the receiver. If the connection fails, the kernel/OS knows how much of it made it (to the other kernel, not necessarily application), but it doesn't tell that information to the process. All that the process knows is that it is later told that the connection is gone. How does it determine from this whether it was successful or unsuccessful with the transfer? Particularly in the case when it is writing additional data after the template being queued up in the kernel/OS. -- david -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu May 15 13:32:01 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA16016 for ; Thu, 15 May 2003 13:32:01 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19GMQI-00050O-00 for ipfix-list@mil.doit.wisc.edu; Thu, 15 May 2003 12:21:42 -0500 Received: from web80002.mail.yahoo.com ([66.163.168.132]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 19GMQH-00050G-00 for ipfix@net.doit.wisc.edu; Thu, 15 May 2003 12:21:41 -0500 Message-ID: <20030515172140.85738.qmail@web80002.mail.yahoo.com> Received: from [68.120.131.143] by web80002.mail.yahoo.com via HTTP; Thu, 15 May 2003 10:21:40 PDT Date: Thu, 15 May 2003 10:21:40 -0700 (PDT) From: Peter Ludemann Subject: Re: [ipfix] Summary up to now To: David Moore , Nevil Brownlee Cc: Mark Thibodeau , ipfix@net.doit.wisc.edu In-Reply-To: <20030515092422.Y46386@login.caida.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Precedence: bulk Sender: majordomo listserver --- David Moore wrote: [snip] > If the process hands a large buffer to the kernel/OS, it > will take it and let the process continue. Some or all of > that data may be successfully transmitted via TCP to the > receiver. If the connection fails, the kernel/OS knows > how much of it made it (to the other kernel, not > necessarily application), but it doesn't tell that > information to the process. [snip] Which is why the various competitors to NetFlow had their own ACKs. BTW, even if the TCP implementation allowed access to the TCP-level ACKs, you'd still want IPFIX-protocol level ACKs because you need *application* level acknowledgment of receipt and processing. Even if you don't want ACKs in the data transmission (despite the the overhead being low), you do want ACKs in the template transmission, for the reasons already given by various people. But this is tricky because you need to ensure that all the receivers agree (you may have multiple receivers for fail-over). CRANE was criticised for its fairly elaborate template negotiation; but even without negotiation, there remains a fair bit of complexity with ACKing template information from receivers. An alternative is to design the protocol so that (a) a receiver can request template information [and be assured it receives ALL the information] and (b) the messages can be handled even if a template is not known (e.g., a template ID, a version ID, and a message length, so the message can be skipped). Some of the rejected protocols had this capabilities already, so you might want to revisit them before redesign-by-committee. - peter -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu May 15 13:52:53 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA16660 for ; Thu, 15 May 2003 13:52:53 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19GMhx-0005R8-00 for ipfix-list@mil.doit.wisc.edu; Thu, 15 May 2003 12:39:57 -0500 Received: from kanfw1.ottawa.alcatel.ca ([192.75.23.69] helo=kanmx2.ca.alcatel.com) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 19GMhv-0005Qz-00 for ipfix@net.doit.wisc.edu; Thu, 15 May 2003 12:39:55 -0500 Received: (qmail 6839 invoked from network); 15 May 2003 17:42:22 -0000 Received: from unknown (HELO alcatel.com) (138.120.51.110) by kanmx2.ca.alcatel.com with SMTP; 15 May 2003 17:42:22 -0000 Message-ID: <3EC3D0CB.F6E77022@alcatel.com> Date: Thu, 15 May 2003 13:39:23 -0400 From: Mark Thibodeau X-Mailer: Mozilla 4.78 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: David Moore CC: Nevil Brownlee , ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Summary up to now References: <0A11633F61BD9F40B43ABCC694004F930218EF04@zsc3c026.us.nortel.com> <3EC39D49.770E896E@alcatel.com> <1053011481.2d22d7efff88b@hotlava.auckland.ac.nz> <20030515092422.Y46386@login.caida.org> Content-Type: multipart/alternative; boundary="------------0F3E74405E5C42F1A640C123" Precedence: bulk Sender: majordomo listserver --------------0F3E74405E5C42F1A640C123 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit I agree with both Nevil and David. The TCP ACK is not enough information to determine if the Template made it to the Server or not. I was originallly thinking of an application where the exporter has access to the guts of the TCP stack. If the TCP ACK never came for the Template, the exporter would resend it until it had positive acknowledgement. But, David is right. Most applications will probably just use a standard stack and forget about it. The information about what was ACKed is not propagated to the exporter. I'm not sure about Nevil's point though. From the IPFIX meeting minutes: The evaluation team has recommended NetFlow v9 as a basis for the IPFIX protocol.......Then, specify the initial IPFIX transport be TCP..... Nevil's email seems to say that we will accept either TCP or UDP ("to make it usable in a minimal environment"). What are we saying? IPFIX recommends TCP, but if an application wants to still use UDP then that's OK. It seems like we haven't made a decision at all on the transport protocol. When IPFIX becomes a standard, what will the transport protocol be? If an application does not use that transport protocol, then I assume that it does not comply with the standard. Or am I totally out to lunch. :-() Cheers, Mark David Moore wrote: > On Fri, May 16, 2003 at 03:11:21AM +1200, Nevil Brownlee wrote: > > Quoting Mark Thibodeau : > > > As for lost template definitions, this should not happen. Since IPFIX > > > will be using TCP, the exporter will know whether or not the template > > > was received successfully. > > > > There's been some apparent confusion on the list about transport for IPFIX. > > Consensus from the last IPFIX meeting was that the 1.0 version of IPFIX > > could use TCP transport [so as to make it easier for developers to implement > > and test], but that version 1.0 would NOT rely on two-way messages between > > exporter and collector [so that it would be usable in a minimal environment]. > > So Mark's statement above is not a sufficient answer. > > I'm probably missing something, but how does just using TCP ensure that > the exporter knows that it was received successfully? (especially > in environments where you can't reach into the guts of TCP). > > If the process hands a large buffer to the kernel/OS, it will take > it and let the process continue. Some or all of that data may be > successfully transmitted via TCP to the receiver. If the connection > fails, the kernel/OS knows how much of it made it (to the other > kernel, not necessarily application), but it doesn't tell that > information to the process. All that the process knows is that it > is later told that the connection is gone. How does it determine > from this whether it was successful or unsuccessful with the > transfer? Particularly in the case when it is writing additional > data after the template being queued up in the kernel/OS. > > -- david > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- **************************************************** * Mark Thibodeau, P Eng. * 670 RSP Development - Firmware Designer * Alcatel CID - (613) 784-5375 * Fax - (613) 599-3696 * mark.thibodeau@alcatel.com **************************************************** --------------0F3E74405E5C42F1A640C123 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit I agree with both Nevil and David.  The TCP ACK is not enough information to determine if the Template made it to the Server or not.

I was originallly thinking of an application where the exporter has access to the guts of the TCP stack.  If the TCP ACK never came for the Template, the exporter would resend it until it had positive acknowledgement.

But, David is right.  Most applications will probably just use a standard stack and forget about it.  The information about what was ACKed is not propagated to the exporter.

I'm not sure about Nevil's point though.  From the IPFIX meeting minutes:
 

The evaluation team has recommended NetFlow v9 as a basis for the IPFIX
protocol.......Then, specify the initial IPFIX transport be TCP.....


Nevil's email seems to say that we will accept either TCP or UDP ("to make it usable in a minimal environment").  What are we saying?  IPFIX recommends TCP, but if an application wants to still use UDP then that's OK.  It seems like we haven't made a decision at all on the transport protocol.  When IPFIX becomes a standard, what will the transport protocol be?  If an application does not use that transport protocol, then I assume that it does not comply with the standard.

Or am I totally out to lunch.  :-()

Cheers,
Mark
 
 

David Moore wrote:

On Fri, May 16, 2003 at 03:11:21AM +1200, Nevil Brownlee wrote:
> Quoting Mark Thibodeau <mark.thibodeau@alcatel.com>:
> > As for lost template definitions, this should not happen.  Since IPFIX
> > will be using TCP, the exporter will know whether or not the template
> > was received successfully.
>
> There's been some apparent confusion on the list about transport for IPFIX.
> Consensus from the last IPFIX meeting was that the 1.0 version of IPFIX
> could use TCP transport [so as to make it easier for developers to implement
> and test], but that version 1.0 would NOT rely on two-way messages between
> exporter and collector [so that it would be usable in a minimal environment].
> So Mark's statement above is not a sufficient answer.

I'm probably missing something, but how does just using TCP ensure that
the exporter knows that it was received successfully?  (especially
in environments where you can't reach into the guts of TCP).

If the process hands a large buffer to the kernel/OS, it will take
it and let the process continue.  Some or all of that data may be
successfully transmitted via TCP to the receiver.  If the connection
fails, the kernel/OS knows how much of it made it (to the other
kernel, not necessarily application), but it doesn't tell that
information to the process.  All that the process knows is that it
is later told that the connection is gone.  How does it determine
from this whether it was successful or unsuccessful with the
transfer?  Particularly in the case when it is writing additional
data after the template being queued up in the kernel/OS.

-- david

--
Help        mailto:majordomo@net.doit.wisc.edu and say "help" in message body
Unsubscribe mailto:majordomo@net.doit.wisc.edu and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

--

****************************************************
* Mark Thibodeau, P Eng.
* 670 RSP Development - Firmware Designer
* Alcatel CID - (613) 784-5375
* Fax - (613) 599-3696
* mark.thibodeau@alcatel.com
****************************************************
  --------------0F3E74405E5C42F1A640C123-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu May 15 14:45:38 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA18247 for ; Thu, 15 May 2003 14:45:38 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19GNUS-0006Uw-00 for ipfix-list@mil.doit.wisc.edu; Thu, 15 May 2003 13:30:04 -0500 Received: from [143.209.238.79] (helo=auds955.usa.alcatel.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19GNUP-0006UO-00 for ipfix@net.doit.wisc.edu; Thu, 15 May 2003 13:30:01 -0500 Received: from alcatel.com (localhost [127.0.0.1]) by auds955.usa.alcatel.com (8.12.9/8.12.9) with ESMTP id h4FITmHI020847; Thu, 15 May 2003 13:29:48 -0500 (CDT) Message-ID: <3EC3DC98.69F952ED@alcatel.com> Date: Thu, 15 May 2003 13:29:45 -0500 From: Alex Audu X-Mailer: Mozilla 4.72 [en] (WinNT; U) X-Accept-Language: en MIME-Version: 1.0 To: Mark Thibodeau CC: David Moore , Nevil Brownlee , ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Summary up to now References: <0A11633F61BD9F40B43ABCC694004F930218EF04@zsc3c026.us.nortel.com> <3EC39D49.770E896E@alcatel.com> <1053011481.2d22d7efff88b@hotlava.auckland.ac.nz> <20030515092422.Y46386@login.caida.org> <3EC3D0CB.F6E77022@alcatel.com> Content-Type: multipart/alternative; boundary="------------5B01015B05D427F256F37F8B" Precedence: bulk Sender: majordomo listserver --------------5B01015B05D427F256F37F8B Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Mark, IPFIX only recommends that the transport protocol be congestion aware. So,..even DCCP (a congestion aware variant of UDP) qualifies in this case as a viable option. Regards, Alex. Mark Thibodeau wrote: > I agree with both Nevil and David. The TCP ACK is not enough > information to determine if the Template made it to the Server or not. > > I was originallly thinking of an application where the exporter has > access to the guts of the TCP stack. If the TCP ACK never came for > the Template, the exporter would resend it until it had positive > acknowledgement. > > But, David is right. Most applications will probably just use a > standard stack and forget about it. The information about what was > ACKed is not propagated to the exporter. > > I'm not sure about Nevil's point though. From the IPFIX meeting > minutes: > > > The evaluation team has recommended NetFlow v9 as a basis > for the IPFIX > protocol.......Then, specify the initial IPFIX transport be > TCP..... > > > Nevil's email seems to say that we will accept either TCP or UDP ("to > make it usable in a minimal environment"). What are we saying? IPFIX > recommends TCP, but if an application wants to still use UDP then > that's OK. It seems like we haven't made a decision at all on the > transport protocol. When IPFIX becomes a standard, what will the > transport protocol be? If an application does not use that transport > protocol, then I assume that it does not comply with the standard. > > Or am I totally out to lunch. :-() > > Cheers, > Mark > > > > David Moore wrote: > >> On Fri, May 16, 2003 at 03:11:21AM +1200, Nevil Brownlee wrote: >> > Quoting Mark Thibodeau : >> > > As for lost template definitions, this should not happen. Since >> IPFIX >> > > will be using TCP, the exporter will know whether or not the >> template >> > > was received successfully. >> > >> > There's been some apparent confusion on the list about transport >> for IPFIX. >> > Consensus from the last IPFIX meeting was that the 1.0 version of >> IPFIX >> > could use TCP transport [so as to make it easier for developers to >> implement >> > and test], but that version 1.0 would NOT rely on two-way messages >> between >> > exporter and collector [so that it would be usable in a minimal >> environment]. >> > So Mark's statement above is not a sufficient answer. >> >> I'm probably missing something, but how does just using TCP ensure >> that >> the exporter knows that it was received successfully? (especially >> in environments where you can't reach into the guts of TCP). >> >> If the process hands a large buffer to the kernel/OS, it will take >> it and let the process continue. Some or all of that data may be >> successfully transmitted via TCP to the receiver. If the connection >> >> fails, the kernel/OS knows how much of it made it (to the other >> kernel, not necessarily application), but it doesn't tell that >> information to the process. All that the process knows is that it >> is later told that the connection is gone. How does it determine >> from this whether it was successful or unsuccessful with the >> transfer? Particularly in the case when it is writing additional >> data after the template being queued up in the kernel/OS. >> >> -- david >> >> -- >> Help mailto:majordomo@net.doit.wisc.edu and say "help" in >> message body >> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >> "unsubscribe ipfix" in message body >> Archive http://ipfix.doit.wisc.edu/archive/ > > -- > > **************************************************** > * Mark Thibodeau, P Eng. > * 670 RSP Development - Firmware Designer > * Alcatel CID - (613) 784-5375 > * Fax - (613) 599-3696 > * mark.thibodeau@alcatel.com > **************************************************** > --------------5B01015B05D427F256F37F8B Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit Mark,

IPFIX only recommends that the transport protocol be congestion aware. So,..even
DCCP (a congestion aware variant of UDP) qualifies in this case as a viable option.

Regards,
Alex.

Mark Thibodeau wrote:

I agree with both Nevil and David.  The TCP ACK is not enough information to determine if the Template made it to the Server or not.

I was originallly thinking of an application where the exporter has access to the guts of the TCP stack.  If the TCP ACK never came for the Template, the exporter would resend it until it had positive acknowledgement.

But, David is right.  Most applications will probably just use a standard stack and forget about it.  The information about what was ACKed is not propagated to the exporter.

I'm not sure about Nevil's point though.  From the IPFIX meeting minutes:
 

The evaluation team has recommended NetFlow v9 as a basis for the IPFIX
protocol.......Then, specify the initial IPFIX transport be TCP.....


Nevil's email seems to say that we will accept either TCP or UDP ("to make it usable in a minimal environment").  What are we saying?  IPFIX recommends TCP, but if an application wants to still use UDP then that's OK.  It seems like we haven't made a decision at all on the transport protocol.  When IPFIX becomes a standard, what will the transport protocol be?  If an application does not use that transport protocol, then I assume that it does not comply with the standard.

Or am I totally out to lunch.  :-()

Cheers,
Mark
 
 

David Moore wrote:

On Fri, May 16, 2003 at 03:11:21AM +1200, Nevil Brownlee wrote:
> Quoting Mark Thibodeau <mark.thibodeau@alcatel.com>:
> > As for lost template definitions, this should not happen.  Since IPFIX
> > will be using TCP, the exporter will know whether or not the template
> > was received successfully.
>
> There's been some apparent confusion on the list about transport for IPFIX.
> Consensus from the last IPFIX meeting was that the 1.0 version of IPFIX
> could use TCP transport [so as to make it easier for developers to implement
> and test], but that version 1.0 would NOT rely on two-way messages between
> exporter and collector [so that it would be usable in a minimal environment].
> So Mark's statement above is not a sufficient answer.

I'm probably missing something, but how does just using TCP ensure that
the exporter knows that it was received successfully?  (especially
in environments where you can't reach into the guts of TCP).

If the process hands a large buffer to the kernel/OS, it will take
it and let the process continue.  Some or all of that data may be
successfully transmitted via TCP to the receiver.  If the connection
fails, the kernel/OS knows how much of it made it (to the other
kernel, not necessarily application), but it doesn't tell that
information to the process.  All that the process knows is that it
is later told that the connection is gone.  How does it determine
from this whether it was successful or unsuccessful with the
transfer?  Particularly in the case when it is writing additional
data after the template being queued up in the kernel/OS.

-- david

--
Help        mailto:majordomo@net.doit.wisc.edu and say "help" in message body
Unsubscribe mailto:majordomo@net.doit.wisc.edu and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

--

****************************************************
* Mark Thibodeau, P Eng.
* 670 RSP Development - Firmware Designer
* Alcatel CID - (613) 784-5375
* Fax - (613) 599-3696
* mark.thibodeau@alcatel.com
****************************************************
 

--------------5B01015B05D427F256F37F8B-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu May 15 21:24:33 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA29030 for ; Thu, 15 May 2003 21:24:33 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19GTdZ-0006W5-00 for ipfix-list@mil.doit.wisc.edu; Thu, 15 May 2003 20:03:53 -0500 Received: from eng4.oar.net ([192.148.244.24]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 19GTdX-0006Vz-00 for ipfix@net.doit.wisc.edu; Thu, 15 May 2003 20:03:51 -0500 Received: (qmail 13131 invoked by uid 4454); 16 May 2003 01:03:51 -0000 Date: Thu, 15 May 2003 21:03:50 -0400 From: Mark Fullmer To: Mark Thibodeau Cc: David Moore , Nevil Brownlee , ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Summary up to now Message-ID: <20030515210350.A13044@net.ohio-state.edu> References: <0A11633F61BD9F40B43ABCC694004F930218EF04@zsc3c026.us.nortel.com> <3EC39D49.770E896E@alcatel.com> <1053011481.2d22d7efff88b@hotlava.auckland.ac.nz> <20030515092422.Y46386@login.caida.org> <3EC3D0CB.F6E77022@alcatel.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: <3EC3D0CB.F6E77022@alcatel.com>; from mark.thibodeau@alcatel.com on Thu, May 15, 2003 at 01:39:23PM -0400 Precedence: bulk Sender: majordomo listserver On Thu, May 15, 2003 at 01:39:23PM -0400, Mark Thibodeau wrote: > I agree with both Nevil and David. The TCP ACK is not enough information to > determine if the Template made it to the Server or not. A periodic application level ACK is only needed to ensure that flow records in flight (in the exporter TCP/IP stack, on the wire, in the collector TCP/IP stack) are not lost when switching from a primary to backup collector, or potentially re-establishing a lost connection to a (primary) collector. Collector ACK's only need to be done if the exporter has the resources to resend the window of flows between ACK's, ie it's optional. I'm assuming for simplicity that an exporter will always send a template before sending data records that use it, I don't think this is an unreasonable assumption. This restriction could probably be lifted if the collector only ACK'd flows it could decode (ie it had the template). If you believe otherwise please provide a concrete example. mark -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu May 15 21:40:40 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA29226 for ; Thu, 15 May 2003 21:40:40 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19GTqH-0006lZ-00 for ipfix-list@mil.doit.wisc.edu; Thu, 15 May 2003 20:17:01 -0500 Received: from eng4.oar.net ([192.148.244.24]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 19GTqF-0006lK-00 for ipfix@net.doit.wisc.edu; Thu, 15 May 2003 20:16:59 -0500 Received: (qmail 13172 invoked by uid 4454); 16 May 2003 01:16:59 -0000 Date: Thu, 15 May 2003 21:16:59 -0400 From: Mark Fullmer To: Tal Givoly Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Lost Flow Records - Synch Message Message-ID: <20030515211659.B13044@net.ohio-state.edu> References: <20030512112040.A90471@net.ohio-state.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: ; from givoly@xacct.com on Mon, May 12, 2003 at 09:01:15AM -0700 Precedence: bulk Sender: majordomo listserver I don't think there's much more than periodic retransmission of templates and some option data to support a lossy control channel. v9 has lifetimes for templates which is one way to solve some issues with a connectionless lossy transport, that is something else that can go away. I agree though, if we don't need to support this I'd rather not, but there's still DCCP. mark On Mon, May 12, 2003 at 09:01:15AM -0700, Tal Givoly wrote: > Mark, > > But even if we do use SCTP-PR, the "control" information (such as templates, > indication of synch messages, etc.) is most likely to be transported over > the reliable SCTP link rather than the partially reliable one. This was > suggested by Benoit in the IETF meeting. In that case, should we introduce > undue complexity by supportinging arbitrary theoretical protocols? > > Tal > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri May 16 02:31:03 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id CAA14986 for ; Fri, 16 May 2003 02:31:02 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19GYcS-0004wC-00 for ipfix-list@mil.doit.wisc.edu; Fri, 16 May 2003 01:23:04 -0500 Received: from usmail.xacct.com ([204.253.100.12]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19GYcQ-0004w1-00 for ipfix@net.doit.wisc.edu; Fri, 16 May 2003 01:23:02 -0500 Received: from Givoly (000-023-831.area1.spcsdns.net [68.24.93.210]) by usmail.xacct.com (8.11.2/8.11.2) with SMTP id h4G6Tbi18483; Thu, 15 May 2003 23:29:43 -0700 From: "Tal Givoly" To: "Mark Fullmer" Cc: Subject: RE: [ipfix] Lost Flow Records - Synch Message Date: Thu, 15 May 2003 23:22:45 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-Reply-To: <20030515211659.B13044@net.ohio-state.edu> Importance: Normal Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Mark, It seems to me pretty obvious that periodic retransmission of templates introduces undue complexity in the collector to buffer data for an arbitrary period of time until such time that it knows how to decode packets for which it has not received the templates. By transmitting only the templates (not necessarily the data) over a transport that is reliable (such as TCP or SCTP), you avoid this undue complexity. This doesn't imply that the data itself doesn't flow over a partially reliable transport (SCTP-PR or UDP). Furthermore, templates don't need lifetimes. Templates could remain in effect from the beginning of a session until changed during that session or by tearing down that session and establishing a new one for the new templates. Lifetimes introduce yet another unnecessary complexity. In general, the way templates are defined in NetFlow v9 (with transient identifier) implies that different record structures (templates) must be identified IMPLICITLY by virtue of the specific set of attributes within the template. So supposing a collector needs to decode a set of templates in a communication - an attribute-by-attribute comparison needs to be used to identify what is the meaning of the data in the template. There is no reference to some externally defined record structure. I believe this is going to be one of the major drawbacks of the current template mechanism (this item has been discussed on the mailing list - http://ipfx.doit.wisc.edu/list/ipfix/archive/1336.html). Tal -----Original Message----- From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu]On Behalf Of Mark Fullmer Sent: Thursday, May 15, 2003 6:17 PM To: Tal Givoly Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Lost Flow Records - Synch Message I don't think there's much more than periodic retransmission of templates and some option data to support a lossy control channel. v9 has lifetimes for templates which is one way to solve some issues with a connectionless lossy transport, that is something else that can go away. I agree though, if we don't need to support this I'd rather not, but there's still DCCP. mark On Mon, May 12, 2003 at 09:01:15AM -0700, Tal Givoly wrote: > Mark, > > But even if we do use SCTP-PR, the "control" information (such as templates, > indication of synch messages, etc.) is most likely to be transported over > the reliable SCTP link rather than the partially reliable one. This was > suggested by Benoit in the IETF meeting. In that case, should we introduce > undue complexity by supportinging arbitrary theoretical protocols? > > Tal > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri May 16 19:00:40 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA14154 for ; Fri, 16 May 2003 19:00:40 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19Gne8-0002hU-00 for ipfix-list@mil.doit.wisc.edu; Fri, 16 May 2003 17:25:48 -0500 Received: from halt-in.cisco.com ([171.70.144.185]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19Gne7-0002hP-00 for ipfix@net.doit.wisc.edu; Fri, 16 May 2003 17:25:47 -0500 Received: from cisco.com (171.71.163.13) by halt-in.cisco.com with ESMTP; 16 May 2003 15:25:38 -0800 Received: from cisco.com (sjc-vpn4-173.cisco.com [10.21.80.173]) by mira-sjc5-f.cisco.com (Mirapoint Messaging Server MOS 3.3.3-GR) with ESMTP id AGH83512; Fri, 16 May 2003 15:32:51 -0700 (PDT) Message-ID: <3EC56569.A9749A23@cisco.com> Date: Fri, 16 May 2003 15:25:45 -0700 From: Ganesh Sadasivan Organization: Cisco Systems Inc X-Mailer: Mozilla 4.76 [en]C-CCK-MCD (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Tal Givoly CC: Mark Fullmer , ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Lost Flow Records - Synch Message References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit - Periodic refreshing would be good - especially in a lossy network and when template config change is taking place even if the transport is reliable. Periodicity can be configured, based on the exporter and the network through which the export data traverses. - The same holds good for life-time for templates. The end-of-life is an alternative to using option templates or some other means to that a template is unconfigured. Thanks Ganesh Tal Givoly wrote: > Mark, > > It seems to me pretty obvious that periodic retransmission of templates > introduces undue complexity in the collector to buffer data for an arbitrary > period of time until such time that it knows how to decode packets for which > it has not received the templates. By transmitting only the templates (not > necessarily the data) over a transport that is reliable (such as TCP or > SCTP), you avoid this undue complexity. This doesn't imply that the data > itself doesn't flow over a partially reliable transport (SCTP-PR or UDP). > > Furthermore, templates don't need lifetimes. Templates could remain in > effect from the beginning of a session until changed during that session or > by tearing down that session and establishing a new one for the new > templates. Lifetimes introduce yet another unnecessary complexity. > > In general, the way templates are defined in NetFlow v9 (with transient > identifier) implies that different record structures (templates) must be > identified IMPLICITLY by virtue of the specific set of attributes within the > template. So supposing a collector needs to decode a set of templates in a > communication - an attribute-by-attribute comparison needs to be used to > identify what is the meaning of the data in the template. There is no > reference to some externally defined record structure. I believe this is > going to be one of the major drawbacks of the current template mechanism > (this item has been discussed on the mailing list - > http://ipfx.doit.wisc.edu/list/ipfix/archive/1336.html). > > Tal > > -----Original Message----- > From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu]On Behalf > Of Mark Fullmer > Sent: Thursday, May 15, 2003 6:17 PM > To: Tal Givoly > Cc: ipfix@net.doit.wisc.edu > Subject: Re: [ipfix] Lost Flow Records - Synch Message > > I don't think there's much more than periodic retransmission > of templates and some option data to support a lossy control channel. > > v9 has lifetimes for templates which is one way to solve some issues > with a connectionless lossy transport, that is something else that > can go away. > > I agree though, if we don't need to support this I'd rather not, but there's > still DCCP. > > mark > > On Mon, May 12, 2003 at 09:01:15AM -0700, Tal Givoly wrote: > > Mark, > > > > But even if we do use SCTP-PR, the "control" information (such as > templates, > > indication of synch messages, etc.) is most likely to be transported over > > the reliable SCTP link rather than the partially reliable one. This was > > suggested by Benoit in the IETF meeting. In that case, should we introduce > > undue complexity by supportinging arbitrary theoretical protocols? > > > > Tal > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message > body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri May 16 19:00:41 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA14156 for ; Fri, 16 May 2003 19:00:41 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19GmrM-0001gd-00 for ipfix-list@mil.doit.wisc.edu; Fri, 16 May 2003 16:35:24 -0500 Received: from halt-in.cisco.com ([171.70.144.185]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19GmrJ-0001gW-00 for ipfix@net.doit.wisc.edu; Fri, 16 May 2003 16:35:22 -0500 Received: from cisco.com (171.71.163.13) by halt-in.cisco.com with ESMTP; 16 May 2003 14:35:10 -0800 Received: from cisco.com (sjc-vpn4-173.cisco.com [10.21.80.173]) by mira-sjc5-f.cisco.com (Mirapoint Messaging Server MOS 3.3.3-GR) with ESMTP id AGH78378; Fri, 16 May 2003 14:42:25 -0700 (PDT) Message-ID: <3EC55996.F610236@cisco.com> Date: Fri, 16 May 2003 14:35:19 -0700 From: Ganesh Sadasivan Organization: Cisco Systems Inc X-Mailer: Mozilla 4.76 [en]C-CCK-MCD (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: ipfix@net.doit.wisc.edu CC: Ganesh Sadasivan Subject: [ipfix] Changes to architecture doc. Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hi, In going through the architecture spec. I find that the following topics needs to be modified. I'll list all of them in one e-mail and then on a topic-by-topic we can have separate e-mail threads. BTW I could not get a copy of the arch. spec from the link http://www.ietf.org/internet-drafts/draft-ietf-ipfix-architecture-02.txt This has been deleted from the internet drafts database because it was not updated for more than 6 months. I guess the same is true with data model document. The alternate link that I got it from is : http://www.ietf.org/proceedings/02nov/I-D/draft-ietf-ipfix-architecture-02.txt * Now that we have chosen the IPFIX protocol, is a separate document needed for architecture and protocols? IMHO yes. The architecture discusses the basic terminology (like flow definition) components of IPFIX framework, different models and association between these components which are fairly static in nature. More over the chosen protocol may not fit into all the models described in the arch. spec. The protocol on the other hand discusses message etc which could keep evolving to newer versions. * Section 2 : Scope There are a few points mentioned here : " * Define the criteria to select the IPFIX Protocol." We have already selected the protocol. Do we still need to keep this? " * Specify the control/data message formats and handshaking details to pass the IP flow information. " This is really protocol specific and should not be discussed in this document. * Section 4: IPFIX Device figure A device could have multiple exporters and each could have 1:n mapping with observation domain. This is not well captured in the figure. * Section 4.3 Observation Domain Functions of Observation domain. " * Encoding the flow records and sending them to the export process. * Encoding the control information into templates and sending them to the export process." The above 2 are already covered by IPFIX protocol. " * Deciding which flow records/control information to export using rules based on time, thresholds, configuration events etc. * Aggregating flow records generated by one or more metering processes." The above 2 are also IPFIX protocol function. So also is flow expiration. Observation point is a logical block which maintains flows for a a set of {observation point, metering process} (in a database form) and maintains aggregate statistics. All the other flow related functions should be handled by IPFIX protocol and metering process. * Section 4.5 : " There MAY be additional rules defined within the observation domain so that only certain flows records are picked up for export. " Though rules are defined w.r.t observation domain, the entity that applies these rules is the IPFIX protocol. * Section 5.1 Selection Criteria for IPFIX Protocol The first paragraph of this section is not required now. But the properties (5.1.1 and 5.1.2) still hold good. * Section 5.1.3 IMO we can remove this. * Section 5.1.4 Remove the line "Once the selection is made from the set of candidate protocols, this section would be replaced by the chosen protocol." * We should add a some description on netflow v9 in section 5 not getting into too much details. * Section 6.2 IPFIX end point authentication Do we really need this section. Doesn't section 6.1.x take care of what is mentioned in 6.2? Is there any plan now or in the future to achieve this through netflow v9? * Section 6.3 Denial of service (DoS) attack prevention This section is quite superficial and I'd suggest we either write one or more concrete ways to achieve this or remove this section. * Section 7 Flow Expiration This should appear as a subsection in IPFIX protocol. Thanks Ganesh -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri May 16 19:31:05 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA14569 for ; Fri, 16 May 2003 19:31:05 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19GoSR-0003lg-00 for ipfix-list@mil.doit.wisc.edu; Fri, 16 May 2003 18:17:47 -0500 Received: from usmail.xacct.com ([204.253.100.12]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19GoSP-0003lR-00 for ipfix@net.doit.wisc.edu; Fri, 16 May 2003 18:17:45 -0500 Received: from Givoly (inside.us.xacct.com [204.253.100.102]) by usmail.xacct.com (8.11.2/8.11.2) with SMTP id h4GNO2i29743; Fri, 16 May 2003 16:24:02 -0700 From: "Tal Givoly" To: "Ganesh Sadasivan" Cc: "Mark Fullmer" , Subject: RE: [ipfix] Lost Flow Records - Synch Message Date: Fri, 16 May 2003 16:17:09 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-Reply-To: <3EC56569.A9749A23@cisco.com> Importance: Normal Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Ganesh, Perhaps it's only me, but I still fail to understand why periodic refreshing is good or even necessary if the control information is sent over TCP or SCTP? Once a connection is established between an exporter and a collector, the templates could be communicated and remain valid for the duration of the session (or until changed). Also, life-time does not seem to be a good solution for changing templates. What if the life-time implies the template is still valid, but the template must be changed at that point? That case already calls for the need to have some method to "reconfigure" the template. If, on the other hand, templates are short-lived, that implies excessive overhead is spent on communicating these templates. Both of these items seem to be UDP-related legacy that add little or no value and undue complexity once you can use either TCP or SCTP (at least for the control information). Tal -----Original Message----- From: Ganesh Sadasivan [mailto:gsadasiv@cisco.com] Sent: Friday, May 16, 2003 3:26 PM To: Tal Givoly Cc: Mark Fullmer; ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Lost Flow Records - Synch Message - Periodic refreshing would be good - especially in a lossy network and when template config change is taking place even if the transport is reliable. Periodicity can be configured, based on the exporter and the network through which the export data traverses. - The same holds good for life-time for templates. The end-of-life is an alternative to using option templates or some other means to that a template is unconfigured. Thanks Ganesh Tal Givoly wrote: > Mark, > > It seems to me pretty obvious that periodic retransmission of templates > introduces undue complexity in the collector to buffer data for an arbitrary > period of time until such time that it knows how to decode packets for which > it has not received the templates. By transmitting only the templates (not > necessarily the data) over a transport that is reliable (such as TCP or > SCTP), you avoid this undue complexity. This doesn't imply that the data > itself doesn't flow over a partially reliable transport (SCTP-PR or UDP). > > Furthermore, templates don't need lifetimes. Templates could remain in > effect from the beginning of a session until changed during that session or > by tearing down that session and establishing a new one for the new > templates. Lifetimes introduce yet another unnecessary complexity. > > In general, the way templates are defined in NetFlow v9 (with transient > identifier) implies that different record structures (templates) must be > identified IMPLICITLY by virtue of the specific set of attributes within the > template. So supposing a collector needs to decode a set of templates in a > communication - an attribute-by-attribute comparison needs to be used to > identify what is the meaning of the data in the template. There is no > reference to some externally defined record structure. I believe this is > going to be one of the major drawbacks of the current template mechanism > (this item has been discussed on the mailing list - > http://ipfx.doit.wisc.edu/list/ipfix/archive/1336.html). > > Tal > > -----Original Message----- > From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu]On Behalf > Of Mark Fullmer > Sent: Thursday, May 15, 2003 6:17 PM > To: Tal Givoly > Cc: ipfix@net.doit.wisc.edu > Subject: Re: [ipfix] Lost Flow Records - Synch Message > > I don't think there's much more than periodic retransmission > of templates and some option data to support a lossy control channel. > > v9 has lifetimes for templates which is one way to solve some issues > with a connectionless lossy transport, that is something else that > can go away. > > I agree though, if we don't need to support this I'd rather not, but there's > still DCCP. > > mark > > On Mon, May 12, 2003 at 09:01:15AM -0700, Tal Givoly wrote: > > Mark, > > > > But even if we do use SCTP-PR, the "control" information (such as > templates, > > indication of synch messages, etc.) is most likely to be transported over > > the reliable SCTP link rather than the partially reliable one. This was > > suggested by Benoit in the IETF meeting. In that case, should we introduce > > undue complexity by supportinging arbitrary theoretical protocols? > > > > Tal > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message > body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri May 16 21:16:58 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA16103 for ; Fri, 16 May 2003 21:16:57 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19GqD7-0005zU-00 for ipfix-list@mil.doit.wisc.edu; Fri, 16 May 2003 20:10:05 -0500 Received: from halt-in.cisco.com ([171.70.144.185]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19GqD5-0005zL-00 for ipfix@net.doit.wisc.edu; Fri, 16 May 2003 20:10:04 -0500 Received: from cisco.com (171.71.163.13) by halt-in.cisco.com with ESMTP; 16 May 2003 18:09:48 -0800 Received: from cisco.com (sjc-vpn4-173.cisco.com [10.21.80.173]) by mira-sjc5-f.cisco.com (Mirapoint Messaging Server MOS 3.3.3-GR) with ESMTP id AGH98421; Fri, 16 May 2003 18:17:08 -0700 (PDT) Message-ID: <3EC58BEA.BED9DD76@cisco.com> Date: Fri, 16 May 2003 18:10:02 -0700 From: Ganesh Sadasivan Organization: Cisco Systems Inc X-Mailer: Mozilla 4.76 [en]C-CCK-MCD (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Tal Givoly CC: Mark Fullmer , ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Lost Flow Records - Synch Message References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Tal, Tal Givoly wrote: > Ganesh, > > Perhaps it's only me, but I still fail to understand why periodic refreshing > is good or even necessary if the control information is sent over TCP or > SCTP? Once a connection is established between an exporter and a collector, > the templates could be communicated and remain valid for the duration of the > session (or until changed). In the event of congestion, though TCP does best effort, packet could be dropped right at the exporter, anywhere in the network or at the collector even though the connection still remains valid. It is quite undesirable if the lost packet is one that contains templates. If there is no periodic update, then collector will never be able to decode flow records corresponding to the lost templates. > > Also, life-time does not seem to be a good solution for changing templates. > What if the life-time implies the template is still valid, but the template > must be changed at that point? That case already calls for the need to have > some method to "reconfigure" the template. If, on the other hand, templates > are short-lived, that implies excessive overhead is spent on communicating > these templates. > The reconfigured templates have a different ID. The old ones silently age out. The life-time value is deployment specific. > > Both of these items seem to be UDP-related legacy that add little or no > value and undue complexity once you can use either TCP or SCTP (at least for > the control information). > > Tal > > -----Original Message----- > From: Ganesh Sadasivan [mailto:gsadasiv@cisco.com] > Sent: Friday, May 16, 2003 3:26 PM > To: Tal Givoly > Cc: Mark Fullmer; ipfix@net.doit.wisc.edu > Subject: Re: [ipfix] Lost Flow Records - Synch Message > > - Periodic refreshing would be good - especially in a lossy network and > when template config change is taking place even if the transport is > reliable. Periodicity can be configured, based on the exporter and the > network through which the export data traverses. > > - The same holds good for life-time for templates. The end-of-life is > an alternative to using option templates or some other means to that > a template is unconfigured. > > Thanks > Ganesh > > Tal Givoly wrote: > > > Mark, > > > > It seems to me pretty obvious that periodic retransmission of templates > > introduces undue complexity in the collector to buffer data for an > arbitrary > > period of time until such time that it knows how to decode packets for > which > > it has not received the templates. By transmitting only the templates (not > > necessarily the data) over a transport that is reliable (such as TCP or > > SCTP), you avoid this undue complexity. This doesn't imply that the data > > itself doesn't flow over a partially reliable transport (SCTP-PR or UDP). > > > > Furthermore, templates don't need lifetimes. Templates could remain in > > effect from the beginning of a session until changed during that session > or > > by tearing down that session and establishing a new one for the new > > templates. Lifetimes introduce yet another unnecessary complexity. > > > > In general, the way templates are defined in NetFlow v9 (with transient > > identifier) implies that different record structures (templates) must be > > identified IMPLICITLY by virtue of the specific set of attributes within > the > > template. So supposing a collector needs to decode a set of templates in a > > communication - an attribute-by-attribute comparison needs to be used to > > identify what is the meaning of the data in the template. There is no > > reference to some externally defined record structure. I believe this is > > going to be one of the major drawbacks of the current template mechanism > > (this item has been discussed on the mailing list - > > http://ipfx.doit.wisc.edu/list/ipfix/archive/1336.html). > > > > Tal > > > > -----Original Message----- > > From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu]On Behalf > > Of Mark Fullmer > > Sent: Thursday, May 15, 2003 6:17 PM > > To: Tal Givoly > > Cc: ipfix@net.doit.wisc.edu > > Subject: Re: [ipfix] Lost Flow Records - Synch Message > > > > I don't think there's much more than periodic retransmission > > of templates and some option data to support a lossy control channel. > > > > v9 has lifetimes for templates which is one way to solve some issues > > with a connectionless lossy transport, that is something else that > > can go away. > > > > I agree though, if we don't need to support this I'd rather not, but > there's > > still DCCP. > > > > mark > > > > On Mon, May 12, 2003 at 09:01:15AM -0700, Tal Givoly wrote: > > > Mark, > > > > > > But even if we do use SCTP-PR, the "control" information (such as > > templates, > > > indication of synch messages, etc.) is most likely to be transported > over > > > the reliable SCTP link rather than the partially reliable one. This was > > > suggested by Benoit in the IETF meeting. In that case, should we > introduce > > > undue complexity by supportinging arbitrary theoretical protocols? > > > > > > Tal > > > > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message > > body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message > body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Sun May 18 12:43:48 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA12500 for ; Sun, 18 May 2003 12:43:48 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19HQk1-0005jN-00 for ipfix-list@mil.doit.wisc.edu; Sun, 18 May 2003 11:10:29 -0500 Received: from usmail.xacct.com ([204.253.100.12]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19HQjz-0005jB-00 for ipfix@net.doit.wisc.edu; Sun, 18 May 2003 11:10:27 -0500 Received: from Givoly ([192.168.0.3]) by usmail.xacct.com (8.11.2/8.11.2) with SMTP id h4IGH0i17565; Sun, 18 May 2003 09:17:01 -0700 From: "Tal Givoly" To: "Ganesh Sadasivan" Cc: "Mark Fullmer" , Subject: RE: [ipfix] Lost Flow Records - Synch Message Date: Sun, 18 May 2003 09:10:05 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-Reply-To: <3EC58BEA.BED9DD76@cisco.com> Importance: Normal Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Ganesh, Responses interspersed below: -----Original Message----- From: Ganesh Sadasivan [mailto:gsadasiv@cisco.com] Sent: Friday, May 16, 2003 6:10 PM To: Tal Givoly Cc: Mark Fullmer; ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Lost Flow Records - Synch Message Tal, Tal Givoly wrote: > Ganesh, > > Perhaps it's only me, but I still fail to understand why periodic refreshing > is good or even necessary if the control information is sent over TCP or > SCTP? Once a connection is established between an exporter and a collector, > the templates could be communicated and remain valid for the duration of the > session (or until changed). In the event of congestion, though TCP does best effort, packet could be dropped right at the exporter, anywhere in the network or at the collector even though the connection still remains valid. It is quite undesirable if the lost packet is one that contains templates. If there is no periodic update, then collector will never be able to decode flow records corresponding to the lost templates. Yes. TCP packets are merely IP packets that could get dropped. However, this will trigger a retransmission by the TCP/IP stack and eventually a disconnection (if the packets don't get through). If it will not disconnect, the template will get to the collector - guaranteed (irrespective of how many packets get lost trying). If it does disconnect, the reconnected stream would send it. It is probably expected that there is no way to send usage data either if these, retransmitted packets, do not get through. > > Also, life-time does not seem to be a good solution for changing templates. > What if the life-time implies the template is still valid, but the template > must be changed at that point? That case already calls for the need to have > some method to "reconfigure" the template. If, on the other hand, templates > are short-lived, that implies excessive overhead is spent on communicating > these templates. > The reconfigured templates have a different ID. The old ones silently age out. The life-time value is deployment specific. Once again, the template mechanism in NetFlow v9 is arbitrary and causes undue complexity and ambiguity in the collector to "discover" the meaning of the template (semantics) from the content. The life-time introduces unnecessary complexity in the collector (for instance, what happens if raw data is stored and reprocessed at a different pace than real-time?). > > Both of these items seem to be UDP-related legacy that add little or no > value and undue complexity once you can use either TCP or SCTP (at least for > the control information). > > Tal > > -----Original Message----- > From: Ganesh Sadasivan [mailto:gsadasiv@cisco.com] > Sent: Friday, May 16, 2003 3:26 PM > To: Tal Givoly > Cc: Mark Fullmer; ipfix@net.doit.wisc.edu > Subject: Re: [ipfix] Lost Flow Records - Synch Message > > - Periodic refreshing would be good - especially in a lossy network and > when template config change is taking place even if the transport is > reliable. Periodicity can be configured, based on the exporter and the > network through which the export data traverses. > > - The same holds good for life-time for templates. The end-of-life is > an alternative to using option templates or some other means to that > a template is unconfigured. > > Thanks > Ganesh > > Tal Givoly wrote: > > > Mark, > > > > It seems to me pretty obvious that periodic retransmission of templates > > introduces undue complexity in the collector to buffer data for an > arbitrary > > period of time until such time that it knows how to decode packets for > which > > it has not received the templates. By transmitting only the templates (not > > necessarily the data) over a transport that is reliable (such as TCP or > > SCTP), you avoid this undue complexity. This doesn't imply that the data > > itself doesn't flow over a partially reliable transport (SCTP-PR or UDP). > > > > Furthermore, templates don't need lifetimes. Templates could remain in > > effect from the beginning of a session until changed during that session > or > > by tearing down that session and establishing a new one for the new > > templates. Lifetimes introduce yet another unnecessary complexity. > > > > In general, the way templates are defined in NetFlow v9 (with transient > > identifier) implies that different record structures (templates) must be > > identified IMPLICITLY by virtue of the specific set of attributes within > the > > template. So supposing a collector needs to decode a set of templates in a > > communication - an attribute-by-attribute comparison needs to be used to > > identify what is the meaning of the data in the template. There is no > > reference to some externally defined record structure. I believe this is > > going to be one of the major drawbacks of the current template mechanism > > (this item has been discussed on the mailing list - > > http://ipfx.doit.wisc.edu/list/ipfix/archive/1336.html). > > > > Tal > > > > -----Original Message----- > > From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu]On Behalf > > Of Mark Fullmer > > Sent: Thursday, May 15, 2003 6:17 PM > > To: Tal Givoly > > Cc: ipfix@net.doit.wisc.edu > > Subject: Re: [ipfix] Lost Flow Records - Synch Message > > > > I don't think there's much more than periodic retransmission > > of templates and some option data to support a lossy control channel. > > > > v9 has lifetimes for templates which is one way to solve some issues > > with a connectionless lossy transport, that is something else that > > can go away. > > > > I agree though, if we don't need to support this I'd rather not, but > there's > > still DCCP. > > > > mark > > > > On Mon, May 12, 2003 at 09:01:15AM -0700, Tal Givoly wrote: > > > Mark, > > > > > > But even if we do use SCTP-PR, the "control" information (such as > > templates, > > > indication of synch messages, etc.) is most likely to be transported > over > > > the reliable SCTP link rather than the partially reliable one. This was > > > suggested by Benoit in the IETF meeting. In that case, should we > introduce > > > undue complexity by supportinging arbitrary theoretical protocols? > > > > > > Tal > > > > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message > > body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message > body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Sun May 18 22:45:15 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA23726 for ; Sun, 18 May 2003 22:45:15 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19HaFy-000215-00 for ipfix-list@mil.doit.wisc.edu; Sun, 18 May 2003 21:20:06 -0500 Received: from halt-in.cisco.com ([171.70.144.185]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19HaFw-00020y-00 for ipfix@net.doit.wisc.edu; Sun, 18 May 2003 21:20:04 -0500 Received: from cisco.com (171.71.163.13) by halt-in.cisco.com with ESMTP; 18 May 2003 19:19:53 -0800 Received: from cisco.com (sjc-vpn3-455.cisco.com [10.21.65.199]) by mira-sjc5-f.cisco.com (Mirapoint Messaging Server MOS 3.3.3-GR) with ESMTP id AGI74535; Sun, 18 May 2003 19:27:07 -0700 (PDT) Message-ID: <3EC83F51.F96AABDB@cisco.com> Date: Sun, 18 May 2003 19:20:01 -0700 From: Ganesh Sadasivan Organization: Cisco Systems Inc X-Mailer: Mozilla 4.76 [en]C-CCK-MCD (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Tal Givoly CC: Mark Fullmer , ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Lost Flow Records - Synch Message References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Tal, Tal Givoly wrote: > Ganesh, > > Responses interspersed below: > > -----Original Message----- > From: Ganesh Sadasivan [mailto:gsadasiv@cisco.com] > Sent: Friday, May 16, 2003 6:10 PM > To: Tal Givoly > Cc: Mark Fullmer; ipfix@net.doit.wisc.edu > Subject: Re: [ipfix] Lost Flow Records - Synch Message > > Tal, > > Tal Givoly wrote: > > > Ganesh, > > > > Perhaps it's only me, but I still fail to understand why periodic > refreshing > > is good or even necessary if the control information is sent over TCP or > > SCTP? Once a connection is established between an exporter and a > collector, > > the templates could be communicated and remain valid for the duration of > the > > session (or until changed). > > In the event of congestion, though TCP does best effort, packet could be > dropped > > right at the exporter, anywhere in the network or at the collector even > though > the > connection still remains valid. It is quite undesirable if the lost packet > is > one that > contains templates. If there is no periodic update, then collector will > never be > able > to decode flow records corresponding to the lost templates. > > Yes. TCP packets are merely IP packets that could get dropped. > However, this will trigger a retransmission by the TCP/IP stack and > eventually a disconnection (if the packets don't get through). If it will > not disconnect, the template will get to the collector - guaranteed > (irrespective of how many packets get lost trying). If it does disconnect, > the reconnected stream would send it. It is probably expected that there is > no way to send usage data either if these, retransmitted packets, do not get > through. > > Yes, I agree that as long as the packet is in TCP send window of the exporter it will get to the collector through retransmission. But it is possible that the exporter is not able to queue the packet into send window due to delays in retransmission which gets dropped in exporter floor. Whether the application (in this case V9) keeps track of this loss or not, it needs to retransmit the template and this needs to happen before too much of flow records pertaining to this template reach the collector. Now since TCP is not the only targeted transport and one could use other congestion aware transport which are partially reliable, and so one can expect data loss in the contorl connection. So a periodic scheme though not super efficient works well regardless of the transport like TCP/SCTP-PR/DCCP used for control connnection. > > > > > Also, life-time does not seem to be a good solution for changing > templates. > > What if the life-time implies the template is still valid, but the > template > > must be changed at that point? That case already calls for the need to > have > > some method to "reconfigure" the template. If, on the other hand, > templates > > are short-lived, that implies excessive overhead is spent on communicating > > these templates. > > > > The reconfigured templates have a different ID. The old ones silently age > out. > The life-time value is deployment specific. > > > Once again, the template mechanism in NetFlow v9 is arbitrary and causes > undue complexity and ambiguity in the collector to "discover" the meaning of > the template (semantics) from the content. The life-time introduces > unnecessary complexity in the collector (for instance, what happens if raw > data is stored and reprocessed at a different pace than real-time?). > > Agreed that there are better means to achive the same. But I don't agree that there is any difficulty in discovering the semantics of the template as the .template ID -> data mapping for a session is unique. .So there should not be any complication whenever the records are processed. If you think this is not the case, please explain. The roll-over case is when a large number of templates are generated in a short period of time. Eg. of one such case which can lead this to happen is ACLs in which each rule maps to a flow . But are we discussing any such cases here? Thanks Ganesh > > > > > Both of these items seem to be UDP-related legacy that add little or no > > value and undue complexity once you can use either TCP or SCTP (at least > for > > the control information). > > > > Tal > > > > -----Original Message----- > > From: Ganesh Sadasivan [mailto:gsadasiv@cisco.com] > > Sent: Friday, May 16, 2003 3:26 PM > > To: Tal Givoly > > Cc: Mark Fullmer; ipfix@net.doit.wisc.edu > > Subject: Re: [ipfix] Lost Flow Records - Synch Message > > > > - Periodic refreshing would be good - especially in a lossy network and > > when template config change is taking place even if the transport is > > reliable. Periodicity can be configured, based on the exporter and the > > network through which the export data traverses. > > > > - The same holds good for life-time for templates. The end-of-life is > > an alternative to using option templates or some other means to that > > a template is unconfigured. > > > > Thanks > > Ganesh > > > > Tal Givoly wrote: > > > > > Mark, > > > > > > It seems to me pretty obvious that periodic retransmission of templates > > > introduces undue complexity in the collector to buffer data for an > > arbitrary > > > period of time until such time that it knows how to decode packets for > > which > > > it has not received the templates. By transmitting only the templates > (not > > > necessarily the data) over a transport that is reliable (such as TCP or > > > SCTP), you avoid this undue complexity. This doesn't imply that the data > > > itself doesn't flow over a partially reliable transport (SCTP-PR or > UDP). > > > > > > Furthermore, templates don't need lifetimes. Templates could remain in > > > effect from the beginning of a session until changed during that session > > or > > > by tearing down that session and establishing a new one for the new > > > templates. Lifetimes introduce yet another unnecessary complexity. > > > > > > In general, the way templates are defined in NetFlow v9 (with transient > > > identifier) implies that different record structures (templates) must be > > > identified IMPLICITLY by virtue of the specific set of attributes within > > the > > > template. So supposing a collector needs to decode a set of templates in > a > > > communication - an attribute-by-attribute comparison needs to be used to > > > identify what is the meaning of the data in the template. There is no > > > reference to some externally defined record structure. I believe this is > > > going to be one of the major drawbacks of the current template mechanism > > > (this item has been discussed on the mailing list - > > > http://ipfx.doit.wisc.edu/list/ipfix/archive/1336.html). > > > > > > Tal > > > > > > -----Original Message----- > > > From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu]On Behalf > > > Of Mark Fullmer > > > Sent: Thursday, May 15, 2003 6:17 PM > > > To: Tal Givoly > > > Cc: ipfix@net.doit.wisc.edu > > > Subject: Re: [ipfix] Lost Flow Records - Synch Message > > > > > > I don't think there's much more than periodic retransmission > > > of templates and some option data to support a lossy control channel. > > > > > > v9 has lifetimes for templates which is one way to solve some issues > > > with a connectionless lossy transport, that is something else that > > > can go away. > > > > > > I agree though, if we don't need to support this I'd rather not, but > > there's > > > still DCCP. > > > > > > mark > > > > > > On Mon, May 12, 2003 at 09:01:15AM -0700, Tal Givoly wrote: > > > > Mark, > > > > > > > > But even if we do use SCTP-PR, the "control" information (such as > > > templates, > > > > indication of synch messages, etc.) is most likely to be transported > > over > > > > the reliable SCTP link rather than the partially reliable one. This > was > > > > suggested by Benoit in the IETF meeting. In that case, should we > > introduce > > > > undue complexity by supportinging arbitrary theoretical protocols? > > > > > > > > Tal > > > > > > > > > > -- > > > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message > > > body > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > "unsubscribe ipfix" in message body > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > -- > > > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message > > body > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > "unsubscribe ipfix" in message body > > > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Sun May 18 23:05:07 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA23937 for ; Sun, 18 May 2003 23:05:07 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19Habq-0002W1-00 for ipfix-list@mil.doit.wisc.edu; Sun, 18 May 2003 21:42:42 -0500 Received: from atlrel6.hp.com ([156.153.255.205]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19Habo-0002Vv-00 for ipfix@net.doit.wisc.edu; Sun, 18 May 2003 21:42:41 -0500 Received: from xatlrelay1.atl.hp.com (xatlrelay1.atl.hp.com [15.45.89.190]) by atlrel6.hp.com (Postfix) with ESMTP id 6ED031C013B4; Sun, 18 May 2003 22:42:40 -0400 (EDT) Received: from xatlbh1.atl.hp.com (xatlbh1.atl.hp.com [15.45.89.186]) by xatlrelay1.atl.hp.com (Postfix) with ESMTP id 35DB51C009D7; Sun, 18 May 2003 22:42:40 -0400 (EDT) Received: by xatlbh1.atl.hp.com with Internet Mail Service (5.5.2655.55) id ; Sun, 18 May 2003 22:42:40 -0400 Message-ID: <1D3D2C371FCBD947A7897FABBD3533A566BAFD@xsun01.ptp.hp.com> From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" To: "'Ganesh Sadasivan'" , "'Tal Givoly'" Cc: "'Mark Fullmer'" , "'ipfix@net.doit.wisc.edu'" Subject: [ipfix] Pull Model for Template Loss Date: Sun, 18 May 2003 22:42:39 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2655.55) Content-Type: text/plain; charset="iso-8859-1" Precedence: bulk Sender: majordomo listserver Hi, An alternative to using a "push model" to enable the collector to locate the latest template, what about a pull based model. I could picture two basic models: 1. SNMP based whereby a template descriptor is made available. Rather than representing this in some binary format, I'd recommend a simple XML format. 2. A URL based scheme where FTP,TFTP or HTTP requests (one or all) would be allowed and templates retrieved. This does not require a file system, obviously the responder on the port can respond to get requests from in memory images of the templates. Alternatively a hybrid approach could be used with some URL(s) available as a MIB variable, and the URL could specify a variety of alternatives such as http://..., snmp://..., ftp://..., tftp://... I think given the rate of change of these templates, and the likelihood of drops in the UDP mode (which is the only one deployed today), enabling asynchronous retrieval would be a good thing. Regards, Jeff Meyer -----Original Message----- From: Ganesh Sadasivan [mailto:gsadasiv@cisco.com] Sent: Friday, May 16, 2003 3:26 PM To: Tal Givoly Cc: Mark Fullmer; ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Lost Flow Records - Synch Message - Periodic refreshing would be good - especially in a lossy network and when template config change is taking place even if the transport is reliable. Periodicity can be configured, based on the exporter and the network through which the export data traverses. - The same holds good for life-time for templates. The end-of-life is an alternative to using option templates or some other means to that a template is unconfigured. Thanks Ganesh Tal Givoly wrote: > Mark, > > It seems to me pretty obvious that periodic retransmission of templates > introduces undue complexity in the collector to buffer data for an arbitrary > period of time until such time that it knows how to decode packets for which > it has not received the templates. By transmitting only the templates (not > necessarily the data) over a transport that is reliable (such as TCP or > SCTP), you avoid this undue complexity. This doesn't imply that the data > itself doesn't flow over a partially reliable transport (SCTP-PR or UDP). > > Furthermore, templates don't need lifetimes. Templates could remain in > effect from the beginning of a session until changed during that session or > by tearing down that session and establishing a new one for the new > templates. Lifetimes introduce yet another unnecessary complexity. > > In general, the way templates are defined in NetFlow v9 (with transient > identifier) implies that different record structures (templates) must be > identified IMPLICITLY by virtue of the specific set of attributes within the > template. So supposing a collector needs to decode a set of templates in a > communication - an attribute-by-attribute comparison needs to be used to > identify what is the meaning of the data in the template. There is no > reference to some externally defined record structure. I believe this is > going to be one of the major drawbacks of the current template mechanism > (this item has been discussed on the mailing list - > http://ipfx.doit.wisc.edu/list/ipfix/archive/1336.html). > > Tal > > -----Original Message----- > From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu]On Behalf > Of Mark Fullmer > Sent: Thursday, May 15, 2003 6:17 PM > To: Tal Givoly > Cc: ipfix@net.doit.wisc.edu > Subject: Re: [ipfix] Lost Flow Records - Synch Message > > I don't think there's much more than periodic retransmission > of templates and some option data to support a lossy control channel. > > v9 has lifetimes for templates which is one way to solve some issues > with a connectionless lossy transport, that is something else that > can go away. > > I agree though, if we don't need to support this I'd rather not, but there's > still DCCP. > > mark > > On Mon, May 12, 2003 at 09:01:15AM -0700, Tal Givoly wrote: > > Mark, > > > > But even if we do use SCTP-PR, the "control" information (such as > templates, > > indication of synch messages, etc.) is most likely to be transported over > > the reliable SCTP link rather than the partially reliable one. This was > > suggested by Benoit in the IETF meeting. In that case, should we introduce > > undue complexity by supportinging arbitrary theoretical protocols? > > > > Tal > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message > body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon May 19 01:23:15 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id BAA25947 for ; Mon, 19 May 2003 01:23:15 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19Hcw2-0005TF-00 for ipfix-list@mil.doit.wisc.edu; Mon, 19 May 2003 00:11:42 -0500 Received: from eng4.oar.net ([192.148.244.24]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 19Hcw0-0005T8-00 for ipfix@net.doit.wisc.edu; Mon, 19 May 2003 00:11:40 -0500 Received: (qmail 41252 invoked by uid 4454); 19 May 2003 05:11:39 -0000 Date: Mon, 19 May 2003 01:11:39 -0400 From: Mark Fullmer To: Tal Givoly Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Lost Flow Records - Synch Message Message-ID: <20030519011139.A41186@net.ohio-state.edu> References: <20030515211659.B13044@net.ohio-state.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: ; from givoly@xacct.com on Thu, May 15, 2003 at 11:22:45PM -0700 Precedence: bulk Sender: majordomo listserver On Thu, May 15, 2003 at 11:22:45PM -0700, Tal Givoly wrote: > Mark, > > It seems to me pretty obvious that periodic retransmission of templates > introduces undue complexity in the collector to buffer data for an arbitrary > period of time until such time that it knows how to decode packets for which > it has not received the templates. By transmitting only the templates (not > necessarily the data) over a transport that is reliable (such as TCP or > SCTP), you avoid this undue complexity. This doesn't imply that the data > itself doesn't flow over a partially reliable transport (SCTP-PR or UDP). I'm not sure the race condition that results in a collector receiving data encoded with a template before the template can be avoided in all situations. With a router the device may be in a situation where it starts forwarding packets and creating flow state before the template gets sent. In this situation it will either have to drop the flows or buffer them to preserve the template first rule, the latter may not be possible. With SCTP I don't think it's possible to give the reliable stream with the templates priority over the stream with flow data. A robust collector will probably want to handle with data before templates by buffering the data. A less robust one will drop the data. I'm just not ready to give up on supporting a totally unreliable transport if it's not difficult to do. > Furthermore, templates don't need lifetimes. Templates could remain in > effect from the beginning of a session until changed during that session or > by tearing down that session and establishing a new one for the new > templates. Lifetimes introduce yet another unnecessary complexity. It's my understanding that the template lifetimes are there to prevent race conditions that would exist if a router rebooted and reused the same template with a different format. Regardless, I agree the template lifetime should go. There are attributes that only need to be sent once, for example an exporter ID or sysUpTime : Real Time mapping. With a connection oriented protocol there are no issues. With a non connection oriented protocal something needs to be done to emulate a session, for example a random ID (cookie) sent along with every packet...If we need to support a non connection oriented transport. This could be done in the existing framework with an option. > In general, the way templates are defined in NetFlow v9 (with transient > identifier) implies that different record structures (templates) must be > identified IMPLICITLY by virtue of the specific set of attributes within the > template. So supposing a collector needs to decode a set of templates in a > communication - an attribute-by-attribute comparison needs to be used to > identify what is the meaning of the data in the template. There is no > reference to some externally defined record structure. I believe this is > going to be one of the major drawbacks of the current template mechanism > (this item has been discussed on the mailing list - > http://ipfx.doit.wisc.edu/list/ipfix/archive/1336.html). Even without transient ID's there are problems, for example what if two access lists mapped packets into two flow cache's that used the same template. The collector would not be able to map the flows back to rules on the router. This is a documentation problem though, not a protocol problem. There are many straightforward ways to accomodate this behavior, such as an id in the flow or supplementary template info. mark -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon May 19 01:25:13 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id BAA25999 for ; Mon, 19 May 2003 01:25:12 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19Hd0U-0005Zm-00 for ipfix-list@mil.doit.wisc.edu; Mon, 19 May 2003 00:16:18 -0500 Received: from eng4.oar.net ([192.148.244.24]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 19Hd0S-0005Zg-00 for ipfix@net.doit.wisc.edu; Mon, 19 May 2003 00:16:16 -0500 Received: (qmail 41289 invoked by uid 4454); 19 May 2003 05:16:16 -0000 Date: Mon, 19 May 2003 01:16:16 -0400 From: Mark Fullmer To: Ganesh Sadasivan Cc: Tal Givoly , ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Lost Flow Records - Synch Message Message-ID: <20030519011616.A41262@net.ohio-state.edu> References: <3EC83F51.F96AABDB@cisco.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: <3EC83F51.F96AABDB@cisco.com>; from gsadasiv@cisco.com on Sun, May 18, 2003 at 07:20:01PM -0700 Precedence: bulk Sender: majordomo listserver If this happens the application layer write() will fail and the template can immediatly be marked for retransmission. I don't think the refresh is needed for a reliable control connection such as TCP or SCTP/PR-SCTP. mark On Sun, May 18, 2003 at 07:20:01PM -0700, Ganesh Sadasivan wrote: > Yes, I agree that as long as the packet is in TCP send window of the exporter > it will get to the collector through retransmission. But it is possible that the > > exporter is not able to queue the packet into send window due to delays > in retransmission which gets dropped in exporter floor. Whether the application > (in this case V9) keeps track of this loss or not, it needs to retransmit the > template > and this needs to happen before too much of flow records pertaining to this > template reach the collector. > Now since TCP is not the only targeted transport and one could use other > congestion aware transport which are partially reliable, and so one can expect > data > loss in the contorl connection. So a periodic scheme though not super efficient > works well regardless of the transport like TCP/SCTP-PR/DCCP used > for control connnection. -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon May 19 14:41:00 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA00276 for ; Mon, 19 May 2003 14:41:00 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19HpH4-0007Wz-00 for ipfix-list@mil.doit.wisc.edu; Mon, 19 May 2003 13:22:14 -0500 Received: from halt-in.cisco.com ([171.70.144.185]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19HpH2-0007Wt-00 for ipfix@net.doit.wisc.edu; Mon, 19 May 2003 13:22:12 -0500 Received: from cisco.com (171.71.163.13) by halt-in.cisco.com with ESMTP; 19 May 2003 11:22:04 -0800 Received: from cisco.com (sjc-vpn3-455.cisco.com [10.21.65.199]) by mira-sjc5-f.cisco.com (Mirapoint Messaging Server MOS 3.3.3-GR) with ESMTP id AGJ25131; Mon, 19 May 2003 11:29:16 -0700 (PDT) Message-ID: <3EC920D2.B8F0A79C@cisco.com> Date: Mon, 19 May 2003 11:22:10 -0700 From: Ganesh Sadasivan Organization: Cisco Systems Inc X-Mailer: Mozilla 4.76 [en]C-CCK-MCD (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Mark Fullmer CC: Tal Givoly , ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Lost Flow Records - Synch Message References: <3EC83F51.F96AABDB@cisco.com> <20030519011616.A41262@net.ohio-state.edu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Mark Fullmer wrote: > If this happens the application layer write() will fail and the template > can immediatly be marked for retransmission. Yes if it is a blocking send. This need not be the case for a non-blocking send. Ganesh > > > I don't think the refresh is needed for a reliable control connection > such as TCP or SCTP/PR-SCTP. > > mark > > On Sun, May 18, 2003 at 07:20:01PM -0700, Ganesh Sadasivan wrote: > > Yes, I agree that as long as the packet is in TCP send window of the exporter > > it will get to the collector through retransmission. But it is possible that the > > > > exporter is not able to queue the packet into send window due to delays > > in retransmission which gets dropped in exporter floor. Whether the application > > (in this case V9) keeps track of this loss or not, it needs to retransmit the > > template > > and this needs to happen before too much of flow records pertaining to this > > template reach the collector. > > Now since TCP is not the only targeted transport and one could use other > > congestion aware transport which are partially reliable, and so one can expect > > data > > loss in the contorl connection. So a periodic scheme though not super efficient > > works well regardless of the transport like TCP/SCTP-PR/DCCP used > > for control connnection. > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue May 20 00:55:38 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA16226 for ; Tue, 20 May 2003 00:55:37 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19Hyo5-0003yg-00 for ipfix-list@mil.doit.wisc.edu; Mon, 19 May 2003 23:32:57 -0500 Received: from web80009.mail.yahoo.com ([66.163.168.139]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 19Hyo3-0003yY-00 for ipfix@net.doit.wisc.edu; Mon, 19 May 2003 23:32:55 -0500 Message-ID: <20030520043254.8957.qmail@web80009.mail.yahoo.com> Received: from [68.120.128.206] by web80009.mail.yahoo.com via HTTP; Mon, 19 May 2003 21:32:54 PDT Date: Mon, 19 May 2003 21:32:54 -0700 (PDT) From: Peter Ludemann Subject: Re: [ipfix] Lost Flow Records - Synch Message To: ipfix@net.doit.wisc.edu In-Reply-To: <20030519011616.A41262@net.ohio-state.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Precedence: bulk Sender: majordomo listserver A small clarification of what Mark Fullmer wrote: a *later* application layer write or close will fail if the data cannot be transmitted at all through repeated (automatic) retries at the TCP layer [which is out of the control of application layer programmers]. This is why even though TCP has transport-layer ACKs, you also need application layer ACK -- TCP will tell you if it cannot deliver data but none of the implementations that I've seen will tell you what data wasn't delivered [and this is also why there is a possibility of duplication with fail-over processing]. Also, TCP can take a very long time to time out, so you may want additional timers running to force fail-over sooner. The common way of handling this is with non-blocking I/O (in Unix-like systems, this is done with select() [the old way] or poll() [the new way]; Java 1.4's "nio" package has "channels" with similar functionality). (Note that the only difference between blocking and non-blocking I/O for output is that blocking will wait if the transmit buffer is full whereas non-blocking will always return ... in terms of detection of inability to determine which bytes have been successfully transmitted, both methods are the same.) I don't know enough details about SCTP implementations to know whether application layer ACKs are necessary or not; but I suspect that they are because it's not the job of the transport layer to wait until the application decides that the data has been sufficiently processed. For excellent references on this, I suggest: W. Richard Stevens: TPC/IP Illustrated, which has the *BSD source code plus excellent over-all commentary on TCP/IP and details. This code forms the basis of the various BSD operating systems, Solaris, Linux, and (I think) Windows-XP [there have been rewrites, but this code is still the base as far as I know]. VxWorks now also BSD-derived code; I don't know about other RTOSs, nor Cisco's IOS. (This book also has the code for UDP, IP, ICMP, etc.) W. Richard Stevens: Unix Network Programming Jon C. Snader, Effective TCP/IP Programming Douglas Comer, Internetworking with TCP/IP. All these books are available in multi-volume sets from Amazon and other booksellers. - peter --- Mark Fullmer wrote: > If this happens the application layer write() will fail and > the template > can immediatly be marked for retransmission. > > I don't think the refresh is needed for a reliable control > connection > such as TCP or SCTP/PR-SCTP. > > mark > > On Sun, May 18, 2003 at 07:20:01PM -0700, Ganesh Sadasivan > wrote: > > Yes, I agree that as long as the packet is in TCP send > window of the exporter > > it will get to the collector through retransmission. But > it is possible that the > > > > exporter is not able to queue the packet into send window > due to delays > > in retransmission which gets dropped in exporter floor. > Whether the application > > (in this case V9) keeps track of this loss or not, it > needs to retransmit the > > template > > and this needs to happen before too much of flow records > pertaining to this > > template reach the collector. > > Now since TCP is not the only targeted transport and one > could use other > > congestion aware transport which are partially reliable, > and so one can expect > > data > > loss in the contorl connection. So a periodic scheme > though not super efficient > > works well regardless of the transport like > TCP/SCTP-PR/DCCP used > > for control connnection. > > -- > Help mailto:majordomo@net.doit.wisc.edu and say > "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue May 20 02:33:43 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id CAA01042 for ; Tue, 20 May 2003 02:33:43 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19I0bp-00069j-00 for ipfix-list@mil.doit.wisc.edu; Tue, 20 May 2003 01:28:25 -0500 Received: from ds20-1.cc.swin.edu.au ([136.186.1.150] helo=swin.edu.au) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19I0bm-00069b-00 for ipfix@net.doit.wisc.edu; Tue, 20 May 2003 01:28:22 -0500 Received: from fokus.fraunhofer.de (szander-laptop.caia.swin.edu.au [136.186.229.90]) by swin.edu.au (8.9.3p2/8.9.3) with ESMTP id QAA1962145 for ; Tue, 20 May 2003 16:28:18 +1000 (EST) Message-ID: <3EC9C98F.6050509@fokus.fraunhofer.de> Date: Tue, 20 May 2003 08:22:07 +0200 From: Sebastian Zander Organization: Fraunhofer FOKUS User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.2) Gecko/20030208 Netscape/7.02 X-Accept-Language: en-us, en MIME-Version: 1.0 To: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Lost Flow Records - Synch Message References: <20030520043254.8957.qmail@web80009.mail.yahoo.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Periodic refreshment of templates is in my opinion pointless when a reliable connection-oriented transport is used. Either the transport is working (the refreshment just wastes bandwidth) or it fails in which case the sender (exporter) knows that it fails and can act accordingly (such as failover and send the templates to the new collector). In this case periodic refreshment just adds unecessary complexity to the exporter. For UDP it is necessary if there are no ACKs from collector to exporter. Periodic refreshments are basically forward error correction. If there would be application layer ACKs anyway (to achieve application layer reliability as explained by other people on the list) why have periodic template refreshments at all? Just my 0.02 cents on the periodic refreshment discussion, Sebastian -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue May 20 11:01:41 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA16841 for ; Tue, 20 May 2003 11:01:40 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19I8FI-00022U-00 for ipfix-list@mil.doit.wisc.edu; Tue, 20 May 2003 09:37:40 -0500 Received: from eng4.oar.net ([192.148.244.24]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 19I8FH-00022N-00 for ipfix@net.doit.wisc.edu; Tue, 20 May 2003 09:37:39 -0500 Received: (qmail 50624 invoked by uid 4454); 20 May 2003 14:37:38 -0000 Date: Tue, 20 May 2003 10:37:38 -0400 From: Mark Fullmer To: Ganesh Sadasivan Cc: Tal Givoly , ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Lost Flow Records - Synch Message Message-ID: <20030520103738.A50602@net.ohio-state.edu> References: <3EC83F51.F96AABDB@cisco.com> <20030519011616.A41262@net.ohio-state.edu> <3EC920D2.B8F0A79C@cisco.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: <3EC920D2.B8F0A79C@cisco.com>; from gsadasiv@cisco.com on Mon, May 19, 2003 at 11:22:10AM -0700 Precedence: bulk Sender: majordomo listserver On Mon, May 19, 2003 at 11:22:10AM -0700, Ganesh Sadasivan wrote: > > > Mark Fullmer wrote: > > > If this happens the application layer write() will fail and the template > > can immediatly be marked for retransmission. > > Yes if it is a blocking send. This need not be the case for a non-blocking send. > If it's a blocking send it will block, a non blocking send will return an error or partial write. Regardless, the point I was trying to make is that a periodic template refresh mechanism is not required for a reliable transport. If buffers are full the application layer will be informed and be able to schedule the failed template send again. mark -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue May 20 21:19:41 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA07871 for ; Tue, 20 May 2003 21:19:41 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19IHiw-0006c1-00 for ipfix-list@mil.doit.wisc.edu; Tue, 20 May 2003 19:44:54 -0500 Received: from palrel12.hp.com ([156.153.255.237]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19IHiu-0006bv-00 for ipfix@net.doit.wisc.edu; Tue, 20 May 2003 19:44:52 -0500 Received: from xparelay2.ptp.hp.com (xparelay2.ptp.hp.com [15.1.28.65]) by palrel12.hp.com (Postfix) with ESMTP id 099C51C01C71; Tue, 20 May 2003 17:44:52 -0700 (PDT) Received: from xpabh1.ptp.hp.com (xpabh1.ptp.hp.com [15.1.28.60]) by xparelay2.ptp.hp.com (Postfix) with ESMTP id BF6661C00A98; Tue, 20 May 2003 17:44:48 -0700 (PDT) Received: by xpabh1.ptp.hp.com with Internet Mail Service (5.5.2655.55) id ; Tue, 20 May 2003 17:44:48 -0700 Message-ID: <1D3D2C371FCBD947A7897FABBD3533A566BB26@xsun01.ptp.hp.com> From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" To: "'Sebastian Zander'" , ipfix@net.doit.wisc.edu Subject: RE: [ipfix] Lost Flow Records - Synch Message Date: Tue, 20 May 2003 17:44:43 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2655.55) Content-Type: text/plain; charset="iso-8859-1" Precedence: bulk Sender: majordomo listserver Sebastian, I completely agree that template refreshment is unnecessary in a TCP (or SCTP) context. In UDP, I'd still propose that the use of a Pull model as I suggested in an earlier e-mail is more effective than periodic retransmissions of templates. Regards, Jeff Meyer -----Original Message----- From: Sebastian Zander [mailto:zander@fokus.fraunhofer.de] Sent: Monday, May 19, 2003 11:22 PM To: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Lost Flow Records - Synch Message Periodic refreshment of templates is in my opinion pointless when a reliable connection-oriented transport is used. Either the transport is working (the refreshment just wastes bandwidth) or it fails in which case the sender (exporter) knows that it fails and can act accordingly (such as failover and send the templates to the new collector). In this case periodic refreshment just adds unecessary complexity to the exporter. For UDP it is necessary if there are no ACKs from collector to exporter. Periodic refreshments are basically forward error correction. If there would be application layer ACKs anyway (to achieve application layer reliability as explained by other people on the list) why have periodic template refreshments at all? Just my 0.02 cents on the periodic refreshment discussion, Sebastian -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue May 20 23:13:07 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA09617 for ; Tue, 20 May 2003 23:13:06 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19IJrt-0001X3-00 for ipfix-list@mil.doit.wisc.edu; Tue, 20 May 2003 22:02:17 -0500 Received: from ds20-1.cc.swin.edu.au ([136.186.1.150] helo=swin.edu.au) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19IJrr-0001Wx-00 for ipfix@net.doit.wisc.edu; Tue, 20 May 2003 22:02:15 -0500 Received: from fokus.fraunhofer.de (szander-laptop.caia.swin.edu.au [136.186.229.90]) by swin.edu.au (8.9.3p2/8.9.3) with ESMTP id NAA2082778; Wed, 21 May 2003 13:01:55 +1000 (EST) Message-ID: <3ECAEAAE.4040908@fokus.fraunhofer.de> Date: Wed, 21 May 2003 04:55:42 +0200 From: Sebastian Zander Organization: Fraunhofer FOKUS User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.2) Gecko/20030208 Netscape/7.02 X-Accept-Language: en-us, en MIME-Version: 1.0 To: "MEYER,JEFFREY D (HP-Cupertino,ex1)" CC: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Lost Flow Records - Synch Message References: <1D3D2C371FCBD947A7897FABBD3533A566BB26@xsun01.ptp.hp.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hi Jeff, MEYER,JEFFREY D (HP-Cupertino,ex1) wrote: > Sebastian, > > I completely agree that template refreshment is unnecessary in a TCP (or > SCTP) context. > > In UDP, I'd still propose that the use of a Pull model as I suggested in > an earlier e-mail is more effective than periodic retransmissions of > templates. I also think that Pull mode is more effective (in my view ACKs + retransmissions are some sort of Pull mode). Because I believe that there is not a constant loss rate but loss is rather bursty. So optimizing the refresh interval for the worst case might result in long periods where redundant data is sent without making use of it. If the protocol is unreliable (no ACKs) your proposal is IMO a better alternative to periodic refreshments. Anyway it is an alternative to be considered. Some nice properties of your proposal: the collector would not need to contact the exporter it could contact some other box which has the templates and the exporter could itself retrieve the templates from that other box at startup (i don't know if that would be feasible for a router though). If the collector can retrieve templates from another box other than the exporter a clear distinction of template IDs and flow IDs is required (an issue raised by Tal recently). Which is also good IMO. Cheers, Sebastian > Regards, > > Jeff Meyer > > > -----Original Message----- > From: Sebastian Zander [mailto:zander@fokus.fraunhofer.de] > Sent: Monday, May 19, 2003 11:22 PM > To: ipfix@net.doit.wisc.edu > Subject: Re: [ipfix] Lost Flow Records - Synch Message > > > Periodic refreshment of templates is in my opinion pointless when a > reliable connection-oriented transport is used. Either the transport > is working (the refreshment just wastes bandwidth) or it fails in > which case the sender (exporter) knows that it fails and can act > accordingly (such as failover and send the templates to the new > collector). In this case periodic refreshment just adds unecessary > complexity to the exporter. > > For UDP it is necessary if there are no ACKs from collector to exporter. > Periodic refreshments are basically forward error correction. If there > would be application layer ACKs anyway (to achieve application layer > reliability as explained by other people on the list) why have periodic > template refreshments at all? > > Just my 0.02 cents on the periodic refreshment discussion, > > Sebastian > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message > body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > -- Sebastian Zander E-mail: zander@fokus.fraunhofer.de Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 D-10589 Berlin, Germany www.fokus.fraunhofer.de/usr/sebastian.zander -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed May 21 06:31:09 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id GAA14071 for ; Wed, 21 May 2003 06:31:09 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19IQPI-0003BN-00 for ipfix-list@mil.doit.wisc.edu; Wed, 21 May 2003 05:01:12 -0500 Received: from tokyo.ccrle.nec.de ([195.37.70.2]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19IQPF-0003BE-00 for ipfix@net.doit.wisc.edu; Wed, 21 May 2003 05:01:09 -0500 Received: from venus.office (venus.office [10.1.1.11]) by tokyo.ccrle.nec.de (8.12.9/8.12.8) with ESMTP id h4LA0xVI042737; Wed, 21 May 2003 12:00:59 +0200 (CEST) Received: from ccrle.nec.de (molina.office [10.1.1.126]) by venus.office (Postfix on SuSE Linux eMail Server 3.0) with ESMTP id 2A87F75009; Wed, 21 May 2003 11:51:12 +0200 (CEST) Message-ID: <3ECB4E5C.9AC24C87@ccrle.nec.de> Date: Wed, 21 May 2003 12:01:00 +0200 From: Maurizio Molina X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Ganesh Sadasivan Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Changes to architecture doc. References: <3EC55996.F610236@cisco.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Ganesh Sadasivan wrote: > Hi, > > In going through the architecture spec. I find that the following > topics > needs to be modified. I'll list all of them in one e-mail and then on a > topic-by-topic we can have separate e-mail threads. > > BTW I could not get a copy of the arch. spec from the link > http://www.ietf.org/internet-drafts/draft-ietf-ipfix-architecture-02.txt > > This has been deleted from the internet drafts database because it was > not updated for more than 6 months. I guess the same is true with data > model > document. > > The alternate link that I got it from is : > http://www.ietf.org/proceedings/02nov/I-D/draft-ietf-ipfix-architecture-02.txt > ........... > > * Section 4.5 : > " There MAY be additional rules defined within the observation domain > so that only certain flows records are picked up for export. " > Though rules are defined w.r.t observation domain, > the entity that applies these rules is the IPFIX protocol. Hi, I'd like to discuss the type of packet sampling/flow selection that seems to be considered by IPFIX so far. Re-reading section 4.5 of the arch. spec., and the previous ones (4.4.2, 4.4.3, 4.4.4), I got the impression that only the following types of sampling/slection at the packet or flow level are considered: 1) Sampling of packets within a flow (sec. 4.4.4): Packets undergo a filter to see if they match a flow to be monitored; if they do, they are (possibly) sampled (to alleviate METER processing load). 2) Selection of flows to export (sec. 4.5): all defined flows are monitored (possibly packet sampling within flows as defined above is applied), then a decisions is taken about whether to export the entire pool of monitored flows or just a subset of it. This, with the goal to alleviate EXPORTER/COLLECTOR load. Imo, what's not explicitly considered is a third case, where there is a packet sampling but "oriented" to flow selection, i.e. packets undergo a filter match but then they're not "blindly" (i.e. n out of N or similar) sampled, but rather sampled or discarded on the basis of some property of the flow they matched. The property may be static (e.g. "flows with high monitoring priority") or dynamically built (e.g. flows carrying more traffic than others->sometime called "elephant" flows). This type of sampling can alleviate both METER and EXPORTER/COLLECTOR processing load. The interest for this type of "sampling/selection" (actually, it's both packet sampling and flow selection) is that for some applications it may be preferred to receive reliable figures of the volume of a subset of flows rather that less reliable estimates of the volume of ALL the flows (as sampling of type 1 would provide). Furthermore, differently from flow selection of type 2), it's not required to keep a flow record for ALL the flows before deciding whether to export them or not, as the decision whether to open a new flow record is taken "on the fly" when packets arrive. Then it's potentially more scalable. That said, what are the issues that a standard body should consider? Imo, they are mainly two: - the definition of policies for seclecting packets on the basis the property of flow they match (i.e. high priority monitoring flow definition, or flows carrying high traffic volumes, or other....). This, without dwelling into the details of algorithms may implement this "on line" detection of properties. - the type of information that has to be exported: e.g. which percentage of traffic is being discarded? Otherwise a collector would only get the report about the monitored flows (high priority or high volume, or other...), but it's different to know that they compose the 99% oe the 10% of the traffic.... In the requirement draft, http://www.ietf.org/internet-drafts/draft-ietf-ipfix-reqs-09.txt, in sec. 5.2 (sampling) and 5.3 (overload behaviour) there's already some text that partially, but not completely and not exactly, supports the type of sampling/selection I've described above. I'd be glad to know whether people in the group share this view that this type of sampling/selection has to be more explicitly considered. The next step would then be to discuss if/what needs to be standardized to support it. Regards, Maurizio -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed May 21 10:21:43 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA23140 for ; Wed, 21 May 2003 10:21:42 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19IUJU-0000ly-00 for ipfix-list@mil.doit.wisc.edu; Wed, 21 May 2003 09:11:28 -0500 Received: from 207-237-237-207.c3-0.nyr-ubr3.nyr.ny.cable.rcn.com ([207.237.237.207] helo=newyork.qosient.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19IUJS-0000lm-00 for ipfix@net.doit.wisc.edu; Wed, 21 May 2003 09:11:26 -0500 Received: from sphynx (sphynx [192.168.0.64]) by newyork.qosient.com (8.11.6/8.11.0) with ESMTP id h4LEBGG11232; Wed, 21 May 2003 10:11:16 -0400 From: "Carter Bullard" To: "'Maurizio Molina'" , "'Ganesh Sadasivan'" Cc: Subject: RE: [ipfix] Changes to architecture doc. Date: Wed, 21 May 2003 10:11:07 -0400 Message-ID: <5C8959A16A71B449AE793CF52FBBED6607E9F4@ptah.newyork.qosient.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.4510 In-Reply-To: <5C8959A16A71B449AE793CF52FBBED66148FA1@ptah.newyork.qosient.com> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Importance: Normal Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by ietf.org id KAA23140 Maurizio, I believe that this is out of scope with IPFIX. We're suppose to be defining transport of flow records, not defining flow monitors. Carter > -----Original Message----- > From: majordomo listserver > [mailto:majordomo@mil.doit.wisc.edu] On Behalf Of Maurizio Molina > Sent: Wednesday, May 21, 2003 6:01 AM > To: Ganesh Sadasivan > Cc: ipfix@net.doit.wisc.edu > Subject: Re: [ipfix] Changes to architecture doc. > > > > > Ganesh Sadasivan wrote: > > > Hi, > > > > In going through the architecture spec. I find that the following > > topics > > needs to be modified. I'll list all of them in one e-mail > and then on a > > topic-by-topic we can have separate e-mail threads. > > > > BTW I could not get a copy of the arch. spec from the link > > > http://www.ietf.org/internet-drafts/draft-ietf-ipfix-architect > ure-02.txt > > > > This has been deleted from the internet drafts database > because it was > > not updated for more than 6 months. I guess the same is > true with data > > model > > document. > > > > The alternate link that I got it from is : > > > http://www.ietf.org/proceedings/02nov/I-D/draft-ietf-ipfix-arc > hitecture-02.txt > > > > ........... > > > > > * Section 4.5 : > > " There MAY be additional rules defined within the > observation domain > > so that only certain flows records are picked up for export. " > > Though rules are defined w.r.t observation domain, > > the entity that applies these rules is the IPFIX protocol. > > Hi, > I'd like to discuss the type of packet sampling/flow > selection that seems to be > considered by IPFIX so far. > Re-reading section 4.5 of the arch. spec., and the previous > ones (4.4.2, 4.4.3, > 4.4.4), I got the impression that only the following types of > sampling/slection at > the packet or flow level are considered: > 1) Sampling of packets within a flow (sec. 4.4.4): Packets > undergo a filter to see > if they match a flow to be monitored; if they do, they are > (possibly) sampled (to > alleviate METER processing load). > 2) Selection of flows to export (sec. 4.5): all defined flows > are monitored > (possibly packet sampling within flows as defined above is > applied), then a > decisions is taken about whether to export the entire pool of > monitored flows or > just a subset of it. This, with the goal to alleviate > EXPORTER/COLLECTOR load. > > Imo, what's not explicitly considered is a third case, where > there is a packet > sampling but "oriented" to flow selection, i.e. packets > undergo a filter match but > then they're not "blindly" (i.e. n out of N or similar) > sampled, but rather > sampled or discarded on the basis of some property of the > flow they matched. The > property may be static (e.g. "flows with high monitoring > priority") or dynamically > built (e.g. flows carrying more traffic than > others->sometime called "elephant" > flows). This type of sampling can alleviate both METER and > EXPORTER/COLLECTOR > processing load. > The interest for this type of "sampling/selection" (actually, > it's both packet > sampling and flow selection) is that for some applications it > may be preferred to > receive reliable figures of the volume of a subset of flows > rather that less > reliable estimates of the volume of ALL the flows (as > sampling of type 1 would > provide). Furthermore, differently from flow selection of > type 2), it's not > required to keep a flow record for ALL the flows before > deciding whether to export > them or not, as the decision whether to open a new flow > record is taken "on the > fly" when packets arrive. Then it's potentially more scalable. > > That said, what are the issues that a standard body should > consider? Imo, they are > mainly two: > - the definition of policies for seclecting packets on the > basis the property of > flow they match (i.e. high priority monitoring flow > definition, or flows carrying > high traffic volumes, or other....). This, without dwelling > into the details of > algorithms may implement this "on line" detection of properties. > - the type of information that has to be exported: e.g. which > percentage of > traffic is being discarded? Otherwise a collector would only > get the report about > the monitored flows (high priority or high volume, or > other...), but it's > different to know that they compose the 99% oe the 10% of the > traffic.... > > In the requirement draft, > http://www.ietf.org/internet-drafts/draft-ietf-ipfix-reqs-09.t xt, in sec. 5.2 (sampling) and 5.3 (overload behaviour) there's already some text that partially, but not completely and not exactly, supports the type of sampling/selection I've described above. I'd be glad to know whether people in the group share this view that this type of sampling/selection has to be more explicitly considered. The next step would then be to discuss if/what needs to be standardized to support it. Regards, Maurizio -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed May 21 12:00:21 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA26275 for ; Wed, 21 May 2003 12:00:20 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19IVtC-0002v4-00 for ipfix-list@mil.doit.wisc.edu; Wed, 21 May 2003 10:52:26 -0500 Received: from auds953.usa.alcatel.com ([143.209.238.6]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19IVtB-0002uy-00 for ipfix@net.doit.wisc.edu; Wed, 21 May 2003 10:52:25 -0500 Received: from alcatel.com (localhost [127.0.0.1]) by auds953.usa.alcatel.com (8.12.8p1/8.12.8) with ESMTP id h4LFq9bD017587; Wed, 21 May 2003 10:52:09 -0500 (CDT) Message-ID: <3ECBA0A6.CB6E2FA@alcatel.com> Date: Wed, 21 May 2003 10:52:06 -0500 From: Alex Audu X-Mailer: Mozilla 4.72 [en] (WinNT; U) X-Accept-Language: en MIME-Version: 1.0 To: Carter Bullard CC: "'Maurizio Molina'" , "'Ganesh Sadasivan'" , ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Changes to architecture doc. References: <5C8959A16A71B449AE793CF52FBBED6607E9F4@ptah.newyork.qosient.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit This, in my view, is a huge limitiation of IPFIX. PSAMP does a better job of specifying the packet selection process as well as the other aspects involved in getting the data to the collector. IPFIX is being suggested as a viable export for PSAMP. If so, it needs to address the issue of data selection. Regards, Alex. Carter Bullard wrote: > Maurizio, > > I believe that this is out of scope with IPFIX. > We're suppose to be defining transport of flow > records, not defining flow monitors. > > Carter > > > -----Original Message----- > > From: majordomo listserver > > [mailto:majordomo@mil.doit.wisc.edu] On Behalf Of Maurizio Molina > > Sent: Wednesday, May 21, 2003 6:01 AM > > To: Ganesh Sadasivan > > Cc: ipfix@net.doit.wisc.edu > > Subject: Re: [ipfix] Changes to architecture doc. > > > > > > > > > > Ganesh Sadasivan wrote: > > > > > Hi, > > > > > > In going through the architecture spec. I find that the following > > > topics > > > needs to be modified. I'll list all of them in one e-mail > > and then on a > > > topic-by-topic we can have separate e-mail threads. > > > > > > BTW I could not get a copy of the arch. spec from the link > > > > > http://www.ietf.org/internet-drafts/draft-ietf-ipfix-architect > > ure-02.txt > > > > > > This has been deleted from the internet drafts database > > because it was > > > not updated for more than 6 months. I guess the same is > > true with data > > > model > > > document. > > > > > > The alternate link that I got it from is : > > > > > http://www.ietf.org/proceedings/02nov/I-D/draft-ietf-ipfix-arc > > hitecture-02.txt > > > > > > > ........... > > > > > > > > * Section 4.5 : > > > " There MAY be additional rules defined within the > > observation domain > > > so that only certain flows records are picked up for export. " > > > Though rules are defined w.r.t observation domain, > > > the entity that applies these rules is the IPFIX protocol. > > > > Hi, > > I'd like to discuss the type of packet sampling/flow > > selection that seems to be > > considered by IPFIX so far. > > Re-reading section 4.5 of the arch. spec., and the previous > > ones (4.4.2, 4.4.3, > > 4.4.4), I got the impression that only the following types of > > sampling/slection at > > the packet or flow level are considered: > > 1) Sampling of packets within a flow (sec. 4.4.4): Packets > > undergo a filter to see > > if they match a flow to be monitored; if they do, they are > > (possibly) sampled (to > > alleviate METER processing load). > > 2) Selection of flows to export (sec. 4.5): all defined flows > > are monitored > > (possibly packet sampling within flows as defined above is > > applied), then a > > decisions is taken about whether to export the entire pool of > > monitored flows or > > just a subset of it. This, with the goal to alleviate > > EXPORTER/COLLECTOR load. > > > > Imo, what's not explicitly considered is a third case, where > > there is a packet > > sampling but "oriented" to flow selection, i.e. packets > > undergo a filter match but > > then they're not "blindly" (i.e. n out of N or similar) > > sampled, but rather > > sampled or discarded on the basis of some property of the > > flow they matched. The > > property may be static (e.g. "flows with high monitoring > > priority") or dynamically > > built (e.g. flows carrying more traffic than > > others->sometime called "elephant" > > flows). This type of sampling can alleviate both METER and > > EXPORTER/COLLECTOR > > processing load. > > The interest for this type of "sampling/selection" (actually, > > it's both packet > > sampling and flow selection) is that for some applications it > > may be preferred to > > receive reliable figures of the volume of a subset of flows > > rather that less > > reliable estimates of the volume of ALL the flows (as > > sampling of type 1 would > > provide). Furthermore, differently from flow selection of > > type 2), it's not > > required to keep a flow record for ALL the flows before > > deciding whether to export > > them or not, as the decision whether to open a new flow > > record is taken "on the > > fly" when packets arrive. Then it's potentially more scalable. > > > > That said, what are the issues that a standard body should > > consider? Imo, they are > > mainly two: > > - the definition of policies for seclecting packets on the > > basis the property of > > flow they match (i.e. high priority monitoring flow > > definition, or flows carrying > > high traffic volumes, or other....). This, without dwelling > > into the details of > > algorithms may implement this "on line" detection of properties. > > - the type of information that has to be exported: e.g. which > > percentage of > > traffic is being discarded? Otherwise a collector would only > > get the report about > > the monitored flows (high priority or high volume, or > > other...), but it's > > different to know that they compose the 99% oe the 10% of the > > traffic.... > > > > In the requirement draft, > > http://www.ietf.org/internet-drafts/draft-ietf-ipfix-reqs-09.t > xt, in sec. 5.2 > (sampling) and 5.3 (overload behaviour) there's already some text that > partially, > but not completely and not exactly, supports the type of sampling/selection > I've > described above. > I'd be glad to know whether people in the group share this view that this > type of > sampling/selection has to be more explicitly considered. The next step would > then > be to discuss if/what needs to be standardized to support it. > > Regards, > Maurizio > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message > body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed May 21 12:01:02 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA26311 for ; Wed, 21 May 2003 12:01:01 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19IVsV-0002ub-00 for ipfix-list@mil.doit.wisc.edu; Wed, 21 May 2003 10:51:43 -0500 Received: from halt-in.cisco.com ([171.70.144.185]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19IVsU-0002uV-00 for ipfix@net.doit.wisc.edu; Wed, 21 May 2003 10:51:42 -0500 Received: from cisco.com (171.71.163.13) by halt-in.cisco.com with ESMTP; 21 May 2003 08:51:42 -0800 Received: from cisco.com (sjc-vpn1-285.cisco.com [10.21.97.29]) by mira-sjc5-f.cisco.com (Mirapoint Messaging Server MOS 3.3.3-GR) with ESMTP id AGL40098; Wed, 21 May 2003 08:58:45 -0700 (PDT) Message-ID: <3ECBA08B.8BD3F1AA@cisco.com> Date: Wed, 21 May 2003 08:51:40 -0700 From: Ganesh Sadasivan Organization: Cisco Systems Inc X-Mailer: Mozilla 4.76 [en]C-CCK-MCD (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Maurizio Molina CC: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Changes to architecture doc. References: <3EC55996.F610236@cisco.com> <3ECB4E5C.9AC24C87@ccrle.nec.de> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hi Maurizio, I can understand that such an information is useful for some applications. But in IPFIX we are focusing on flow collection and export. Flow discard is something out of scope of IPFIX discussion. That does not preclude a vendor from implementing what you mentioned below and use a vendor specific template to report this to the collector. Thanks Ganesh Maurizio Molina wrote: > Ganesh Sadasivan wrote: > > > Hi, > > > > In going through the architecture spec. I find that the following > > topics > > needs to be modified. I'll list all of them in one e-mail and then on a > > topic-by-topic we can have separate e-mail threads. > > > > BTW I could not get a copy of the arch. spec from the link > > http://www.ietf.org/internet-drafts/draft-ietf-ipfix-architecture-02.txt > > > > This has been deleted from the internet drafts database because it was > > not updated for more than 6 months. I guess the same is true with data > > model > > document. > > > > The alternate link that I got it from is : > > http://www.ietf.org/proceedings/02nov/I-D/draft-ietf-ipfix-architecture-02.txt > > > > ........... > > > > > * Section 4.5 : > > " There MAY be additional rules defined within the observation domain > > so that only certain flows records are picked up for export. " > > Though rules are defined w.r.t observation domain, > > the entity that applies these rules is the IPFIX protocol. > > Hi, > I'd like to discuss the type of packet sampling/flow selection that seems to be > considered by IPFIX so far. > Re-reading section 4.5 of the arch. spec., and the previous ones (4.4.2, 4.4.3, > 4.4.4), I got the impression that only the following types of sampling/slection at > the packet or flow level are considered: > 1) Sampling of packets within a flow (sec. 4.4.4): Packets undergo a filter to see > if they match a flow to be monitored; if they do, they are (possibly) sampled (to > alleviate METER processing load). > 2) Selection of flows to export (sec. 4.5): all defined flows are monitored > (possibly packet sampling within flows as defined above is applied), then a > decisions is taken about whether to export the entire pool of monitored flows or > just a subset of it. This, with the goal to alleviate EXPORTER/COLLECTOR load. > > Imo, what's not explicitly considered is a third case, where there is a packet > sampling but "oriented" to flow selection, i.e. packets undergo a filter match but > then they're not "blindly" (i.e. n out of N or similar) sampled, but rather > sampled or discarded on the basis of some property of the flow they matched. The > property may be static (e.g. "flows with high monitoring priority") or dynamically > built (e.g. flows carrying more traffic than others->sometime called "elephant" > flows). This type of sampling can alleviate both METER and EXPORTER/COLLECTOR > processing load. > The interest for this type of "sampling/selection" (actually, it's both packet > sampling and flow selection) is that for some applications it may be preferred to > receive reliable figures of the volume of a subset of flows rather that less > reliable estimates of the volume of ALL the flows (as sampling of type 1 would > provide). Furthermore, differently from flow selection of type 2), it's not > required to keep a flow record for ALL the flows before deciding whether to export > them or not, as the decision whether to open a new flow record is taken "on the > fly" when packets arrive. Then it's potentially more scalable. > > That said, what are the issues that a standard body should consider? Imo, they are > mainly two: > - the definition of policies for seclecting packets on the basis the property of > flow they match (i.e. high priority monitoring flow definition, or flows carrying > high traffic volumes, or other....). This, without dwelling into the details of > algorithms may implement this "on line" detection of properties. > - the type of information that has to be exported: e.g. which percentage of > traffic is being discarded? Otherwise a collector would only get the report about > the monitored flows (high priority or high volume, or other...), but it's > different to know that they compose the 99% oe the 10% of the traffic.... > > In the requirement draft, > http://www.ietf.org/internet-drafts/draft-ietf-ipfix-reqs-09.txt, in sec. 5.2 > (sampling) and 5.3 (overload behaviour) there's already some text that partially, > but not completely and not exactly, supports the type of sampling/selection I've > described above. > I'd be glad to know whether people in the group share this view that this type of > sampling/selection has to be more explicitly considered. The next step would then > be to discuss if/what needs to be standardized to support it. > > Regards, > Maurizio > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed May 21 12:05:31 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA26416 for ; Wed, 21 May 2003 12:05:30 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19IVye-00032r-00 for ipfix-list@mil.doit.wisc.edu; Wed, 21 May 2003 10:58:04 -0500 Received: from 207-237-237-207.c3-0.nyr-ubr3.nyr.ny.cable.rcn.com ([207.237.237.207] helo=newyork.qosient.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19IVyd-00032k-00 for ipfix@net.doit.wisc.edu; Wed, 21 May 2003 10:58:03 -0500 Received: from sphynx (sphynx [192.168.0.64]) by newyork.qosient.com (8.11.6/8.11.0) with ESMTP id h4LFvsG11762; Wed, 21 May 2003 11:57:54 -0400 From: "Carter Bullard" To: "'Alex Audu'" Cc: "'Maurizio Molina'" , "'Ganesh Sadasivan'" , Subject: RE: [ipfix] Changes to architecture doc. Date: Wed, 21 May 2003 11:57:43 -0400 Message-ID: <5C8959A16A71B449AE793CF52FBBED6607E9FA@ptah.newyork.qosient.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.4510 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Importance: Normal In-Reply-To: <5C8959A16A71B449AE793CF52FBBED66149108@ptah.newyork.qosient.com> Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hey Alex, What is the relationship between transport of flow records and the generation of flow records? IPFIX does not define a flow generation process, it defines a flow record transport protocol. Carter > -----Original Message----- > From: Alex Audu [mailto:Alex.Audu@alcatel.com] > Sent: Wednesday, May 21, 2003 11:52 AM > To: Carter Bullard > Cc: 'Maurizio Molina'; 'Ganesh Sadasivan'; ipfix@net.doit.wisc.edu > Subject: Re: [ipfix] Changes to architecture doc. > > > This, in my view, is a huge limitiation of IPFIX. PSAMP does > a better job of > specifying > the packet selection process as well as the other aspects > involved in getting the > data to > the collector. IPFIX is being suggested as a viable export > for PSAMP. If so, it > needs to > address the issue of data selection. > > Regards, > Alex. > > Carter Bullard wrote: > > > Maurizio, > > > > I believe that this is out of scope with IPFIX. > > We're suppose to be defining transport of flow > > records, not defining flow monitors. > > > > Carter > > > > > -----Original Message----- > > > From: majordomo listserver > > > [mailto:majordomo@mil.doit.wisc.edu] On Behalf Of Maurizio Molina > > > Sent: Wednesday, May 21, 2003 6:01 AM > > > To: Ganesh Sadasivan > > > Cc: ipfix@net.doit.wisc.edu > > > Subject: Re: [ipfix] Changes to architecture doc. > > > > > > > > > > > > > > > Ganesh Sadasivan wrote: > > > > > > > Hi, > > > > > > > > In going through the architecture spec. I find that > the following > > > > topics > > > > needs to be modified. I'll list all of them in one e-mail > > > and then on a > > > > topic-by-topic we can have separate e-mail threads. > > > > > > > > BTW I could not get a copy of the arch. spec from the link > > > > > > > http://www.ietf.org/internet-drafts/draft-ietf-ipfix-architect > > > ure-02.txt > > > > > > > > This has been deleted from the internet drafts database > > > because it was > > > > not updated for more than 6 months. I guess the same is > > > true with data > > > > model > > > > document. > > > > > > > > The alternate link that I got it from is : > > > > > > > http://www.ietf.org/proceedings/02nov/I-D/draft-ietf-ipfix-arc > > > hitecture-02.txt > > > > > > > > > > ........... > > > > > > > > > > > * Section 4.5 : > > > > " There MAY be additional rules defined within the > > > observation domain > > > > so that only certain flows records are picked up for > export. " > > > > Though rules are defined w.r.t observation domain, > > > > the entity that applies these rules is the IPFIX protocol. > > > > > > Hi, > > > I'd like to discuss the type of packet sampling/flow > > > selection that seems to be > > > considered by IPFIX so far. > > > Re-reading section 4.5 of the arch. spec., and the previous > > > ones (4.4.2, 4.4.3, > > > 4.4.4), I got the impression that only the following types of > > > sampling/slection at > > > the packet or flow level are considered: > > > 1) Sampling of packets within a flow (sec. 4.4.4): Packets > > > undergo a filter to see > > > if they match a flow to be monitored; if they do, they are > > > (possibly) sampled (to > > > alleviate METER processing load). > > > 2) Selection of flows to export (sec. 4.5): all defined flows > > > are monitored > > > (possibly packet sampling within flows as defined above is > > > applied), then a > > > decisions is taken about whether to export the entire pool of > > > monitored flows or > > > just a subset of it. This, with the goal to alleviate > > > EXPORTER/COLLECTOR load. > > > > > > Imo, what's not explicitly considered is a third case, where > > > there is a packet > > > sampling but "oriented" to flow selection, i.e. packets > > > undergo a filter match but > > > then they're not "blindly" (i.e. n out of N or similar) > > > sampled, but rather > > > sampled or discarded on the basis of some property of the > > > flow they matched. The > > > property may be static (e.g. "flows with high monitoring > > > priority") or dynamically > > > built (e.g. flows carrying more traffic than > > > others->sometime called "elephant" > > > flows). This type of sampling can alleviate both METER and > > > EXPORTER/COLLECTOR > > > processing load. > > > The interest for this type of "sampling/selection" (actually, > > > it's both packet > > > sampling and flow selection) is that for some applications it > > > may be preferred to > > > receive reliable figures of the volume of a subset of flows > > > rather that less > > > reliable estimates of the volume of ALL the flows (as > > > sampling of type 1 would > > > provide). Furthermore, differently from flow selection of > > > type 2), it's not > > > required to keep a flow record for ALL the flows before > > > deciding whether to export > > > them or not, as the decision whether to open a new flow > > > record is taken "on the > > > fly" when packets arrive. Then it's potentially more scalable. > > > > > > That said, what are the issues that a standard body should > > > consider? Imo, they are > > > mainly two: > > > - the definition of policies for seclecting packets on the > > > basis the property of > > > flow they match (i.e. high priority monitoring flow > > > definition, or flows carrying > > > high traffic volumes, or other....). This, without dwelling > > > into the details of > > > algorithms may implement this "on line" detection of properties. > > > - the type of information that has to be exported: e.g. which > > > percentage of > > > traffic is being discarded? Otherwise a collector would only > > > get the report about > > > the monitored flows (high priority or high volume, or > > > other...), but it's > > > different to know that they compose the 99% oe the 10% of the > > > traffic.... > > > > > > In the requirement draft, > > > http://www.ietf.org/internet-drafts/draft-ietf-ipfix-reqs-09.t > > xt, in sec. 5.2 > > (sampling) and 5.3 (overload behaviour) there's already > some text that > > partially, > > but not completely and not exactly, supports the type of > sampling/selection > > I've > > described above. > > I'd be glad to know whether people in the group share this > view that this > > type of > > sampling/selection has to be more explicitly considered. > The next step would > > then > > be to discuss if/what needs to be standardized to support it. > > > > Regards, > > Maurizio > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say > "help" in message > > body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say > "help" in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed May 21 12:49:51 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA27947 for ; Wed, 21 May 2003 12:49:51 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19IWdt-00040I-00 for ipfix-list@mil.doit.wisc.edu; Wed, 21 May 2003 11:40:41 -0500 Received: from tokyo.ccrle.nec.de ([195.37.70.2]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19IWdr-00040D-00 for ipfix@net.doit.wisc.edu; Wed, 21 May 2003 11:40:39 -0500 Received: from venus.office (venus.office [10.1.1.11]) by tokyo.ccrle.nec.de (8.12.9/8.12.8) with ESMTP id h4LGe7VI068824; Wed, 21 May 2003 18:40:07 +0200 (CEST) Received: from ccrle.nec.de (molina.office [10.1.1.126]) by venus.office (Postfix on SuSE Linux eMail Server 3.0) with ESMTP id 78B0F722FD; Wed, 21 May 2003 18:30:17 +0200 (CEST) Message-ID: <3ECBABE9.E16191C8@ccrle.nec.de> Date: Wed, 21 May 2003 18:40:09 +0200 From: Maurizio Molina X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Carter Bullard Cc: "'Alex Audu'" , "'Ganesh Sadasivan'" , ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Changes to architecture doc. References: <5C8959A16A71B449AE793CF52FBBED6607E9FA@ptah.newyork.qosient.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Carter Bullard wrote: > Hey Alex, > What is the relationship between transport of > flow records and the generation of flow records? > IPFIX does not define a flow generation process, > it defines a flow record transport protocol. > > Carter Hi Carter, then what is your interpretation of the following part of the requirement draft (5.2), that basically says that information about (packet) sampling MUST be well defined and reported to the collector if the sampling changes? This information is about the generation of flow records, isn't it? Then if it's out of scope, we should drop this text from the requirements as well... Maurizio **************** From req. draft ********** The metering process MAY support packet sampling. If sampling is supported the sampling configuration MUST be well defined. The sampling configuration includes the sampling method and all its parameters. If the sampling configuration is changed during operation, the new sampling configuration with its parameters MUST be indicated to all collecting processes receiving the affected flow records. Changing the sampling configuration includes: adding a sampling function to the metering process, removing a sampling function from the metering process, change sampling method, and change sampling parameter(s). In case of any change in the sampling configuration, all flow records metered by the previous sampling configuration MUST be terminated and exported according to the export configuration. The metering process MUST NOT merge the flow records generated with the new sampling configuration with the flow records generated with the previous sampling configuration. ******** End From Req. draft ***************** > > > > -----Original Message----- > > From: Alex Audu [mailto:Alex.Audu@alcatel.com] > > Sent: Wednesday, May 21, 2003 11:52 AM > > To: Carter Bullard > > Cc: 'Maurizio Molina'; 'Ganesh Sadasivan'; ipfix@net.doit.wisc.edu > > Subject: Re: [ipfix] Changes to architecture doc. > > > > > > This, in my view, is a huge limitiation of IPFIX. PSAMP does > > a better job of > > specifying > > the packet selection process as well as the other aspects > > involved in getting the > > data to > > the collector. IPFIX is being suggested as a viable export > > for PSAMP. If so, it > > needs to > > address the issue of data selection. > > > > Regards, > > Alex. > > > > Carter Bullard wrote: > > > > > Maurizio, > > > > > > I believe that this is out of scope with IPFIX. > > > We're suppose to be defining transport of flow > > > records, not defining flow monitors. > > > > > > Carter > > > > > > > -----Original Message----- > > > > From: majordomo listserver > > > > [mailto:majordomo@mil.doit.wisc.edu] On Behalf Of Maurizio Molina > > > > Sent: Wednesday, May 21, 2003 6:01 AM > > > > To: Ganesh Sadasivan > > > > Cc: ipfix@net.doit.wisc.edu > > > > Subject: Re: [ipfix] Changes to architecture doc. > > > > > > > > > > > > > > > > > > > > Ganesh Sadasivan wrote: > > > > > > > > > Hi, > > > > > > > > > > In going through the architecture spec. I find that > > the following > > > > > topics > > > > > needs to be modified. I'll list all of them in one e-mail > > > > and then on a > > > > > topic-by-topic we can have separate e-mail threads. > > > > > > > > > > BTW I could not get a copy of the arch. spec from the link > > > > > > > > > http://www.ietf.org/internet-drafts/draft-ietf-ipfix-architect > > > > ure-02.txt > > > > > > > > > > This has been deleted from the internet drafts database > > > > because it was > > > > > not updated for more than 6 months. I guess the same is > > > > true with data > > > > > model > > > > > document. > > > > > > > > > > The alternate link that I got it from is : > > > > > > > > > http://www.ietf.org/proceedings/02nov/I-D/draft-ietf-ipfix-arc > > > > hitecture-02.txt > > > > > > > > > > > > > ........... > > > > > > > > > > > > > > * Section 4.5 : > > > > > " There MAY be additional rules defined within the > > > > observation domain > > > > > so that only certain flows records are picked up for > > export. " > > > > > Though rules are defined w.r.t observation domain, > > > > > the entity that applies these rules is the IPFIX protocol. > > > > > > > > Hi, > > > > I'd like to discuss the type of packet sampling/flow > > > > selection that seems to be > > > > considered by IPFIX so far. > > > > Re-reading section 4.5 of the arch. spec., and the previous > > > > ones (4.4.2, 4.4.3, > > > > 4.4.4), I got the impression that only the following types of > > > > sampling/slection at > > > > the packet or flow level are considered: > > > > 1) Sampling of packets within a flow (sec. 4.4.4): Packets > > > > undergo a filter to see > > > > if they match a flow to be monitored; if they do, they are > > > > (possibly) sampled (to > > > > alleviate METER processing load). > > > > 2) Selection of flows to export (sec. 4.5): all defined flows > > > > are monitored > > > > (possibly packet sampling within flows as defined above is > > > > applied), then a > > > > decisions is taken about whether to export the entire pool of > > > > monitored flows or > > > > just a subset of it. This, with the goal to alleviate > > > > EXPORTER/COLLECTOR load. > > > > > > > > Imo, what's not explicitly considered is a third case, where > > > > there is a packet > > > > sampling but "oriented" to flow selection, i.e. packets > > > > undergo a filter match but > > > > then they're not "blindly" (i.e. n out of N or similar) > > > > sampled, but rather > > > > sampled or discarded on the basis of some property of the > > > > flow they matched. The > > > > property may be static (e.g. "flows with high monitoring > > > > priority") or dynamically > > > > built (e.g. flows carrying more traffic than > > > > others->sometime called "elephant" > > > > flows). This type of sampling can alleviate both METER and > > > > EXPORTER/COLLECTOR > > > > processing load. > > > > The interest for this type of "sampling/selection" (actually, > > > > it's both packet > > > > sampling and flow selection) is that for some applications it > > > > may be preferred to > > > > receive reliable figures of the volume of a subset of flows > > > > rather that less > > > > reliable estimates of the volume of ALL the flows (as > > > > sampling of type 1 would > > > > provide). Furthermore, differently from flow selection of > > > > type 2), it's not > > > > required to keep a flow record for ALL the flows before > > > > deciding whether to export > > > > them or not, as the decision whether to open a new flow > > > > record is taken "on the > > > > fly" when packets arrive. Then it's potentially more scalable. > > > > > > > > That said, what are the issues that a standard body should > > > > consider? Imo, they are > > > > mainly two: > > > > - the definition of policies for seclecting packets on the > > > > basis the property of > > > > flow they match (i.e. high priority monitoring flow > > > > definition, or flows carrying > > > > high traffic volumes, or other....). This, without dwelling > > > > into the details of > > > > algorithms may implement this "on line" detection of properties. > > > > - the type of information that has to be exported: e.g. which > > > > percentage of > > > > traffic is being discarded? Otherwise a collector would only > > > > get the report about > > > > the monitored flows (high priority or high volume, or > > > > other...), but it's > > > > different to know that they compose the 99% oe the 10% of the > > > > traffic.... > > > > > > > > In the requirement draft, > > > > http://www.ietf.org/internet-drafts/draft-ietf-ipfix-reqs-09.t > > > xt, in sec. 5.2 > > > (sampling) and 5.3 (overload behaviour) there's already > > some text that > > > partially, > > > but not completely and not exactly, supports the type of > > sampling/selection > > > I've > > > described above. > > > I'd be glad to know whether people in the group share this > > view that this > > > type of > > > sampling/selection has to be more explicitly considered. > > The next step would > > > then > > > be to discuss if/what needs to be standardized to support it. > > > > > > Regards, > > > Maurizio > > > > > > -- > > > Help mailto:majordomo@net.doit.wisc.edu and say > > "help" in message > > > body > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > "unsubscribe ipfix" in message body > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > -- > > > Help mailto:majordomo@net.doit.wisc.edu and say > > "help" in message body > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > "unsubscribe ipfix" in message body > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed May 21 13:10:17 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA28507 for ; Wed, 21 May 2003 13:10:17 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19IWyv-0004Tg-00 for ipfix-list@mil.doit.wisc.edu; Wed, 21 May 2003 12:02:25 -0500 Received: from 207-237-237-207.c3-0.nyr-ubr3.nyr.ny.cable.rcn.com ([207.237.237.207] helo=newyork.qosient.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19IWyt-0004TV-00 for ipfix@net.doit.wisc.edu; Wed, 21 May 2003 12:02:24 -0500 Received: from sphynx (sphynx [192.168.0.64]) by newyork.qosient.com (8.11.6/8.11.0) with ESMTP id h4LH1ZG11878; Wed, 21 May 2003 13:01:35 -0400 From: "Carter Bullard" To: "'Maurizio Molina'" Cc: "'Alex Audu'" , "'Ganesh Sadasivan'" , Subject: RE: [ipfix] Changes to architecture doc. Date: Wed, 21 May 2003 13:01:24 -0400 Message-ID: <5C8959A16A71B449AE793CF52FBBED6607E9FB@ptah.newyork.qosient.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.4510 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Importance: Normal In-Reply-To: <5C8959A16A71B449AE793CF52FBBED6614913F@ptah.newyork.qosient.com> Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hey Maurizio, The requirement to indicate to collectors that a configuration parameter has changed does specify in anyway what the configuration is or what the configuration supports or how the device uses the configuration. Carter > -----Original Message----- > From: Maurizio Molina [mailto:molina@ccrle.nec.de] > Sent: Wednesday, May 21, 2003 12:40 PM > To: Carter Bullard > Cc: 'Alex Audu'; 'Ganesh Sadasivan'; ipfix@net.doit.wisc.edu > Subject: Re: [ipfix] Changes to architecture doc. > > > Carter Bullard wrote: > > > Hey Alex, > > What is the relationship between transport of > > flow records and the generation of flow records? > > IPFIX does not define a flow generation process, > > it defines a flow record transport protocol. > > > > Carter > > Hi Carter, > then what is your interpretation of the following part of the > requirement > draft (5.2), that basically says that information about > (packet) sampling > MUST be well defined and reported to the collector if the > sampling changes? > This information is about the generation of flow records, > isn't it? Then if > it's out of scope, we should drop this text from the requirements as > well... > Maurizio > > **************** From req. draft ********** > The metering process MAY support packet sampling. If sampling is > supported the sampling configuration MUST be well defined. The > sampling configuration includes the sampling method and all its > parameters. > > If the sampling configuration is changed during operation, the new > sampling configuration with its parameters MUST be indicated to all > collecting processes receiving the affected flow records. Changing > the sampling configuration includes: adding a sampling function to > the metering process, removing a sampling function from > the metering > process, change sampling method, and change sampling parameter(s). > > In case of any change in the sampling configuration, all > flow records > metered by the previous sampling configuration MUST be > terminated and > exported according to the export configuration. The > metering process > MUST NOT merge the flow records generated with the new sampling > configuration with the flow records generated with the previous > sampling configuration. > ******** End From Req. draft ***************** > > > > > > > > > -----Original Message----- > > > From: Alex Audu [mailto:Alex.Audu@alcatel.com] > > > Sent: Wednesday, May 21, 2003 11:52 AM > > > To: Carter Bullard > > > Cc: 'Maurizio Molina'; 'Ganesh Sadasivan'; ipfix@net.doit.wisc.edu > > > Subject: Re: [ipfix] Changes to architecture doc. > > > > > > > > > This, in my view, is a huge limitiation of IPFIX. PSAMP does > > > a better job of > > > specifying > > > the packet selection process as well as the other aspects > > > involved in getting the > > > data to > > > the collector. IPFIX is being suggested as a viable export > > > for PSAMP. If so, it > > > needs to > > > address the issue of data selection. > > > > > > Regards, > > > Alex. > > > > > > Carter Bullard wrote: > > > > > > > Maurizio, > > > > > > > > I believe that this is out of scope with IPFIX. > > > > We're suppose to be defining transport of flow > > > > records, not defining flow monitors. > > > > > > > > Carter > > > > > > > > > -----Original Message----- > > > > > From: majordomo listserver > > > > > [mailto:majordomo@mil.doit.wisc.edu] On Behalf Of > Maurizio Molina > > > > > Sent: Wednesday, May 21, 2003 6:01 AM > > > > > To: Ganesh Sadasivan > > > > > Cc: ipfix@net.doit.wisc.edu > > > > > Subject: Re: [ipfix] Changes to architecture doc. > > > > > > > > > > > > > > > > > > > > > > > > > Ganesh Sadasivan wrote: > > > > > > > > > > > Hi, > > > > > > > > > > > > In going through the architecture spec. I find that > > > the following > > > > > > topics > > > > > > needs to be modified. I'll list all of them in one e-mail > > > > > and then on a > > > > > > topic-by-topic we can have separate e-mail threads. > > > > > > > > > > > > BTW I could not get a copy of the arch. spec from the link > > > > > > > > > > > http://www.ietf.org/internet-drafts/draft-ietf-ipfix-architect > > > > > ure-02.txt > > > > > > > > > > > > This has been deleted from the internet drafts database > > > > > because it was > > > > > > not updated for more than 6 months. I guess the same is > > > > > true with data > > > > > > model > > > > > > document. > > > > > > > > > > > > The alternate link that I got it from is : > > > > > > > > > > > http://www.ietf.org/proceedings/02nov/I-D/draft-ietf-ipfix-arc > > > > > hitecture-02.txt > > > > > > > > > > > > > > > > ........... > > > > > > > > > > > > > > > > > * Section 4.5 : > > > > > > " There MAY be additional rules defined within the > > > > > observation domain > > > > > > so that only certain flows records are picked up for > > > export. " > > > > > > Though rules are defined w.r.t observation domain, > > > > > > the entity that applies these rules is the IPFIX protocol. > > > > > > > > > > Hi, > > > > > I'd like to discuss the type of packet sampling/flow > > > > > selection that seems to be > > > > > considered by IPFIX so far. > > > > > Re-reading section 4.5 of the arch. spec., and the previous > > > > > ones (4.4.2, 4.4.3, > > > > > 4.4.4), I got the impression that only the following types of > > > > > sampling/slection at > > > > > the packet or flow level are considered: > > > > > 1) Sampling of packets within a flow (sec. 4.4.4): Packets > > > > > undergo a filter to see > > > > > if they match a flow to be monitored; if they do, they are > > > > > (possibly) sampled (to > > > > > alleviate METER processing load). > > > > > 2) Selection of flows to export (sec. 4.5): all defined flows > > > > > are monitored > > > > > (possibly packet sampling within flows as defined above is > > > > > applied), then a > > > > > decisions is taken about whether to export the entire pool of > > > > > monitored flows or > > > > > just a subset of it. This, with the goal to alleviate > > > > > EXPORTER/COLLECTOR load. > > > > > > > > > > Imo, what's not explicitly considered is a third case, where > > > > > there is a packet > > > > > sampling but "oriented" to flow selection, i.e. packets > > > > > undergo a filter match but > > > > > then they're not "blindly" (i.e. n out of N or similar) > > > > > sampled, but rather > > > > > sampled or discarded on the basis of some property of the > > > > > flow they matched. The > > > > > property may be static (e.g. "flows with high monitoring > > > > > priority") or dynamically > > > > > built (e.g. flows carrying more traffic than > > > > > others->sometime called "elephant" > > > > > flows). This type of sampling can alleviate both METER and > > > > > EXPORTER/COLLECTOR > > > > > processing load. > > > > > The interest for this type of "sampling/selection" (actually, > > > > > it's both packet > > > > > sampling and flow selection) is that for some applications it > > > > > may be preferred to > > > > > receive reliable figures of the volume of a subset of flows > > > > > rather that less > > > > > reliable estimates of the volume of ALL the flows (as > > > > > sampling of type 1 would > > > > > provide). Furthermore, differently from flow selection of > > > > > type 2), it's not > > > > > required to keep a flow record for ALL the flows before > > > > > deciding whether to export > > > > > them or not, as the decision whether to open a new flow > > > > > record is taken "on the > > > > > fly" when packets arrive. Then it's potentially more scalable. > > > > > > > > > > That said, what are the issues that a standard body should > > > > > consider? Imo, they are > > > > > mainly two: > > > > > - the definition of policies for seclecting packets on the > > > > > basis the property of > > > > > flow they match (i.e. high priority monitoring flow > > > > > definition, or flows carrying > > > > > high traffic volumes, or other....). This, without dwelling > > > > > into the details of > > > > > algorithms may implement this "on line" detection of > properties. > > > > > - the type of information that has to be exported: e.g. which > > > > > percentage of > > > > > traffic is being discarded? Otherwise a collector would only > > > > > get the report about > > > > > the monitored flows (high priority or high volume, or > > > > > other...), but it's > > > > > different to know that they compose the 99% oe the 10% of the > > > > > traffic.... > > > > > > > > > > In the requirement draft, > > > > > http://www.ietf.org/internet-drafts/draft-ietf-ipfix-reqs-09.t > > > > xt, in sec. 5.2 > > > > (sampling) and 5.3 (overload behaviour) there's already > > > some text that > > > > partially, > > > > but not completely and not exactly, supports the type of > > > sampling/selection > > > > I've > > > > described above. > > > > I'd be glad to know whether people in the group share this > > > view that this > > > > type of > > > > sampling/selection has to be more explicitly considered. > > > The next step would > > > > then > > > > be to discuss if/what needs to be standardized to support it. > > > > > > > > Regards, > > > > Maurizio > > > > > > > > -- > > > > Help mailto:majordomo@net.doit.wisc.edu and say > > > "help" in message > > > > body > > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > > "unsubscribe ipfix" in message body > > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > -- > > > > Help mailto:majordomo@net.doit.wisc.edu and say > > > "help" in message body > > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > > "unsubscribe ipfix" in message body > > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed May 21 14:48:32 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA01242 for ; Wed, 21 May 2003 14:48:31 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19IYVB-0006PT-00 for ipfix-list@mil.doit.wisc.edu; Wed, 21 May 2003 13:39:49 -0500 Received: from auds952.usa.alcatel.com ([143.209.238.7]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19IYV5-0006PF-00 for ipfix@net.doit.wisc.edu; Wed, 21 May 2003 13:39:43 -0500 Received: from alcatel.com (localhost [127.0.0.1]) by auds952.usa.alcatel.com (8.12.8p1/8.12.8) with ESMTP id h4LIcnaS002228; Wed, 21 May 2003 13:38:49 -0500 (CDT) Message-ID: <3ECBC7B9.F6D6F421@alcatel.com> Date: Wed, 21 May 2003 13:38:49 -0500 From: Alex Audu X-Mailer: Mozilla 4.72 [en] (WinNT; U) X-Accept-Language: en MIME-Version: 1.0 To: Carter Bullard CC: "'Maurizio Molina'" , "'Ganesh Sadasivan'" , ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Changes to architecture doc. References: <5C8959A16A71B449AE793CF52FBBED6607E9FA@ptah.newyork.qosient.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Carter, Transport and generation of flow records seem to be orthogonal processes, until you start asking for ways to control or modify the generation process itself, or to communicate the character of the generated flows to the remote collector. Some of the issues currently being discussed (what to do about lost flow records) could be linked to the lack of specification of the data model used in generating the flows. Alex.. Carter Bullard wrote: > Hey Alex, > What is the relationship between transport of > flow records and the generation of flow records? > IPFIX does not define a flow generation process, > it defines a flow record transport protocol. > > Carter > > > -----Original Message----- > > From: Alex Audu [mailto:Alex.Audu@alcatel.com] > > Sent: Wednesday, May 21, 2003 11:52 AM > > To: Carter Bullard > > Cc: 'Maurizio Molina'; 'Ganesh Sadasivan'; ipfix@net.doit.wisc.edu > > Subject: Re: [ipfix] Changes to architecture doc. > > > > > > This, in my view, is a huge limitiation of IPFIX. PSAMP does > > a better job of > > specifying > > the packet selection process as well as the other aspects > > involved in getting the > > data to > > the collector. IPFIX is being suggested as a viable export > > for PSAMP. If so, it > > needs to > > address the issue of data selection. > > > > Regards, > > Alex. > > > > Carter Bullard wrote: > > > > > Maurizio, > > > > > > I believe that this is out of scope with IPFIX. > > > We're suppose to be defining transport of flow > > > records, not defining flow monitors. > > > > > > Carter > > > > > > > -----Original Message----- > > > > From: majordomo listserver > > > > [mailto:majordomo@mil.doit.wisc.edu] On Behalf Of Maurizio Molina > > > > Sent: Wednesday, May 21, 2003 6:01 AM > > > > To: Ganesh Sadasivan > > > > Cc: ipfix@net.doit.wisc.edu > > > > Subject: Re: [ipfix] Changes to architecture doc. > > > > > > > > > > > > > > > > > > > > Ganesh Sadasivan wrote: > > > > > > > > > Hi, > > > > > > > > > > In going through the architecture spec. I find that > > the following > > > > > topics > > > > > needs to be modified. I'll list all of them in one e-mail > > > > and then on a > > > > > topic-by-topic we can have separate e-mail threads. > > > > > > > > > > BTW I could not get a copy of the arch. spec from the link > > > > > > > > > http://www.ietf.org/internet-drafts/draft-ietf-ipfix-architect > > > > ure-02.txt > > > > > > > > > > This has been deleted from the internet drafts database > > > > because it was > > > > > not updated for more than 6 months. I guess the same is > > > > true with data > > > > > model > > > > > document. > > > > > > > > > > The alternate link that I got it from is : > > > > > > > > > http://www.ietf.org/proceedings/02nov/I-D/draft-ietf-ipfix-arc > > > > hitecture-02.txt > > > > > > > > > > > > > ........... > > > > > > > > > > > > > > * Section 4.5 : > > > > > " There MAY be additional rules defined within the > > > > observation domain > > > > > so that only certain flows records are picked up for > > export. " > > > > > Though rules are defined w.r.t observation domain, > > > > > the entity that applies these rules is the IPFIX protocol. > > > > > > > > Hi, > > > > I'd like to discuss the type of packet sampling/flow > > > > selection that seems to be > > > > considered by IPFIX so far. > > > > Re-reading section 4.5 of the arch. spec., and the previous > > > > ones (4.4.2, 4.4.3, > > > > 4.4.4), I got the impression that only the following types of > > > > sampling/slection at > > > > the packet or flow level are considered: > > > > 1) Sampling of packets within a flow (sec. 4.4.4): Packets > > > > undergo a filter to see > > > > if they match a flow to be monitored; if they do, they are > > > > (possibly) sampled (to > > > > alleviate METER processing load). > > > > 2) Selection of flows to export (sec. 4.5): all defined flows > > > > are monitored > > > > (possibly packet sampling within flows as defined above is > > > > applied), then a > > > > decisions is taken about whether to export the entire pool of > > > > monitored flows or > > > > just a subset of it. This, with the goal to alleviate > > > > EXPORTER/COLLECTOR load. > > > > > > > > Imo, what's not explicitly considered is a third case, where > > > > there is a packet > > > > sampling but "oriented" to flow selection, i.e. packets > > > > undergo a filter match but > > > > then they're not "blindly" (i.e. n out of N or similar) > > > > sampled, but rather > > > > sampled or discarded on the basis of some property of the > > > > flow they matched. The > > > > property may be static (e.g. "flows with high monitoring > > > > priority") or dynamically > > > > built (e.g. flows carrying more traffic than > > > > others->sometime called "elephant" > > > > flows). This type of sampling can alleviate both METER and > > > > EXPORTER/COLLECTOR > > > > processing load. > > > > The interest for this type of "sampling/selection" (actually, > > > > it's both packet > > > > sampling and flow selection) is that for some applications it > > > > may be preferred to > > > > receive reliable figures of the volume of a subset of flows > > > > rather that less > > > > reliable estimates of the volume of ALL the flows (as > > > > sampling of type 1 would > > > > provide). Furthermore, differently from flow selection of > > > > type 2), it's not > > > > required to keep a flow record for ALL the flows before > > > > deciding whether to export > > > > them or not, as the decision whether to open a new flow > > > > record is taken "on the > > > > fly" when packets arrive. Then it's potentially more scalable. > > > > > > > > That said, what are the issues that a standard body should > > > > consider? Imo, they are > > > > mainly two: > > > > - the definition of policies for seclecting packets on the > > > > basis the property of > > > > flow they match (i.e. high priority monitoring flow > > > > definition, or flows carrying > > > > high traffic volumes, or other....). This, without dwelling > > > > into the details of > > > > algorithms may implement this "on line" detection of properties. > > > > - the type of information that has to be exported: e.g. which > > > > percentage of > > > > traffic is being discarded? Otherwise a collector would only > > > > get the report about > > > > the monitored flows (high priority or high volume, or > > > > other...), but it's > > > > different to know that they compose the 99% oe the 10% of the > > > > traffic.... > > > > > > > > In the requirement draft, > > > > http://www.ietf.org/internet-drafts/draft-ietf-ipfix-reqs-09.t > > > xt, in sec. 5.2 > > > (sampling) and 5.3 (overload behaviour) there's already > > some text that > > > partially, > > > but not completely and not exactly, supports the type of > > sampling/selection > > > I've > > > described above. > > > I'd be glad to know whether people in the group share this > > view that this > > > type of > > > sampling/selection has to be more explicitly considered. > > The next step would > > > then > > > be to discuss if/what needs to be standardized to support it. > > > > > > Regards, > > > Maurizio > > > > > > -- > > > Help mailto:majordomo@net.doit.wisc.edu and say > > "help" in message > > > body > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > "unsubscribe ipfix" in message body > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > -- > > > Help mailto:majordomo@net.doit.wisc.edu and say > > "help" in message body > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > "unsubscribe ipfix" in message body > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed May 21 16:06:02 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA06980 for ; Wed, 21 May 2003 16:06:02 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19IZMY-0007VB-00 for ipfix-list@mil.doit.wisc.edu; Wed, 21 May 2003 14:34:58 -0500 Received: from palrel13.hp.com ([156.153.255.238]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19IZMW-0007V4-00 for ipfix@net.doit.wisc.edu; Wed, 21 May 2003 14:34:56 -0500 Received: from xparelay1.ptp.hp.com (xparelay1.ptp.hp.com [15.1.28.62]) by palrel13.hp.com (Postfix) with ESMTP id EB1211C021D7; Wed, 21 May 2003 12:34:55 -0700 (PDT) Received: from xpabh3.ptp.hp.com (xpabh3.ptp.hp.com [15.1.28.63]) by xparelay1.ptp.hp.com (Postfix) with ESMTP id 674FD100556D; Wed, 21 May 2003 12:34:55 -0700 (PDT) Received: by xpabh3.ptp.hp.com with Internet Mail Service (5.5.2655.55) id ; Wed, 21 May 2003 12:34:55 -0700 Message-ID: <1D3D2C371FCBD947A7897FABBD3533A566BB36@xsun01.ptp.hp.com> From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" To: "'Alex Audu'" , Carter Bullard Cc: "'Maurizio Molina'" , "'Ganesh Sadasivan'" , ipfix@net.doit.wisc.edu Subject: RE: [ipfix] Changes to architecture doc. Date: Wed, 21 May 2003 12:34:52 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2655.55) Content-Type: text/plain; charset="iso-8859-1" Precedence: bulk Sender: majordomo listserver Hi, I'd say the purpose of the IPFIX protocol is to get a data record from the exporter to the collector. So lost flow records (at least those which the exporter has acquired from the observer) are in the domain of the IPFIX protocol, as some degree of reliability is required. The operation of the metering process, e.g. does it sample, does it filter, etc., are separate concerns. This is pretty explicit in the base IPFIX requirements document: http://www.ietf.org/internet-drafts/draft-ietf-ipfix-reqs-09.txt section 7.1. -- Jeff -----Original Message----- From: Alex Audu [mailto:Alex.Audu@alcatel.com] Sent: Wednesday, May 21, 2003 11:39 AM To: Carter Bullard Cc: 'Maurizio Molina'; 'Ganesh Sadasivan'; ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Changes to architecture doc. Carter, Transport and generation of flow records seem to be orthogonal processes, until you start asking for ways to control or modify the generation process itself, or to communicate the character of the generated flows to the remote collector. Some of the issues currently being discussed (what to do about lost flow records) could be linked to the lack of specification of the data model used in generating the flows. Alex.. Carter Bullard wrote: > Hey Alex, > What is the relationship between transport of > flow records and the generation of flow records? > IPFIX does not define a flow generation process, > it defines a flow record transport protocol. > > Carter > > > -----Original Message----- > > From: Alex Audu [mailto:Alex.Audu@alcatel.com] > > Sent: Wednesday, May 21, 2003 11:52 AM > > To: Carter Bullard > > Cc: 'Maurizio Molina'; 'Ganesh Sadasivan'; ipfix@net.doit.wisc.edu > > Subject: Re: [ipfix] Changes to architecture doc. > > > > > > This, in my view, is a huge limitiation of IPFIX. PSAMP does > > a better job of > > specifying > > the packet selection process as well as the other aspects > > involved in getting the > > data to > > the collector. IPFIX is being suggested as a viable export > > for PSAMP. If so, it > > needs to > > address the issue of data selection. > > > > Regards, > > Alex. > > > > Carter Bullard wrote: > > > > > Maurizio, > > > > > > I believe that this is out of scope with IPFIX. > > > We're suppose to be defining transport of flow > > > records, not defining flow monitors. > > > > > > Carter > > > > > > > -----Original Message----- > > > > From: majordomo listserver > > > > [mailto:majordomo@mil.doit.wisc.edu] On Behalf Of Maurizio Molina > > > > Sent: Wednesday, May 21, 2003 6:01 AM > > > > To: Ganesh Sadasivan > > > > Cc: ipfix@net.doit.wisc.edu > > > > Subject: Re: [ipfix] Changes to architecture doc. > > > > > > > > > > > > > > > > > > > > Ganesh Sadasivan wrote: > > > > > > > > > Hi, > > > > > > > > > > In going through the architecture spec. I find that > > the following > > > > > topics > > > > > needs to be modified. I'll list all of them in one e-mail > > > > and then on a > > > > > topic-by-topic we can have separate e-mail threads. > > > > > > > > > > BTW I could not get a copy of the arch. spec from the link > > > > > > > > > http://www.ietf.org/internet-drafts/draft-ietf-ipfix-architect > > > > ure-02.txt > > > > > > > > > > This has been deleted from the internet drafts database > > > > because it was > > > > > not updated for more than 6 months. I guess the same is > > > > true with data > > > > > model > > > > > document. > > > > > > > > > > The alternate link that I got it from is : > > > > > > > > > http://www.ietf.org/proceedings/02nov/I-D/draft-ietf-ipfix-arc > > > > hitecture-02.txt > > > > > > > > > > > > > ........... > > > > > > > > > > > > > > * Section 4.5 : > > > > > " There MAY be additional rules defined within the > > > > observation domain > > > > > so that only certain flows records are picked up for > > export. " > > > > > Though rules are defined w.r.t observation domain, > > > > > the entity that applies these rules is the IPFIX protocol. > > > > > > > > Hi, > > > > I'd like to discuss the type of packet sampling/flow > > > > selection that seems to be > > > > considered by IPFIX so far. > > > > Re-reading section 4.5 of the arch. spec., and the previous > > > > ones (4.4.2, 4.4.3, > > > > 4.4.4), I got the impression that only the following types of > > > > sampling/slection at > > > > the packet or flow level are considered: > > > > 1) Sampling of packets within a flow (sec. 4.4.4): Packets > > > > undergo a filter to see > > > > if they match a flow to be monitored; if they do, they are > > > > (possibly) sampled (to > > > > alleviate METER processing load). > > > > 2) Selection of flows to export (sec. 4.5): all defined flows > > > > are monitored > > > > (possibly packet sampling within flows as defined above is > > > > applied), then a > > > > decisions is taken about whether to export the entire pool of > > > > monitored flows or > > > > just a subset of it. This, with the goal to alleviate > > > > EXPORTER/COLLECTOR load. > > > > > > > > Imo, what's not explicitly considered is a third case, where > > > > there is a packet > > > > sampling but "oriented" to flow selection, i.e. packets > > > > undergo a filter match but > > > > then they're not "blindly" (i.e. n out of N or similar) > > > > sampled, but rather > > > > sampled or discarded on the basis of some property of the > > > > flow they matched. The > > > > property may be static (e.g. "flows with high monitoring > > > > priority") or dynamically > > > > built (e.g. flows carrying more traffic than > > > > others->sometime called "elephant" > > > > flows). This type of sampling can alleviate both METER and > > > > EXPORTER/COLLECTOR > > > > processing load. > > > > The interest for this type of "sampling/selection" (actually, > > > > it's both packet > > > > sampling and flow selection) is that for some applications it > > > > may be preferred to > > > > receive reliable figures of the volume of a subset of flows > > > > rather that less > > > > reliable estimates of the volume of ALL the flows (as > > > > sampling of type 1 would > > > > provide). Furthermore, differently from flow selection of > > > > type 2), it's not > > > > required to keep a flow record for ALL the flows before > > > > deciding whether to export > > > > them or not, as the decision whether to open a new flow > > > > record is taken "on the > > > > fly" when packets arrive. Then it's potentially more scalable. > > > > > > > > That said, what are the issues that a standard body should > > > > consider? Imo, they are > > > > mainly two: > > > > - the definition of policies for seclecting packets on the > > > > basis the property of > > > > flow they match (i.e. high priority monitoring flow > > > > definition, or flows carrying > > > > high traffic volumes, or other....). This, without dwelling > > > > into the details of > > > > algorithms may implement this "on line" detection of properties. > > > > - the type of information that has to be exported: e.g. which > > > > percentage of > > > > traffic is being discarded? Otherwise a collector would only > > > > get the report about > > > > the monitored flows (high priority or high volume, or > > > > other...), but it's > > > > different to know that they compose the 99% oe the 10% of the > > > > traffic.... > > > > > > > > In the requirement draft, > > > > http://www.ietf.org/internet-drafts/draft-ietf-ipfix-reqs-09.t > > > xt, in sec. 5.2 > > > (sampling) and 5.3 (overload behaviour) there's already > > some text that > > > partially, > > > but not completely and not exactly, supports the type of > > sampling/selection > > > I've > > > described above. > > > I'd be glad to know whether people in the group share this > > view that this > > > type of > > > sampling/selection has to be more explicitly considered. > > The next step would > > > then > > > be to discuss if/what needs to be standardized to support it. > > > > > > Regards, > > > Maurizio > > > > > > -- > > > Help mailto:majordomo@net.doit.wisc.edu and say > > "help" in message > > > body > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > "unsubscribe ipfix" in message body > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > -- > > > Help mailto:majordomo@net.doit.wisc.edu and say > > "help" in message body > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > "unsubscribe ipfix" in message body > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed May 21 16:46:19 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA08368 for ; Wed, 21 May 2003 16:46:19 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19IaIT-00012c-00 for ipfix-list@mil.doit.wisc.edu; Wed, 21 May 2003 15:34:49 -0500 Received: from atlrel8.hp.com ([156.153.255.206]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19IaIR-00012V-00 for ipfix@net.doit.wisc.edu; Wed, 21 May 2003 15:34:47 -0500 Received: from xatlrelay2.atl.hp.com (xatlrelay2.atl.hp.com [15.45.89.191]) by atlrel8.hp.com (Postfix) with ESMTP id 445A01C011D4; Tue, 27 May 2003 14:44:21 -0400 (EDT) Received: from xatlbh3.atl.hp.com (xatlbh3.atl.hp.com [15.45.89.188]) by xatlrelay2.atl.hp.com (Postfix) with ESMTP id 9D6BC1C00B40; Wed, 21 May 2003 16:34:46 -0400 (EDT) Received: by xatlbh3.atl.hp.com with Internet Mail Service (5.5.2655.55) id ; Wed, 21 May 2003 16:34:46 -0400 Message-ID: <1D3D2C371FCBD947A7897FABBD3533A566BB3A@xsun01.ptp.hp.com> From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" To: "'Sebastian Zander'" , ipfix@net.doit.wisc.edu Subject: RE: [ipfix] extensibility of the ipfix protocol Date: Wed, 21 May 2003 16:34:37 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2655.55) Content-Type: text/plain; charset="iso-8859-1" Precedence: bulk Sender: majordomo listserver Sebastian, One of the issues of talking about extensibility in the generic sense is that it creates the temptation to specify generality without any driving use case. "Someone may want to ..." seems like a slippery slope in bounding requirements. In particular the case (1) extending fields in the header sounds dubious. Since IPFIX is about getting flow records from the exporter to the collector, what is the purpose of allowing arbitrary header extensions? In case (2), I'd certainly recommend option (b), making an explicit separation of namespaces (in this case integer values) by using unique vendor ids as independent "namespace authorities" seems like common sense. SNMP's been doing this for years, and Diameter is also following this convention as you point out. As for case (3), my understanding is that explicit type information in the payload/template itself did not achieve consensus. This does mean that the numeric id (and optional vendorId) needs some concept of type, in a supporting document. Having each attribute specify its type by reference to a type space defined in the base IPFIX Information Model is the way to go here (in my opinion). When a vendor extends the set of attributes, chosing a well defined type would be preferred to inventing a new type. If the vendor cannot resist the overwhelming temptation to create new types, it can be defined in their own information model. Regards, Jeff Meyer -----Original Message----- From: Sebastian Zander [mailto:zander@fokus.fraunhofer.de] Sent: Tuesday, May 06, 2003 6:55 PM To: ipfix@net.doit.wisc.edu Subject: [ipfix] extensibility of the ipfix protocol Hi, although extensibility is mandatory by the requirements there hasn't been a lot discussion on how to do it and the chosen candidate protocol doesn't say much about extensibility. So here are my thoughts about extensibility. The protocol specification must define how the protocol can be extended. I see the following (but there may be more) ways of extending the protocol: 1. Extending the fixed packet header This would allow to add new header fields without changing the protocol specification. A bit (X) could be stolen from version to indicate the presence of extension header(s). Extension header(s) follow the fixed header if X=1. Extension header(s) consists of type, length and value. 2. Extending the attribute/field space This is certainly needed. It could be done at least in two ways: a) A flat number space where numbers are specified in extension RFCs and registered by IANA. A nice thing would a dynamic space where numbers are not defined by IANA but could be negotiated out of band (see payload type in the RTP specification). Such a space could also be used as safe playground. b) A number space which is defined in extension RFCs (IANA) and the possibility of using vendor specific attributes by having an optional vendor ID field in the template specification. The vendor ID would be IANA assigned but the vendor then could use its private field ID space (see the Diameter Base protocol specification). Adds more complexity and overhead to the protocol but could be very useful when the protocol becomes widely adopted and a lot of extensions are done. 3. Extending the type space (if there are types at all) Probably there won't be much need to extent this because the protocol specification could do a good job of specifying all the basic types. If this really needs to be extended it would be probably done with a revised protocol specification. Comments? Opinions? Cheers, Sebastian -- Sebastian Zander E-mail: zander@fokus.fraunhofer.de Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 D-10589 Berlin, Germany www.fokus.fraunhofer.de/usr/sebastian.zander -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed May 21 20:35:34 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA13949 for ; Wed, 21 May 2003 20:35:34 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19IdtV-0005bY-00 for ipfix-list@mil.doit.wisc.edu; Wed, 21 May 2003 19:25:17 -0500 Received: from ds20-1.cc.swin.edu.au ([136.186.1.150] helo=swin.edu.au) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19IdtT-0005bO-00 for ipfix@net.doit.wisc.edu; Wed, 21 May 2003 19:25:15 -0500 Received: from fokus.fraunhofer.de (szander-laptop.caia.swin.edu.au [136.186.229.90]) by swin.edu.au (8.9.3p2/8.9.3) with ESMTP id KAA1675946; Thu, 22 May 2003 10:25:10 +1000 (EST) Message-ID: <3ECC1773.9090703@fokus.fraunhofer.de> Date: Thu, 22 May 2003 02:18:59 +0200 From: Sebastian Zander Organization: Fraunhofer FOKUS User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.2) Gecko/20030208 Netscape/7.02 X-Accept-Language: en-us, en MIME-Version: 1.0 To: "MEYER,JEFFREY D (HP-Cupertino,ex1)" CC: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] extensibility of the ipfix protocol References: <1D3D2C371FCBD947A7897FABBD3533A566BB3A@xsun01.ptp.hp.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Jeff, MEYER,JEFFREY D (HP-Cupertino,ex1) wrote: > Sebastian, > > One of the issues of talking about extensibility in the generic sense is > that it creates the temptation to specify generality without any driving > use case. "Someone may want to ..." seems like a slippery slope in bounding > requirements. > > In particular the case (1) extending fields in the header sounds dubious. > > Since IPFIX is about getting flow records from the exporter to the > collector, > what is the purpose of allowing arbitrary header extensions? The purpose is to send information per packet which is associated to or necessary for using the rest of the information contained in the packet. For me its pretty obvious that you need such an extension mechanism. And as Mark pointed out this is already supported by the use of option templates. netflow certainly meets the extensibility reqs but the netflow drafts do *not* a good job IMO in explaining its extensibility compared to e.g. what Diameter does. Instead of letting the reader guess the ipfix draft should have more verbage. > In case (2), I'd certainly recommend option (b), making an explicit > separation of namespaces (in this case integer values) by using unique > vendor ids as independent "namespace authorities" seems like common sense. > SNMP's been doing this for years, and Diameter is also following this > convention as you point out. > > As for case (3), my understanding is that explicit type information in > the payload/template itself did not achieve consensus. This does mean > that the numeric id (and optional vendorId) needs some concept of type, > in a supporting document. Having each attribute specify its type by > reference to a type space defined in the base IPFIX Information Model > is the way to go here (in my opinion). Yeah, but the protocol draft must be aligned somehow to the information model draft e.g. where are the types defined, where is the encoding defined... Cheers, Sebastian > When a vendor extends the set of attributes, chosing a well defined > type would be preferred to inventing a new type. If the vendor cannot > resist the overwhelming temptation to create new types, it can be > defined in their own information model. > > > Regards, > > Jeff Meyer > > -----Original Message----- > From: Sebastian Zander [mailto:zander@fokus.fraunhofer.de] > Sent: Tuesday, May 06, 2003 6:55 PM > To: ipfix@net.doit.wisc.edu > Subject: [ipfix] extensibility of the ipfix protocol > > > Hi, > > although extensibility is mandatory by the requirements there hasn't > been a lot discussion on how to do it and the chosen candidate protocol > doesn't say much about extensibility. So here are my thoughts about > extensibility. > > The protocol specification must define how the protocol can be extended. > > I see the following (but there may be more) ways of extending the > protocol: > > 1. Extending the fixed packet header > This would allow to add new header fields without changing the > protocol specification. > > A bit (X) could be stolen from version to indicate the presence of > extension header(s). Extension header(s) follow the fixed header if > X=1. Extension header(s) consists of type, length and value. > > 2. Extending the attribute/field space > This is certainly needed. It could be done at least in two ways: > a) A flat number space where numbers are specified in extension > RFCs and registered by IANA. A nice thing would a dynamic > space where numbers are not defined by IANA but could be negotiated > out of band (see payload type in the RTP specification). Such a > space could also be used as safe playground. > b) A number space which is defined in extension RFCs (IANA) and the > possibility of using vendor specific attributes by having an optional > vendor ID field in the template specification. The vendor ID would > be IANA assigned but the vendor then could use its private field ID > space (see the Diameter Base protocol specification). Adds more > complexity and overhead to the protocol but could be very useful > when the protocol becomes widely adopted and a lot of extensions > are done. > > 3. Extending the type space (if there are types at all) > Probably there won't be much need to extent this because the protocol > specification could do a good job of specifying all the basic types. > If this really needs to be extended it would be probably done with a > revised protocol specification. > > Comments? Opinions? > > Cheers, > > Sebastian > -- Sebastian Zander E-mail: zander@fokus.fraunhofer.de Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 D-10589 Berlin, Germany www.fokus.fraunhofer.de/usr/sebastian.zander -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed May 21 20:54:56 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA14334 for ; Wed, 21 May 2003 20:54:55 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19IeEa-00064O-00 for ipfix-list@mil.doit.wisc.edu; Wed, 21 May 2003 19:47:04 -0500 Received: from palrel11.hp.com ([156.153.255.246]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19IeEY-00064G-00 for ipfix@net.doit.wisc.edu; Wed, 21 May 2003 19:47:02 -0500 Received: from xparelay2.ptp.hp.com (xparelay2.ptp.hp.com [15.1.28.65]) by palrel11.hp.com (Postfix) with ESMTP id 51F721C011FB; Wed, 21 May 2003 17:47:01 -0700 (PDT) Received: from xpabh3.ptp.hp.com (xpabh3.ptp.hp.com [15.1.28.63]) by xparelay2.ptp.hp.com (Postfix) with ESMTP id 0F8F31C00AD9; Wed, 21 May 2003 17:47:01 -0700 (PDT) Received: by xpabh3.ptp.hp.com with Internet Mail Service (5.5.2655.55) id ; Wed, 21 May 2003 17:47:00 -0700 Message-ID: <1D3D2C371FCBD947A7897FABBD3533A566BB3F@xsun01.ptp.hp.com> From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" To: "'Sebastian Zander'" , "MEYER,JEFFREY D (HP-Cupertino,ex1)" Cc: ipfix@net.doit.wisc.edu Subject: RE: [ipfix] extensibility of the ipfix protocol Date: Wed, 21 May 2003 17:46:58 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2655.55) Content-Type: text/plain; charset="iso-8859-1" Precedence: bulk Sender: majordomo listserver Sebastian, I'd agree that making the extensibility model more explicit is desireable. I'm still unclear on when or why one would extend information into the header vs. extending the data template sent itself. I view the Option template as flagging "pseudo-flow records" which have a set of attributes which carry information other than actual flow details, e.g. configuration info or counters. But isn't the combination of having Extensible Flow Records (for "real" data) and Extensible Option Records (for "other" data) sufficient to move any form of data the exporter wants to export? I guess this is what Mark was saying. So I'd agree to that. I would see the same "Information Model" and extension model being used for both Flow Record and Options. So to clarify, the header is fixed. There is a distinction between flow records and "other" data transmitted by the exporter. And a common way of defining how attributes which appear in either flow or option records will be used. I'm not sure on why the "Scope" options should be treated differently than any other attribute which may appear in an Option Record (Section 6.1) in NFv9 specification. Seems like things would be simpler and just as effective if there were just a "Scope" attribute which could appear in Option records. In this case an Options template would be identical to a Flow Template except it would have a FlowSet id of 1 vs. 0. Does that work? Regards, Jeff Meyer -----Original Message----- From: Sebastian Zander [mailto:zander@fokus.fraunhofer.de] Sent: Wednesday, May 21, 2003 5:19 PM To: MEYER,JEFFREY D (HP-Cupertino,ex1) Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] extensibility of the ipfix protocol Jeff, MEYER,JEFFREY D (HP-Cupertino,ex1) wrote: > Sebastian, > > One of the issues of talking about extensibility in the generic sense is > that it creates the temptation to specify generality without any driving > use case. "Someone may want to ..." seems like a slippery slope in bounding > requirements. > > In particular the case (1) extending fields in the header sounds dubious. > > Since IPFIX is about getting flow records from the exporter to the > collector, > what is the purpose of allowing arbitrary header extensions? The purpose is to send information per packet which is associated to or necessary for using the rest of the information contained in the packet. For me its pretty obvious that you need such an extension mechanism. And as Mark pointed out this is already supported by the use of option templates. netflow certainly meets the extensibility reqs but the netflow drafts do *not* a good job IMO in explaining its extensibility compared to e.g. what Diameter does. Instead of letting the reader guess the ipfix draft should have more verbage. > In case (2), I'd certainly recommend option (b), making an explicit > separation of namespaces (in this case integer values) by using unique > vendor ids as independent "namespace authorities" seems like common sense. > SNMP's been doing this for years, and Diameter is also following this > convention as you point out. > > As for case (3), my understanding is that explicit type information in > the payload/template itself did not achieve consensus. This does mean > that the numeric id (and optional vendorId) needs some concept of type, > in a supporting document. Having each attribute specify its type by > reference to a type space defined in the base IPFIX Information Model > is the way to go here (in my opinion). Yeah, but the protocol draft must be aligned somehow to the information model draft e.g. where are the types defined, where is the encoding defined... Cheers, Sebastian > When a vendor extends the set of attributes, chosing a well defined > type would be preferred to inventing a new type. If the vendor cannot > resist the overwhelming temptation to create new types, it can be > defined in their own information model. > > > Regards, > > Jeff Meyer > > -----Original Message----- > From: Sebastian Zander [mailto:zander@fokus.fraunhofer.de] > Sent: Tuesday, May 06, 2003 6:55 PM > To: ipfix@net.doit.wisc.edu > Subject: [ipfix] extensibility of the ipfix protocol > > > Hi, > > although extensibility is mandatory by the requirements there hasn't > been a lot discussion on how to do it and the chosen candidate protocol > doesn't say much about extensibility. So here are my thoughts about > extensibility. > > The protocol specification must define how the protocol can be extended. > > I see the following (but there may be more) ways of extending the > protocol: > > 1. Extending the fixed packet header > This would allow to add new header fields without changing the > protocol specification. > > A bit (X) could be stolen from version to indicate the presence of > extension header(s). Extension header(s) follow the fixed header if > X=1. Extension header(s) consists of type, length and value. > > 2. Extending the attribute/field space > This is certainly needed. It could be done at least in two ways: > a) A flat number space where numbers are specified in extension > RFCs and registered by IANA. A nice thing would a dynamic > space where numbers are not defined by IANA but could be negotiated > out of band (see payload type in the RTP specification). Such a > space could also be used as safe playground. > b) A number space which is defined in extension RFCs (IANA) and the > possibility of using vendor specific attributes by having an optional > vendor ID field in the template specification. The vendor ID would > be IANA assigned but the vendor then could use its private field ID > space (see the Diameter Base protocol specification). Adds more > complexity and overhead to the protocol but could be very useful > when the protocol becomes widely adopted and a lot of extensions > are done. > > 3. Extending the type space (if there are types at all) > Probably there won't be much need to extent this because the protocol > specification could do a good job of specifying all the basic types. > If this really needs to be extended it would be probably done with a > revised protocol specification. > > Comments? Opinions? > > Cheers, > > Sebastian > -- Sebastian Zander E-mail: zander@fokus.fraunhofer.de Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 D-10589 Berlin, Germany www.fokus.fraunhofer.de/usr/sebastian.zander -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed May 21 22:04:24 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA15554 for ; Wed, 21 May 2003 22:04:23 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19IfJn-0007QG-00 for ipfix-list@mil.doit.wisc.edu; Wed, 21 May 2003 20:56:31 -0500 Received: from ds20-1.cc.swin.edu.au ([136.186.1.150] helo=swin.edu.au) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19IfJk-0007Q6-00 for ipfix@net.doit.wisc.edu; Wed, 21 May 2003 20:56:29 -0500 Received: from fokus.fraunhofer.de (szander-laptop.caia.swin.edu.au [136.186.229.90]) by swin.edu.au (8.9.3p2/8.9.3) with ESMTP id LAA1687445; Thu, 22 May 2003 11:56:07 +1000 (EST) Message-ID: <3ECC2CC1.8090900@fokus.fraunhofer.de> Date: Thu, 22 May 2003 03:49:53 +0200 From: Sebastian Zander Organization: Fraunhofer FOKUS User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.2) Gecko/20030208 Netscape/7.02 X-Accept-Language: en-us, en MIME-Version: 1.0 To: "MEYER,JEFFREY D (HP-Cupertino,ex1)" CC: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] extensibility of the ipfix protocol References: <1D3D2C371FCBD947A7897FABBD3533A566BB3F@xsun01.ptp.hp.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Jeff, comments below MEYER,JEFFREY D (HP-Cupertino,ex1) wrote: > Sebastian, > > I'd agree that making the extensibility model more explicit is desireable. > > I'm still unclear on when or why one would extend information into the > header > vs. extending the data template sent itself. I view the Option template as > flagging "pseudo-flow records" which have a set of attributes which carry > information other than actual flow details, e.g. configuration info or > counters. > > But isn't the combination of having Extensible Flow Records (for "real" > data) and Extensible Option Records (for "other" data) sufficient to move > any form of data the exporter wants to export? > > I guess this is what Mark was saying. So I'd agree to that. I also agree > I would see the same "Information Model" and extension model being used > for both Flow Record and Options. > > > So to clarify, the header is fixed. There is a distinction between flow > records and "other" data transmitted by the exporter. And a common way > of defining how attributes which appear in either flow or option records > will be used. > > I'm not sure on why the "Scope" options should be treated differently > than any other attribute which may appear in an Option Record (Section 6.1) > in NFv9 specification. Seems like things would be simpler and just as > effective if there were just a "Scope" attribute which could appear in > Option records. In this case an Options template would be identical > to a Flow Template except it would have a FlowSet id of 1 vs. 0. > > Does that work? I must admit that i don't 100% understand how scope is supposed to work. The example in 10.4 and 10.5 seems somewhat flawed. The draft says there are N scope fields and N option fields. Must the number of scope fields be equal to the number of option fields? If yes this seems to waste lots of space if the scope does not change (and it is not the case in the example). If not (probably the authors meant M and N) the question is how to map the scope fields to the options fields when there is more than 1 scope field (seems impossibe)? Making scope another attribute sounds somewhat good. However has scope some kind of grouping functionality or must it explicitly appear before each option? Can scopes be combined such as e.g. this option is per interface vs. this option is per template per interface? I think my questions are about grouping because scope seems to have some kind of grouping function... Cheers, Sebastian > > Regards, > > Jeff Meyer > > -----Original Message----- > From: Sebastian Zander [mailto:zander@fokus.fraunhofer.de] > Sent: Wednesday, May 21, 2003 5:19 PM > To: MEYER,JEFFREY D (HP-Cupertino,ex1) > Cc: ipfix@net.doit.wisc.edu > Subject: Re: [ipfix] extensibility of the ipfix protocol > > > Jeff, > > MEYER,JEFFREY D (HP-Cupertino,ex1) wrote: > >>Sebastian, >> >> One of the issues of talking about extensibility in the generic sense is >>that it creates the temptation to specify generality without any driving >>use case. "Someone may want to ..." seems like a slippery slope in > > bounding > >>requirements. >> >> In particular the case (1) extending fields in the header sounds > > dubious. > >> Since IPFIX is about getting flow records from the exporter to the >>collector, >>what is the purpose of allowing arbitrary header extensions? > > > The purpose is to send information per packet which is associated to or > necessary for using the rest of the information contained in the packet. For > me its pretty obvious that you need such an extension mechanism. And as Mark > pointed out this is already supported by the use of option templates. > > netflow certainly meets the extensibility reqs but the netflow drafts do > *not* > a good job IMO in explaining its extensibility compared to e.g. what > Diameter > does. Instead of letting the reader guess the ipfix draft should have more > verbage. > > >> In case (2), I'd certainly recommend option (b), making an explicit >>separation of namespaces (in this case integer values) by using unique >>vendor ids as independent "namespace authorities" seems like common sense. >>SNMP's been doing this for years, and Diameter is also following this >>convention as you point out. >> >> As for case (3), my understanding is that explicit type information in >>the payload/template itself did not achieve consensus. This does mean >>that the numeric id (and optional vendorId) needs some concept of type, >>in a supporting document. Having each attribute specify its type by >>reference to a type space defined in the base IPFIX Information Model >>is the way to go here (in my opinion). > > > Yeah, but the protocol draft must be aligned somehow to the information > model draft e.g. where are the types defined, where is the encoding > defined... > > Cheers, > > Sebastian > > >> When a vendor extends the set of attributes, chosing a well defined >>type would be preferred to inventing a new type. If the vendor cannot >>resist the overwhelming temptation to create new types, it can be >>defined in their own information model. >> >> >>Regards, >> >> Jeff Meyer >> >>-----Original Message----- >>From: Sebastian Zander [mailto:zander@fokus.fraunhofer.de] >>Sent: Tuesday, May 06, 2003 6:55 PM >>To: ipfix@net.doit.wisc.edu >>Subject: [ipfix] extensibility of the ipfix protocol >> >> >>Hi, >> >>although extensibility is mandatory by the requirements there hasn't >>been a lot discussion on how to do it and the chosen candidate protocol >>doesn't say much about extensibility. So here are my thoughts about >>extensibility. >> >>The protocol specification must define how the protocol can be extended. >> >>I see the following (but there may be more) ways of extending the >>protocol: >> >>1. Extending the fixed packet header >>This would allow to add new header fields without changing the >>protocol specification. >> >>A bit (X) could be stolen from version to indicate the presence of >>extension header(s). Extension header(s) follow the fixed header if >>X=1. Extension header(s) consists of type, length and value. >> >>2. Extending the attribute/field space >>This is certainly needed. It could be done at least in two ways: >>a) A flat number space where numbers are specified in extension >> RFCs and registered by IANA. A nice thing would a dynamic >> space where numbers are not defined by IANA but could be negotiated >> out of band (see payload type in the RTP specification). Such a >> space could also be used as safe playground. >>b) A number space which is defined in extension RFCs (IANA) and the >> possibility of using vendor specific attributes by having an optional >> vendor ID field in the template specification. The vendor ID would >> be IANA assigned but the vendor then could use its private field ID >> space (see the Diameter Base protocol specification). Adds more >> complexity and overhead to the protocol but could be very useful >> when the protocol becomes widely adopted and a lot of extensions >> are done. >> >>3. Extending the type space (if there are types at all) >>Probably there won't be much need to extent this because the protocol >>specification could do a good job of specifying all the basic types. >>If this really needs to be extended it would be probably done with a >>revised protocol specification. >> >>Comments? Opinions? >> >>Cheers, >> >>Sebastian >> > > > -- Sebastian Zander E-mail: zander@fokus.fraunhofer.de Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 D-10589 Berlin, Germany www.fokus.fraunhofer.de/usr/sebastian.zander -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu May 22 09:16:24 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA12430 for ; Thu, 22 May 2003 09:16:24 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19IpOP-0006Dz-00 for ipfix-list@mil.doit.wisc.edu; Thu, 22 May 2003 07:41:57 -0500 Received: from tokyo.ccrle.nec.de ([195.37.70.2]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19IpON-0006Dq-00 for ipfix@net.doit.wisc.edu; Thu, 22 May 2003 07:41:56 -0500 Received: from venus.office (venus.office [10.1.1.11]) by tokyo.ccrle.nec.de (8.12.9/8.12.8) with ESMTP id h4MCfsVI004537 for ; Thu, 22 May 2003 14:41:54 +0200 (CEST) Received: from [10.1.1.128] (n-quittek.office [10.1.1.128]) by venus.office (Postfix on SuSE Linux eMail Server 3.0) with ESMTP id 9C6919FCB0 for ; Thu, 22 May 2003 14:31:56 +0200 (CEST) Date: Thu, 22 May 2003 14:43:49 +0200 From: Juergen Quittek To: ipfix@net.doit.wisc.edu Subject: [ipfix] IPFIX requirements - final changes Message-ID: <17788047.1053614629@[10.1.1.128]> X-Mailer: Mulberry/2.1.2 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Dear all, Addressing the comments received from IESG review, we are working on a new - and hopefully final version of the requirements document. For reference, you will find a complete list of issues and their status at the end of this message. Several comments could be addressed by editorial changes and clarifications. They are already committed. But five comments suggest changes on which I want to have WG consensus before committing them. Please state if you see a problem with any of the suggested changes listed below (or if you have a better alternative). - More concrete definition of byte counter old text: "8. byte counter Which bytes of a packet are counted MUST be defined exactly." suggested replacement: "8. byte counter The sum of the total length of all IP packets belonging to the flow. The total length of a packet covers IP header and IP payload." - More concrete definition of multicast replication factor old text: "20. multicast replication factor the number of outgoing packets originating from a single incoming multicast packet. This is a dynamic property of multicast flows, that may change over time. For unicast flows it has the constant value 1. The computation of the factor MUST be clearly defined." suggestion: Replace last sentence by "The reported value MUST be the value of the factor at the time the flow record is exported." - More concrete definition of BGP-related attributes old text: "In addition, the exporting process MAY be able to report attributes related to inter-autonomous system routing of a flow, for example by reporting BGP Autonomous System numbers [RFC1771]." suggestion: replace text by three new MAY attributes "27. source BGP Autonomous System number [RFC1771] 28. destination BGP Autonomous System number 29. next hop BGP Autonomous System number" - Confidentiality from SHOULD to MUST suggestion: Change confidentiality requirement in section 6.3.3 from SCHOULD to MUST. - Anonymization from MAY to MUST suggestion: Change anonymization requirement in section 6.7 from MAY to MUST. Cheers, Juergen Here is the complete list of changes: ============================== IPFIX Requirements Open Issues ============================== ======================================================================== 1. timestamp resolution reference ======================================================================== Problem Description: ------------------------------------------------------------------------ 5.4. Timestamps The metering process MUST be able to generate timestamps for the first and the last observation of a packet of a flow at the observation point. The timestamp resolution MUST be at least the one of the sysUpTime [RFC1213], which is one centisecond. I think I'd rather see a reference to RFC3418 instead of RFC1213. We can do so by RFC-Editor note. May want to check with authors too. ======================================================================== Suggested solution: ------------------------------------------------------------------------ replace ref to RFC1213 by ref to RFC3418 ======================================================================== Status: solution committed ======================================================================== ======================================================================== 2. reporting TOS/traffic class octet or reporting DSCP ======================================================================== Problem Description: ------------------------------------------------------------------------ On page 14 I see: 9. type of service octet (in case of IPv4), traffic class octet (in case of IPv6) Is that not better state as 9. DSCP, which is set in the type of service octet (in case of IPv4), or in the traffic class octet (in case of IPv6) ======================================================================== Suggested solution: ------------------------------------------------------------------------ According to RFC 2474, the DSCP covers only 6 bit of the type of service octet, and in some cases you want to see the entire byte. This is be pointed out by a comment added to item 9.: "According to RFC 2474 these octets include the DiffServ Code Point that has a length of 6 bit." ======================================================================== Status: solution committed ======================================================================== ======================================================================== 3. location of reference to RFC 2119 ======================================================================== Problem Description: ------------------------------------------------------------------------ I would prefer to see the paragraph referencing RFC 2119 in section 2. ======================================================================== Suggested solution: ------------------------------------------------------------------------ Move reference to RFC 2119 to section 2 ======================================================================== Status: solution committed ======================================================================== ======================================================================== 4. bad wording in section 6.3.3, last paragraph ======================================================================== Problem Description: ------------------------------------------------------------------------ Last paragraph in section 6.3.3 needs to be reworded. ======================================================================== Suggested solution: ------------------------------------------------------------------------ replace "The security threats from which these 3 requirements are deducted are explained in the "Security Considerations" (Section 10)." by "The security requirements have been derived from an analysis of potential security threads. The analysis is summarized in Section 10." ======================================================================== Status: solution committed ======================================================================== ======================================================================== 5. describe potential problem of faked DoS attack ======================================================================== Problem Description: ------------------------------------------------------------------------ In section 10.2, an important point is left unsaid. If the injection of IPFIX traffic fool the network provided into thinking that a DOS attack is underway, the countermeasures employed by the network provider may actually deny service to real customers. ======================================================================== Suggested solution: ------------------------------------------------------------------------ no suggestion for a wording, yet ======================================================================== Status: solution required ======================================================================== ======================================================================== 6. location of appendix ======================================================================== Problem Description: ------------------------------------------------------------------------ The appendix needs to be moved up in the document. ======================================================================== Suggested solution: ------------------------------------------------------------------------ move appendix above references, copyright, and authors' addresses ======================================================================== Status: solution to be committed ======================================================================== ======================================================================== 7. consistency of appendix ======================================================================== Problem Description: ------------------------------------------------------------------------ The appendix is outdated. Alignment with the text sections and coverage of all requirements need to be checked and fixed. ======================================================================== Suggested solution: ------------------------------------------------------------------------ no suggestion, yet ======================================================================== Status: solution required ======================================================================== ======================================================================== 8. More concrete definition of byte counter ======================================================================== Problem Description: ------------------------------------------------------------------------ This document goes to a lot of trouble to say exactly what is exported (e.g. what fields, etc), except for two cases: 8. byte counter Which bytes of a packet are counted MUST be defined exactly. This document is defining what fields are being reported on; why can't it also define these two things that must also be defined? Is it useful to have different ipfix protocols exporting different measurements? ======================================================================== Suggested solution: ------------------------------------------------------------------------ replace "Which bytes of a packet are counted MUST be defined exactly." by "The sum of the total length of all IP packets belonging to the flow. The total length of a packet covers IP header and IP payload." ======================================================================== Status: solution to be agreed on ======================================================================== ======================================================================== 9. More concrete definition of multicast replication factor ======================================================================== Problem Description: ------------------------------------------------------------------------ This document goes to a lot of trouble to say exactly what is exported (e.g. what fields, etc), except for two cases: 20. multicast replication factor the number of outgoing packets originating from a single incoming multicast packet. This is a dynamic property of multicast flows, that may change over time. For unicast flows it has the constant value 1. The computation of the factor MUST be clearly defined. This document is defining what fields are being reported on; why can't it also define these two things that must also be defined? Is it useful to have different ipfix protocols exporting different measurements? ======================================================================== Suggested solution: ------------------------------------------------------------------------ replace last sentence by "The reported value MUST be the value of the factor at the time the flow record is exported." ======================================================================== Status: solution to be agreed on ======================================================================== ======================================================================== 10. More concrete definition of BGP-related attributes ======================================================================== Problem Description: ------------------------------------------------------------------------ Also, at the end of 6.1 it kind of says "Plus other stuff related to inter-AS routing if you want". It's already in the MAY section; can't they just list some concrete items related to inter-AS routing as 27, 28, ...? I don't think it's useful to say what amounts to "plus anything else that you might feel like adding that you think is relevant", since that doesn't encourage different people to make the same choices. ======================================================================== Suggested solution: ------------------------------------------------------------------------ add three more MAY attributes: - 27. source BGP Autonomous System number [RFC1771] - 28. destination BGP Autonomous System number - 29. next hop BGP Autonomous System number ======================================================================== Status: solution to be agreed on ======================================================================== ======================================================================== 11. Address the reliability issue of usage based accounting ======================================================================== Problem Description: ------------------------------------------------------------------------ The applications include usage-based accounting. They should cite RFC 2975, and indicate that the particular niche for usage-based accounting would not be met if reliability was not very high. Section 5.1 reliability requirements do not meet the needs, nor do the timestamping of first and last packet of a flow, because the flow may have been mischaracterized and that is first and last of different flows. The way to satisfy this issue is to caveat the discussion of usage-based accounting with strong words on reliability that counter the comments on lesser reliability and more probabilistic techniques for other applications of ipfix. ======================================================================== Suggested solution: ------------------------------------------------------------------------ We need to state clearly, that many accounting applications require higher reliability than what is provided by the MANDATORY requirements. Where vwould be a good place to state this? Already in section 3.1? In section 6.3.2? ======================================================================== Status: solution required ======================================================================== ======================================================================== 12. Confidentiality from SHOULD to MUST ======================================================================== Problem Description: ------------------------------------------------------------------------ Requirements on the ipfix implementation - the document is long and I wonder if the working group really meant the protocol's confidentiality and anonymization features to be so optional - SHOULD confidentiality, MAY anonymization. Just for implementation. ======================================================================== Suggested solution: ------------------------------------------------------------------------ change confidentiality requirement in section 6.3.3 from SCHOULD to MUST ======================================================================== Status: solution to be agreed on ======================================================================== ======================================================================== 13. Anonymization from MAY to MUST ======================================================================== Problem Description: ------------------------------------------------------------------------ Requirements on the ipfix implementation - the document is long and I wonder if the working group really meant the protocol's confidentiality and anonymization features to be so optional - SHOULD confidentiality, MAY anonymization. Just for implementation. ======================================================================== Suggested solution: ------------------------------------------------------------------------ change anonymization requirement in section 6.7 from MAY to MUST ======================================================================== Status: solution to be agreed on ======================================================================== -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu May 22 09:49:50 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA13937 for ; Thu, 22 May 2003 09:49:50 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19IpyF-000707-00 for ipfix-list@mil.doit.wisc.edu; Thu, 22 May 2003 08:18:59 -0500 Received: from tokyo.ccrle.nec.de ([195.37.70.2]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19IpyD-000700-00 for ipfix@net.doit.wisc.edu; Thu, 22 May 2003 08:18:57 -0500 Received: from venus.office (venus.office [10.1.1.11]) by tokyo.ccrle.nec.de (8.12.9/8.12.8) with ESMTP id h4MDIdVI007842; Thu, 22 May 2003 15:18:41 +0200 (CEST) Received: from [10.1.1.128] (n-quittek.office [10.1.1.128]) by venus.office (Postfix on SuSE Linux eMail Server 3.0) with ESMTP id A498A2C96F; Thu, 22 May 2003 15:08:41 +0200 (CEST) Date: Thu, 22 May 2003 15:20:34 +0200 From: Juergen Quittek To: "MEYER,JEFFREY D (HP-Cupertino,ex1)" , calato@riverstonenet.com Cc: "'Nevil Brownlee'" , ipfix@net.doit.wisc.edu Subject: RE: [ipfix] Editors for IPFIX drafts Message-ID: <19993589.1053616834@[10.1.1.128]> In-Reply-To: <1D3D2C371FCBD947A7897FABBD3533A566BAD3@xsun01.ptp.hp.com> References: <1D3D2C371FCBD947A7897FABBD3533A566BAD3@xsun01.ptp.hp.com> X-Mailer: Mulberry/2.1.2 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Jeff, -- MEYER,JEFFREY D (HP-Cupertino,ex1) wrote on 12 May 2003 11:51 -0400: > Juergen, > > I'm using a basic text editor. Since there are typical bounding markup > tags I typically cut/paste blocks of tag and fill in the meaningful detail. > > Not fancy, but I've found it workable. Agreed. I did this for some time. However, you will speed up by at least a factor of three if you use a reasonable tool. Juergen > -- Jeff > >> -----Original Message----- >> From: Juergen Quittek [mailto:quittek@ccrle.nec.de] >> Sent: Monday, May 12, 2003 3:53 AM >> To: calato@riverstonenet.com; MEYER,JEFFREY D (HP-Cupertino,ex1) >> Cc: 'Nevil Brownlee'; ipfix@net.doit.wisc.edu >> Subject: Re: [ipfix] Editors for IPFIX drafts >> >> >> Paul, >> >> From my experience, it does not make sense to write a largish >> XML specification without an XML editor, that automates >> typing of 80% of the lengthy XML DTD or schema syntax. >> >> Just using a Word or ASCII editor would be a significant >> waste of time. >> >> I'm using a commercial editor (TurboXML) but there are certainly >> also free ones. Does anyone know a free tool? Jeff? >> >> Cheers, >> >> Juergen >> >> -- calato@riverstonenet.com wrote on 06 May 2003 11:00 -0400: >> >> > >> > Seems like a reasonable model. I'd say lets head that way >> > and see how it goes. >> > >> > Are there any MS word templates for creating the XML part? >> > >> > >> > Paul >> > >> > "MEYER,JEFFREY D (HP-Cupertino,ex1)" wrote: >> >> >> >> Hi, >> >> >> >> If the human readable part is generated from the XML appendix, >> >> then I guess it doesn't matter which part you call normative, >> >> since they both originate from the same source. >> >> >> >> The development model I see is the following: >> >> >> >> Generate XML Info Model >> >> Run model through XSLT (xalan) translater to produce prose >> >> (consistent with RFC 2629) [I'll volunteer to write the >> >> style sheet translation) >> >> Incorporate the raw model as appendix and the generated prose >> >> description into the appropriate section (consistent w/ >> >> RFC 2629) along with other text of the Information >> Model Draft. >> >> Run Marshall's tool http://xml.resource.org/ to >> produce nroff style >> >> and HTML versions of document. Submit nroff style to >> RFC editor, >> >> post HTML style in addition to ipfix website (easier >> to navigate). >> >> >> >> For examples of the dual formatted output, see the IPDR >> submissions >> >> to IPDR located at: >> >> >> >> http://www.ipdr.org/documents/ipfix >> >> >> >> Regards, >> >> >> >> Jeff Meyer >> >> >> >> > -----Original Message----- >> >> > From: Juergen Quittek [mailto:quittek@ccrle.nec.de] >> >> > Sent: Thursday, May 01, 2003 8:21 AM >> >> > To: calato@riverstonenet.com >> >> > Cc: MEYER,JEFFREY D (HP-Cupertino,ex1); 'Nevil Brownlee'; >> >> > ipfix@net.doit.wisc.edu >> >> > Subject: Re: [ipfix] Editors for IPFIX drafts >> >> > >> >> > >> >> > Paul, >> >> > >> >> > I agree to Jeff, that there is some value in having an XML >> >> > representation >> >> > of the specification. However, in the end this document will >> >> > become an RFC >> >> > and it must be well readable to human beings. >> >> > >> >> > Therefore, I think that the main body must be human readable, >> >> > but we can >> >> > provide an appendix, that contains an equivalent machine >> >> > readable spec. >> >> > >> >> > Different to Jeff, I think that the hunam readable main >> body should be >> >> > the normative part, while the XML appendix is just given as >> >> > support for >> >> > users of XML tools allowing them to genrate protocol stacks >> >> > and/or tools >> >> > for consistency checking. >> >> > >> >> > As Jeff suggested, we can work with an XML representation >> >> > when developing >> >> > the spec and generate human readable text automatically from >> >> > the XML code. >> >> > This will help us avoiding fomal mistakes in the human >> >> > readable spec and we >> >> > would always have consistency between human readable and >> XML part. >> >> > >> >> > Juergen >> >> > >> >> > >> >> > -- calato@riverstonenet.com wrote on 01 May 2003 11:00 -0400: >> >> > >> >> > > >> >> > > I'm a little confused. Are we talking about XML descriptions >> >> > > as an appendix with prose as the main body or XML as the >> >> > > main body with tools translating it into a more readable >> >> > > form? Or XML as an appendix with tools translating it for >> >> > > use as sections mixed in with the prose? >> >> > > >> >> > > >> >> > > Paul >> >> > > >> >> > > Juergen Quittek wrote: >> >> > >> >> >> > >> Jeff, >> >> > >> >> >> > >> -- MEYER,JEFFREY D (HP-Cupertino,ex1) wrote on 28 April >> >> > 2003 09:20 -0700: >> >> > >> > >> >> > >> > Juergen, >> >> > >> > >> >> > >> > Thanks for the response. >> >> > >> > >> >> > >> > I agree XML in and of itself is marginally human readable. >> >> > >> > >> >> > >> > I think that basic tools such as Xalan >> >> > (http://xml.apache.org/), can >> >> > >> > be used to "transform" a base XML document into a >> >> > variety of formats, >> >> > >> > including formatting consistent with RFC's. Note that >> >> > all the IPDR >> >> > >> > RFC's were written in XML using Marshall Rose's >> xml2rfc tools >> >> > >> > (http://xml.resource.org/). >> >> > >> >> >> > >> I also like Marshall's tools. >> >> > >> If Paul agrees, we should use them for the data model >> document. >> >> > >> >> >> > >> > So the model I would picture is having the Normative >> >> > XML data model >> >> > >> > as an appendix, and having generated information from >> >> > this data model >> >> > >> > populate some of the body sections to aid human readers. >> >> > >> >> >> > >> This looks like a reasonable way to go. >> >> > >> >> >> > >> Juergen >> >> > >> >> >> > >> > Regards, >> >> > >> > >> >> > >> > Jeff Meyer >> >> > >> > >> >> > >> >> -----Original Message----- >> >> > >> >> From: Juergen Quittek [mailto:quittek@ccrle.nec.de] >> >> > >> >> Sent: Monday, April 28, 2003 8:24 AM >> >> > >> >> To: MEYER,JEFFREY D (HP-Cupertino,ex1); 'Nevil Brownlee'; >> >> > >> >> ipfix@net.doit.wisc.edu >> >> > >> >> Subject: RE: [ipfix] Editors for IPFIX drafts >> >> > >> >> >> >> > >> >> >> >> > >> >> Jeff, >> >> > >> >> >> >> > >> >> I do see a value in such a formal description. But >> there is a >> >> > >> >> trafe-off. >> >> > >> >> >> >> > >> >> I also see a degradation of human readability. You >> >> > loose means of >> >> > >> >> structuring your document text when using XML, for >> >> > example grouping >> >> > >> >> attributes that are related. (Of course you could do so in >> >> > >> >> XML, but it is >> >> > >> >> much harder to find the heading "Port-related >> Attributes" in >> >> > >> >> an XML document, >> >> > >> >> because headings cannot be well highlighted and >> numbering of >> >> > >> >> headings is >> >> > >> >> not well supported. >> >> > >> >> >> >> > >> >> Anyway, I like the idea of an XML encoding of the >> model, but >> >> > >> >> I we should >> >> > >> >> not go this way if we cannot find a good way of getting it >> >> > >> >> hunam readable >> >> > >> >> (maybe by applying formatting rules). >> >> > >> >> >> >> > >> >> An option would also be adding the XML code to the document >> >> > >> >> as an appendix. >> >> > >> >> >> >> > >> >> Juergen >> >> > >> >> >> >> > >> >> >> >> > >> >> -- MEYER,JEFFREY D (HP-Cupertino,ex1) wrote on 28 >> April 2003 >> >> > >> >> 10:57 -0400: >> >> > >> >> >> >> > >> >> > Hi, >> >> > >> >> > >> >> > >> >> > My particular interest on working on the >> >> > information model would >> >> > >> >> > be on developing a more formal and standard way of >> >> > describing the >> >> > >> >> > information elements which are used in IPFIX >> communication. >> >> > >> >> > >> >> > >> >> > The modelling of information elements should >> >> > ideally be done >> >> > >> >> > using a formal description language, so that tools can be >> >> > >> >> constructed >> >> > >> >> > which do not require humans to transcribe this >> information. >> >> > >> >> > >> >> > >> >> > I would propose the use of XML-Schema to provide >> >> > this formal >> >> > >> >> > description. Through the use of XSL processors, prose >> >> > >> >> based descriptions >> >> > >> >> > and tables can be produced if these are >> considered desireable. >> >> > >> >> > >> >> > >> >> > NOTE: describing the information model formally >> >> > does NOT mean >> >> > >> >> > that encoding and transport for IPFIX need to change >> >> > in any way. It >> >> > >> >> > simply means that rather than having as the base >> >> > >> >> description of information >> >> > >> >> > items some prose in a document, you instead have a >> >> > machine readable >> >> > >> >> > representation. >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > As an example of a formal (but currently >> >> > incomplete) information >> >> > >> >> > model for IPFIX, see: >> >> > >> >> > >> http://www.ipdr.org/documents/ipfix/ipfixService-20020902.xml >> >> > >> >> > >> >> > >> >> > I would recommend making such a representation the >> >> > normative form >> >> > >> >> > of the IPFIX information model, much like SNMP MIBs. >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > Please let me know if people see value in this >> exercise. >> >> > >> >> > >> >> > >> >> > Regards, >> >> > >> >> > >> >> > >> >> > Jeff Meyer >> >> > >> >> > >> >> > >> >> >> -----Original Message----- >> >> > >> >> >> From: Nevil Brownlee [mailto:n.brownlee@auckland.ac.nz] >> >> > >> >> >> Sent: Sunday, April 27, 2003 8:44 PM >> >> > >> >> >> To: ipfix@net.doit.wisc.edu >> >> > >> >> >> Subject: [ipfix] Editors for IPFIX drafts >> >> > >> >> >> >> >> > >> >> >> >> >> > >> >> >> >> >> > >> >> >> Hello all: >> >> > >> >> >> >> >> > >> >> >> Following the IPFIX meeting at the San Francisco >> >> > IETF we asked for >> >> > >> >> >> volunteers to edit the IPFIX drafts. The >> current list is: >> >> > >> >> >> >> >> > >> >> >> Applicability: Tanja Zseby, Reinaldo Penno >> >> > >> >> >> >> >> > >> >> >> Architecture: Ganesh Sadasivan, Nevil Brownlee >> >> > >> >> >> >> >> > >> >> >> Information Model: Paul Calato, Jeff Meyer, >> Juergen Quittek >> >> > >> >> >> >> >> > >> >> >> Protocol: Mark Fulmer, Paul Calato, Reinaldo Penno >> >> > >> >> >> >> >> > >> >> >> At this stage we already have old (expired) >> versions of the >> >> > >> >> >> Architecture (-02.txt) and Data Model (-01.txt) >> >> > drafts on the IPFIX >> >> > >> >> >> web page; would the editors of all four drafts >> please email >> >> > >> >> >> me an initial >> >> > >> >> >> version of their new drafts to published as placeholders >> >> > >> >> >> within the next >> >> > >> >> >> few weeks. >> >> > >> >> >> >> >> > >> >> >> After that, the editors will work on developing better >> >> > >> >> versions of the >> >> > >> >> >> drafts which can be discussed on the IPFIX list >> and submitted >> >> > >> >> >> by mid-June, >> >> > >> >> >> i.e. in plenty of time for the Vienna IETF >> meeting in July. >> >> > >> >> >> >> >> > >> >> >> Cheers, Nevil >> >> > >> >> >> >> >> > >> >> >> PS: Good to see some real discussion on the list. Lets >> >> > >> >> concentrate on >> >> > >> >> >> producing text the draft editors can use! >> >> > >> >> >> >> >> > >> >> >> >> >> > -------------------------------------------------------------- >> >> > >> >> >> --------- >> >> > >> >> >> Nevil Brownlee Director, Technology >> >> > >> >> Development >> >> > >> >> >> Phone: +64 9 373 7599 x88941 ITSS, The University >> >> > >> >> of Auckland >> >> > >> >> >> FAX: +64 9 373 7021 Private Bag 92019, Auckland, >> >> > >> >> New Zealand >> >> > >> >> >> >> >> > >> >> >> >> >> > >> >> >> ------------------------------------------------- >> >> > >> >> >> This mail sent through University of Auckland >> >> > >> >> >> http://www.auckland.ac.nz/ >> >> > >> >> >> >> >> > >> >> >> -- >> >> > >> >> >> Help mailto:majordomo@net.doit.wisc.edu >> and say "help" >> >> > >> >> >> in message body >> >> > >> >> >> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >> >> > >> >> >> "unsubscribe ipfix" in message body >> >> > >> >> >> Archive http://ipfix.doit.wisc.edu/archive/ >> >> > >> >> >> >> >> > >> >> > >> >> > >> >> > -- >> >> > >> >> > Help mailto:majordomo@net.doit.wisc.edu and say >> >> > >> >> "help" in message body >> >> > >> >> > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >> >> > >> >> > "unsubscribe ipfix" in message body >> >> > >> >> > Archive http://ipfix.doit.wisc.edu/archive/ >> >> > >> >> >> >> > >> >> >> >> > >> >> >> > >> -- >> >> > >> Help mailto:majordomo@net.doit.wisc.edu and say >> >> > "help" in message body >> >> > >> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >> >> > >> "unsubscribe ipfix" in message body >> >> > >> Archive http://ipfix.doit.wisc.edu/archive/ >> >> > >> >> > >> >> -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu May 22 11:13:25 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA18061 for ; Thu, 22 May 2003 11:13:25 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19IrWk-0001Io-00 for ipfix-list@mil.doit.wisc.edu; Thu, 22 May 2003 09:58:42 -0500 Received: from tokyo.ccrle.nec.de ([195.37.70.2]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19IrWi-0001Ih-00 for ipfix@net.doit.wisc.edu; Thu, 22 May 2003 09:58:41 -0500 Received: from venus.office (venus.office [10.1.1.11]) by tokyo.ccrle.nec.de (8.12.9/8.12.8) with ESMTP id h4MEweVI014233 for ; Thu, 22 May 2003 16:58:40 +0200 (CEST) Received: from [10.1.1.128] (n-quittek.office [10.1.1.128]) by venus.office (Postfix on SuSE Linux eMail Server 3.0) with ESMTP id 8391F2E762 for ; Thu, 22 May 2003 16:48:41 +0200 (CEST) Date: Thu, 22 May 2003 17:00:34 +0200 From: Juergen Quittek To: ipfix@net.doit.wisc.edu Subject: text on reliability (was: Re: [ipfix] IPFIX requirements - final changes) Message-ID: <25993696.1053622834@[10.1.1.128]> In-Reply-To: <17788047.1053614629@[10.1.1.128]> References: <17788047.1053614629@[10.1.1.128]> X-Mailer: Mulberry/2.1.2 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Dear all, I the message I posted earlier today there is an open issue that does not need further discussion (at least I hope so) but some concrete editing. It concerns a statement on reliability. Since we had so many controversal discussions on this issue, I ask on the list for suggestions how to phrase this well. If there is no response within a few days, I will write something by myself. Please find below a description of this issue raised by the IESG reviewers. Juergen -- Juergen Quittek wrote on 22 May 2003 14:43 +0200: > ======================================================================== > 11. Address the reliability issue of usage based accounting > ======================================================================== > Problem Description: > ------------------------------------------------------------------------ > The applications include usage-based accounting. They should cite RFC 2975, > and indicate that the particular niche for usage-based accounting would not > be met if reliability was not very high. Section 5.1 reliability requirements > do not meet the needs, nor do the timestamping of first and last packet of > a flow, because the flow may have been mischaracterized and that is first and > last of different flows. The way to satisfy this issue is to caveat the > discussion of usage-based accounting with strong words on reliability that > counter the comments on lesser reliability and more probabilistic techniques > for other applications of ipfix. > ======================================================================== > Suggested solution: > ------------------------------------------------------------------------ > We need to state clearly, that many accounting applications require > higher reliability than what is provided by the MANDATORY requirements. > Where vwould be a good place to state this? > Already in section 3.1? > In section 6.3.2? > ======================================================================== > Status: solution required > ======================================================================== -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu May 22 22:07:00 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA08452 for ; Thu, 22 May 2003 22:06:59 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19J1Qr-000650-00 for ipfix-list@mil.doit.wisc.edu; Thu, 22 May 2003 20:33:17 -0500 Received: from ds20-1.cc.swin.edu.au ([136.186.1.150] helo=swin.edu.au) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19J1Qo-00064r-00 for ipfix@net.doit.wisc.edu; Thu, 22 May 2003 20:33:14 -0500 Received: from fokus.fraunhofer.de (szander-laptop.caia.swin.edu.au [136.186.229.90]) by swin.edu.au (8.9.3p2/8.9.3) with ESMTP id LAA1860163; Fri, 23 May 2003 11:33:03 +1000 (EST) Message-ID: <3ECD78DB.7020308@fokus.fraunhofer.de> Date: Fri, 23 May 2003 03:26:51 +0200 From: Sebastian Zander Organization: Fraunhofer FOKUS User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.2) Gecko/20030208 Netscape/7.02 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Juergen Quittek CC: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] IPFIX requirements - final changes References: <17788047.1053614629@[10.1.1.128]> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hi Juergen, comments below Juergen Quittek wrote: > Dear all, > > Addressing the comments received from IESG review, we are working > on a new - and hopefully final version of the requirements document. > > For reference, you will find a complete list of issues and > their status at the end of this message. > > Several comments could be addressed by editorial changes and > clarifications. They are already committed. But five comments > suggest changes on which I want to have WG consensus before > committing them. Please state if you see a problem with any > of the suggested changes listed below (or if you have a > better alternative). > > - More concrete definition of byte counter > old text: > "8. byte counter > Which bytes of a packet are counted MUST be > defined exactly." > suggested replacement: > "8. byte counter > The sum of the total length of all IP packets > belonging to the flow. The total length of a > packet covers IP header and IP payload." I just miss the word "bytes" in the definition: The total length of an IP packet is the number of bytes in the IP header and IP payload. Cheers, Sebastian > - More concrete definition of multicast replication factor > old text: > "20. multicast replication factor > the number of outgoing packets originating from > a single incoming multicast packet. This is a > dynamic property of multicast flows, that may > change over time. For unicast flows it has the > constant value 1. The computation of the factor > MUST be clearly defined." > suggestion: Replace last sentence by > "The reported value MUST be the value of the > factor at the time the flow record is exported." > > - More concrete definition of BGP-related attributes > old text: > "In addition, the exporting process MAY be able to > report attributes related to inter-autonomous > system routing of a flow, for example by reporting > BGP Autonomous System numbers [RFC1771]." > suggestion: replace text by three new MAY attributes > "27. source BGP Autonomous System number [RFC1771] > 28. destination BGP Autonomous System number > 29. next hop BGP Autonomous System number" > > - Confidentiality from SHOULD to MUST > suggestion: Change confidentiality requirement in > section 6.3.3 from SCHOULD to MUST. > > - Anonymization from MAY to MUST > suggestion: Change anonymization requirement in > section 6.7 from MAY to MUST. > > Cheers, > > Juergen > > > > Here is the complete list of changes: > > ============================== > IPFIX Requirements Open Issues > ============================== > > > > ======================================================================== > 1. timestamp resolution reference > ======================================================================== > Problem Description: > ------------------------------------------------------------------------ > 5.4. Timestamps > > The metering process MUST be able to generate timestamps for the > first and the last observation of a packet of a flow at the > observation point. The timestamp resolution MUST be at least the one > of the sysUpTime [RFC1213], which is one centisecond. > > I think I'd rather see a reference to RFC3418 instead of RFC1213. > We can do so by RFC-Editor note. May want to check with authors too. > ======================================================================== > Suggested solution: > ------------------------------------------------------------------------ > replace ref to RFC1213 by ref to RFC3418 > ======================================================================== > Status: solution committed > ======================================================================== > > > > ======================================================================== > 2. reporting TOS/traffic class octet or reporting DSCP > ======================================================================== > Problem Description: > ------------------------------------------------------------------------ > On page 14 I see: > > 9. type of service octet (in case of IPv4), traffic class > octet (in case of IPv6) > > Is that not better state as > > 9. DSCP, which is set in the type of service octet (in case of IPv4), > or in the traffic class octet (in case of IPv6) > ======================================================================== > Suggested solution: > ------------------------------------------------------------------------ > According to RFC 2474, the DSCP covers only 6 bit of the type of > service octet, and in some cases you want to see the entire byte. > This is be pointed out by a comment added to item 9.: > "According to RFC 2474 these octets include the DiffServ Code Point > that has a length of 6 bit." > ======================================================================== > Status: solution committed > ======================================================================== > > > > ======================================================================== > 3. location of reference to RFC 2119 > ======================================================================== > Problem Description: > ------------------------------------------------------------------------ > I would prefer to see the paragraph referencing RFC 2119 in section 2. > ======================================================================== > Suggested solution: > ------------------------------------------------------------------------ > Move reference to RFC 2119 to section 2 > ======================================================================== > Status: solution committed > ======================================================================== > > > > ======================================================================== > 4. bad wording in section 6.3.3, last paragraph > ======================================================================== > Problem Description: > ------------------------------------------------------------------------ > Last paragraph in section 6.3.3 needs to be reworded. > ======================================================================== > Suggested solution: > ------------------------------------------------------------------------ > replace > "The security threats from which these 3 requirements are deducted > are explained in the "Security Considerations" (Section 10)." > by > "The security requirements have been derived from an analysis of > potential security threads. The analysis is summarized in > Section 10." > ======================================================================== > Status: solution committed > ======================================================================== > > > > ======================================================================== > 5. describe potential problem of faked DoS attack > ======================================================================== > Problem Description: > ------------------------------------------------------------------------ > In section 10.2, an important point is left unsaid. If the injection of > IPFIX traffic fool the network provided into thinking that a DOS attack is > underway, the countermeasures employed by the network provider may actually > deny service to real customers. > ======================================================================== > Suggested solution: > ------------------------------------------------------------------------ > no suggestion for a wording, yet > ======================================================================== > Status: solution required > ======================================================================== > > > > ======================================================================== > 6. location of appendix > ======================================================================== > Problem Description: > ------------------------------------------------------------------------ > The appendix needs to be moved up in the document. > ======================================================================== > Suggested solution: > ------------------------------------------------------------------------ > move appendix above references, copyright, and authors' addresses > ======================================================================== > Status: solution to be committed > ======================================================================== > > > > ======================================================================== > 7. consistency of appendix > ======================================================================== > Problem Description: > ------------------------------------------------------------------------ > The appendix is outdated. Alignment with the text sections and > coverage of all requirements need to be checked and fixed. > ======================================================================== > Suggested solution: > ------------------------------------------------------------------------ > no suggestion, yet > ======================================================================== > Status: solution required > ======================================================================== > > > > ======================================================================== > 8. More concrete definition of byte counter > ======================================================================== > Problem Description: > ------------------------------------------------------------------------ > This document goes to a lot of trouble to say exactly what is > exported (e.g. what fields, etc), except for two cases: > > 8. byte counter > Which bytes of a packet are counted MUST be defined exactly. > > This document is defining what fields are being reported on; > why can't it also define these two things that must also be > defined? Is it useful to have different ipfix protocols exporting > different measurements? > ======================================================================== > Suggested solution: > ------------------------------------------------------------------------ > replace "Which bytes of a packet are counted MUST be defined > exactly." by "The sum of the total length of all IP packets > belonging to the flow. The total length of a packet covers > IP header and IP payload." > ======================================================================== > Status: solution to be agreed on > ======================================================================== > > > > ======================================================================== > 9. More concrete definition of multicast replication factor > ======================================================================== > Problem Description: > ------------------------------------------------------------------------ > This document goes to a lot of trouble to say exactly what is > exported (e.g. what fields, etc), except for two cases: > > 20. multicast replication factor > the number of outgoing packets originating from a single > incoming multicast packet. This is a dynamic property of > multicast flows, that may change over time. For unicast flows > it has the constant value 1. The computation of the factor MUST > be clearly defined. > > This document is defining what fields are being reported on; > why can't it also define these two things that must also be > defined? Is it useful to have different ipfix protocols exporting > different measurements? > ======================================================================== > Suggested solution: > ------------------------------------------------------------------------ > replace last sentence by "The reported value MUST be the value of the > factor at the time the flow record is exported." > ======================================================================== > Status: solution to be agreed on > ======================================================================== > > > > ======================================================================== > 10. More concrete definition of BGP-related attributes > ======================================================================== > Problem Description: > ------------------------------------------------------------------------ > Also, at the end of 6.1 it kind of says "Plus other stuff related to > inter-AS > routing if you want". It's already in the MAY section; can't they just > list some concrete items related to inter-AS routing as 27, 28, ...? > I don't think it's useful to say what amounts to "plus anything else that > you might feel like adding that you think is relevant", since that doesn't > encourage different people to make the same choices. > ======================================================================== > Suggested solution: > ------------------------------------------------------------------------ > add three more MAY attributes: > - 27. source BGP Autonomous System number [RFC1771] > - 28. destination BGP Autonomous System number > - 29. next hop BGP Autonomous System number > ======================================================================== > Status: solution to be agreed on > ======================================================================== > > > > ======================================================================== > 11. Address the reliability issue of usage based accounting > ======================================================================== > Problem Description: > ------------------------------------------------------------------------ > The applications include usage-based accounting. They should cite RFC > 2975, > and indicate that the particular niche for usage-based accounting would not > be met if reliability was not very high. Section 5.1 reliability > requirements > do not meet the needs, nor do the timestamping of first and last packet of > a flow, because the flow may have been mischaracterized and that is > first and > last of different flows. The way to satisfy this issue is to caveat the > discussion of usage-based accounting with strong words on reliability that > counter the comments on lesser reliability and more probabilistic > techniques > for other applications of ipfix. > ======================================================================== > Suggested solution: > ------------------------------------------------------------------------ > We need to state clearly, that many accounting applications require > higher reliability than what is provided by the MANDATORY requirements. > Where vwould be a good place to state this? > Already in section 3.1? > In section 6.3.2? > ======================================================================== > Status: solution required > ======================================================================== > > > > ======================================================================== > 12. Confidentiality from SHOULD to MUST > ======================================================================== > Problem Description: > ------------------------------------------------------------------------ > Requirements on the ipfix implementation - the document is long and I > wonder if > the working group really meant the protocol's confidentiality and > anonymization > features to be so optional - SHOULD confidentiality, MAY anonymization. > Just for implementation. > ======================================================================== > Suggested solution: > ------------------------------------------------------------------------ > change confidentiality requirement in section 6.3.3 > from SCHOULD to MUST > ======================================================================== > Status: solution to be agreed on > ======================================================================== > > > > ======================================================================== > 13. Anonymization from MAY to MUST > ======================================================================== > Problem Description: > ------------------------------------------------------------------------ > Requirements on the ipfix implementation - the document is long and I > wonder if > the working group really meant the protocol's confidentiality and > anonymization > features to be so optional - SHOULD confidentiality, MAY anonymization. > Just for implementation. > ======================================================================== > Suggested solution: > ------------------------------------------------------------------------ > change anonymization requirement in section 6.7 from MAY to MUST > ======================================================================== > Status: solution to be agreed on > ======================================================================== > > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message > body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > -- Sebastian Zander E-mail: zander@fokus.fraunhofer.de Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 D-10589 Berlin, Germany www.fokus.fraunhofer.de/usr/sebastian.zander -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri May 23 05:16:40 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id FAA29449 for ; Fri, 23 May 2003 05:16:39 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19J8V5-0007mo-00 for ipfix-list@mil.doit.wisc.edu; Fri, 23 May 2003 04:06:07 -0500 Received: from tokyo.ccrle.nec.de ([195.37.70.2]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19J8V3-0007mh-00 for ipfix@net.doit.wisc.edu; Fri, 23 May 2003 04:06:05 -0500 Received: from venus.office (venus.office [10.1.1.11]) by tokyo.ccrle.nec.de (8.12.9/8.12.8) with ESMTP id h4N95vVI043637; Fri, 23 May 2003 11:05:57 +0200 (CEST) Received: from [10.1.1.128] (n-quittek.office [10.1.1.128]) by venus.office (Postfix on SuSE Linux eMail Server 3.0) with ESMTP id 426929FC4A; Fri, 23 May 2003 10:55:51 +0200 (CEST) Date: Fri, 23 May 2003 11:07:51 +0200 From: Juergen Quittek To: Sebastian Zander Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] IPFIX requirements - final changes Message-ID: <6557569.1053688071@[10.1.1.128]> In-Reply-To: <3ECD78DB.7020308@fokus.fraunhofer.de> References: <3ECD78DB.7020308@fokus.fraunhofer.de> X-Mailer: Mulberry/2.1.2 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Sebastian, -- Sebastian Zander wrote on 23 May 2003 03:26 +0200: >> - More concrete definition of byte counter >> old text: >> "8. byte counter >> Which bytes of a packet are counted MUST be >> defined exactly." >> suggested replacement: >> "8. byte counter >> The sum of the total length of all IP packets >> belonging to the flow. The total length of a >> packet covers IP header and IP payload." > > I just miss the word "bytes" in the definition: > The total length of an IP packet is the number of bytes in the > IP header and IP payload. You are right! What about "8. byte counter The sum of the total length in bytes of all IP packets belonging to the flow. The total length of a packet covers IP header and IP payload." Juergen -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri May 23 05:37:34 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id FAA29841 for ; Fri, 23 May 2003 05:37:33 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19J8rk-0000Zp-00 for ipfix-list@mil.doit.wisc.edu; Fri, 23 May 2003 04:29:32 -0500 Received: from tokyo.ccrle.nec.de ([195.37.70.2]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19J8ri-0000Zj-00 for ipfix@net.doit.wisc.edu; Fri, 23 May 2003 04:29:30 -0500 Received: from venus.office (venus.office [10.1.1.11]) by tokyo.ccrle.nec.de (8.12.9/8.12.8) with ESMTP id h4N9TTVI045100; Fri, 23 May 2003 11:29:29 +0200 (CEST) Received: from ccrle.nec.de (tartarelli.office [10.1.1.107]) by venus.office (Postfix on SuSE Linux eMail Server 3.0) with ESMTP id DF851582DB; Fri, 23 May 2003 11:19:22 +0200 (CEST) Message-ID: <3ECDE9FA.8050705@ccrle.nec.de> Date: Fri, 23 May 2003 11:29:30 +0200 From: Sandra Tartarelli User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; de-DE; rv:1.0.1) Gecko/20020823 Netscape/7.0 X-Accept-Language: de-de, de MIME-Version: 1.0 To: ipfix@net.doit.wisc.edu Subject: [ipfix] IPFIX reqs appendix: consistency check Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Dear all, I checked the consistency of the appendix of the IPFIX requirements document. This was listed as open issue in Juergen's email "IPFIX requirement: final changes": > ======================================================================== > 7. consistency of appendix > ======================================================================== > Problem Description: > ------------------------------------------------------------------------ > The appendix is outdated. Alignment with the text sections and > coverage of all requirements need to be checked and fixed. > ======================================================================== > Suggested solution: > ------------------------------------------------------------------------ > no suggestion, yet > ======================================================================== > Status: solution required > ======================================================================== > In the following, you find my suggestions to update it. Part of the suggestions consider the comments in Juergen's email "IPFIX requirement: final changes". Best regards, Sandra ============================================================================== --- 5.2 Sampling > Add note (j): "If sampling is supported, sampling configuration changes MUST be indicated to all collecting processes" --- 5.3 Overload behavior > Add note (k): "If overload is supported and it induces changes in the metering process behavior, the overload behavior MUST be clearly defined" --- Requirement on packet fragmentation state at the meter is missing (section 5.8) > Add "5.8 Packet Fragmentation state O O - - - O" --- 6.1 Byte counter > Add note (l): "which bytes of a packet are counted MUST be defined clearly" This note is not required if the suggestion from Juergen to replace the byte counter definition is accepted. From Juergen's email: > More concrete definition of byte counter > old text: > 8. byte counter > Which bytes of a packet are counted MUST be > defined exactly." > suggested replacement: > 8. byte counter: > The sum of the total length of all IP packets > belonging to the flow. The total length of a > packet covers IP header and IP payload. --- 6.1 Dropped packet counter: This requirement appears as a MUST in the appendix, but it is a MAY in current section 6.1 > Replace with: "6.1 Dropped packet counter (h,i) O O O O O O" --- 6.1 ToS byte and DSCP: they appear as separate requirements in the appendix. Besides, Traffic Class (IPv6) is not mentioned. However, Juergen wrote: > On page 14 I see: > > 9. type of service octet (in case of IPv4), traffic class > octet (in case of IPv6) > > Is that not better state as > > 9. DSCP, which is set in the type of service octet (in case of > IPv4), > or in the traffic class octet (in case of IPv6) > If this is accepted, then remove line "6.1 ToS Byte M S M O M M" and leave only "6.1 DSCP (a) M S M O M M" --- 6.1 BGP AS# is given as MUST in the appendix, but it is a MAY in current section 6.1 If the following suggestion from Juergen is accepted: > - More concrete definition of BGP-related attributes > old text: > In addition, the exporting process MAY be able to > report attributes related to inter-autonomous > system routing of a flow, for example by reporting > BGP Autonomous System numbers [RFC1771]. > suggestion: replace text by three new MAY attributes > 27. source BGP Autonomous System number [RFC1771] > 28. destination BGP Autonomous System number > 29. next hop BGP Autonomous System number > then replace "6.1 BGP AS# - S M - - M" with the following line: "6.1 src/dst/next hop BGP AS # - O O - - O" --- For the Information Model, requirements on ICMP type and in/out interface are missing: > Add "6.1 ICMP type and code (l) S S - S S S" > Add note: "(l) If protocol type is ICMP" > Add "6.1 input/output interface (m) S S S S S S" > Add note: "(m) This requirement does not apply if the observation point is located at a probe device" --- For the Data Model the following requirement is missing: > "6.2 Independency of the transport protocol S S S S S S" --- "6.7 Anonymization O O O O O O" > Add note: "(n) anonymization MUST be clearly indicated to all receiving collecting processes" Juergen suggested: > - Anonymization from MAY to MUST > suggestion: Change anonymization requirement in > section 6.7 from MAY to MUST. >If Juergen's suggestion is accepted then also change the req. into: "6.7 Anonymization M M M M M M" --- Configuration requirements for metering process: > Add "7.1 Config Overload behavior (a) O O O O O O" --- Configuration reqs. for exporting process: > Config Notifications and Config Anonymization should be changed to SHOULD (they are listed as MAY) >Add the following missing reqs: >"Config Collecting Processes S S S S S S" >"Config reporting interval (a) S S S S S S" -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri May 23 11:16:36 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA09441 for ; Fri, 23 May 2003 11:16:36 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19JDxh-0007cF-00 for ipfix-list@mil.doit.wisc.edu; Fri, 23 May 2003 09:56:01 -0500 Received: from host60-112.pool8173.interbusiness.it ([81.73.112.60] helo=localhost.localdomain) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19JDxe-0007c7-00 for ipfix@net.doit.wisc.edu; Fri, 23 May 2003 09:55:59 -0500 Received: from ntop.org (athlon [127.0.0.1]) by localhost.localdomain (8.12.8/8.12.8) with ESMTP id h4NEtuLl011559 for ; Fri, 23 May 2003 16:55:57 +0200 Message-ID: <3ECE367C.30608@ntop.org> Date: Fri, 23 May 2003 16:55:56 +0200 From: Luca Deri Organization: ntop.org User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2.1) Gecko/20030225 X-Accept-Language: en-us, en MIME-Version: 1.0 To: ipfix@net.doit.wisc.edu Subject: [ipfix] nFlow Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Dear all, I have decided to define a new flow format named nFlow (see http://www.nflow.org/) that extends the NetFlow format adding new features such as: - support of both IPv4 and IPv6. - compressed (RFC 1950, 1952) flow format for reducing network traffic. - flow authentication, protection against forging, and non repudiation (RFC 1321). - flow payload information. - accurate ICMP access. - packet sampling for easing probe scalability. - performance (network and application) information. - MPLS and VLAN tag info. I have defined it mostly because NetFlow is not rich enough for ntop (a GPL passive network monitoring tool). Currently I have a GPL software Netflow+nFlow probe (http://www.ntop.org/nProbe.html) and a collector available. I plan to make ntop nFlow-aware in the near future. I would like to know whether: - we could work together so that IPFIX and nFlow could merge (I am open to change it) - I could show up at the next IETF meeting in Vienna and present nFlow Your feedback is appreciated. Regards, Luca -- Luca Deri http://luca.ntop.org/ Hacker: someone who loves to program and enjoys being clever about it - Richard Stallman -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Sun May 25 12:35:53 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA05678 for ; Sun, 25 May 2003 12:35:53 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19Jy6z-0002yl-00 for ipfix-list@mil.doit.wisc.edu; Sun, 25 May 2003 11:12:41 -0500 Received: from dhcp1.se.kurtis.pp.se ([195.43.225.70] helo=laptop2.kurtis.autonomica.se) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19Jy6w-0002yd-00 for ipfix@net.doit.wisc.edu; Sun, 25 May 2003 11:12:38 -0500 Received: from kurtis.pp.se (localhost [127.0.0.1]) by laptop2.kurtis.autonomica.se (8.12.9/8.10.2) with ESMTP id h4PGCARp004974; Sun, 25 May 2003 18:12:11 +0200 (CEST) Date: Sun, 25 May 2003 18:12:08 +0200 Subject: Re: [ipfix] Summary up to now Content-Type: text/plain; charset=ISO-8859-1; format=flowed Mime-Version: 1.0 (Apple Message framework v552) Cc: Mark Thibodeau , ipfix@net.doit.wisc.edu To: Nevil Brownlee From: Kurt Erik Lindqvist In-Reply-To: <1053011481.2d22d7efff88b@hotlava.auckland.ac.nz> Message-Id: X-Mailer: Apple Mail (2.552) Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by ietf.org id MAA05678 On torsdag, maj 15, 2003, at 17:11 Europe/Stockholm, Nevil Brownlee wrote: >> >> As for lost template definitions, this should not happen. Since IPFIX >> will be using TCP, the exporter will know whether or not the template >> was received successfully. > > There's been some apparent confusion on the list about transport for > IPFIX. > Consensus from the last IPFIX meeting was that the 1.0 version of IPFIX > could use TCP transport [so as to make it easier for developers to > implement > and test], but that version 1.0 would NOT rely on two-way messages > between > exporter and collector [so that it would be usable in a minimal > environment]. > So Mark's statement above is not a sufficient answer. > Ok, so I am way behind on email and I am not following this list very closely. And I am getting a déja vu feeling on this question... One thing I am struggling (still) to understand is the use of TCP over UDP. I asked this question in San Fransisco and I think the answer was that reliable transport was needed. I am not sure I agree (still) with this. Isnt't it better to loose some data instead of in worst case congestion control coming into play? - kurtis - -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Sun May 25 12:45:09 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA05864 for ; Sun, 25 May 2003 12:45:09 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19JyED-00037G-00 for ipfix-list@mil.doit.wisc.edu; Sun, 25 May 2003 11:20:09 -0500 Received: from dhcp1.se.kurtis.pp.se ([195.43.225.70] helo=laptop2.kurtis.autonomica.se) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19JyEC-000379-00 for ipfix@net.doit.wisc.edu; Sun, 25 May 2003 11:20:08 -0500 Received: from kurtis.pp.se (localhost [127.0.0.1]) by laptop2.kurtis.autonomica.se (8.12.9/8.10.2) with ESMTP id h4PGJqRp004978; Sun, 25 May 2003 18:19:53 +0200 (CEST) Date: Sun, 25 May 2003 18:19:50 +0200 Subject: Re: [ipfix] Summary up to now Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) Cc: David Moore , Nevil Brownlee , Mark Thibodeau , ipfix@net.doit.wisc.edu To: Peter Ludemann From: Kurt Erik Lindqvist In-Reply-To: <20030515172140.85738.qmail@web80002.mail.yahoo.com> Message-Id: Content-Transfer-Encoding: 7bit X-Mailer: Apple Mail (2.552) Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit > Which is why the various competitors to NetFlow had their own > ACKs. BTW, even if the TCP implementation allowed access to > the TCP-level ACKs, you'd still want IPFIX-protocol level > ACKs because you need *application* level acknowledgment of > receipt and processing. > What if you do this over a bad line, let's say a Ethernet that have full/half duplex misconfigured or other problems so that you start loosing large amounts of data? - kurtis - -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Sun May 25 12:47:31 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA05915 for ; Sun, 25 May 2003 12:47:31 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19JyGI-0003AA-00 for ipfix-list@mil.doit.wisc.edu; Sun, 25 May 2003 11:22:18 -0500 Received: from dhcp1.se.kurtis.pp.se ([195.43.225.70] helo=laptop2.kurtis.autonomica.se) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19JyGG-0003A2-00 for ipfix@net.doit.wisc.edu; Sun, 25 May 2003 11:22:16 -0500 Received: from kurtis.pp.se (localhost [127.0.0.1]) by laptop2.kurtis.autonomica.se (8.12.9/8.10.2) with ESMTP id h4PGMARp004981; Sun, 25 May 2003 18:22:10 +0200 (CEST) Date: Sun, 25 May 2003 18:22:08 +0200 Subject: Re: [ipfix] Summary up to now Content-Type: text/plain; charset=ISO-8859-1; format=flowed Mime-Version: 1.0 (Apple Message framework v552) Cc: Mark Thibodeau , David Moore , Nevil Brownlee , ipfix@net.doit.wisc.edu To: Alex Audu From: Kurt Erik Lindqvist In-Reply-To: <3EC3DC98.69F952ED@alcatel.com> Message-Id: <0823EB5D-8ECD-11D7-B63E-000393A638B2@kurtis.pp.se> X-Mailer: Apple Mail (2.552) Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by ietf.org id MAA05915 congestion aware for the template only? What do you do when congestion happens? - kurtis - On torsdag, maj 15, 2003, at 20:29 Europe/Stockholm, Alex Audu wrote: > Mark, > > IPFIX only recommends that the transport protocol be congestion aware. > So,..even > DCCP (a congestion aware variant of UDP) qualifies in this case as a > viable option. > > Regards, > Alex. > > Mark Thibodeau wrote: > > I agree with both Nevil and David.  The TCP ACK is not enough > information to determine if the Template made it to the Server or not. > > I was originallly thinking of an application where the exporter has > access to the guts of the TCP stack.  If the TCP ACK never came for > the Template, the exporter would resend it until it had positive > acknowledgement. > > But, David is right.  Most applications will probably just use a > standard stack and forget about it.  The information about what was > ACKed is not propagated to the exporter. > > I'm not sure about Nevil's point though.  From the IPFIX meeting > minutes: >   > > The evaluation team has recommended NetFlow v9 as a basis for the IPFIX > protocol.......Then, specify the initial IPFIX transport be TCP..... > > > Nevil's email seems to say that we will accept either TCP or UDP ("to > make it usable in a minimal environment").  What are we saying?  IPFIX > recommends TCP, but if an application wants to still use UDP then > that's OK.  It seems like we haven't made a decision at all on the > transport protocol.  When IPFIX becomes a standard, what will the > transport protocol be?  If an application does not use that transport > protocol, then I assume that it does not comply with the standard. > > Or am I totally out to lunch.  :-() > > Cheers, > Mark >   >   > > David Moore wrote: > > On Fri, May 16, 2003 at 03:11:21AM +1200, Nevil Brownlee wrote: > > Quoting Mark Thibodeau : > > > As for lost template definitions, this should not happen.  Since > IPFIX > > > will be using TCP, the exporter will know whether or not the > template > > > was received successfully. > > > > There's been some apparent confusion on the list about transport for > IPFIX. > > Consensus from the last IPFIX meeting was that the 1.0 version of > IPFIX > > could use TCP transport [so as to make it easier for developers to > implement > > and test], but that version 1.0 would NOT rely on two-way messages > between > > exporter and collector [so that it would be usable in a minimal > environment]. > > So Mark's statement above is not a sufficient answer. > > I'm probably missing something, but how does just using TCP ensure that > the exporter knows that it was received successfully?  (especially > in environments where you can't reach into the guts of TCP). > > If the process hands a large buffer to the kernel/OS, it will take > it and let the process continue.  Some or all of that data may be > successfully transmitted via TCP to the receiver.  If the connection > fails, the kernel/OS knows how much of it made it (to the other > kernel, not necessarily application), but it doesn't tell that > information to the process.  All that the process knows is that it > is later told that the connection is gone.  How does it determine > from this whether it was successful or unsuccessful with the > transfer?  Particularly in the case when it is writing additional > data after the template being queued up in the kernel/OS. > > -- david > > -- > Help        mailto:majordomo@net.doit.wisc.edu and say "help" in > message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive     http://ipfix.doit.wisc.edu/archive/ > > -- > > **************************************************** > * Mark Thibodeau, P Eng. > * 670 RSP Development - Firmware Designer > * Alcatel CID - (613) 784-5375 > * Fax - (613) 599-3696 > * mark.thibodeau@alcatel.com > **************************************************** >   > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Sun May 25 13:26:56 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA06462 for ; Sun, 25 May 2003 13:26:56 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19Jys7-00040c-00 for ipfix-list@mil.doit.wisc.edu; Sun, 25 May 2003 12:01:23 -0500 Received: from web80002.mail.yahoo.com ([66.163.168.132]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 19Jys6-00040X-00 for ipfix@net.doit.wisc.edu; Sun, 25 May 2003 12:01:22 -0500 Message-ID: <20030525170121.48124.qmail@web80002.mail.yahoo.com> Received: from [67.124.50.156] by web80002.mail.yahoo.com via HTTP; Sun, 25 May 2003 10:01:21 PDT Date: Sun, 25 May 2003 10:01:21 -0700 (PDT) From: Peter Ludemann Subject: Re: [ipfix] Summary up to now To: Kurt Erik Lindqvist Cc: David Moore , Nevil Brownlee , Mark Thibodeau , ipfix@net.doit.wisc.edu In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Precedence: bulk Sender: majordomo listserver --- Kurt Erik Lindqvist wrote: > > Which is why the various competitors to NetFlow had their > > own ACKs. BTW, even if the TCP implementation allowed > > access to the TCP-level ACKs, you'd still want IPFIX- > > protocol level ACKs because you need *application* level > > acknowledgment of receipt and processing. > > > > What if you do this over a bad line, let's say a Ethernet > that have full/half duplex misconfigured or other problems > so that you start loosing large amounts of data? > > - kurtis - In this case, you'll have a lot of retransmits. Most TCP implementations provide some method of monitoring the retransmits: for example, "netstat -s" on Solaris, /proc/net/* on Linux. [NB: if you were trying to be as reliable as possible with UDP, you'd also have lots of retransmits.] If there is congestion, the application (IPFIX) layer will see its transmit queue growing (this is separate from the kernel buffer for TCP). It can make a decision, for example to break the connection and reestablish it, to fail-over to another receiver, to throw away old records, throw away new records, use a higher level of aggregation, etc. I don't think that these behaviours should be specified in the protocol; they should simply be allowed. And the protocol should be general enough to allow informing the receiver of the behaviour (e.g., a template with summary of non-transmitted data). - peter -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Sun May 25 16:17:15 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA10532 for ; Sun, 25 May 2003 16:17:15 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19K1ja-0007Yc-00 for ipfix-list@mil.doit.wisc.edu; Sun, 25 May 2003 15:04:46 -0500 Received: from dhcp1.se.kurtis.pp.se ([195.43.225.70] helo=laptop2.kurtis.autonomica.se) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19K1jY-0007YU-00 for ipfix@net.doit.wisc.edu; Sun, 25 May 2003 15:04:45 -0500 Received: from kurtis.pp.se (localhost [127.0.0.1]) by laptop2.kurtis.autonomica.se (8.12.9/8.10.2) with ESMTP id h4PK4bRp005175; Sun, 25 May 2003 22:04:37 +0200 (CEST) Date: Sun, 25 May 2003 22:04:35 +0200 Subject: Re: [ipfix] Summary up to now Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) Cc: David Moore , Nevil Brownlee , Mark Thibodeau , ipfix@net.doit.wisc.edu To: Peter Ludemann From: Kurt Erik Lindqvist In-Reply-To: <20030525170121.48124.qmail@web80002.mail.yahoo.com> Message-Id: <1B510A38-8EEC-11D7-B63E-000393A638B2@kurtis.pp.se> Content-Transfer-Encoding: 7bit X-Mailer: Apple Mail (2.552) Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit >>> Which is why the various competitors to NetFlow had their >>> own ACKs. BTW, even if the TCP implementation allowed >>> access to the TCP-level ACKs, you'd still want IPFIX- >>> protocol level ACKs because you need *application* level >>> acknowledgment of receipt and processing. >>> >> >> What if you do this over a bad line, let's say a Ethernet >> that have full/half duplex misconfigured or other problems >> so that you start loosing large amounts of data? >> >> - kurtis - > > In this case, you'll have a lot of retransmits. Most TCP > implementations provide some method of monitoring the > retransmits: for example, "netstat -s" on Solaris, > /proc/net/* on Linux. [NB: if you were trying to be as > reliable as possible with UDP, you'd also have lots of > retransmits.] > > If there is congestion, the application (IPFIX) layer will > see its transmit queue growing (this is separate from the > kernel buffer for TCP). It can make a decision, for example > to break the connection and reestablish it, to fail-over to > another receiver, to throw away old records, throw away new > records, use a higher level of aggregation, etc. > > I don't think that these behaviours should be specified in > the protocol; they should simply be allowed. And the protocol > should be general enough to allow informing the receiver of > the behaviour (e.g., a template with summary of > non-transmitted data). > Well, this is what worries me though. If this is data from high speed interfaces like 10G, you will produce quite a bit of data in worst case. If what we are doing is handling congestion, then we also need to queue the data somewhere. For high speed interfaces, this will be _a lot_ of data. I would think that we for most of this would be better to drop the data but give the receiver/collector a mechanism to detect that it have lost some (sampled) data. Maybe even an indication of how much data. - kurtis - -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Sun May 25 18:53:27 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA14727 for ; Sun, 25 May 2003 18:53:27 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19K4BI-0002lD-00 for ipfix-list@mil.doit.wisc.edu; Sun, 25 May 2003 17:41:32 -0500 Received: from web80006.mail.yahoo.com ([66.163.168.136]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 19K4BG-0002l7-00 for ipfix@net.doit.wisc.edu; Sun, 25 May 2003 17:41:30 -0500 Message-ID: <20030525224129.15061.qmail@web80006.mail.yahoo.com> Received: from [68.120.131.185] by web80006.mail.yahoo.com via HTTP; Sun, 25 May 2003 15:41:29 PDT Date: Sun, 25 May 2003 15:41:29 -0700 (PDT) From: Peter Ludemann Subject: Re: [ipfix] Summary up to now To: Kurt Erik Lindqvist Cc: ipfix@net.doit.wisc.edu In-Reply-To: <1B510A38-8EEC-11D7-B63E-000393A638B2@kurtis.pp.se> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Precedence: bulk Sender: majordomo listserver Let's try an actual situation: 500 byte records 10,000 records/sec (5 Mbs) 10 second time-out to start fail-over 10 seconds to do fail-over Then, the transmit buffer would need to be 100MB to ensure no data loss. For safety and to allow for "catch-up" transmission on fail-over, double that: 200MB. I can buy 256MB of high quality ECC memory for less than US$50, so I don't see any reason for concern. (If you're sending more than 10,000 rec/sec from your device, you should probably reconsider how you're aggregating.) The TCP buffer is irrelevant to these calculations (typically, it's 64K-256K) because we cannot remove from our transmit buffer until we get an ACK from the other end. If we cannot do a fail-over within the time given (e.g., 20 sec), then we will have to throw away information. Whether we throw away the oldest, newest, or random: that's a matter for the implementation. If we want to guard against longer network failures (e.g., 10 minutes), then we must have more memory for our transmit buffer. It doesn't matter what transport we are using ... if we want reliability, we must provide a buffer of this size. A separate set of data (template) can contain information such as time period information, total number of bytes sent, number of bytes acknowledged, etc. This can be used to communicate data loss information. But this is an issue of the data model, not of the transport (except that it should probably be kept in a separate buffer, or sent on a separate channel, to ensure that it isn't lost). - peter --- Kurt Erik Lindqvist wrote: > >>> Which is why the various competitors to NetFlow had > their > >>> own ACKs. BTW, even if the TCP implementation allowed > >>> access to the TCP-level ACKs, you'd still want IPFIX- > >>> protocol level ACKs because you need *application* > level > >>> acknowledgment of receipt and processing. > >>> > >> > >> What if you do this over a bad line, let's say a > Ethernet > >> that have full/half duplex misconfigured or other > problems > >> so that you start loosing large amounts of data? > >> > >> - kurtis - > > > > In this case, you'll have a lot of retransmits. Most TCP > > implementations provide some method of monitoring the > > retransmits: for example, "netstat -s" on Solaris, > > /proc/net/* on Linux. [NB: if you were trying to be as > > reliable as possible with UDP, you'd also have lots of > > retransmits.] > > > > If there is congestion, the application (IPFIX) layer > will > > see its transmit queue growing (this is separate from the > > kernel buffer for TCP). It can make a decision, for > example > > to break the connection and reestablish it, to fail-over > to > > another receiver, to throw away old records, throw away > new > > records, use a higher level of aggregation, etc. > > > > I don't think that these behaviours should be specified > in > > the protocol; they should simply be allowed. And the > protocol > > should be general enough to allow informing the receiver > of > > the behaviour (e.g., a template with summary of > > non-transmitted data). > > > > Well, this is what worries me though. If this is data from > high speed > interfaces like 10G, you will produce quite a bit of data > in worst > case. If what we are doing is handling congestion, then we > also need to > queue the data somewhere. For high speed interfaces, this > will be _a > lot_ of data. > > I would think that we for most of this would be better to > drop the data > but give the receiver/collector a mechanism to detect that > it have lost > some (sampled) data. Maybe even an indication of how much > data. > > - kurtis - > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon May 26 03:06:00 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id DAA04207 for ; Mon, 26 May 2003 03:06:00 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19KBZA-00047n-00 for ipfix-list@mil.doit.wisc.edu; Mon, 26 May 2003 01:34:40 -0500 Received: from laptop2.kurtis.autonomica.se ([192.71.80.74]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19KBZ8-00047f-00 for ipfix@net.doit.wisc.edu; Mon, 26 May 2003 01:34:38 -0500 Received: from kurtis.pp.se (localhost [127.0.0.1]) by laptop2.kurtis.autonomica.se (8.12.9/8.10.2) with ESMTP id h4Q6YfRp005567; Mon, 26 May 2003 08:34:42 +0200 (CEST) Date: Mon, 26 May 2003 08:34:40 +0200 Subject: Re: [ipfix] Summary up to now Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) Cc: ipfix@net.doit.wisc.edu To: Peter Ludemann From: Kurt Erik Lindqvist In-Reply-To: <20030525224129.15061.qmail@web80006.mail.yahoo.com> Message-Id: <20D86C33-8F44-11D7-857A-000393A638B2@kurtis.pp.se> Content-Transfer-Encoding: 7bit X-Mailer: Apple Mail (2.552) Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit > Let's try an actual situation: > > 500 byte records > 10,000 records/sec (5 Mbs) > 10 second time-out to start fail-over > 10 seconds to do fail-over > > Then, the transmit buffer would need to be 100MB to ensure no > data loss. For safety and to allow for "catch-up" > transmission on fail-over, double that: 200MB. I can buy > 256MB of high quality ECC memory for less than US$50, so I > don't see any reason for concern. (If you're sending more > than 10,000 rec/sec from your device, you should probably > reconsider how you're aggregating.) If this is a router, 256MB is a lot, and it will cost you way much more than $50. > If we cannot do a fail-over within the time given (e.g., 20 > sec), then we will have to throw away information. Whether we > throw away the oldest, newest, or random: that's a matter for > the implementation. > > If we want to guard against longer network failures (e.g., 10 > minutes), then we must have more memory for our transmit > buffer. It doesn't matter what transport we are using ... if > we want reliability, we must provide a buffer of this size. What I am questioning is if you really want that level of reliability. At least in all cases. The costs of doing this are quite high. - kurtis - -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon May 26 11:15:07 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA16064 for ; Mon, 26 May 2003 11:15:07 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19KJU8-0000DZ-00 for ipfix-list@mil.doit.wisc.edu; Mon, 26 May 2003 10:02:00 -0500 Received: from weird-brew.cisco.com ([144.254.15.118] helo=strange-brew.cisco.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19KJU7-0000DT-00 for ipfix@net.doit.wisc.edu; Mon, 26 May 2003 10:01:59 -0500 Received: from cisco.com (dhcp-peg3-vl30-144-254-7-236.cisco.com [144.254.7.236]) by strange-brew.cisco.com (8.11.6+Sun/8.8.8) with ESMTP id h4QF1pR06970; Mon, 26 May 2003 17:01:51 +0200 (CEST) Message-ID: <3ED22C5F.9090208@cisco.com> Date: Mon, 26 May 2003 17:01:51 +0200 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.3) Gecko/20030312 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Juergen Quittek CC: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] IPFIX requirements - final changes References: <17788047.1053614629@[10.1.1.128]> In-Reply-To: <17788047.1053614629@[10.1.1.128]> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Juergen, > ======================================================================== > 12. Confidentiality from SHOULD to MUST > ======================================================================== > Problem Description: > ------------------------------------------------------------------------ > Requirements on the ipfix implementation - the document is long and I > wonder if > the working group really meant the protocol's confidentiality and > anonymization > features to be so optional - SHOULD confidentiality, MAY anonymization. > Just for implementation. > ======================================================================== > Suggested solution: > ------------------------------------------------------------------------ > change confidentiality requirement in section 6.3.3 > from SHOULD to MUST > ======================================================================== > Status: solution to be agreed on > ======================================================================== I'm just wondering if a MUST is not a little bit extreme. The draft would become: "Confidentiality of flow specific data transferred from an exporting process to a collecting process MUST be ensured." We all agree that over the Internet confidentiality is compulsory but this sentence would mean that even in a private LAN or environment, we must have confidentiality to be IPFIX compliant. And we know that, with the software based encryption, the throughput would be lowered. Ok, then we should have an crypto engine in hardware on the exporter to export the numerous flow export packets but there is a price to it. My point is that, as Confidentiality is not needed in all cases, why make it a MUST? > > > > ======================================================================== > 13. Anonymization from MAY to MUST > ======================================================================== > Problem Description: > ------------------------------------------------------------------------ > Requirements on the ipfix implementation - the document is long and I > wonder if > the working group really meant the protocol's confidentiality and > anonymization > features to be so optional - SHOULD confidentiality, MAY anonymization. > Just for implementation. > ======================================================================== > Suggested solution: > ------------------------------------------------------------------------ > change anonymization requirement in section 6.7 from MAY to MUST > ======================================================================== > Status: solution to be agreed on > ======================================================================== Again, I have the same type of comments. Anonymization is not required in all cases. So why make it a MUST? Regards, Benoit. -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon May 26 12:42:06 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA18073 for ; Mon, 26 May 2003 12:42:05 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19KKes-0001eY-00 for ipfix-list@mil.doit.wisc.edu; Mon, 26 May 2003 11:17:10 -0500 Received: from zrc2s0jx.nortelnetworks.com ([47.103.122.112]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19KKeq-0001eQ-00 for ipfix@net.doit.wisc.edu; Mon, 26 May 2003 11:17:08 -0500 Received: from zsc3c028.us.nortel.com (zsc3c028.us.nortel.com [47.81.138.28]) by zrc2s0jx.nortelnetworks.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id h4QGGja18851; Mon, 26 May 2003 11:16:45 -0500 (CDT) Received: from zsc3c026.us.nortel.com ([47.81.138.26]) by zsc3c028.us.nortel.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id LGT14361; Mon, 26 May 2003 09:16:46 -0700 Received: from private2xsth6c (artpt5kj.us.nortel.com [47.140.52.4]) by zsc3c026.us.nortel.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id J4HALV63; Mon, 26 May 2003 09:16:44 -0700 Message-ID: <002a01c323a2$3a808080$04348c2f@private2xsth6c> X-Sybari-Space: 00000000 00000000 00000000 From: "Reinaldo Penno" To: "Benoit Claise" , "Juergen Quittek" Cc: References: <17788047.1053614629@[10.1.1.128]> <3ED22C5F.9090208@cisco.com> Subject: Re: [ipfix] IPFIX requirements - final changes Date: Mon, 26 May 2003 12:16:55 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1106 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit I agree that in a LAN confidentiality might not be an issue, although anonymization might be. Anyway, I agree with the MUST but we should append a "be able to" after it. So a "IPfix Exporter MUST be able to provide confidentiality and anonymization". This makes clear that will not be used all the time but it's there if somebody needs it (beside being interoperable, etc, etc). regards, Reinaldo ----- Original Message ----- From: "Benoit Claise" To: "Juergen Quittek" Cc: Sent: Monday, May 26, 2003 11:01 AM Subject: Re: [ipfix] IPFIX requirements - final changes > Juergen, > > > ======================================================================== > > 12. Confidentiality from SHOULD to MUST > > ======================================================================== > > Problem Description: > > ------------------------------------------------------------------------ > > Requirements on the ipfix implementation - the document is long and I > > wonder if > > the working group really meant the protocol's confidentiality and > > anonymization > > features to be so optional - SHOULD confidentiality, MAY anonymization. > > Just for implementation. > > ======================================================================== > > Suggested solution: > > ------------------------------------------------------------------------ > > change confidentiality requirement in section 6.3.3 > > from SHOULD to MUST > > ======================================================================== > > Status: solution to be agreed on > > ======================================================================== > > I'm just wondering if a MUST is not a little bit extreme. > The draft would become: > "Confidentiality of flow specific data transferred from an exporting > process to a collecting process MUST be ensured." > We all agree that over the Internet confidentiality is compulsory but > this sentence would mean that even in a private LAN or environment, we > must have confidentiality to be IPFIX compliant. And we know that, with > the software based encryption, the throughput would be lowered. Ok, then > we should have an crypto engine in hardware on the exporter to export > the numerous flow export packets but there is a price to it. > My point is that, as Confidentiality is not needed in all cases, why > make it a MUST? > > > > > > > > > ======================================================================== > > 13. Anonymization from MAY to MUST > > ======================================================================== > > Problem Description: > > ------------------------------------------------------------------------ > > Requirements on the ipfix implementation - the document is long and I > > wonder if > > the working group really meant the protocol's confidentiality and > > anonymization > > features to be so optional - SHOULD confidentiality, MAY anonymization. > > Just for implementation. > > ======================================================================== > > Suggested solution: > > ------------------------------------------------------------------------ > > change anonymization requirement in section 6.7 from MAY to MUST > > ======================================================================== > > Status: solution to be agreed on > > ======================================================================== > > Again, I have the same type of comments. Anonymization is not required > in all cases. So why make it a MUST? > > Regards, Benoit. > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon May 26 13:11:01 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA18440 for ; Mon, 26 May 2003 13:11:01 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19KL93-0002FS-00 for ipfix-list@mil.doit.wisc.edu; Mon, 26 May 2003 11:48:21 -0500 Received: from usmail.xacct.com ([204.253.100.12]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19KL91-0002FJ-00 for ipfix@net.doit.wisc.edu; Mon, 26 May 2003 11:48:19 -0500 Received: from Givoly ([192.168.0.3]) by usmail.xacct.com (8.11.2/8.11.2) with SMTP id h4QGnti09292; Mon, 26 May 2003 09:49:56 -0700 From: "Tal Givoly" To: "Benoit Claise" , "Juergen Quittek" Cc: Subject: RE: [ipfix] IPFIX requirements - final changes Date: Mon, 26 May 2003 09:42:55 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-Reply-To: <3ED22C5F.9090208@cisco.com> Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Juergen, I agree with Benoit - it doesn't make sense to make confidentiality and anonymization mandatory. In many deployments, there could be physical and other security measures that addresses these issues. Tal -----Original Message----- From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu]On Behalf Of Benoit Claise Sent: Monday, May 26, 2003 8:02 AM To: Juergen Quittek Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] IPFIX requirements - final changes Juergen, > ======================================================================== > 12. Confidentiality from SHOULD to MUST > ======================================================================== > Problem Description: > ------------------------------------------------------------------------ > Requirements on the ipfix implementation - the document is long and I > wonder if > the working group really meant the protocol's confidentiality and > anonymization > features to be so optional - SHOULD confidentiality, MAY anonymization. > Just for implementation. > ======================================================================== > Suggested solution: > ------------------------------------------------------------------------ > change confidentiality requirement in section 6.3.3 > from SHOULD to MUST > ======================================================================== > Status: solution to be agreed on > ======================================================================== I'm just wondering if a MUST is not a little bit extreme. The draft would become: "Confidentiality of flow specific data transferred from an exporting process to a collecting process MUST be ensured." We all agree that over the Internet confidentiality is compulsory but this sentence would mean that even in a private LAN or environment, we must have confidentiality to be IPFIX compliant. And we know that, with the software based encryption, the throughput would be lowered. Ok, then we should have an crypto engine in hardware on the exporter to export the numerous flow export packets but there is a price to it. My point is that, as Confidentiality is not needed in all cases, why make it a MUST? > > > > ======================================================================== > 13. Anonymization from MAY to MUST > ======================================================================== > Problem Description: > ------------------------------------------------------------------------ > Requirements on the ipfix implementation - the document is long and I > wonder if > the working group really meant the protocol's confidentiality and > anonymization > features to be so optional - SHOULD confidentiality, MAY anonymization. > Just for implementation. > ======================================================================== > Suggested solution: > ------------------------------------------------------------------------ > change anonymization requirement in section 6.7 from MAY to MUST > ======================================================================== > Status: solution to be agreed on > ======================================================================== Again, I have the same type of comments. Anonymization is not required in all cases. So why make it a MUST? Regards, Benoit. -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon May 26 13:11:56 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA18472 for ; Mon, 26 May 2003 13:11:55 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19KLIK-0002RT-00 for ipfix-list@mil.doit.wisc.edu; Mon, 26 May 2003 11:57:56 -0500 Received: from usmail.xacct.com ([204.253.100.12]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19KLIJ-0002RO-00 for ipfix@net.doit.wisc.edu; Mon, 26 May 2003 11:57:55 -0500 Received: from Givoly ([192.168.0.3]) by usmail.xacct.com (8.11.2/8.11.2) with SMTP id h4QH4Li09421; Mon, 26 May 2003 10:04:23 -0700 From: "Tal Givoly" To: "Kurt Erik Lindqvist" , "Peter Ludemann" Cc: Subject: RE: [ipfix] Summary up to now Date: Mon, 26 May 2003 09:57:21 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-Reply-To: <20D86C33-8F44-11D7-857A-000393A638B2@kurtis.pp.se> Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit I should remind you that records may be compactly encoded in 50-100 bytes. It doesn't fundamentally change the requirement for buffering, however, it changes the requirement, in the example from 256 MB to 12.5-25.6 MB. This is not insignificant, but is definitely more feasible. Kurt, I agree completely that many applications do not justify this level of reliability (and associated costs). On the other hand, billing and accounting in many environments may. For aggregate records, typically not exposed at 10K rec/sec, it is very much desired as each record may describe much more cost. This level of reliability is not a must for the protocol - as was agreed, the protocol must be extensible enough to support extensions that provide this level of reliability. Tal -----Original Message----- From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu]On Behalf Of Kurt Erik Lindqvist Sent: Sunday, May 25, 2003 11:35 PM To: Peter Ludemann Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Summary up to now > Let's try an actual situation: > > 500 byte records > 10,000 records/sec (5 Mbs) > 10 second time-out to start fail-over > 10 seconds to do fail-over > > Then, the transmit buffer would need to be 100MB to ensure no > data loss. For safety and to allow for "catch-up" > transmission on fail-over, double that: 200MB. I can buy > 256MB of high quality ECC memory for less than US$50, so I > don't see any reason for concern. (If you're sending more > than 10,000 rec/sec from your device, you should probably > reconsider how you're aggregating.) If this is a router, 256MB is a lot, and it will cost you way much more than $50. > If we cannot do a fail-over within the time given (e.g., 20 > sec), then we will have to throw away information. Whether we > throw away the oldest, newest, or random: that's a matter for > the implementation. > > If we want to guard against longer network failures (e.g., 10 > minutes), then we must have more memory for our transmit > buffer. It doesn't matter what transport we are using ... if > we want reliability, we must provide a buffer of this size. What I am questioning is if you really want that level of reliability. At least in all cases. The costs of doing this are quite high. - kurtis - -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue May 27 07:07:21 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id HAA07962 for ; Tue, 27 May 2003 07:07:21 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19KbuN-0001Hg-00 for ipfix-list@mil.doit.wisc.edu; Tue, 27 May 2003 05:42:19 -0500 Received: from laptop2.kurtis.autonomica.se ([192.71.80.74]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19KbuL-0001HV-00 for ipfix@net.doit.wisc.edu; Tue, 27 May 2003 05:42:17 -0500 Received: from kurtis.pp.se (localhost [127.0.0.1]) by laptop2.kurtis.autonomica.se (8.12.9/8.10.2) with ESMTP id h4RAgDRp006626; Tue, 27 May 2003 12:42:14 +0200 (CEST) Date: Tue, 27 May 2003 12:42:12 +0200 Subject: Re: [ipfix] Summary up to now Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v552) Cc: "Peter Ludemann" , To: "Tal Givoly" From: Kurt Erik Lindqvist In-Reply-To: Message-Id: Content-Transfer-Encoding: 7bit X-Mailer: Apple Mail (2.552) Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit > I should remind you that records may be compactly encoded in 50-100 > bytes. > It doesn't fundamentally change the requirement for buffering, > however, it > changes the requirement, in the example from 256 MB to 12.5-25.6 MB. > This is > not insignificant, but is definitely more feasible. Way more so. > Kurt, I agree completely that many applications do not justify this > level of > reliability (and associated costs). On the other hand, billing and > accounting in many environments may. For aggregate records, typically > not > exposed at 10K rec/sec, it is very much desired as each record may > describe > much more cost. > > This level of reliability is not a must for the protocol - as was > agreed, > the protocol must be extensible enough to support extensions that > provide > this level of reliability. > I am sure there are situations where this level of reliability is wished for, but let those that want it pay that price. From what I was reading the suggestion was to make TCP the default behavior and I think already there we have a problem. Made optional, fine by me. - kurtis - -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue May 27 11:44:49 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA17513 for ; Tue, 27 May 2003 11:44:49 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19KgQ3-0006xC-00 for ipfix-list@mil.doit.wisc.edu; Tue, 27 May 2003 10:31:19 -0500 Received: from auds951.usa.alcatel.com ([143.209.238.80]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19KgQ2-0006x5-00 for ipfix@net.doit.wisc.edu; Tue, 27 May 2003 10:31:18 -0500 Received: from alcatel.com (localhost [127.0.0.1]) by auds951.usa.alcatel.com (8.12.8p1/8.12.8) with ESMTP id h4RFUYb2014044; Tue, 27 May 2003 10:30:34 -0500 (CDT) Message-ID: <3ED38495.8804C2C1@alcatel.com> Date: Tue, 27 May 2003 10:30:29 -0500 From: Alex Audu Reply-To: alex.audu@alcatel.com X-Mailer: Mozilla 4.79 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Reinaldo Penno CC: Benoit Claise , Juergen Quittek , ipfix@net.doit.wisc.edu Subject: Re: [ipfix] IPFIX requirements - final changes References: <17788047.1053614629@[10.1.1.128]> <3ED22C5F.9090208@cisco.com> <002a01c323a2$3a808080$04348c2f@private2xsth6c> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit I support Reinaldo's comments on this issue. IPFix Exporter MUST support confidentiality and anonymization. It is left for configuration to turn these on as needed. Regards, Alex. Reinaldo Penno wrote: > I agree that in a LAN confidentiality might not be an issue, although > anonymization might be. > > Anyway, I agree with the MUST but we should append a "be able to" after it. > So a "IPfix Exporter MUST be able to provide confidentiality and > anonymization". This makes clear that will not be used all the time but it's > there if somebody needs it (beside being interoperable, etc, etc). > > regards, > > Reinaldo > > ----- Original Message ----- > From: "Benoit Claise" > To: "Juergen Quittek" > Cc: > Sent: Monday, May 26, 2003 11:01 AM > Subject: Re: [ipfix] IPFIX requirements - final changes > > > Juergen, > > > > > ======================================================================== > > > 12. Confidentiality from SHOULD to MUST > > > ======================================================================== > > > Problem Description: > > > ------------------------------------------------------------------------ > > > Requirements on the ipfix implementation - the document is long and I > > > wonder if > > > the working group really meant the protocol's confidentiality and > > > anonymization > > > features to be so optional - SHOULD confidentiality, MAY anonymization. > > > Just for implementation. > > > ======================================================================== > > > Suggested solution: > > > ------------------------------------------------------------------------ > > > change confidentiality requirement in section 6.3.3 > > > from SHOULD to MUST > > > ======================================================================== > > > Status: solution to be agreed on > > > ======================================================================== > > > > I'm just wondering if a MUST is not a little bit extreme. > > The draft would become: > > "Confidentiality of flow specific data transferred from an exporting > > process to a collecting process MUST be ensured." > > We all agree that over the Internet confidentiality is compulsory but > > this sentence would mean that even in a private LAN or environment, we > > must have confidentiality to be IPFIX compliant. And we know that, with > > the software based encryption, the throughput would be lowered. Ok, then > > we should have an crypto engine in hardware on the exporter to export > > the numerous flow export packets but there is a price to it. > > My point is that, as Confidentiality is not needed in all cases, why > > make it a MUST? > > > > > > > > > > > > > > ======================================================================== > > > 13. Anonymization from MAY to MUST > > > ======================================================================== > > > Problem Description: > > > ------------------------------------------------------------------------ > > > Requirements on the ipfix implementation - the document is long and I > > > wonder if > > > the working group really meant the protocol's confidentiality and > > > anonymization > > > features to be so optional - SHOULD confidentiality, MAY anonymization. > > > Just for implementation. > > > ======================================================================== > > > Suggested solution: > > > ------------------------------------------------------------------------ > > > change anonymization requirement in section 6.7 from MAY to MUST > > > ======================================================================== > > > Status: solution to be agreed on > > > ======================================================================== > > > > Again, I have the same type of comments. Anonymization is not required > > in all cases. So why make it a MUST? > > > > Regards, Benoit. > > > > > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message > body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue May 27 14:34:51 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA23351 for ; Tue, 27 May 2003 14:34:51 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19Kj8G-0002Wi-00 for ipfix-list@mil.doit.wisc.edu; Tue, 27 May 2003 13:25:08 -0500 Received: from atlrel9.hp.com ([156.153.255.214]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19Kj8F-0002Wd-00 for ipfix@net.doit.wisc.edu; Tue, 27 May 2003 13:25:07 -0500 Received: from xatlrelay2.atl.hp.com (xatlrelay2.atl.hp.com [15.45.89.191]) by atlrel9.hp.com (Postfix) with ESMTP id 7BDE21C01756; Tue, 27 May 2003 14:25:06 -0400 (EDT) Received: from xatlbh3.atl.hp.com (xatlbh3.atl.hp.com [15.45.89.188]) by xatlrelay2.atl.hp.com (Postfix) with ESMTP id DC2A31C000AF; Tue, 27 May 2003 14:25:05 -0400 (EDT) Received: by xatlbh3.atl.hp.com with Internet Mail Service (5.5.2655.55) id ; Tue, 27 May 2003 14:25:05 -0400 Message-ID: <1D3D2C371FCBD947A7897FABBD3533A566BB56@xsun01.ptp.hp.com> From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" To: "'Tal Givoly'" , Benoit Claise , Juergen Quittek Cc: ipfix@net.doit.wisc.edu Subject: RE: [ipfix] IPFIX requirements - final changes Date: Tue, 27 May 2003 14:25:03 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2655.55) Content-Type: text/plain; charset="iso-8859-1" Precedence: bulk Sender: majordomo listserver Hi, I agree w/ Tal and Benoit on this one. Regards, Jeff Meyer > -----Original Message----- > From: Tal Givoly [mailto:givoly@xacct.com] > Sent: Monday, May 26, 2003 9:43 AM > To: Benoit Claise; Juergen Quittek > Cc: ipfix@net.doit.wisc.edu > Subject: RE: [ipfix] IPFIX requirements - final changes > > > Juergen, > > I agree with Benoit - it doesn't make sense to make > confidentiality and > anonymization mandatory. In many deployments, there could be > physical and > other security measures that addresses these issues. > > Tal > > -----Original Message----- > From: majordomo listserver > [mailto:majordomo@mil.doit.wisc.edu]On Behalf > Of Benoit Claise > Sent: Monday, May 26, 2003 8:02 AM > To: Juergen Quittek > Cc: ipfix@net.doit.wisc.edu > Subject: Re: [ipfix] IPFIX requirements - final changes > > > Juergen, > > > > ============================================================== > ========== > > 12. Confidentiality from SHOULD to MUST > > > ============================================================== > ========== > > Problem Description: > > > -------------------------------------------------------------- > ---------- > > Requirements on the ipfix implementation - the document is > long and I > > wonder if > > the working group really meant the protocol's confidentiality and > > anonymization > > features to be so optional - SHOULD confidentiality, MAY > anonymization. > > Just for implementation. > > > ============================================================== > ========== > > Suggested solution: > > > -------------------------------------------------------------- > ---------- > > change confidentiality requirement in section 6.3.3 > > from SHOULD to MUST > > > ============================================================== > ========== > > Status: solution to be agreed on > > > ============================================================== > ========== > > I'm just wondering if a MUST is not a little bit extreme. > The draft would become: > "Confidentiality of flow specific data transferred from an > exporting > process to a collecting process MUST be ensured." > We all agree that over the Internet confidentiality is compulsory but > this sentence would mean that even in a private LAN or environment, we > must have confidentiality to be IPFIX compliant. And we know > that, with > the software based encryption, the throughput would be > lowered. Ok, then > we should have an crypto engine in hardware on the exporter to export > the numerous flow export packets but there is a price to it. > My point is that, as Confidentiality is not needed in all cases, why > make it a MUST? > > > > > > > > > > ============================================================== > ========== > > 13. Anonymization from MAY to MUST > > > ============================================================== > ========== > > Problem Description: > > > -------------------------------------------------------------- > ---------- > > Requirements on the ipfix implementation - the document is > long and I > > wonder if > > the working group really meant the protocol's confidentiality and > > anonymization > > features to be so optional - SHOULD confidentiality, MAY > anonymization. > > Just for implementation. > > > ============================================================== > ========== > > Suggested solution: > > > -------------------------------------------------------------- > ---------- > > change anonymization requirement in section 6.7 from MAY to MUST > > > ============================================================== > ========== > > Status: solution to be agreed on > > > ============================================================== > ========== > > Again, I have the same type of comments. Anonymization is not required > in all cases. So why make it a MUST? > > Regards, Benoit. > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" > in message > body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" > in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue May 27 15:36:56 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA27707 for ; Tue, 27 May 2003 15:36:56 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19Kk7i-0003l0-00 for ipfix-list@mil.doit.wisc.edu; Tue, 27 May 2003 14:28:38 -0500 Received: from mailhost2.auckland.ac.nz ([130.216.191.4]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19Kk7g-0003kr-00 for ipfix@net.doit.wisc.edu; Tue, 27 May 2003 14:28:36 -0500 Received: from mailhost.auckland.ac.nz (IDENT:mirapoint@mailhost [130.216.191.61]) by mailhost2.auckland.ac.nz (8.12.9/8.12.9/8.12.9-ua) with ESMTP id h4RJSN8W012217; Wed, 28 May 2003 07:28:24 +1200 (NZST) Received: from hotlava.auckland.ac.nz (hotlava.auckland.ac.nz [130.216.191.123]) by mailhost.auckland.ac.nz (Mirapoint Messaging Server MOS 3.3.5-GR) with ESMTP id AQL79623; Wed, 28 May 2003 07:28:22 +1200 (NZST) Received: (from apache@localhost) by hotlava.auckland.ac.nz (8.11.6/8.11.6) id h4RJSMt28335; Wed, 28 May 2003 07:28:22 +1200 X-Authentication-Warning: hotlava.auckland.ac.nz: apache set sender to n.brownlee@auckland.ac.nz using -f Received: from wallaby.caida.org (wallaby.caida.org [192.172.226.111]) by hotlava.auckland.ac.nz (Horde) with HTTP for ; Wed, 28 May 2003 07:28:22 +1200 Message-ID: <1054063702.00bdd37681917@hotlava.auckland.ac.nz> Date: Wed, 28 May 2003 07:28:22 +1200 From: Nevil Brownlee To: "MEYER,JEFFREY D (HP-Cupertino,ex1)" Cc: ipfix@net.doit.wisc.edu Subject: RE: [ipfix] IPFIX requirements - final changes References: <1D3D2C371FCBD947A7897FABBD3533A566BB56@xsun01.ptp.hp.com> In-Reply-To: <1D3D2C371FCBD947A7897FABBD3533A566BB56@xsun01.ptp.hp.com> MIME-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Internet Messaging Program (IMP) 4.0-cvs X-Originating-IP: 192.172.226.111 Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Responding to Jeff, who said: > I agree w/ Tal and Benoit on this one. The IESG comment was about confidentiality and anonymisation in IPFIX being "so optional." Maybe a better way to improve this would be to write a few parargraphs referring to RFC 2975, making it clear that building an accounting system invloves much more than just selecting an export protocol (i.e. other techniques like using a separate network may be sufficient in many cases), which is why the IPFIX requirements only say SHOULD for these. Of course, if a vendor implements them, users can decide whether or not to actually use them! If others agree with this, I could write some text in a day or two ... Juergen's other proposed changes to the draft look OK to me. Cheers, Nevil ----------------------------------------------------------------------- Nevil Brownlee Director, Technology Development Phone: +64 9 373 7599 x88941 ITSS, The University of Auckland FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand ------------------------------------------------- This mail sent through University of Auckland http://www.auckland.ac.nz/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue May 27 15:58:47 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA28741 for ; Tue, 27 May 2003 15:58:47 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19KkDn-0003ue-00 for ipfix-list@mil.doit.wisc.edu; Tue, 27 May 2003 14:34:55 -0500 Received: from palrel12.hp.com ([156.153.255.237]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19KkDl-0003uR-00 for ipfix@net.doit.wisc.edu; Tue, 27 May 2003 14:34:53 -0500 Received: from xparelay1.ptp.hp.com (xparelay1.ptp.hp.com [15.1.28.62]) by palrel12.hp.com (Postfix) with ESMTP id B0F601C01AB3; Tue, 27 May 2003 12:34:52 -0700 (PDT) Received: from xpabh1.ptp.hp.com (xpabh1.ptp.hp.com [15.1.28.60]) by xparelay1.ptp.hp.com (Postfix) with ESMTP id 6E54310054CE; Tue, 27 May 2003 12:34:52 -0700 (PDT) Received: by xpabh1.ptp.hp.com with Internet Mail Service (5.5.2655.55) id ; Tue, 27 May 2003 12:34:52 -0700 Message-ID: <1D3D2C371FCBD947A7897FABBD3533A566BB59@xsun01.ptp.hp.com> From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" To: "'Nevil Brownlee'" , "MEYER,JEFFREY D (HP-Cupertino,ex1)" Cc: ipfix@net.doit.wisc.edu Subject: RE: [ipfix] IPFIX requirements - final changes Date: Tue, 27 May 2003 12:34:45 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2655.55) Content-Type: text/plain; charset="iso-8859-1" Precedence: bulk Sender: majordomo listserver I agree with Nevil, Tal and Benoit on this... -- Jeff Meyer > -----Original Message----- > From: Nevil Brownlee [mailto:n.brownlee@auckland.ac.nz] > Sent: Tuesday, May 27, 2003 12:28 PM > To: MEYER,JEFFREY D (HP-Cupertino,ex1) > Cc: ipfix@net.doit.wisc.edu > Subject: RE: [ipfix] IPFIX requirements - final changes > > > Responding to Jeff, who said: > > > I agree w/ Tal and Benoit on this one. > > The IESG comment was about confidentiality and anonymisation in IPFIX > being "so optional." Maybe a better way to improve this would be to > write a few parargraphs referring to RFC 2975, making it clear that > building an accounting system invloves much more than just selecting > an export protocol (i.e. other techniques like using a > separate network > may be sufficient in many cases), which is why the IPFIX > requirements only > say SHOULD for these. Of course, if a vendor implements them, users > can decide whether or not to actually use them! If others > agree with this, > I could write some text in a day or two ... > > Juergen's other proposed changes to the draft look OK to me. > > Cheers, Nevil > > -------------------------------------------------------------- > --------- > Nevil Brownlee Director, Technology Development > Phone: +64 9 373 7599 x88941 ITSS, The University of Auckland > FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand > > > ------------------------------------------------- > This mail sent through University of Auckland > http://www.auckland.ac.nz/ > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed May 28 04:34:24 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id EAA21228 for ; Wed, 28 May 2003 04:34:23 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19KvvZ-0002tu-00 for ipfix-list@mil.doit.wisc.edu; Wed, 28 May 2003 03:04:53 -0500 Received: from tokyo.ccrle.nec.de ([195.37.70.2]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19KvvW-0002tc-00 for ipfix@net.doit.wisc.edu; Wed, 28 May 2003 03:04:50 -0500 Received: from venus.office (venus.office [10.1.1.11]) by tokyo.ccrle.nec.de (8.12.9/8.12.8) with ESMTP id h4S84bVI043301; Wed, 28 May 2003 10:04:38 +0200 (CEST) Received: from [10.1.1.128] (n-quittek.office [10.1.1.128]) by venus.office (Postfix on SuSE Linux eMail Server 3.0) with ESMTP id 46516A7A43; Wed, 28 May 2003 09:53:44 +0200 (CEST) Date: Wed, 28 May 2003 10:06:39 +0200 From: Juergen Quittek To: Tal Givoly , Benoit Claise Cc: ipfix@net.doit.wisc.edu Subject: RE: [ipfix] IPFIX requirements - final changes Message-ID: <3758163.1054116399@[10.1.1.128]> In-Reply-To: References: X-Mailer: Mulberry/2.1.2 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Tal, Reinaldo, and Benoit, Do we all have the following in mind? Anonymization and confidentiality MUST be covered by the protocol specification. But the protocol specifiction specification defines it as a MAY feature (anonymization) or SHOULD feature (confidentiality) for IPFIX protocol implementations. I guess this is what the IESG reviewers were referring to. What is probably missing in the requirements document is a statement like the following: Many requirements in this document are not explicitly stated as IPFIX protocol requirements, but as requirements for the metering process, the exporting process, or for other traffic measurement components. However, every requirement that needs support from the IPFIX protocol MUST be covered by the IPFIX protocol specification independent of the significance of the requirement, which can be MANDATOYRY, RECOMMENDED, or OPTIONAL. Note that the protocol specification itself also assigns significance attributes to its features, such that a protocol implementation does not necessarily need to implement all features. Any comment on this? Juergen -- Tal Givoly wrote on 26 May 2003 09:42 -0700: > Juergen, > > I agree with Benoit - it doesn't make sense to make confidentiality and > anonymization mandatory. In many deployments, there could be physical and > other security measures that addresses these issues. > > Tal > > -----Original Message----- > From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu]On Behalf > Of Benoit Claise > Sent: Monday, May 26, 2003 8:02 AM > To: Juergen Quittek > Cc: ipfix@net.doit.wisc.edu > Subject: Re: [ipfix] IPFIX requirements - final changes > > > Juergen, > >> ======================================================================== >> 12. Confidentiality from SHOULD to MUST >> ======================================================================== >> Problem Description: >> ------------------------------------------------------------------------ >> Requirements on the ipfix implementation - the document is long and I >> wonder if >> the working group really meant the protocol's confidentiality and >> anonymization >> features to be so optional - SHOULD confidentiality, MAY anonymization. >> Just for implementation. >> ======================================================================== >> Suggested solution: >> ------------------------------------------------------------------------ >> change confidentiality requirement in section 6.3.3 >> from SHOULD to MUST >> ======================================================================== >> Status: solution to be agreed on >> ======================================================================== > > I'm just wondering if a MUST is not a little bit extreme. > The draft would become: > "Confidentiality of flow specific data transferred from an exporting > process to a collecting process MUST be ensured." > We all agree that over the Internet confidentiality is compulsory but > this sentence would mean that even in a private LAN or environment, we > must have confidentiality to be IPFIX compliant. And we know that, with > the software based encryption, the throughput would be lowered. Ok, then > we should have an crypto engine in hardware on the exporter to export > the numerous flow export packets but there is a price to it. > My point is that, as Confidentiality is not needed in all cases, why > make it a MUST? > >> >> >> >> ======================================================================== >> 13. Anonymization from MAY to MUST >> ======================================================================== >> Problem Description: >> ------------------------------------------------------------------------ >> Requirements on the ipfix implementation - the document is long and I >> wonder if >> the working group really meant the protocol's confidentiality and >> anonymization >> features to be so optional - SHOULD confidentiality, MAY anonymization. >> Just for implementation. >> ======================================================================== >> Suggested solution: >> ------------------------------------------------------------------------ >> change anonymization requirement in section 6.7 from MAY to MUST >> ======================================================================== >> Status: solution to be agreed on >> ======================================================================== > > Again, I have the same type of comments. Anonymization is not required > in all cases. So why make it a MUST? > > Regards, Benoit. > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message > body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed May 28 06:33:52 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id GAA24287 for ; Wed, 28 May 2003 06:33:52 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19Ky7Y-0007F6-00 for ipfix-list@mil.doit.wisc.edu; Wed, 28 May 2003 05:25:24 -0500 Received: from ds20-1.cc.swin.edu.au ([136.186.1.150] helo=swin.edu.au) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19Ky7V-0007Eq-00 for ipfix@net.doit.wisc.edu; Wed, 28 May 2003 05:25:21 -0500 Received: from fokus.fraunhofer.de (szander-laptop.caia.swin.edu.au [136.186.229.90]) by swin.edu.au (8.9.3p2/8.9.3) with ESMTP id UAA1974569; Wed, 28 May 2003 20:25:14 +1000 (EST) Message-ID: <3ED48D12.9030603@fokus.fraunhofer.de> Date: Wed, 28 May 2003 12:18:58 +0200 From: Sebastian Zander Organization: Fraunhofer FOKUS User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.2) Gecko/20030208 Netscape/7.02 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Juergen Quittek CC: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] IPFIX requirements - final changes References: <3758163.1054116399@[10.1.1.128]> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hi Juergen, i think your second paragraph is a very good idea. But i don't understand the last 2 sentences. My understanding is that the ipfix requirement draft defines the ipfix protocol requirements. So i don't understand why the protocol specification must cover all requirements. E.g. when a requirementis optional why must it be covered in the protocol specification? I agree to say its a MUST for the protocol but not all exporters and collectors must support it. But i do *not* agree at all that in general anonymization and confidentiality is not needed in a LAN. Anonymization can be required in a LAN and even in a LAN confidentiality may be needed because i don't want everybody who has access to my LAN to have access to my IPFIX data. Cheers, Sebastian Juergen Quittek wrote: > Tal, Reinaldo, and Benoit, > > Do we all have the following in mind? > > Anonymization and confidentiality MUST be covered by the > protocol specification. But the protocol specifiction > specification defines it as a MAY feature (anonymization) > or SHOULD feature (confidentiality) for IPFIX protocol > implementations. > > I guess this is what the IESG reviewers were referring to. > > What is probably missing in the requirements document is a > statement like the following: > > Many requirements in this document are not explicitly > stated as IPFIX protocol requirements, but as requirements > for the metering process, the exporting process, or for > other traffic measurement components. However, every > requirement that needs support from the IPFIX protocol > MUST be covered by the IPFIX protocol specification > independent of the significance of the requirement, > which can be MANDATOYRY, RECOMMENDED, or OPTIONAL. > Note that the protocol specification itself also assigns > significance attributes to its features, such that a > protocol implementation does not necessarily need to > implement all features. > > Any comment on this? > > Juergen > > > -- Tal Givoly wrote on 26 May 2003 09:42 -0700: > >> Juergen, >> >> I agree with Benoit - it doesn't make sense to make confidentiality and >> anonymization mandatory. In many deployments, there could be physical and >> other security measures that addresses these issues. >> >> Tal >> >> -----Original Message----- >> From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu]On Behalf >> Of Benoit Claise >> Sent: Monday, May 26, 2003 8:02 AM >> To: Juergen Quittek >> Cc: ipfix@net.doit.wisc.edu >> Subject: Re: [ipfix] IPFIX requirements - final changes >> >> >> Juergen, >> >>> ======================================================================== >>> 12. Confidentiality from SHOULD to MUST >>> ======================================================================== >>> Problem Description: >>> ------------------------------------------------------------------------ >>> Requirements on the ipfix implementation - the document is long and I >>> wonder if >>> the working group really meant the protocol's confidentiality and >>> anonymization >>> features to be so optional - SHOULD confidentiality, MAY anonymization. >>> Just for implementation. >>> ======================================================================== >>> Suggested solution: >>> ------------------------------------------------------------------------ >>> change confidentiality requirement in section 6.3.3 >>> from SHOULD to MUST >>> ======================================================================== >>> Status: solution to be agreed on >>> ======================================================================== >> >> >> I'm just wondering if a MUST is not a little bit extreme. >> The draft would become: >> "Confidentiality of flow specific data transferred from an exporting >> process to a collecting process MUST be ensured." >> We all agree that over the Internet confidentiality is compulsory but >> this sentence would mean that even in a private LAN or environment, we >> must have confidentiality to be IPFIX compliant. And we know that, with >> the software based encryption, the throughput would be lowered. Ok, then >> we should have an crypto engine in hardware on the exporter to export >> the numerous flow export packets but there is a price to it. >> My point is that, as Confidentiality is not needed in all cases, why >> make it a MUST? >> >>> >>> >>> >>> ======================================================================== >>> 13. Anonymization from MAY to MUST >>> ======================================================================== >>> Problem Description: >>> ------------------------------------------------------------------------ >>> Requirements on the ipfix implementation - the document is long and I >>> wonder if >>> the working group really meant the protocol's confidentiality and >>> anonymization >>> features to be so optional - SHOULD confidentiality, MAY anonymization. >>> Just for implementation. >>> ======================================================================== >>> Suggested solution: >>> ------------------------------------------------------------------------ >>> change anonymization requirement in section 6.7 from MAY to MUST >>> ======================================================================== >>> Status: solution to be agreed on >>> ======================================================================== >> >> >> Again, I have the same type of comments. Anonymization is not required >> in all cases. So why make it a MUST? >> >> Regards, Benoit. >> >> >> >> -- >> Help mailto:majordomo@net.doit.wisc.edu and say "help" in message >> body >> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >> "unsubscribe ipfix" in message body >> Archive http://ipfix.doit.wisc.edu/archive/ >> > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message > body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > -- Sebastian Zander E-mail: zander@fokus.fraunhofer.de Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 D-10589 Berlin, Germany www.fokus.fraunhofer.de/usr/sebastian.zander -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed May 28 09:41:33 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA02943 for ; Wed, 28 May 2003 09:41:33 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19L0wY-00036w-00 for ipfix-list@mil.doit.wisc.edu; Wed, 28 May 2003 08:26:14 -0500 Received: from tokyo.ccrle.nec.de ([195.37.70.2]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19L0wV-00036n-00 for ipfix@net.doit.wisc.edu; Wed, 28 May 2003 08:26:12 -0500 Received: from venus.office (venus.office [10.1.1.11]) by tokyo.ccrle.nec.de (8.12.9/8.12.8) with ESMTP id h4SDQ1VI073315; Wed, 28 May 2003 15:26:02 +0200 (CEST) Received: from [10.1.1.128] (n-quittek.office [10.1.1.128]) by venus.office (Postfix on SuSE Linux eMail Server 3.0) with ESMTP id 74290A7CEE; Wed, 28 May 2003 15:15:05 +0200 (CEST) Date: Wed, 28 May 2003 15:28:01 +0200 From: Juergen Quittek To: Sebastian Zander Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] IPFIX requirements - final changes Message-ID: <23040159.1054135681@[10.1.1.128]> In-Reply-To: <3ED48D12.9030603@fokus.fraunhofer.de> References: <3ED48D12.9030603@fokus.fraunhofer.de> X-Mailer: Mulberry/2.1.2 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Sebastian, -- Sebastian Zander wrote on 28 May 2003 12:18 +0200: > Hi Juergen, > > i think your second paragraph is a very good idea. But i don't > understand the last 2 sentences. My understanding is that the ipfix > requirement draft defines the ipfix protocol requirements. So i don't > understand why the protocol specification must cover all requirements. > E.g. when a requirementis optional why must it be covered in the protocol > specification? If the requirement doc says "the exporter MAY anonymize", then the protocol specification MUST support anonymization in order to make sure, that anonymization is interoperable. For example the protocol spec MUST define how to inform a collecting process that the data it receives is anonymized. Analogously, the protocol spec should specify how to encrypt. In both cases, the protocol specification can state that a concrete implementation of the protocol MAY support features that serve anonymization. Then an implementor can choose whether or not to implement this OPTIONAL feature of the protocol. > I agree to say its a MUST for the protocol but not all exporters > and collectors must support it. Agreed. > But i do *not* agree at all that in general anonymization and > confidentiality is not needed in a LAN. Anonymization can be required > in a LAN and even in a LAN confidentiality may be needed because i > don't want everybody who has access to my LAN to have access to my > IPFIX data. Also agreed. Best wishes, Juergen > Cheers, > > Sebastian > > Juergen Quittek wrote: >> Tal, Reinaldo, and Benoit, >> >> Do we all have the following in mind? >> >> Anonymization and confidentiality MUST be covered by the >> protocol specification. But the protocol specifiction >> specification defines it as a MAY feature (anonymization) >> or SHOULD feature (confidentiality) for IPFIX protocol >> implementations. >> >> I guess this is what the IESG reviewers were referring to. >> >> What is probably missing in the requirements document is a >> statement like the following: >> >> Many requirements in this document are not explicitly >> stated as IPFIX protocol requirements, but as requirements >> for the metering process, the exporting process, or for >> other traffic measurement components. However, every >> requirement that needs support from the IPFIX protocol >> MUST be covered by the IPFIX protocol specification >> independent of the significance of the requirement, >> which can be MANDATOYRY, RECOMMENDED, or OPTIONAL. >> Note that the protocol specification itself also assigns >> significance attributes to its features, such that a >> protocol implementation does not necessarily need to >> implement all features. >> >> Any comment on this? >> >> Juergen >> >> >> -- Tal Givoly wrote on 26 May 2003 09:42 -0700: >> >>> Juergen, >>> >>> I agree with Benoit - it doesn't make sense to make confidentiality and >>> anonymization mandatory. In many deployments, there could be physical and >>> other security measures that addresses these issues. >>> >>> Tal >>> >>> -----Original Message----- >>> From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu]On Behalf >>> Of Benoit Claise >>> Sent: Monday, May 26, 2003 8:02 AM >>> To: Juergen Quittek >>> Cc: ipfix@net.doit.wisc.edu >>> Subject: Re: [ipfix] IPFIX requirements - final changes >>> >>> >>> Juergen, >>> >>>> ======================================================================== >>>> 12. Confidentiality from SHOULD to MUST >>>> ======================================================================== >>>> Problem Description: >>>> ------------------------------------------------------------------------ >>>> Requirements on the ipfix implementation - the document is long and I >>>> wonder if >>>> the working group really meant the protocol's confidentiality and >>>> anonymization >>>> features to be so optional - SHOULD confidentiality, MAY anonymization. >>>> Just for implementation. >>>> ======================================================================== >>>> Suggested solution: >>>> ------------------------------------------------------------------------ >>>> change confidentiality requirement in section 6.3.3 >>>> from SHOULD to MUST >>>> ======================================================================== >>>> Status: solution to be agreed on >>>> ======================================================================== >>> >>> >>> I'm just wondering if a MUST is not a little bit extreme. >>> The draft would become: >>> "Confidentiality of flow specific data transferred from an exporting >>> process to a collecting process MUST be ensured." >>> We all agree that over the Internet confidentiality is compulsory but >>> this sentence would mean that even in a private LAN or environment, we >>> must have confidentiality to be IPFIX compliant. And we know that, with >>> the software based encryption, the throughput would be lowered. Ok, then >>> we should have an crypto engine in hardware on the exporter to export >>> the numerous flow export packets but there is a price to it. >>> My point is that, as Confidentiality is not needed in all cases, why >>> make it a MUST? >>> >>>> >>>> >>>> >>>> ======================================================================== >>>> 13. Anonymization from MAY to MUST >>>> ======================================================================== >>>> Problem Description: >>>> ------------------------------------------------------------------------ >>>> Requirements on the ipfix implementation - the document is long and I >>>> wonder if >>>> the working group really meant the protocol's confidentiality and >>>> anonymization >>>> features to be so optional - SHOULD confidentiality, MAY anonymization. >>>> Just for implementation. >>>> ======================================================================== >>>> Suggested solution: >>>> ------------------------------------------------------------------------ >>>> change anonymization requirement in section 6.7 from MAY to MUST >>>> ======================================================================== >>>> Status: solution to be agreed on >>>> ======================================================================== >>> >>> >>> Again, I have the same type of comments. Anonymization is not required >>> in all cases. So why make it a MUST? >>> >>> Regards, Benoit. >>> >>> >>> >>> -- >>> Help mailto:majordomo@net.doit.wisc.edu and say "help" in message >>> body >>> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >>> "unsubscribe ipfix" in message body >>> Archive http://ipfix.doit.wisc.edu/archive/ >>> >> >> >> >> -- >> Help mailto:majordomo@net.doit.wisc.edu and say "help" in message >> body >> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >> "unsubscribe ipfix" in message body >> Archive http://ipfix.doit.wisc.edu/archive/ >> > > > -- > Sebastian Zander E-mail: zander@fokus.fraunhofer.de > Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 > Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 > D-10589 Berlin, Germany www.fokus.fraunhofer.de/usr/sebastian.zander > > > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed May 28 11:30:35 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA08485 for ; Wed, 28 May 2003 11:30:35 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19L2as-0005BZ-00 for ipfix-list@mil.doit.wisc.edu; Wed, 28 May 2003 10:11:58 -0500 Received: from usmail.xacct.com ([204.253.100.12]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19L2aq-0005BS-00 for ipfix@net.doit.wisc.edu; Wed, 28 May 2003 10:11:56 -0500 Received: from Givoly ([192.168.0.3]) by usmail.xacct.com (8.11.2/8.11.2) with SMTP id h4SFIei06056; Wed, 28 May 2003 08:18:41 -0700 From: "Tal Givoly" To: "Juergen Quittek" , "Benoit Claise" Cc: Subject: RE: [ipfix] IPFIX requirements - final changes Date: Wed, 28 May 2003 08:11:38 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-Reply-To: <3758163.1054116399@[10.1.1.128]> Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Juergen, I believe that what you described is definitely the way to go - extensions that are fully specified to avoid ambiguity (a single well-defined way to implement for interoperability), but not mandatory (either optional or recommended). In that case, should we also consider to fully specify the reliability extensions in the same manner? how else would we assure that we comply with the mandatory requirement that the protocol be extensible to support reliability without providing at least a valid example, if not a complete specification of a standard extension? Tal -----Original Message----- From: Juergen Quittek [mailto:quittek@ccrle.nec.de] Sent: Wednesday, May 28, 2003 1:07 AM To: Tal Givoly; Benoit Claise Cc: ipfix@net.doit.wisc.edu Subject: RE: [ipfix] IPFIX requirements - final changes Tal, Reinaldo, and Benoit, Do we all have the following in mind? Anonymization and confidentiality MUST be covered by the protocol specification. But the protocol specifiction specification defines it as a MAY feature (anonymization) or SHOULD feature (confidentiality) for IPFIX protocol implementations. I guess this is what the IESG reviewers were referring to. What is probably missing in the requirements document is a statement like the following: Many requirements in this document are not explicitly stated as IPFIX protocol requirements, but as requirements for the metering process, the exporting process, or for other traffic measurement components. However, every requirement that needs support from the IPFIX protocol MUST be covered by the IPFIX protocol specification independent of the significance of the requirement, which can be MANDATOYRY, RECOMMENDED, or OPTIONAL. Note that the protocol specification itself also assigns significance attributes to its features, such that a protocol implementation does not necessarily need to implement all features. Any comment on this? Juergen -- Tal Givoly wrote on 26 May 2003 09:42 -0700: > Juergen, > > I agree with Benoit - it doesn't make sense to make confidentiality and > anonymization mandatory. In many deployments, there could be physical and > other security measures that addresses these issues. > > Tal > > -----Original Message----- > From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu]On Behalf > Of Benoit Claise > Sent: Monday, May 26, 2003 8:02 AM > To: Juergen Quittek > Cc: ipfix@net.doit.wisc.edu > Subject: Re: [ipfix] IPFIX requirements - final changes > > > Juergen, > >> ======================================================================== >> 12. Confidentiality from SHOULD to MUST >> ======================================================================== >> Problem Description: >> ------------------------------------------------------------------------ >> Requirements on the ipfix implementation - the document is long and I >> wonder if >> the working group really meant the protocol's confidentiality and >> anonymization >> features to be so optional - SHOULD confidentiality, MAY anonymization. >> Just for implementation. >> ======================================================================== >> Suggested solution: >> ------------------------------------------------------------------------ >> change confidentiality requirement in section 6.3.3 >> from SHOULD to MUST >> ======================================================================== >> Status: solution to be agreed on >> ======================================================================== > > I'm just wondering if a MUST is not a little bit extreme. > The draft would become: > "Confidentiality of flow specific data transferred from an exporting > process to a collecting process MUST be ensured." > We all agree that over the Internet confidentiality is compulsory but > this sentence would mean that even in a private LAN or environment, we > must have confidentiality to be IPFIX compliant. And we know that, with > the software based encryption, the throughput would be lowered. Ok, then > we should have an crypto engine in hardware on the exporter to export > the numerous flow export packets but there is a price to it. > My point is that, as Confidentiality is not needed in all cases, why > make it a MUST? > >> >> >> >> ======================================================================== >> 13. Anonymization from MAY to MUST >> ======================================================================== >> Problem Description: >> ------------------------------------------------------------------------ >> Requirements on the ipfix implementation - the document is long and I >> wonder if >> the working group really meant the protocol's confidentiality and >> anonymization >> features to be so optional - SHOULD confidentiality, MAY anonymization. >> Just for implementation. >> ======================================================================== >> Suggested solution: >> ------------------------------------------------------------------------ >> change anonymization requirement in section 6.7 from MAY to MUST >> ======================================================================== >> Status: solution to be agreed on >> ======================================================================== > > Again, I have the same type of comments. Anonymization is not required > in all cases. So why make it a MUST? > > Regards, Benoit. > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message > body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed May 28 14:07:00 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA14066 for ; Wed, 28 May 2003 14:07:00 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19L4qg-00001f-00 for ipfix-list@mil.doit.wisc.edu; Wed, 28 May 2003 12:36:26 -0500 Received: from atlrel6.hp.com ([156.153.255.205]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19L4qd-00001Z-00 for ipfix@net.doit.wisc.edu; Wed, 28 May 2003 12:36:23 -0500 Received: from xatlrelay1.atl.hp.com (xatlrelay1.atl.hp.com [15.45.89.190]) by atlrel6.hp.com (Postfix) with ESMTP id 341991C0183C; Wed, 28 May 2003 13:36:23 -0400 (EDT) Received: from xatlbh1.atl.hp.com (xatlbh1.atl.hp.com [15.45.89.186]) by xatlrelay1.atl.hp.com (Postfix) with ESMTP id 000921C000A8; Wed, 28 May 2003 13:36:22 -0400 (EDT) Received: by xatlbh1.atl.hp.com with Internet Mail Service (5.5.2655.55) id ; Wed, 28 May 2003 13:36:22 -0400 Message-ID: <1D3D2C371FCBD947A7897FABBD3533A50295FFC2@xsun01.ptp.hp.com> From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" To: "'Juergen Quittek'" , Sebastian Zander Cc: ipfix@net.doit.wisc.edu Subject: RE: [ipfix] IPFIX requirements - final changes Date: Wed, 28 May 2003 13:36:12 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2655.55) Content-Type: text/plain; charset="iso-8859-1" Precedence: bulk Sender: majordomo listserver Hi, How about a little "separation of concern" on the topics of anonymization and confidentiality? IPSEC provides a means to make confidential either UDP based or TCP based protocol conversations (presumably SCTP as well). Can we simply say that the protocol itself will not introduce any specific confidentiality mechanisms, but will rely on underlying mechanisms? As far as anonymization, is this a protocol issue or a data model issue? Since the protocol itself can carry more or less arbitrary sets of structured (first normal form) information. Define new attributes for: anonSourceAddress anonDestAddress anonSourceAddressV6 anonDestAddressV6 Since these are the only user identifying fields in the data model today (that I know of), then creating their anonymous analog (which presumes some model on the exporter/observer, to transform addr's) should be sufficient. Regards, Jeff Meyer > -----Original Message----- > From: Juergen Quittek [mailto:quittek@ccrle.nec.de] > Sent: Wednesday, May 28, 2003 6:28 AM > To: Sebastian Zander > Cc: ipfix@net.doit.wisc.edu > Subject: Re: [ipfix] IPFIX requirements - final changes > > > Sebastian, > > -- Sebastian Zander wrote on 28 May 2003 12:18 +0200: > > > Hi Juergen, > > > > i think your second paragraph is a very good idea. But i don't > > understand the last 2 sentences. My understanding is that the ipfix > > requirement draft defines the ipfix protocol requirements. > So i don't > > understand why the protocol specification must cover all > requirements. > > E.g. when a requirementis optional why must it be covered > in the protocol > > specification? > > If the requirement doc says "the exporter MAY anonymize", then the > protocol specification MUST support anonymization in order to make > sure, that anonymization is interoperable. For example the protocol > spec MUST define how to inform a collecting process that the data it > receives is anonymized. Analogously, the protocol spec should specify > how to encrypt. > > In both cases, the protocol specification can state that a concrete > implementation of the protocol MAY support features that serve > anonymization. Then an implementor can choose whether or not > to implement > this OPTIONAL feature of the protocol. > > > I agree to say its a MUST for the protocol but not all exporters > > and collectors must support it. > > Agreed. > > > But i do *not* agree at all that in general anonymization and > > confidentiality is not needed in a LAN. Anonymization can > be required > > in a LAN and even in a LAN confidentiality may be needed because i > > don't want everybody who has access to my LAN to have access to my > > IPFIX data. > > Also agreed. > > Best wishes, > > Juergen > > > Cheers, > > > > Sebastian > > > > Juergen Quittek wrote: > >> Tal, Reinaldo, and Benoit, > >> > >> Do we all have the following in mind? > >> > >> Anonymization and confidentiality MUST be covered by the > >> protocol specification. But the protocol specifiction > >> specification defines it as a MAY feature (anonymization) > >> or SHOULD feature (confidentiality) for IPFIX protocol > >> implementations. > >> > >> I guess this is what the IESG reviewers were referring to. > >> > >> What is probably missing in the requirements document is a > >> statement like the following: > >> > >> Many requirements in this document are not explicitly > >> stated as IPFIX protocol requirements, but as requirements > >> for the metering process, the exporting process, or for > >> other traffic measurement components. However, every > >> requirement that needs support from the IPFIX protocol > >> MUST be covered by the IPFIX protocol specification > >> independent of the significance of the requirement, > >> which can be MANDATOYRY, RECOMMENDED, or OPTIONAL. > >> Note that the protocol specification itself also assigns > >> significance attributes to its features, such that a > >> protocol implementation does not necessarily need to > >> implement all features. > >> > >> Any comment on this? > >> > >> Juergen > >> > >> > >> -- Tal Givoly wrote on 26 May 2003 09:42 -0700: > >> > >>> Juergen, > >>> > >>> I agree with Benoit - it doesn't make sense to make > confidentiality and > >>> anonymization mandatory. In many deployments, there could > be physical and > >>> other security measures that addresses these issues. > >>> > >>> Tal > >>> > >>> -----Original Message----- > >>> From: majordomo listserver > [mailto:majordomo@mil.doit.wisc.edu]On Behalf > >>> Of Benoit Claise > >>> Sent: Monday, May 26, 2003 8:02 AM > >>> To: Juergen Quittek > >>> Cc: ipfix@net.doit.wisc.edu > >>> Subject: Re: [ipfix] IPFIX requirements - final changes > >>> > >>> > >>> Juergen, > >>> > >>>> > ============================================================== > ========== > >>>> 12. Confidentiality from SHOULD to MUST > >>>> > ============================================================== > ========== > >>>> Problem Description: > >>>> > -------------------------------------------------------------- > ---------- > >>>> Requirements on the ipfix implementation - the document > is long and I > >>>> wonder if > >>>> the working group really meant the protocol's confidentiality and > >>>> anonymization > >>>> features to be so optional - SHOULD confidentiality, > MAY anonymization. > >>>> Just for implementation. > >>>> > ============================================================== > ========== > >>>> Suggested solution: > >>>> > -------------------------------------------------------------- > ---------- > >>>> change confidentiality requirement in section 6.3.3 > >>>> from SHOULD to MUST > >>>> > ============================================================== > ========== > >>>> Status: solution to be agreed on > >>>> > ============================================================== > ========== > >>> > >>> > >>> I'm just wondering if a MUST is not a little bit extreme. > >>> The draft would become: > >>> "Confidentiality of flow specific data transferred > from an exporting > >>> process to a collecting process MUST be ensured." > >>> We all agree that over the Internet confidentiality is > compulsory but > >>> this sentence would mean that even in a private LAN or > environment, we > >>> must have confidentiality to be IPFIX compliant. And we > know that, with > >>> the software based encryption, the throughput would be > lowered. Ok, then > >>> we should have an crypto engine in hardware on the > exporter to export > >>> the numerous flow export packets but there is a price to it. > >>> My point is that, as Confidentiality is not needed in all > cases, why > >>> make it a MUST? > >>> > >>>> > >>>> > >>>> > >>>> > ============================================================== > ========== > >>>> 13. Anonymization from MAY to MUST > >>>> > ============================================================== > ========== > >>>> Problem Description: > >>>> > -------------------------------------------------------------- > ---------- > >>>> Requirements on the ipfix implementation - the document > is long and I > >>>> wonder if > >>>> the working group really meant the protocol's confidentiality and > >>>> anonymization > >>>> features to be so optional - SHOULD confidentiality, > MAY anonymization. > >>>> Just for implementation. > >>>> > ============================================================== > ========== > >>>> Suggested solution: > >>>> > -------------------------------------------------------------- > ---------- > >>>> change anonymization requirement in section 6.7 from > MAY to MUST > >>>> > ============================================================== > ========== > >>>> Status: solution to be agreed on > >>>> > ============================================================== > ========== > >>> > >>> > >>> Again, I have the same type of comments. Anonymization is > not required > >>> in all cases. So why make it a MUST? > >>> > >>> Regards, Benoit. > >>> > >>> > >>> > >>> -- > >>> Help mailto:majordomo@net.doit.wisc.edu and say > "help" in message > >>> body > >>> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > >>> "unsubscribe ipfix" in message body > >>> Archive http://ipfix.doit.wisc.edu/archive/ > >>> > >> > >> > >> > >> -- > >> Help mailto:majordomo@net.doit.wisc.edu and say > "help" in message > >> body > >> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > >> "unsubscribe ipfix" in message body > >> Archive http://ipfix.doit.wisc.edu/archive/ > >> > > > > > > -- > > Sebastian Zander E-mail: > zander@fokus.fraunhofer.de > > Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 > > Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 > > D-10589 Berlin, Germany www.fokus.fraunhofer.de/usr/sebastian.zander > > > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed May 28 21:56:38 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA02791 for ; Wed, 28 May 2003 21:56:38 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19LCKo-0001cI-00 for ipfix-list@mil.doit.wisc.edu; Wed, 28 May 2003 20:36:02 -0500 Received: from ds20-1.cc.swin.edu.au ([136.186.1.150] helo=swin.edu.au) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19LCKm-0001cD-00 for ipfix@net.doit.wisc.edu; Wed, 28 May 2003 20:36:00 -0500 Received: from fokus.fraunhofer.de (szander-laptop.caia.swin.edu.au [136.186.229.90]) by swin.edu.au (8.9.3p2/8.9.3) with ESMTP id LAA2053063; Thu, 29 May 2003 11:35:53 +1000 (EST) Message-ID: <3ED56280.90204@fokus.fraunhofer.de> Date: Thu, 29 May 2003 03:29:36 +0200 From: Sebastian Zander Organization: Fraunhofer FOKUS User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.2) Gecko/20030208 Netscape/7.02 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Juergen Quittek CC: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] IPFIX requirements - final changes References: <3ED48D12.9030603@fokus.fraunhofer.de> <23040159.1054135681@[10.1.1.128]> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Juergen, Juergen Quittek wrote: > Sebastian, > > -- Sebastian Zander wrote on 28 May 2003 12:18 +0200: > >> Hi Juergen, >> >> i think your second paragraph is a very good idea. But i don't >> understand the last 2 sentences. My understanding is that the ipfix >> requirement draft defines the ipfix protocol requirements. So i don't >> understand why the protocol specification must cover all requirements. >> E.g. when a requirementis optional why must it be covered in the protocol >> specification? > > > If the requirement doc says "the exporter MAY anonymize", then the > protocol specification MUST support anonymization in order to make > sure, that anonymization is interoperable. For example the protocol > spec MUST define how to inform a collecting process that the data it > receives is anonymized. Analogously, the protocol spec should specify > how to encrypt. > > In both cases, the protocol specification can state that a concrete > implementation of the protocol MAY support features that serve > anonymization. Then an implementor can choose whether or not to implement > this OPTIONAL feature of the protocol. You are right. The req draft contains protocol specification requirements (which is sometimes confusing and the reason why we get those comments). But i suggest to change the middle sentence saying that all protocol specification requirements from the req draft must be taken over into the protocol draft *including their significance*. Otherwise why do we spend a long time discussing the significance of some reqs if it is not used later in the protocol draft? Furthermore some of the reqs may be targeting the information model or achitecture draft instead of the protocol spec. So the sentence should not only mention the protocol spec but *also the information model etc*. (Jeff just gave an example that anonymization may influence the information model.) This statement must be put in the very beginning of the req draft. The last sentence is not necessary in my opinion because its pretty obvious that the protocol draft and the other drafts will use more (and more detailed) reqs. Then there is no need to change the requirements to MUST. Cheers, Sebastian >> I agree to say its a MUST for the protocol but not all exporters >> and collectors must support it. > > > Agreed. > >> But i do *not* agree at all that in general anonymization and >> confidentiality is not needed in a LAN. Anonymization can be required >> in a LAN and even in a LAN confidentiality may be needed because i >> don't want everybody who has access to my LAN to have access to my >> IPFIX data. > > > Also agreed. > > Best wishes, > > Juergen > >> Cheers, >> >> Sebastian >> >> Juergen Quittek wrote: >> >>> Tal, Reinaldo, and Benoit, >>> >>> Do we all have the following in mind? >>> >>> Anonymization and confidentiality MUST be covered by the >>> protocol specification. But the protocol specifiction >>> specification defines it as a MAY feature (anonymization) >>> or SHOULD feature (confidentiality) for IPFIX protocol >>> implementations. >>> >>> I guess this is what the IESG reviewers were referring to. >>> >>> What is probably missing in the requirements document is a >>> statement like the following: >>> >>> Many requirements in this document are not explicitly >>> stated as IPFIX protocol requirements, but as requirements >>> for the metering process, the exporting process, or for >>> other traffic measurement components. However, every >>> requirement that needs support from the IPFIX protocol >>> MUST be covered by the IPFIX protocol specification >>> independent of the significance of the requirement, >>> which can be MANDATOYRY, RECOMMENDED, or OPTIONAL. >>> Note that the protocol specification itself also assigns >>> significance attributes to its features, such that a >>> protocol implementation does not necessarily need to >>> implement all features. >>> >>> Any comment on this? >>> >>> Juergen >>> >>> >>> -- Tal Givoly wrote on 26 May 2003 09:42 -0700: >>> >>>> Juergen, >>>> >>>> I agree with Benoit - it doesn't make sense to make confidentiality and >>>> anonymization mandatory. In many deployments, there could be >>>> physical and >>>> other security measures that addresses these issues. >>>> >>>> Tal >>>> >>>> -----Original Message----- >>>> From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu]On >>>> Behalf >>>> Of Benoit Claise >>>> Sent: Monday, May 26, 2003 8:02 AM >>>> To: Juergen Quittek >>>> Cc: ipfix@net.doit.wisc.edu >>>> Subject: Re: [ipfix] IPFIX requirements - final changes >>>> >>>> >>>> Juergen, >>>> >>>>> ======================================================================== >>>>> >>>>> 12. Confidentiality from SHOULD to MUST >>>>> ======================================================================== >>>>> >>>>> Problem Description: >>>>> ------------------------------------------------------------------------ >>>>> >>>>> Requirements on the ipfix implementation - the document is long and I >>>>> wonder if >>>>> the working group really meant the protocol's confidentiality and >>>>> anonymization >>>>> features to be so optional - SHOULD confidentiality, MAY >>>>> anonymization. >>>>> Just for implementation. >>>>> ======================================================================== >>>>> >>>>> Suggested solution: >>>>> ------------------------------------------------------------------------ >>>>> >>>>> change confidentiality requirement in section 6.3.3 >>>>> from SHOULD to MUST >>>>> ======================================================================== >>>>> >>>>> Status: solution to be agreed on >>>>> ======================================================================== >>>>> >>>> >>>> >>>> >>>> I'm just wondering if a MUST is not a little bit extreme. >>>> The draft would become: >>>> "Confidentiality of flow specific data transferred from an exporting >>>> process to a collecting process MUST be ensured." >>>> We all agree that over the Internet confidentiality is compulsory but >>>> this sentence would mean that even in a private LAN or environment, we >>>> must have confidentiality to be IPFIX compliant. And we know that, with >>>> the software based encryption, the throughput would be lowered. Ok, >>>> then >>>> we should have an crypto engine in hardware on the exporter to export >>>> the numerous flow export packets but there is a price to it. >>>> My point is that, as Confidentiality is not needed in all cases, why >>>> make it a MUST? >>>> >>>>> >>>>> >>>>> >>>>> ======================================================================== >>>>> >>>>> 13. Anonymization from MAY to MUST >>>>> ======================================================================== >>>>> >>>>> Problem Description: >>>>> ------------------------------------------------------------------------ >>>>> >>>>> Requirements on the ipfix implementation - the document is long and I >>>>> wonder if >>>>> the working group really meant the protocol's confidentiality and >>>>> anonymization >>>>> features to be so optional - SHOULD confidentiality, MAY >>>>> anonymization. >>>>> Just for implementation. >>>>> ======================================================================== >>>>> >>>>> Suggested solution: >>>>> ------------------------------------------------------------------------ >>>>> >>>>> change anonymization requirement in section 6.7 from MAY to MUST >>>>> ======================================================================== >>>>> >>>>> Status: solution to be agreed on >>>>> ======================================================================== >>>>> >>>> >>>> >>>> >>>> Again, I have the same type of comments. Anonymization is not required >>>> in all cases. So why make it a MUST? >>>> >>>> Regards, Benoit. >>>> >>>> >>>> >>>> -- >>>> Help mailto:majordomo@net.doit.wisc.edu and say "help" in >>>> message >>>> body >>>> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >>>> "unsubscribe ipfix" in message body >>>> Archive http://ipfix.doit.wisc.edu/archive/ >>>> >>> >>> >>> >>> -- >>> Help mailto:majordomo@net.doit.wisc.edu and say "help" in message >>> body >>> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >>> "unsubscribe ipfix" in message body >>> Archive http://ipfix.doit.wisc.edu/archive/ >>> >> >> >> -- >> Sebastian Zander E-mail: >> zander@fokus.fraunhofer.de >> Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 >> Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 >> D-10589 Berlin, Germany >> www.fokus.fraunhofer.de/usr/sebastian.zander >> >> >> >> > > > -- Sebastian Zander E-mail: zander@fokus.fraunhofer.de Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 D-10589 Berlin, Germany www.fokus.fraunhofer.de/usr/sebastian.zander -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed May 28 22:00:21 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA02906 for ; Wed, 28 May 2003 22:00:20 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19LCS5-0001jz-00 for ipfix-list@mil.doit.wisc.edu; Wed, 28 May 2003 20:43:33 -0500 Received: from mailhost2.auckland.ac.nz ([130.216.191.4]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19LCS3-0001jq-00 for ipfix@net.doit.wisc.edu; Wed, 28 May 2003 20:43:31 -0500 Received: from mailhost.auckland.ac.nz (IDENT:mirapoint@mailhost [130.216.191.61]) by mailhost2.auckland.ac.nz (8.12.9/8.12.9/8.12.9-ua) with ESMTP id h4T1hP8W024011; Thu, 29 May 2003 13:43:25 +1200 (NZST) Received: from hotlava.auckland.ac.nz (hotlava.auckland.ac.nz [130.216.191.123]) by mailhost.auckland.ac.nz (Mirapoint Messaging Server MOS 3.3.5-GR) with ESMTP id AQN47689; Thu, 29 May 2003 13:43:18 +1200 (NZST) Received: (from apache@localhost) by hotlava.auckland.ac.nz (8.11.6/8.11.6) id h4T1hHG05512; Thu, 29 May 2003 13:43:17 +1200 X-Authentication-Warning: hotlava.auckland.ac.nz: apache set sender to n.brownlee@auckland.ac.nz using -f Received: from dyn41.caida.org (dyn41.caida.org [192.172.226.41]) by hotlava.auckland.ac.nz (Horde) with HTTP for ; Thu, 29 May 2003 13:43:17 +1200 Message-ID: <1054172597.0d4147af570a5@hotlava.auckland.ac.nz> Date: Thu, 29 May 2003 13:43:17 +1200 From: Nevil Brownlee To: Ganesh Sadasivan Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Changes to architecture doc. References: <3EC55996.F610236@cisco.com> In-Reply-To: <3EC55996.F610236@cisco.com> MIME-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Internet Messaging Program (IMP) 4.0-cvs X-Originating-IP: 192.172.226.41 Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hello all: > * Now that we have chosen the IPFIX protocol, is a separate document > needed for architecture and protocols? > > IMHO yes. The architecture discusses the basic terminology (like flow > definition) components of IPFIX framework, different models and > association between these components which are fairly static in nature. > More over the chosen protocol may not fit into all the models described > in the arch. spec. The protocol on the other hand discusses message > etc which could keep evolving to newer versions. Agreed. *Architecture* defines terms (including 'IPFIX flow), and explains how the overall system works. *Information Model* explains how IPFIX flow data is made up, i.e. what data elements may be part of a flow, what they mean, how they are represented, and how they are combined into a flow's data structure. *Protocol* specifies how IPFIX flow data and control information is carried (via a congestion-aware transport protocol) from IPFIX exporter to IPFIX collector. [If you thought IPFIX is only about how to transport flow data, have another look at the WG charter.] > * Section 2 : Scope > There are a few points mentioned here : > > " * Define the criteria to select the IPFIX Protocol." > We have already selected the protocol. Do we still need > to keep this? We put that phrase into the Architecture document as a placeholder until we'd selected a protocol as the starting point for IPFIX. It's time to take it out. > " * Specify the control/data message formats and handshaking details > to pass the IP flow information. " > This is really protocol specific and should not be discussed in this > document. Agreed. > * Section 4: IPFIX Device figure > A device could have multiple exporters and each could have 1:n > mapping with observation domain. This is not well captured in the > figure. There are two diagrams in the introduction to section 4. Both seem fairly clear, but I think that they both would benefit from a paragraph or two of explanatory text. I'll write some, real soon now .. Also, there's a comment in the text about "interfaces shown as <~~>," but I can't see any like that on the diagrams ??? > * Section 4.3 Observation Domain > Functions of Observation domain. > " * Encoding the flow records and sending them to the export process. > * Encoding the control information into templates and sending them to > the export process." > The above 2 are already covered by IPFIX protocol. > " * Deciding which flow records/control information to export using > rules based on time, thresholds, configuration events etc. > * Aggregating flow records generated by one or more metering > processes." > The above 2 are also IPFIX protocol function. So also is flow > expiration. > > Observation point is a logical block which maintains flows for a > a set of {observation point, metering process} (in a database form) > and maintains aggregate statistics. All the other flow related > functions should be handled by IPFIX protocol and metering process. No. Section 3 defines 'Observation Point' as "a location in the network where IP packets can be observed," and that seems fine to me, let's leave it like that. As for section 4.3's list of 'typical functions of an observation domain,' I think it's helpful to keep the list there - but we should also add a reference to the *Protocol* document. [Come to think of it, we should probably refer to the *Protocol* and *Information Model* documents in quite a few other sections through the *Architecture* document.] > * Section 4.5 : > " There MAY be additional rules defined within the observation domain > so that only certain flows records are picked up for export. " > Though rules are defined w.r.t observation domain, > the entity that applies these rules is the IPFIX protocol. Surely it's the metering process that applies selection rules? After all, if a flow is not going to be exported, there's no point in a meter wasting resources on it?? The metering process, i.e. deciding how to actually build flows by executing a sequence of functions (with specified parameters) on the observed packets, is surely part of the Information model. Or is it? Maybe it fits better here in the *Architecture* document. It certainly needs to be defined somewhere, it's not something which can be left floating as an 'implementation' detail! > * Section 5.1 Selection Criteria for IPFIX Protocol > The first paragraph of this section is not required now. But the > properties (5.1.1 and 5.1.2) still hold good. Agreed > * Section 5.1.3 IMO we can remove this. Agreed > * Section 5.1.4 Remove the line > "Once the selection is made from the set of candidate protocols, this > section would be replaced by the chosen protocol." > * We should add a some description on netflow v9 in section 5 not > getting into too much details. Yes indeed. Let's ask Benoit to provide some text for that (remembering that we only need an outline, and a reference to the *Protocol* document). > * Section 6.2 IPFIX end point authentication > Do we really need this section. Doesn't section 6.1.x take care of > what is mentioned in 6.2? Is there any plan now or in the future to > achieve this through netflow v9? Hmm. 6.2 does seem to be covered by 6.1.2, maybe we should move its text to there. 6.1.2 says that authentication could be achieved by transport-layer means such as IP Authentication Header, so there's not really any point in IPFIX doing this for itself! > * Section 6.3 Denial of service (DoS) attack prevention > This section is quite superficial and I'd suggest we either write > one or more concrete ways to achieve this or remove this section. Yes, it is superficial. But it's better than nothing. It ends with the remark "to be discussed on the general (i.e. IPFIX) list," so does anyone have any suitable text we could re-use for this? > * Section 7 Flow Expiration > This should appear as a subsection in IPFIX protocol. This seems very similar to section 4.5 (see above) - it's part of the metering process. How about combining 4.5 and 7 into a new section called 'The Metering Process;' it would go between the current sections 4 and 5 ???? Cheers, Nevil ----------------------------------------------------------------------- Nevil Brownlee Director, Technology Development Phone: +64 9 373 7599 x88941 ITSS, The University of Auckland FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand ------------------------------------------------- This mail sent through University of Auckland http://www.auckland.ac.nz/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu May 29 14:18:18 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA12773 for ; Thu, 29 May 2003 14:18:17 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19LROH-0006Rq-00 for ipfix-list@mil.doit.wisc.edu; Thu, 29 May 2003 12:40:37 -0500 Received: from halt-in.cisco.com ([171.70.144.185]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19LROE-0006Ri-00 for ipfix@net.doit.wisc.edu; Thu, 29 May 2003 12:40:34 -0500 Received: from cisco.com (171.71.163.13) by halt-in.cisco.com with ESMTP; 29 May 2003 10:40:14 -0800 Received: from cisco.com (dhcp-171-71-137-107.cisco.com [171.71.137.107]) by mira-sjc5-f.cisco.com (Mirapoint Messaging Server MOS 3.3.3-GR) with ESMTP id AGR73555; Thu, 29 May 2003 10:47:33 -0700 (PDT) Message-ID: <3ED64610.8BD96FE@cisco.com> Date: Thu, 29 May 2003 10:40:32 -0700 From: Ganesh Sadasivan Organization: Cisco Systems Inc X-Mailer: Mozilla 4.76 [en]C-CCK-MCD (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Nevil Brownlee CC: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Changes to architecture doc. References: <3EC55996.F610236@cisco.com> <1054172597.0d4147af570a5@hotlava.auckland.ac.nz> Content-Type: multipart/alternative; boundary="------------16BB50E09FFA21761F06ED78" Precedence: bulk Sender: majordomo listserver --------------16BB50E09FFA21761F06ED78 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Hi Nevil, See inline. Nevil Brownlee wrote: > Hello all: > > > * Now that we have chosen the IPFIX protocol, is a separate document > > needed for architecture and protocols? > > > > IMHO yes. The architecture discusses the basic terminology (like flow > > definition) components of IPFIX framework, different models and > > association between these components which are fairly static in nature. > > More over the chosen protocol may not fit into all the models described > > in the arch. spec. The protocol on the other hand discusses message > > etc which could keep evolving to newer versions. > > Agreed. *Architecture* defines terms (including 'IPFIX flow), and explains > how the overall system works. *Information Model* explains how IPFIX flow > data is made up, i.e. what data elements may be part of a flow, what they > mean, how they are represented, and how they are combined into a flow's > data structure. *Protocol* specifies how IPFIX flow data and control > information is carried (via a congestion-aware transport protocol) from > IPFIX exporter to IPFIX collector. [If you thought IPFIX is only about > how to transport flow data, have another look at the WG charter.] > > > * Section 2 : Scope > > There are a few points mentioned here : > > > > " * Define the criteria to select the IPFIX Protocol." > > We have already selected the protocol. Do we still need > > to keep this? > > We put that phrase into the Architecture document as a placeholder > until we'd selected a protocol as the starting point for IPFIX. > It's time to take it out. > > > " * Specify the control/data message formats and handshaking details > > to pass the IP flow information. " > > This is really protocol specific and should not be discussed in this > > document. > > Agreed. > > > * Section 4: IPFIX Device figure > > A device could have multiple exporters and each could have 1:n > > mapping with observation domain. This is not well captured in the > > figure. > > There are two diagrams in the introduction to section 4. Both seem > fairly clear, but I think that they both would benefit from a paragraph > or two of explanatory text. I'll write some, real soon now .. > > Also, there's a comment in the text about "interfaces shown as <~~>," > but I can't see any like that on the diagrams ??? Between "Collector [1]" and "[Application1] .. [Application n]". > > > > * Section 4.3 Observation Domain > > Functions of Observation domain. > > " * Encoding the flow records and sending them to the export process. > > * Encoding the control information into templates and sending them to > > the export process." > > The above 2 are already covered by IPFIX protocol. > > " * Deciding which flow records/control information to export using > > rules based on time, thresholds, configuration events etc. > > * Aggregating flow records generated by one or more metering > > processes." > > The above 2 are also IPFIX protocol function. So also is flow > > expiration. > > > > Observation point is a logical block which maintains flows for a > > a set of {observation point, metering process} (in a database form) > > and maintains aggregate statistics. All the other flow related > > functions should be handled by IPFIX protocol and metering process. > > No. Section 3 defines 'Observation Point' as "a location in the network > where IP packets can be observed," and that seems fine to me, let's > leave it like that. As for section 4.3's list of 'typical functions > of an observation domain,' I think it's helpful to keep the list there - > but we should also add a reference to the *Protocol* document. Oops that was a typing mistake . I meant "Observation Domain". > > > [Come to think of it, we should probably refer to the *Protocol* and > *Information Model* documents in quite a few other sections through > the *Architecture* document.] > Yes. > > > * Section 4.5 : > > " There MAY be additional rules defined within the observation domain > > so that only certain flows records are picked up for export. " > > Though rules are defined w.r.t observation domain, > > the entity that applies these rules is the IPFIX protocol. > > Surely it's the metering process that applies selection rules? After > all, if a flow is not going to be exported, there's no point in a meter > wasting resources on it?? > I was refering to the post-flow selection and pre-export phase. This is optional and may be implemented in some cases only. > > The metering process, i.e. deciding how to actually build flows by > executing a sequence of functions (with specified parameters) on > the observed packets, is surely part of the Information model. Or is > it? Maybe it fits better here in the *Architecture* document. > It certainly needs to be defined somewhere, it's not something which > can be left floating as an 'implementation' detail! The figure after 4.4.4 in the architecture doc provides some details of this. Do we want to go into more details? > > > > * Section 5.1 Selection Criteria for IPFIX Protocol > > The first paragraph of this section is not required now. But the > > properties (5.1.1 and 5.1.2) still hold good. > > Agreed > > > * Section 5.1.3 IMO we can remove this. > > Agreed > > > * Section 5.1.4 Remove the line > > "Once the selection is made from the set of candidate protocols, this > > section would be replaced by the chosen protocol." > > * We should add a some description on netflow v9 in section 5 not > > getting into too much details. > > Yes indeed. Let's ask Benoit to provide some text for that (remembering > that we only need an outline, and a reference to the *Protocol* document). I can write this up. > > > > * Section 6.2 IPFIX end point authentication > > Do we really need this section. Doesn't section 6.1.x take care of > > what is mentioned in 6.2? Is there any plan now or in the future to > > achieve this through netflow v9? > > Hmm. 6.2 does seem to be covered by 6.1.2, maybe we should move its > text to there. 6.1.2 says that authentication could be achieved by > transport-layer means such as IP Authentication Header, so there's > not really any point in IPFIX doing this for itself! > > > * Section 6.3 Denial of service (DoS) attack prevention > > This section is quite superficial and I'd suggest we either write > > one or more concrete ways to achieve this or remove this section. > > Yes, it is superficial. But it's better than nothing. It ends with > the remark "to be discussed on the general (i.e. IPFIX) list," so > does anyone have any suitable text we could re-use for this? > > > * Section 7 Flow Expiration > > This should appear as a subsection in IPFIX protocol. > > This seems very similar to section 4.5 (see above) - it's part of the > metering process. How about combining 4.5 and 7 into a new section > called 'The Metering Process;' it would go between the current sections > 4 and 5 ???? I am a little confused here. Here are my thoughts on Metering, Observation Domain & IPFIX protocol and please correct me if I'm wrong. +-----------------------------+----------------------------------------+ | packet header capturing | | | METERING PROCESS | | timestamping | | | | | v | | +----->+ | | | | | | | sampling Si (1:1 in case of no sampling) | | | | | | | classifying Fi (NULL when No criteria) | | | | | | +------+ | | | | +-----------------------------+----------------------------------------+ | v Flows | | | +-----------------------------+----------------------------------------+ | V OBSERVATION DOMAIN | | +----------------------+ +------------------+ | | | Flow data base |<--------|Provide non-flow | | | | (includes flows | | information (Eg. | | | | from all obs. | | router state) | | | | points in an obs. | +------------------+ | | | domain) | | | | | +------------------+ | | | |<--------|Maintain aggregate| | | | | | statistics | | | +----------------------+ +------------------+ | | | +-----------------------------+----------------------------------------+ | Flow Database +-----------------------------+----------------------------------------+ | V IPFIX PROTOCOL | | +----------------------+ | | | Rules for 1. sending | | | | templates 2. sending | | | | data records 3. timi-| | | | out flows 4. Encoding| | | | template & data. | | | | | | | | | | | | | | | +----------------------+ | | | +----------------------------------------------------------------------+ Thanks Ganesh > > > Cheers, Nevil > > ----------------------------------------------------------------------- > Nevil Brownlee Director, Technology Development > Phone: +64 9 373 7599 x88941 ITSS, The University of Auckland > FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand > > ------------------------------------------------- > This mail sent through University of Auckland > http://www.auckland.ac.nz/ --------------16BB50E09FFA21761F06ED78 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit  
Hi Nevil,
   See inline.

Nevil Brownlee wrote:

Hello all:

> * Now that we have chosen the IPFIX protocol, is a separate document
> needed for architecture and protocols?
>
> IMHO yes. The architecture discusses the basic terminology (like flow
> definition) components of IPFIX framework, different models and
> association between these components which are fairly static in nature.
> More over the chosen protocol may not fit into all the models described
> in the arch. spec. The protocol on the other hand discusses message
> etc which could keep evolving to newer versions.

Agreed.  *Architecture* defines terms (including 'IPFIX flow), and explains
how the overall system works.  *Information Model*  explains how IPFIX flow
data is made up, i.e. what data elements may be part of a flow, what they
mean, how they are represented, and how they are combined into a flow's
data structure.  *Protocol* specifies how IPFIX flow data and control
information is carried (via a congestion-aware transport protocol) from
IPFIX exporter to IPFIX collector. [If you thought IPFIX is only about
how to transport flow data, have another look at the WG charter.]

> * Section 2 : Scope
> There are a few points mentioned here :
>
> " * Define the criteria to select the IPFIX Protocol."
> We have already selected the protocol. Do we still need
> to keep this?

We put that phrase into the Architecture document as a placeholder
until we'd selected a protocol as the starting point for IPFIX.
It's time to take it out.

> " * Specify the control/data message formats and handshaking details
>   to pass the IP flow information. "
> This is really protocol specific and should not be discussed in this
> document.

Agreed.

> * Section 4: IPFIX Device figure
> A device could have multiple exporters and each could have 1:n
> mapping with observation domain. This is not well captured in the
> figure.

There are two diagrams in the introduction to section 4.  Both seem
fairly clear, but I think that they both would benefit from a paragraph
or two of explanatory text.  I'll write some, real soon now ..

Also, there's a comment in the text about "interfaces shown as <~~>,"
but I can't see any like that on the diagrams ???


Between "Collector [1]" and "[Application1] .. [Application n]".

 

> * Section 4.3 Observation Domain
> Functions of Observation domain.
> " * Encoding the flow records and sending them to the export process.
>   * Encoding the control information into templates and sending them to
>   the export process."
> The above 2 are already covered by IPFIX protocol.
> "  * Deciding which flow records/control information to export using
>      rules based on time, thresholds, configuration events etc.
>    * Aggregating flow records generated by one or more metering
>      processes."
> The above 2 are also IPFIX protocol function. So also is flow
> expiration.
>
> Observation point is a logical block which maintains flows for a
> a set of {observation point, metering process} (in a database form)
> and maintains aggregate statistics. All the other flow related
> functions should be handled by IPFIX protocol and metering process.

No.  Section 3 defines 'Observation Point' as "a location in the network
where IP packets can be observed," and that seems fine to me, let's
leave it like that.  As for section 4.3's list of 'typical functions
of an observation domain,' I think it's helpful to keep the list there -
but we should also add a reference to the *Protocol* document.

Oops that was a typing mistake . I meant "Observation Domain".
 

[Come to think of it, we should probably refer to the *Protocol* and
*Information Model* documents in quite a few other sections through
the *Architecture* document.]
 

Yes.
 
> * Section 4.5 :
> "  There MAY be additional rules defined within the observation domain
>    so that only certain flows records are picked up for export. "
> Though rules are defined w.r.t observation domain,
> the entity that applies these rules is the IPFIX protocol.

Surely it's the metering process that applies selection rules?  After
all, if a flow is not going to be exported, there's no point in a meter
wasting resources on it??
 

I was refering to the post-flow selection and pre-export phase. This is
optional and may be implemented in some cases only.
 
The metering process, i.e. deciding how to actually build flows by
executing a sequence of functions (with specified parameters) on
the observed packets, is surely part of the Information model.  Or is
it?  Maybe it fits better here in the *Architecture* document.
It certainly needs to be defined somewhere, it's not something which
can be left floating as an 'implementation' detail!
The figure after 4.4.4 in the architecture doc provides some details of this.
Do we want to go into more details?
 

> * Section 5.1 Selection Criteria for IPFIX Protocol
> The first paragraph of this section is not required now. But the
> properties (5.1.1 and 5.1.2) still hold good.

Agreed

> * Section 5.1.3 IMO we can remove this.

Agreed

> * Section 5.1.4 Remove the line
> "Once the selection is made from the set of candidate protocols, this
>  section would be replaced by the chosen protocol."
> * We should add a some description on netflow v9 in section 5 not
> getting into too much details.

Yes indeed.  Let's ask Benoit to provide some text for that (remembering
that we only need an outline, and a reference to the *Protocol* document).


I can write this up.
 

 

> * Section 6.2 IPFIX end point authentication
> Do we really need this section. Doesn't section 6.1.x take care of
> what is mentioned in 6.2? Is there any plan now or in the future to
> achieve this through netflow v9?

Hmm.  6.2 does seem to be covered by 6.1.2, maybe we should move its
text to there.  6.1.2 says that authentication could be achieved by
transport-layer means such as IP Authentication Header, so there's
not really any point in IPFIX doing this for itself!

> * Section 6.3 Denial of service (DoS) attack prevention
> This section is quite superficial and I'd suggest we either write
> one or more concrete ways to achieve this or remove this section.

Yes, it is superficial.  But it's better than nothing.  It ends with
the remark "to be discussed on the general (i.e. IPFIX) list," so
does anyone have any suitable text we could re-use for this?

> * Section 7 Flow Expiration
> This should appear as a subsection in IPFIX protocol.

This seems very similar to section 4.5 (see above) - it's part of the
metering process.  How about combining 4.5 and 7 into a new section
called 'The Metering Process;' it would go between the current sections
4 and 5 ????

I am a little confused here.
Here are my thoughts on Metering, Observation Domain & IPFIX protocol
and please correct me if I'm wrong.
 
 

   +-----------------------------+----------------------------------------+
   |              packet header capturing                                 |
   |                             |               METERING PROCESS         |
   |                        timestamping                                  |
   |                             |                                        |
   |                             v                                        |
   |                      +----->+                                        |
   |                      |      |                                        |
   |                      |   sampling Si (1:1 in case of no sampling)    |
   |                      |      |                                        |
   |                      | classifying Fi (NULL when No criteria)        |
   |                      |      |                                        |
   |                      +------+                                        |
   |                             |                                        |
   +-----------------------------+----------------------------------------+
                                 |
                                 v
                               Flows
                                 |
                                 |
                                 |
   +-----------------------------+----------------------------------------+
   |                             V    OBSERVATION DOMAIN                  |
   |             +----------------------+         +------------------+    |
   |             | Flow data base       |<--------|Provide non-flow  |    |
   |             | (includes flows      |         | information (Eg. |    |
   |             | from all obs.        |         | router state)    |    |
   |             | points in an obs.    |         +------------------+    |
   |             | domain)              |                                 |
   |             |                      |         +------------------+    |
   |             |                      |<--------|Maintain aggregate|    |
   |             |                      |         | statistics       |    |
   |             +----------------------+         +------------------+    |
   |                                                                      |
   +-----------------------------+----------------------------------------+
                                 |
                            Flow Database
   +-----------------------------+----------------------------------------+
   |                             V    IPFIX PROTOCOL                      |
   |             +----------------------+                                 |
   |             | Rules for 1. sending |                                 |
   |             | templates 2. sending |                                 |
   |             | data records 3. timi-|                                 |
   |             | out flows 4. Encoding|                                 |
   |             | template & data.     |                                 |
   |             |                      |                                 |
   |             |                      |                                 |
   |             |                      |                                 |
   |             +----------------------+                                 |
   |                                                                      |
   +----------------------------------------------------------------------+

Thanks
Ganesh

 

Cheers, Nevil

-----------------------------------------------------------------------
   Nevil Brownlee                   Director, Technology Development
   Phone: +64 9 373 7599 x88941     ITSS, The University of Auckland
   FAX: +64 9 373 7021      Private Bag 92019, Auckland, New Zealand

-------------------------------------------------
This mail sent through University of Auckland
http://www.auckland.ac.nz/

--------------16BB50E09FFA21761F06ED78-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri May 30 09:17:28 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA05643 for ; Fri, 30 May 2003 09:17:27 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19LjJ4-0006lH-00 for ipfix-list@mil.doit.wisc.edu; Fri, 30 May 2003 07:48:26 -0500 Received: from tokyo.ccrle.nec.de ([195.37.70.2]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19LjJ2-0006ky-00 for ipfix@net.doit.wisc.edu; Fri, 30 May 2003 07:48:24 -0500 Received: from venus.office (venus.office [10.1.1.11]) by tokyo.ccrle.nec.de (8.12.9/8.12.8) with ESMTP id h4UCm6VI048488; Fri, 30 May 2003 14:48:07 +0200 (CEST) Received: from [10.1.1.128] (n-quittek.office [10.1.1.128]) by venus.office (Postfix on SuSE Linux eMail Server 3.0) with ESMTP id 3336EA88F3; Fri, 30 May 2003 14:36:52 +0200 (CEST) Date: Fri, 30 May 2003 14:50:10 +0200 From: Juergen Quittek To: Sebastian Zander Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] IPFIX requirements - final changes Message-ID: <18754447.1054306210@[10.1.1.128]> In-Reply-To: <3ED56280.90204@fokus.fraunhofer.de> References: <3ED56280.90204@fokus.fraunhofer.de> X-Mailer: Mulberry/2.1.2 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Sebastian, -- Sebastian Zander wrote on 29 May 2003 03:29 +0200: > Juergen, > > Juergen Quittek wrote: >> Sebastian, >> >> -- Sebastian Zander wrote on 28 May 2003 12:18 +0200: >> >>> Hi Juergen, >>> >>> i think your second paragraph is a very good idea. But i don't >>> understand the last 2 sentences. My understanding is that the ipfix >>> requirement draft defines the ipfix protocol requirements. So i don't >>> understand why the protocol specification must cover all requirements. >>> E.g. when a requirementis optional why must it be covered in the protocol >>> specification? >> >> >> If the requirement doc says "the exporter MAY anonymize", then the >> protocol specification MUST support anonymization in order to make >> sure, that anonymization is interoperable. For example the protocol >> spec MUST define how to inform a collecting process that the data it >> receives is anonymized. Analogously, the protocol spec should specify >> how to encrypt. >> >> In both cases, the protocol specification can state that a concrete >> implementation of the protocol MAY support features that serve >> anonymization. Then an implementor can choose whether or not to implement >> this OPTIONAL feature of the protocol. > > You are right. The req draft contains protocol specification requirements > (which is sometimes confusing and the reason why we get those comments). > But i suggest to change the middle sentence saying that all protocol > specification requirements from the req draft must be taken over into the > protocol draft *including their significance*. Otherwise why do we spend a > long time discussing the significance of some reqs if it is not used later > in the protocol draft? Furthermore some of the reqs may be targeting the > information model or achitecture draft instead of the protocol spec. So > the sentence should not only mention the protocol spec but *also the > information model etc*. (Jeff just gave an example that anonymization may > influence the information model.) I see your point, but I would suggest a different phrasing for the following reason: - I do not expect that requirement items can be mapped one-to-one onto protocol features. Some individual protocol feature will cover several requirement items at once. So here you cannot map the significance one-to-one, but need something like a minimum significance to be considered. - My experience from designing other protocols is that - compared to the requirements - you usually increase significance of some features for technical or orther reasons. We must make sure that the protocol definition process has this level of freedom. Therefore, I suggest to extend my initially suggested paragraph "Many requirements in this document are not explicitly stated as IPFIX protocol requirements, but as requirements for the metering process, the exporting process, or for other traffic measurement components. However, every requirement that needs support from the IPFIX protocol MUST be covered by the IPFIX protocol specification independent of the significance of the requirement, which can be MANDATOYRY, RECOMMENDED, or OPTIONAL. Note that the protocol specification itself also assigns significance attributes to its features, such that a protocol implementation does not necessarily need to implement all features." by appending this sentence or a similar one: "However, the significance assigned to a protocol feature MUST NOT be lower than the significance of the corresponding requirements." Cheers, Juergen > This statement must be put in the very beginning of the req draft. > > The last sentence is not necessary in my opinion because its pretty obvious > that the protocol draft and the other drafts will use more (and more > detailed) reqs. > > Then there is no need to change the requirements to MUST. > > Cheers, > > Sebastian > >>> I agree to say its a MUST for the protocol but not all exporters >>> and collectors must support it. >> >> >> Agreed. >> >>> But i do *not* agree at all that in general anonymization and >>> confidentiality is not needed in a LAN. Anonymization can be required >>> in a LAN and even in a LAN confidentiality may be needed because i >>> don't want everybody who has access to my LAN to have access to my >>> IPFIX data. >> >> >> Also agreed. >> >> Best wishes, >> >> Juergen >> >>> Cheers, >>> >>> Sebastian >>> >>> Juergen Quittek wrote: >>> >>>> Tal, Reinaldo, and Benoit, >>>> >>>> Do we all have the following in mind? >>>> >>>> Anonymization and confidentiality MUST be covered by the >>>> protocol specification. But the protocol specifiction >>>> specification defines it as a MAY feature (anonymization) >>>> or SHOULD feature (confidentiality) for IPFIX protocol >>>> implementations. >>>> >>>> I guess this is what the IESG reviewers were referring to. >>>> >>>> What is probably missing in the requirements document is a >>>> statement like the following: >>>> >>>> Many requirements in this document are not explicitly >>>> stated as IPFIX protocol requirements, but as requirements >>>> for the metering process, the exporting process, or for >>>> other traffic measurement components. However, every >>>> requirement that needs support from the IPFIX protocol >>>> MUST be covered by the IPFIX protocol specification >>>> independent of the significance of the requirement, >>>> which can be MANDATOYRY, RECOMMENDED, or OPTIONAL. >>>> Note that the protocol specification itself also assigns >>>> significance attributes to its features, such that a >>>> protocol implementation does not necessarily need to >>>> implement all features. >>>> >>>> Any comment on this? >>>> >>>> Juergen >>>> >>>> >>>> -- Tal Givoly wrote on 26 May 2003 09:42 -0700: >>>> >>>>> Juergen, >>>>> >>>>> I agree with Benoit - it doesn't make sense to make confidentiality and >>>>> anonymization mandatory. In many deployments, there could be >>>>> physical and >>>>> other security measures that addresses these issues. >>>>> >>>>> Tal >>>>> >>>>> -----Original Message----- >>>>> From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu]On >>>>> Behalf >>>>> Of Benoit Claise >>>>> Sent: Monday, May 26, 2003 8:02 AM >>>>> To: Juergen Quittek >>>>> Cc: ipfix@net.doit.wisc.edu >>>>> Subject: Re: [ipfix] IPFIX requirements - final changes >>>>> >>>>> >>>>> Juergen, >>>>> >>>>>> ======================================================================== >>>>>> >>>>>> 12. Confidentiality from SHOULD to MUST >>>>>> ======================================================================== >>>>>> >>>>>> Problem Description: >>>>>> ------------------------------------------------------------------------ >>>>>> >>>>>> Requirements on the ipfix implementation - the document is long and I >>>>>> wonder if >>>>>> the working group really meant the protocol's confidentiality and >>>>>> anonymization >>>>>> features to be so optional - SHOULD confidentiality, MAY >>>>>> anonymization. >>>>>> Just for implementation. >>>>>> ======================================================================== >>>>>> >>>>>> Suggested solution: >>>>>> ------------------------------------------------------------------------ >>>>>> >>>>>> change confidentiality requirement in section 6.3.3 >>>>>> from SHOULD to MUST >>>>>> ======================================================================== >>>>>> >>>>>> Status: solution to be agreed on >>>>>> ======================================================================== >>>>>> >>>>> >>>>> >>>>> >>>>> I'm just wondering if a MUST is not a little bit extreme. >>>>> The draft would become: >>>>> "Confidentiality of flow specific data transferred from an exporting >>>>> process to a collecting process MUST be ensured." >>>>> We all agree that over the Internet confidentiality is compulsory but >>>>> this sentence would mean that even in a private LAN or environment, we >>>>> must have confidentiality to be IPFIX compliant. And we know that, with >>>>> the software based encryption, the throughput would be lowered. Ok, >>>>> then >>>>> we should have an crypto engine in hardware on the exporter to export >>>>> the numerous flow export packets but there is a price to it. >>>>> My point is that, as Confidentiality is not needed in all cases, why >>>>> make it a MUST? >>>>> >>>>>> >>>>>> >>>>>> >>>>>> ======================================================================== >>>>>> >>>>>> 13. Anonymization from MAY to MUST >>>>>> ======================================================================== >>>>>> >>>>>> Problem Description: >>>>>> ------------------------------------------------------------------------ >>>>>> >>>>>> Requirements on the ipfix implementation - the document is long and I >>>>>> wonder if >>>>>> the working group really meant the protocol's confidentiality and >>>>>> anonymization >>>>>> features to be so optional - SHOULD confidentiality, MAY >>>>>> anonymization. >>>>>> Just for implementation. >>>>>> ======================================================================== >>>>>> >>>>>> Suggested solution: >>>>>> ------------------------------------------------------------------------ >>>>>> >>>>>> change anonymization requirement in section 6.7 from MAY to MUST >>>>>> ======================================================================== >>>>>> >>>>>> Status: solution to be agreed on >>>>>> ======================================================================== >>>>>> >>>>> >>>>> >>>>> >>>>> Again, I have the same type of comments. Anonymization is not required >>>>> in all cases. So why make it a MUST? >>>>> >>>>> Regards, Benoit. >>>>> >>>>> >>>>> >>>>> -- >>>>> Help mailto:majordomo@net.doit.wisc.edu and say "help" in >>>>> message >>>>> body >>>>> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >>>>> "unsubscribe ipfix" in message body >>>>> Archive http://ipfix.doit.wisc.edu/archive/ >>>>> >>>> >>>> >>>> >>>> -- >>>> Help mailto:majordomo@net.doit.wisc.edu and say "help" in message >>>> body >>>> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >>>> "unsubscribe ipfix" in message body >>>> Archive http://ipfix.doit.wisc.edu/archive/ >>>> >>> >>> >>> -- >>> Sebastian Zander E-mail: >>> zander@fokus.fraunhofer.de >>> Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 >>> Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 >>> D-10589 Berlin, Germany >>> www.fokus.fraunhofer.de/usr/sebastian.zander >>> >>> >>> >>> >> >> >> > > > -- > Sebastian Zander E-mail: zander@fokus.fraunhofer.de > Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 > Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 > D-10589 Berlin, Germany www.fokus.fraunhofer.de/usr/sebastian.zander > > > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri May 30 09:58:59 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA07754 for ; Fri, 30 May 2003 09:58:59 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19Lk54-00007M-00 for ipfix-list@mil.doit.wisc.edu; Fri, 30 May 2003 08:38:02 -0500 Received: from tokyo.ccrle.nec.de ([195.37.70.2]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19Lk51-00007C-00 for ipfix@net.doit.wisc.edu; Fri, 30 May 2003 08:37:59 -0500 Received: from venus.office (venus.office [10.1.1.11]) by tokyo.ccrle.nec.de (8.12.9/8.12.8) with ESMTP id h4UDbdVI051912; Fri, 30 May 2003 15:37:39 +0200 (CEST) Received: from [10.1.1.128] (n-quittek.office [10.1.1.128]) by venus.office (Postfix on SuSE Linux eMail Server 3.0) with ESMTP id 1AE22A895D; Fri, 30 May 2003 15:26:24 +0200 (CEST) Date: Fri, 30 May 2003 15:39:43 +0200 From: Juergen Quittek To: Ganesh Sadasivan , ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Changes to architecture doc. Message-ID: <21726941.1054309183@[10.1.1.128]> In-Reply-To: <3EC55996.F610236@cisco.com> References: <3EC55996.F610236@cisco.com> X-Mailer: Mulberry/2.1.2 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Ganesh -- Ganesh Sadasivan wrote on 16 May 2003 14:35 -0700: > * Section 4.3 Observation Domain > Functions of Observation domain. > " * Encoding the flow records and sending them to the export process. > * Encoding the control information into templates and sending them to > the export process." > The above 2 are already covered by IPFIX protocol. > > " * Deciding which flow records/control information to export using > rules based on time, thresholds, configuration events etc. > * Aggregating flow records generated by one or more metering > processes." > The above 2 are also IPFIX protocol function. So also is flow > expiration. > > Observation point is a logical block which maintains flows for a Here you mean observation domain, not observation point. > a set of {observation point, metering process} (in a database form) > and maintains aggregate statistics. All the other flow related > functions should be handled by IPFIX protocol and metering process. > I have a problem with the concept of an observation domain. You say it is a 'functional block', but in the figure it includes observation points which are locations in the network and no functional blocks. So the concept of the observation domain is not a clean one. I see an observation domain as a useful concept for implementation design of IPFIX on a device with multiple line cards, but not useful for the general IPFIX architecture. Also I do not see a reason, why the general architecture should define an internal function block for "sending them [flow records] from [the metering process] to the exporting process". Why sending them, if they are co-located? Whether or not you locate metering process and exporting process on different boards and need to send them over an internal bus is a detail of an implementation design, but IMHO should not be part of the general IPFIX architecture. And we are definitely not defining a netwrok protocol between metering process and exporting process. The requirements do intentionally not discuss how flow records are shared between metering process and exporting process, because there are many different potential hardware and software device architectures that require different ways of communication between these processes. I do not think the IPFIX architecture should request that the there is an instance sending flow records from one process to the other, because there are several alternatives which would become excluded without need, for example shared memory (including remote memory access from the exporter to a metering card). In the requirements document we mapped requirements concerning the reports to the collecting process (for example the reporting period) to the section of requirements for the exporting process. Consequently, I would see several of the functions you listed for the observation records as part of the exporting process (for example generating templates). For all these reasons I suggest to use the following maping of functions to metering process and exporting process. The metering process is a process triggered by packet arrival. It performs time critical wire-speed processing of packets and creating/updating of the corresponding flow records. Compared to the metering process, the export process is a rather low speed process triggered by flow timeouts and reporting interval timers (and probably by some other events). It encodes flow records into packets, generates templates and sends everything securely to the collecting process. Now, coming back to the point that probably was a motivation for inventing the observation domain: If you want to split the exporting process in a portion that acts on router line cards (generating templetes and reports and a part acting on the central processor board of a router (securely exporting all reports generated on different line cards), then I suggest to split the exporting process (as defined in the requirements) into a reporting process and sending process (you might like to change terms). Then the chain of observation point, metering process and exporting process O - M - E would then be replaced by the chain observation point, metering process, reporitng process and sending process O - M - R - S Advantages: - it is in line with the requirements - it is consistent (not merging functions and locations) - it still allows you to place parts of the exporting process on line cards, while the rest remains central - it is much in line with the PSAMP architecture increasing synergy between both WGs Cheers, Juergen -- Juergen Quittek quittek@ccrle.nec.de Tel: +49 6221 90511-15 NEC Europe Ltd., Network Laboratories Fax: +49 6221 90511-55 Kurfuersten-Anlage 36, 69115 Heidelberg, Germany http://www.ccrle.nec.de -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri May 30 10:28:30 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA10284 for ; Fri, 30 May 2003 10:28:29 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19Lkkm-00015m-00 for ipfix-list@mil.doit.wisc.edu; Fri, 30 May 2003 09:21:08 -0500 Received: from tokyo.ccrle.nec.de ([195.37.70.2]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19Lkkk-00015g-00 for ipfix@net.doit.wisc.edu; Fri, 30 May 2003 09:21:06 -0500 Received: from venus.office (venus.office [10.1.1.11]) by tokyo.ccrle.nec.de (8.12.9/8.12.8) with ESMTP id h4UEL4VI055269; Fri, 30 May 2003 16:21:04 +0200 (CEST) Received: from [10.1.1.128] (n-quittek.office [10.1.1.128]) by venus.office (Postfix on SuSE Linux eMail Server 3.0) with ESMTP id B70DEA89AF; Fri, 30 May 2003 16:09:49 +0200 (CEST) Date: Fri, 30 May 2003 16:23:08 +0200 From: Juergen Quittek To: Sandra Tartarelli , ipfix@net.doit.wisc.edu Subject: Re: [ipfix] IPFIX reqs appendix: consistency check Message-ID: <24332478.1054311788@[10.1.1.128]> In-Reply-To: <3ECDE9FA.8050705@ccrle.nec.de> References: <3ECDE9FA.8050705@ccrle.nec.de> X-Mailer: Mulberry/2.1.2 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Dear all, There have been no objections to Sandra's comments on the appendix of the requirements document. I will commit all changes she suggested. Juergen -- Juergen Quittek quittek@ccrle.nec.de Tel: +49 6221 90511-15 NEC Europe Ltd., Network Laboratories Fax: +49 6221 90511-55 Kurfuersten-Anlage 36, 69115 Heidelberg, Germany http://www.ccrle.nec.de -- Sandra Tartarelli wrote on 23 May 2003 11:29 +0200: > Dear all, > > I checked the consistency of the appendix of the IPFIX requirements document. This was listed as open issue in Juergen's email "IPFIX requirement: final changes": > >> ======================================================================== >> 7. consistency of appendix >> ======================================================================== >> Problem Description: >> ------------------------------------------------------------------------ >> The appendix is outdated. Alignment with the text sections and >> coverage of all requirements need to be checked and fixed. >> ======================================================================== >> Suggested solution: >> ------------------------------------------------------------------------ >> no suggestion, yet >> ======================================================================== >> Status: solution required >> ======================================================================== >> > > In the following, you find my suggestions to update it. Part of the suggestions consider the comments in Juergen's email "IPFIX requirement: final changes". > > Best regards, > Sandra > > ============================================================================== > --- 5.2 Sampling > > Add note (j): "If sampling is supported, sampling configuration changes MUST be indicated to all collecting processes" > > --- 5.3 Overload behavior > > Add note (k): "If overload is supported and it induces changes in the metering process behavior, the overload behavior MUST be clearly defined" > > --- Requirement on packet fragmentation state at the meter is missing (section 5.8) > > Add "5.8 Packet Fragmentation state O O - - - O" > > --- 6.1 Byte counter > > Add note (l): "which bytes of a packet are counted MUST be defined clearly" > This note is not required if the suggestion from Juergen to replace the byte counter definition is accepted. > From Juergen's email: > >> More concrete definition of byte counter >> old text: >> 8. byte counter >> Which bytes of a packet are counted MUST be >> defined exactly." >> suggested replacement: >> 8. byte counter: >> The sum of the total length of all IP packets >> belonging to the flow. The total length of a >> packet covers IP header and IP payload. > > > > --- 6.1 Dropped packet counter: This requirement appears as a MUST in the appendix, but it is a MAY in current section 6.1 > > Replace with: "6.1 Dropped packet counter (h,i) O O O O O O" > > --- 6.1 ToS byte and DSCP: they appear as separate requirements in the appendix. Besides, Traffic Class (IPv6) is not mentioned. > However, Juergen wrote: > >> On page 14 I see: >> >> 9. type of service octet (in case of IPv4), traffic class >> octet (in case of IPv6) >> >> Is that not better state as >> >> 9. DSCP, which is set in the type of service octet (in case of >> IPv4), >> or in the traffic class octet (in case of IPv6) > > > > > If this is accepted, then remove line "6.1 ToS Byte M S M O M M" and leave only "6.1 DSCP (a) M S M O M M" > > --- 6.1 BGP AS# is given as MUST in the appendix, but it is a MAY in current section 6.1 > If the following suggestion from Juergen is accepted: > >> - More concrete definition of BGP-related attributes >> old text: >> In addition, the exporting process MAY be able to >> report attributes related to inter-autonomous >> system routing of a flow, for example by reporting >> BGP Autonomous System numbers [RFC1771]. >> suggestion: replace text by three new MAY attributes >> 27. source BGP Autonomous System number [RFC1771] >> 28. destination BGP Autonomous System number >> 29. next hop BGP Autonomous System number > > > > > > then replace "6.1 BGP AS# - S M - - M" with the following line: > "6.1 src/dst/next hop BGP AS # - O O - - O" > > --- For the Information Model, requirements on ICMP type and in/out interface are missing: > > Add "6.1 ICMP type and code (l) S S - S S S" > > Add note: "(l) If protocol type is ICMP" > > > Add "6.1 input/output interface (m) S S S S S S" > > Add note: "(m) This requirement does not apply if the observation point is located at a probe device" > > --- For the Data Model the following requirement is missing: > > "6.2 Independency of the transport protocol S S S S S S" > > --- "6.7 Anonymization O O O O O O" > > Add note: "(n) anonymization MUST be clearly indicated to all receiving collecting processes" > > Juergen suggested: > >> - Anonymization from MAY to MUST >> suggestion: Change anonymization requirement in >> section 6.7 from MAY to MUST. > > > > >If Juergen's suggestion is accepted then also change the req. into: "6.7 Anonymization M M M M M M" > > --- Configuration requirements for metering process: > > Add "7.1 Config Overload behavior (a) O O O O O O" > > --- Configuration reqs. for exporting process: > > Config Notifications and Config Anonymization should be changed to SHOULD (they are listed as MAY) > > >Add the following missing reqs: > >"Config Collecting Processes S S S S S S" > >"Config reporting interval (a) S S S S S S" > > > > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri May 30 11:48:21 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA13911 for ; Fri, 30 May 2003 11:48:21 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19LlrH-0002gA-00 for ipfix-list@mil.doit.wisc.edu; Fri, 30 May 2003 10:31:55 -0500 Received: from halt-in.cisco.com ([171.70.144.185]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19LlrF-0002g5-00 for ipfix@net.doit.wisc.edu; Fri, 30 May 2003 10:31:53 -0500 Received: from cisco.com (171.71.163.13) by halt-in.cisco.com with ESMTP; 30 May 2003 08:31:56 -0800 Received: from cisco.com (sjc-vpn1-550.cisco.com [10.21.98.38]) by mira-sjc5-f.cisco.com (Mirapoint Messaging Server MOS 3.3.3-GR) with ESMTP id AGS73296; Fri, 30 May 2003 08:38:50 -0700 (PDT) Message-ID: <3ED77965.4012EF7@cisco.com> Date: Fri, 30 May 2003 08:31:49 -0700 From: Ganesh Sadasivan Organization: Cisco Systems Inc X-Mailer: Mozilla 4.79 [en]C-CCK-MCD (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Juergen Quittek CC: ipfix@net.doit.wisc.edu Subject: Observation Domain [ Was Re: [ipfix] Changes to architecture doc.] References: <3EC55996.F610236@cisco.com> <21726941.1054309183@[10.1.1.128]> Content-Type: multipart/alternative; boundary="------------313DDE62E878B4F031A942F6" Precedence: bulk Sender: majordomo listserver --------------313DDE62E878B4F031A942F6 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Juergen, Juergen Quittek wrote: > Ganesh > > -- Ganesh Sadasivan wrote on 16 May 2003 14:35 -0700: > > > * Section 4.3 Observation Domain > > Functions of Observation domain. > > " * Encoding the flow records and sending them to the export process. > > * Encoding the control information into templates and sending them to > > the export process." > > The above 2 are already covered by IPFIX protocol. > > > > " * Deciding which flow records/control information to export using > > rules based on time, thresholds, configuration events etc. > > * Aggregating flow records generated by one or more metering > > processes." > > The above 2 are also IPFIX protocol function. So also is flow > > expiration. > > > > Observation point is a logical block which maintains flows for a > > Here you mean observation domain, not observation point. Yes. sorry for the confusion. > > > > a set of {observation point, metering process} (in a database form) > > and maintains aggregate statistics. All the other flow related > > functions should be handled by IPFIX protocol and metering process. > > > > I have a problem with the concept of an observation domain. You say it > is a 'functional block', but in the figure it includes observation points > which are locations in the network and no functional blocks. So the > concept of the observation domain is not a clean one. I see an observation > domain as a useful concept for implementation design of IPFIX on a > device with multiple line cards, but not useful for the general IPFIX > architecture. The idea is to present a single identity for a group of observation points to the exporter. That is why it is mentioned as a logical block which encompasses a set of observation points. It does not matter whether the observation points are in a single board or different board or even different nodes. It is a question as to how the collector identifies a group of observation points. Ofcourse a degenrate case is a single observation point in an observation domain. > > > Also I do not see a reason, why the general architecture should define > an internal function block for "sending them [flow records] from [the > metering process] to the exporting process". Why sending them, if they > are co-located? Whether or not you locate metering process and exporting > process on different boards and need to send them over an internal bus > is a detail of an implementation design, but IMHO should not be part > of the general IPFIX architecture. And we are definitely not defining > a netwrok protocol between metering process and exporting process. > I am copying the figure down once again with slight modifications for clarity and please explain how this restricts to a class of implementtations. +-----------------------------+-------------------------------------+ | packet header capturing | | | METERING PROCESS | | timestamping on an | | | observation point | | v | | +----->+ | | | | | | | sampling Si (1:1 in case of no sampling) | | | | | | | classifying Fi (NULL when No criteria) | | | | | | +------+ | | | | +-----------------------------+-------------------------------------+ | v . Flows . +--------------------------------------+... | | +-----------------------------+----------------------------------------+ | V OBSERVATION DOMAIN | | +----------------------+ +------------------+ | | | Flow data base |<--------|Provide non-flow | | | | (includes flows | | information (Eg. | | | | from all obs. | | router state) | | | | points in an obs. | +------------------+ | | | domain) | | | | | +------------------+ | | | |<--------|Maintain aggregate| | | | | | statistics | | | +----------------------+ +------------------+ | | | +-----------------------------+----------------------------------------+ | Flow Database (identified by observation domain) +-----------------------------+----------------------------------------+ | V IPFIX PROTOCOL | | +----------------------+ | | | Rules for 1. picking&| | | | sending templates | | | | 2.picking& sending | | | | data records 3. timi-| | | | out flows 4. Encoding| | | | template & data. | | | | | | | | | | | | | | | +----------------------+ | | | +----------------------------------------------------------------------+ > > The requirements do intentionally not discuss how flow records are > shared between metering process and exporting process, because there > are many different potential hardware and software device architectures > that require different ways of communication between these processes. > I do not think the IPFIX architecture should request that the there > is an instance sending flow records from one process to the other, > because there are several alternatives which would become excluded > without need, for example shared memory (including remote memory access > from the exporter to a metering card). > The idea is that any function mentioned in the figure above as part of an observation domain is optional, but if such a functionality exists, it exists in the context of an observation domain. It is like pieces in a metering process. Not every one would want to do sampling and classifying. But they are still listed as part of metering. > > In the requirements document we mapped requirements concerning the > reports to the collecting process (for example the reporting period) > to the section of requirements for the exporting process. Consequently, > I would see several of the functions you listed for the observation > records as part of the exporting process (for example generating templates). If you are refering to section 4.3 of the arch. spec, yes they specify functions which are not done by the observation domain. I had mentioned in my first e-mail on this topic "> * Section 4.3 Observation Domain > Functions of Observation domain. > " * Encoding the flow records and sending them to the export process. > * Encoding the control information into templates and sending them to > the export process." > The above 2 are already covered by IPFIX protocol. > " * Deciding which flow records/control information to export using > rules based on time, thresholds, configuration events etc. > * Aggregating flow records generated by one or more metering > processes." > The above 2 are also IPFIX protocol function. So also is flow > expiration. " But then we have to mention that IPFIX protocol happens in the context of export process. > > > For all these reasons I suggest to use the following maping of functions > to metering process and exporting process. > > The metering process is a process triggered by packet arrival. It performs > time critical wire-speed processing of packets and creating/updating of > the corresponding flow records. > > Compared to the metering process, the export process is a rather low speed > process triggered by flow timeouts and reporting interval timers (and > probably by some other events). It encodes flow records into packets, > generates templates and sends everything securely to the collecting process. > Agreed. > > Now, coming back to the point that probably was a motivation for inventing > the observation domain: If you want to split the exporting process in > a portion that acts on router line cards (generating templetes and reports > and a part acting on the central processor board of a router (securely > exporting all reports generated on different line cards), then I suggest > to split the exporting process (as defined in the requirements) into a > reporting process and sending process (you might like to change terms). If observation domain were to do functions in 4.3 yes then there a good reason. But IMO is not the function of observation domain. As far the export process is concerened (which includes the IPFIX protocol), itcan be split the way you mentioned above. But not mentioning this explicitly does not limit the implementation on distributed systems (or does it?) . Thanks Ganesh > > > > Then the chain of observation point, metering process and exporting > process > > O - M - E > > would then be replaced by the chain observation point, metering process, > reporitng process and sending process > > O - M - R - S > > Advantages: > - it is in line with the requirements > - it is consistent (not merging functions and locations) > - it still allows you to place parts of the exporting > process on line cards, while the rest remains central > - it is much in line with the PSAMP architecture > increasing synergy between both WGs > > Cheers, > > Juergen > -- > Juergen Quittek quittek@ccrle.nec.de Tel: +49 6221 90511-15 > NEC Europe Ltd., Network Laboratories Fax: +49 6221 90511-55 > Kurfuersten-Anlage 36, 69115 Heidelberg, Germany http://www.ccrle.nec.de > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ --------------313DDE62E878B4F031A942F6 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit Juergen,
 

Juergen Quittek wrote:

Ganesh

-- Ganesh Sadasivan wrote on 16 May 2003 14:35 -0700:

> * Section 4.3 Observation Domain
> Functions of Observation domain.
> " * Encoding the flow records and sending them to the export process.
>   * Encoding the control information into templates and sending them to
>   the export process."
> The above 2 are already covered by IPFIX protocol.
>
> "  * Deciding which flow records/control information to export using
>      rules based on time, thresholds, configuration events etc.
>    * Aggregating flow records generated by one or more metering
>      processes."
> The above 2 are also IPFIX protocol function. So also is flow
> expiration.
>
> Observation point is a logical block which maintains flows for a

Here you mean observation domain, not observation point.

Yes. sorry for the confusion.
 

> a set of {observation point, metering process} (in a database form)
> and maintains aggregate statistics. All the other flow related
> functions should be handled by IPFIX protocol and metering process.
>

I have a problem with the concept of an observation domain.  You say it
is a 'functional block', but in the figure it includes observation points
which are locations in the network and no functional blocks.  So the
concept of the observation domain is not a clean one.  I see an observation
domain as a useful concept for implementation design of IPFIX on a
device with multiple line cards, but not useful for the general IPFIX
architecture.


The idea is to present a single identity for a group of observation points
to the exporter. That is why it is mentioned as a logical block which
encompasses a set of observation points. It does not matter whether the
observation points are in a single board or different board or even different
nodes. It is a question as to how the collector identifies a group of observation
points.
Ofcourse a degenrate case is a single observation point in an observation
domain.

 

Also I do not see a reason, why the general architecture should define
an internal function block for "sending them [flow records] from [the
metering process] to the exporting process".  Why sending them, if they
are co-located?  Whether or not you locate metering process and exporting
process on different boards and need to send them over an internal bus
is a detail of an implementation design, but IMHO should not be part
of the general IPFIX architecture. And we are definitely not defining
a netwrok protocol between metering process and exporting process.
 

I am copying the figure down once again with slight modifications for clarity
and please explain how this restricts to a class of implementtations.

   +-----------------------------+-------------------------------------+
   |              packet header capturing                              |
   |                             |               METERING PROCESS      |
   |                        timestamping         on an                 |
   |                             |               observation point     |
   |                             v                                     |
   |                      +----->+                                     |
   |                      |      |                                     |
   |                      |   sampling Si (1:1 in case of no sampling) |
   |                      |      |                                     |
   |                      | classifying Fi (NULL when No criteria)     |
   |                      |      |                                     |
   |                      +------+                                     |
   |                             |                                     |
   +-----------------------------+-------------------------------------+
                                 |
                                 v                                      .
                               Flows                                    .
                                 +--------------------------------------+...
                                 |
                                 |
   +-----------------------------+----------------------------------------+
   |                             V    OBSERVATION DOMAIN                  |
   |             +----------------------+         +------------------+    |
   |             | Flow data base       |<--------|Provide non-flow  |    |
   |             | (includes flows      |         | information (Eg. |    |
   |             | from all obs.        |         | router state)    |    |
   |             | points in an obs.    |         +------------------+    |
   |             | domain)              |                                 |
   |             |                      |         +------------------+    |
   |             |                      |<--------|Maintain aggregate|    |
   |             |                      |         | statistics       |    |
   |             +----------------------+         +------------------+    |
   |                                                                      |
   +-----------------------------+----------------------------------------+
                                 |
                            Flow Database (identified by observation domain)
   +-----------------------------+----------------------------------------+
   |                             V    IPFIX PROTOCOL                      |
   |             +----------------------+                                 |
   |             | Rules for 1. picking&|                                 |
   |             | sending templates    |                                 |
   |             | 2.picking& sending   |                                 |
   |             | data records 3. timi-|                                 |
   |             | out flows 4. Encoding|                                 |
   |             | template & data.     |                                 |
   |             |                      |                                 |
   |             |                      |                                 |
   |             |                      |                                 |
   |             +----------------------+                                 |
   |                                                                      |
   +----------------------------------------------------------------------+

 
The requirements do intentionally not discuss how flow records are
shared between metering process and exporting process, because there
are many different potential hardware and software device architectures
that require different ways of communication between these processes.
I do not think the IPFIX architecture should request that the there
is an instance sending flow records from one process to the other,
because there are several alternatives which would become excluded
without need, for example shared memory (including remote memory access
from the exporter to a metering card).
 
 The idea is that any  function mentioned in the figure above
as part of an observation domain is optional, but if such a functionality exists,
it exists in the context of an observation domain. It is like pieces in a metering
process. Not every one would want to do sampling and classifying. But they
are still listed as part of metering.
 
In the requirements document we mapped requirements concerning the
reports to the collecting process (for example the reporting period)
to the section of requirements for the exporting process. Consequently,
I would see several of the functions you listed for the observation
records as part of the exporting process (for example generating templates).
If you are refering to section 4.3 of the arch. spec, yes they specify functions which
are not done by the observation domain. I had mentioned in my first e-mail
on this topic
"> * Section 4.3 Observation Domain
  > Functions of Observation domain.
  > " * Encoding the flow records and sending them to the export process.
  >   * Encoding the control information into templates and sending them to
  >   the export process."
  > The above 2 are already covered by IPFIX protocol.
  > "  * Deciding which flow records/control information to export using
  >      rules based on time, thresholds, configuration events etc.
  >    * Aggregating flow records generated by one or more metering
  >      processes."
  > The above 2 are also IPFIX protocol function. So also is flow
  > expiration. "
But then we have to mention that IPFIX protocol happens in the context of
export process.
 

For all these reasons I suggest to use the following maping of functions
to metering process and exporting process.

The metering process is a process triggered by packet arrival. It performs
time critical wire-speed processing of packets and creating/updating of
the corresponding flow records.

Compared to the metering process, the export process is a rather low speed
process triggered by flow timeouts and reporting interval timers (and
probably by some other events). It encodes flow records into packets,
generates templates and sends everything securely to the collecting process.
 

Agreed.
 
Now, coming back to the point that probably was a motivation for inventing
the observation domain:  If you want to split the exporting process in
a portion that acts on router line cards (generating templetes and reports
and a part acting on the central processor board of a router (securely
exporting all reports generated on different line cards), then I suggest
to split the exporting process (as defined in the requirements) into a
reporting process and sending process (you might like to change terms).
If observation domain were to do functions in 4.3 yes then there a good reason.
But IMO is not the function of observation domain. As far the export process
is concerened (which includes the IPFIX protocol), itcan be split the way
you mentioned above. But not mentioning this explicitly does not limit the
implementation on distributed systems (or does it?) .

Thanks
Ganesh

 
 

Then the chain of observation point, metering process and exporting
process

  O - M - E

would then be replaced by the chain observation point, metering process,
reporitng process and sending process

  O - M - R - S

Advantages:
  - it is in line with the requirements
  - it is consistent (not merging functions and locations)
  - it still allows you to place parts of the exporting
    process on line cards, while the rest remains central
  - it is much in line with the PSAMP architecture
    increasing synergy between both WGs

Cheers,

    Juergen
--
Juergen Quittek        quittek@ccrle.nec.de        Tel: +49 6221 90511-15
NEC Europe Ltd.,       Network Laboratories        Fax: +49 6221 90511-55
Kurfuersten-Anlage 36, 69115 Heidelberg, Germany   http://www.ccrle.nec.de

--
Help        mailto:majordomo@net.doit.wisc.edu and say "help" in message body
Unsubscribe mailto:majordomo@net.doit.wisc.edu and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

--------------313DDE62E878B4F031A942F6-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri May 30 14:57:48 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA21510 for ; Fri, 30 May 2003 14:57:46 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19Loqf-0006ez-00 for ipfix-list@mil.doit.wisc.edu; Fri, 30 May 2003 13:43:29 -0500 Received: from palrel13.hp.com ([156.153.255.238]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19Loqc-0006ep-00 for ipfix@net.doit.wisc.edu; Fri, 30 May 2003 13:43:26 -0500 Received: from xparelay2.ptp.hp.com (xparelay2.ptp.hp.com [15.1.28.65]) by palrel13.hp.com (Postfix) with ESMTP id ED9261C010EC; Fri, 30 May 2003 11:43:23 -0700 (PDT) Received: from xpabh3.ptp.hp.com (xpabh3.ptp.hp.com [15.1.28.63]) by xparelay2.ptp.hp.com (Postfix) with ESMTP id A7F3D1C00A6D; Fri, 30 May 2003 11:43:23 -0700 (PDT) Received: by xpabh3.ptp.hp.com with Internet Mail Service (5.5.2655.55) id ; Fri, 30 May 2003 11:43:23 -0700 Message-ID: <1D3D2C371FCBD947A7897FABBD3533A50295FFEB@xsun01.ptp.hp.com> From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" To: "'Juergen Quittek'" , Sebastian Zander Cc: ipfix@net.doit.wisc.edu Subject: RE: [ipfix] IPFIX requirements - final changes Date: Fri, 30 May 2003 11:43:12 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2655.55) Content-Type: text/plain; charset="iso-8859-1" Precedence: bulk Sender: majordomo listserver Juergen, Your phrasing sounds good: "Many requirements in this document are not explicitly stated as IPFIX protocol requirements, but as requirements for the metering process, the exporting process, or for other traffic measurement components. However, every requirement that needs support from the IPFIX protocol MUST be covered by the IPFIX protocol specification independent of the significance of the requirement, which can be MANDATOYRY, RECOMMENDED, or OPTIONAL. Note that the protocol specification itself also assigns significance attributes to its features, such that a protocol implementation does not necessarily need to implement all features." One change I would suggest is to recognize that the specification contains both a protocol and an (extendable) information model. E.g. anonymization can be addressed w/o any change to the protocol. So I'd rather see one of the sentences modified as: However, every requirement that needs support from the IPFIX protocol or information model MUST be covered by the IPFIX protocol or information model specification independent of the significance of the requirement Regards, Jeff Meyer > -----Original Message----- > From: Juergen Quittek [mailto:quittek@ccrle.nec.de] > Sent: Friday, May 30, 2003 5:50 AM > To: Sebastian Zander > Cc: ipfix@net.doit.wisc.edu > Subject: Re: [ipfix] IPFIX requirements - final changes > > > Sebastian, > > -- Sebastian Zander wrote on 29 May 2003 03:29 +0200: > > > Juergen, > > > > Juergen Quittek wrote: > >> Sebastian, > >> > >> -- Sebastian Zander wrote on 28 May 2003 12:18 +0200: > >> > >>> Hi Juergen, > >>> > >>> i think your second paragraph is a very good idea. But i don't > >>> understand the last 2 sentences. My understanding is that > the ipfix > >>> requirement draft defines the ipfix protocol > requirements. So i don't > >>> understand why the protocol specification must cover all > requirements. > >>> E.g. when a requirementis optional why must it be covered > in the protocol > >>> specification? > >> > >> > >> If the requirement doc says "the exporter MAY anonymize", then the > >> protocol specification MUST support anonymization in order to make > >> sure, that anonymization is interoperable. For example the protocol > >> spec MUST define how to inform a collecting process that > the data it > >> receives is anonymized. Analogously, the protocol spec > should specify > >> how to encrypt. > >> > >> In both cases, the protocol specification can state that a concrete > >> implementation of the protocol MAY support features that serve > >> anonymization. Then an implementor can choose whether or > not to implement > >> this OPTIONAL feature of the protocol. > > > > You are right. The req draft contains protocol > specification requirements > > (which is sometimes confusing and the reason why we get > those comments). > > But i suggest to change the middle sentence saying that all protocol > > specification requirements from the req draft must be taken > over into the > > protocol draft *including their significance*. Otherwise > why do we spend a > > long time discussing the significance of some reqs if it is > not used later > > in the protocol draft? Furthermore some of the reqs may be > targeting the > > information model or achitecture draft instead of the > protocol spec. So > > the sentence should not only mention the protocol spec but *also the > > information model etc*. (Jeff just gave an example that > anonymization may > > influence the information model.) > > I see your point, but I would suggest a different phrasing for the > following reason: > - I do not expect that requirement items can be mapped > one-to-one onto > protocol features. Some individual protocol feature will > cover several > requirement items at once. So here you cannot map the significance > one-to-one, but need something like a minimum significance to be > considered. > - My experience from designing other protocols is that - > compared to the > requirements - you usually increase significance of some features > for technical or orther reasons. We must make sure that > the protocol > definition process has this level of freedom. > > Therefore, I suggest to extend my initially suggested paragraph > > "Many requirements in this document are not explicitly > stated as IPFIX protocol requirements, but as requirements > for the metering process, the exporting process, or for > other traffic measurement components. However, every > requirement that needs support from the IPFIX protocol > MUST be covered by the IPFIX protocol specification > independent of the significance of the requirement, > which can be MANDATOYRY, RECOMMENDED, or OPTIONAL. > Note that the protocol specification itself also assigns > significance attributes to its features, such that a > protocol implementation does not necessarily need to > implement all features." > > by appending this sentence or a similar one: > > "However, the significance assigned to a protocol feature > MUST NOT be lower than the significance of the corresponding > requirements." > > Cheers, > > Juergen > > > > This statement must be put in the very beginning of the req draft. > > > > The last sentence is not necessary in my opinion because > its pretty obvious > > that the protocol draft and the other drafts will use more (and more > > detailed) reqs. > > > > Then there is no need to change the requirements to MUST. > > > > Cheers, > > > > Sebastian > > > >>> I agree to say its a MUST for the protocol but not all exporters > >>> and collectors must support it. > >> > >> > >> Agreed. > >> > >>> But i do *not* agree at all that in general anonymization and > >>> confidentiality is not needed in a LAN. Anonymization can > be required > >>> in a LAN and even in a LAN confidentiality may be needed because i > >>> don't want everybody who has access to my LAN to have access to my > >>> IPFIX data. > >> > >> > >> Also agreed. > >> > >> Best wishes, > >> > >> Juergen > >> > >>> Cheers, > >>> > >>> Sebastian > >>> > >>> Juergen Quittek wrote: > >>> > >>>> Tal, Reinaldo, and Benoit, > >>>> > >>>> Do we all have the following in mind? > >>>> > >>>> Anonymization and confidentiality MUST be covered by the > >>>> protocol specification. But the protocol specifiction > >>>> specification defines it as a MAY feature (anonymization) > >>>> or SHOULD feature (confidentiality) for IPFIX protocol > >>>> implementations. > >>>> > >>>> I guess this is what the IESG reviewers were referring to. > >>>> > >>>> What is probably missing in the requirements document is a > >>>> statement like the following: > >>>> > >>>> Many requirements in this document are not explicitly > >>>> stated as IPFIX protocol requirements, but as requirements > >>>> for the metering process, the exporting process, or for > >>>> other traffic measurement components. However, every > >>>> requirement that needs support from the IPFIX protocol > >>>> MUST be covered by the IPFIX protocol specification > >>>> independent of the significance of the requirement, > >>>> which can be MANDATOYRY, RECOMMENDED, or OPTIONAL. > >>>> Note that the protocol specification itself also assigns > >>>> significance attributes to its features, such that a > >>>> protocol implementation does not necessarily need to > >>>> implement all features. > >>>> > >>>> Any comment on this? > >>>> > >>>> Juergen > >>>> > >>>> > >>>> -- Tal Givoly wrote on 26 May 2003 09:42 -0700: > >>>> > >>>>> Juergen, > >>>>> > >>>>> I agree with Benoit - it doesn't make sense to make > confidentiality and > >>>>> anonymization mandatory. In many deployments, there could be > >>>>> physical and > >>>>> other security measures that addresses these issues. > >>>>> > >>>>> Tal > >>>>> > >>>>> -----Original Message----- > >>>>> From: majordomo listserver > [mailto:majordomo@mil.doit.wisc.edu]On > >>>>> Behalf > >>>>> Of Benoit Claise > >>>>> Sent: Monday, May 26, 2003 8:02 AM > >>>>> To: Juergen Quittek > >>>>> Cc: ipfix@net.doit.wisc.edu > >>>>> Subject: Re: [ipfix] IPFIX requirements - final changes > >>>>> > >>>>> > >>>>> Juergen, > >>>>> > >>>>>> > ============================================================== > ========== > >>>>>> > >>>>>> 12. Confidentiality from SHOULD to MUST > >>>>>> > ============================================================== > ========== > >>>>>> > >>>>>> Problem Description: > >>>>>> > -------------------------------------------------------------- > ---------- > >>>>>> > >>>>>> Requirements on the ipfix implementation - the > document is long and I > >>>>>> wonder if > >>>>>> the working group really meant the protocol's > confidentiality and > >>>>>> anonymization > >>>>>> features to be so optional - SHOULD confidentiality, MAY > >>>>>> anonymization. > >>>>>> Just for implementation. > >>>>>> > ============================================================== > ========== > >>>>>> > >>>>>> Suggested solution: > >>>>>> > -------------------------------------------------------------- > ---------- > >>>>>> > >>>>>> change confidentiality requirement in section 6.3.3 > >>>>>> from SHOULD to MUST > >>>>>> > ============================================================== > ========== > >>>>>> > >>>>>> Status: solution to be agreed on > >>>>>> > ============================================================== > ========== > >>>>>> > >>>>> > >>>>> > >>>>> > >>>>> I'm just wondering if a MUST is not a little bit extreme. > >>>>> The draft would become: > >>>>> "Confidentiality of flow specific data transferred > from an exporting > >>>>> process to a collecting process MUST be ensured." > >>>>> We all agree that over the Internet confidentiality is > compulsory but > >>>>> this sentence would mean that even in a private LAN or > environment, we > >>>>> must have confidentiality to be IPFIX compliant. And we > know that, with > >>>>> the software based encryption, the throughput would be > lowered. Ok, > >>>>> then > >>>>> we should have an crypto engine in hardware on the > exporter to export > >>>>> the numerous flow export packets but there is a price to it. > >>>>> My point is that, as Confidentiality is not needed in > all cases, why > >>>>> make it a MUST? > >>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > ============================================================== > ========== > >>>>>> > >>>>>> 13. Anonymization from MAY to MUST > >>>>>> > ============================================================== > ========== > >>>>>> > >>>>>> Problem Description: > >>>>>> > -------------------------------------------------------------- > ---------- > >>>>>> > >>>>>> Requirements on the ipfix implementation - the > document is long and I > >>>>>> wonder if > >>>>>> the working group really meant the protocol's > confidentiality and > >>>>>> anonymization > >>>>>> features to be so optional - SHOULD confidentiality, MAY > >>>>>> anonymization. > >>>>>> Just for implementation. > >>>>>> > ============================================================== > ========== > >>>>>> > >>>>>> Suggested solution: > >>>>>> > -------------------------------------------------------------- > ---------- > >>>>>> > >>>>>> change anonymization requirement in section 6.7 from > MAY to MUST > >>>>>> > ============================================================== > ========== > >>>>>> > >>>>>> Status: solution to be agreed on > >>>>>> > ============================================================== > ========== > >>>>>> > >>>>> > >>>>> > >>>>> > >>>>> Again, I have the same type of comments. Anonymization > is not required > >>>>> in all cases. So why make it a MUST? > >>>>> > >>>>> Regards, Benoit. > >>>>> > >>>>> > >>>>> > >>>>> -- > >>>>> Help mailto:majordomo@net.doit.wisc.edu and say "help" in > >>>>> message > >>>>> body > >>>>> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > >>>>> "unsubscribe ipfix" in message body > >>>>> Archive http://ipfix.doit.wisc.edu/archive/ > >>>>> > >>>> > >>>> > >>>> > >>>> -- > >>>> Help mailto:majordomo@net.doit.wisc.edu and say > "help" in message > >>>> body > >>>> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > >>>> "unsubscribe ipfix" in message body > >>>> Archive http://ipfix.doit.wisc.edu/archive/ > >>>> > >>> > >>> > >>> -- > >>> Sebastian Zander E-mail: > >>> zander@fokus.fraunhofer.de > >>> Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 > >>> Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 > >>> D-10589 Berlin, Germany > >>> www.fokus.fraunhofer.de/usr/sebastian.zander > >>> > >>> > >>> > >>> > >> > >> > >> > > > > > > -- > > Sebastian Zander E-mail: > zander@fokus.fraunhofer.de > > Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 > > Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 > > D-10589 Berlin, Germany > www.fokus.fraunhofer.de/usr/sebastian.zander > > > > > > > > > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" > in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri May 30 19:14:51 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA02838 for ; Fri, 30 May 2003 19:14:50 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19LsK6-0003Q0-00 for ipfix-list@mil.doit.wisc.edu; Fri, 30 May 2003 17:26:06 -0500 Received: from tokyo.ccrle.nec.de ([195.37.70.2]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19LsK2-0003Po-00 for ipfix@net.doit.wisc.edu; Fri, 30 May 2003 17:26:03 -0500 Received: from venus.office (venus.office [10.1.1.11]) by tokyo.ccrle.nec.de (8.12.9/8.12.8) with ESMTP id h4UMPgVI072954; Sat, 31 May 2003 00:25:43 +0200 (CEST) Received: from [10.1.1.26] (dial02.office [10.1.1.26]) by venus.office (Postfix on SuSE Linux eMail Server 3.0) with ESMTP id BAA03A8A9D; Sat, 31 May 2003 00:14:18 +0200 (CEST) Date: Sat, 31 May 2003 00:27:39 +0200 From: Juergen Quittek To: "MEYER,JEFFREY D (HP-Cupertino,ex1)" , Sebastian Zander Cc: ipfix@net.doit.wisc.edu Subject: RE: [ipfix] IPFIX requirements - final changes Message-ID: <24024064.1054340859@[10.1.1.26]> In-Reply-To: <1D3D2C371FCBD947A7897FABBD3533A50295FFEB@xsun01.ptp.hp.com> References: <1D3D2C371FCBD947A7897FABBD3533A50295FFEB@xsun01.ptp.hp.com> X-Mailer: Mulberry/2.1.2 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Jeff, -- MEYER,JEFFREY D (HP-Cupertino,ex1) wrote on 30 May 2003 11:43 -0700: > Juergen, > > Your phrasing sounds good: > > "Many requirements in this document are not explicitly > stated as IPFIX protocol requirements, but as requirements > for the metering process, the exporting process, or for > other traffic measurement components. However, every > requirement that needs support from the IPFIX protocol > MUST be covered by the IPFIX protocol specification > independent of the significance of the requirement, > which can be MANDATOYRY, RECOMMENDED, or OPTIONAL. > Note that the protocol specification itself also assigns > significance attributes to its features, such that a > protocol implementation does not necessarily need to > implement all features." > > One change I would suggest is to recognize that the specification > contains both a protocol and an (extendable) information model. > E.g. anonymization can be addressed w/o any change to the > protocol. > > So I'd rather see one of the sentences modified as: > > However, every requirement that needs support from the > IPFIX protocol or information model MUST be covered by the > IPFIX protocol or information model specification > independent of the significance of the requirement Agreed. Maybe we should replace two times 'or' by 'and/or'. Juergen > Regards, > > Jeff Meyer > >> -----Original Message----- >> From: Juergen Quittek [mailto:quittek@ccrle.nec.de] >> Sent: Friday, May 30, 2003 5:50 AM >> To: Sebastian Zander >> Cc: ipfix@net.doit.wisc.edu >> Subject: Re: [ipfix] IPFIX requirements - final changes >> >> >> Sebastian, >> >> -- Sebastian Zander wrote on 29 May 2003 03:29 +0200: >> >> > Juergen, >> > >> > Juergen Quittek wrote: >> >> Sebastian, >> >> >> >> -- Sebastian Zander wrote on 28 May 2003 12:18 +0200: >> >> >> >>> Hi Juergen, >> >>> >> >>> i think your second paragraph is a very good idea. But i don't >> >>> understand the last 2 sentences. My understanding is that >> the ipfix >> >>> requirement draft defines the ipfix protocol >> requirements. So i don't >> >>> understand why the protocol specification must cover all >> requirements. >> >>> E.g. when a requirementis optional why must it be covered >> in the protocol >> >>> specification? >> >> >> >> >> >> If the requirement doc says "the exporter MAY anonymize", then the >> >> protocol specification MUST support anonymization in order to make >> >> sure, that anonymization is interoperable. For example the protocol >> >> spec MUST define how to inform a collecting process that >> the data it >> >> receives is anonymized. Analogously, the protocol spec >> should specify >> >> how to encrypt. >> >> >> >> In both cases, the protocol specification can state that a concrete >> >> implementation of the protocol MAY support features that serve >> >> anonymization. Then an implementor can choose whether or >> not to implement >> >> this OPTIONAL feature of the protocol. >> > >> > You are right. The req draft contains protocol >> specification requirements >> > (which is sometimes confusing and the reason why we get >> those comments). >> > But i suggest to change the middle sentence saying that all protocol >> > specification requirements from the req draft must be taken >> over into the >> > protocol draft *including their significance*. Otherwise >> why do we spend a >> > long time discussing the significance of some reqs if it is >> not used later >> > in the protocol draft? Furthermore some of the reqs may be >> targeting the >> > information model or achitecture draft instead of the >> protocol spec. So >> > the sentence should not only mention the protocol spec but *also the >> > information model etc*. (Jeff just gave an example that >> anonymization may >> > influence the information model.) >> >> I see your point, but I would suggest a different phrasing for the >> following reason: >> - I do not expect that requirement items can be mapped >> one-to-one onto >> protocol features. Some individual protocol feature will >> cover several >> requirement items at once. So here you cannot map the significance >> one-to-one, but need something like a minimum significance to be >> considered. >> - My experience from designing other protocols is that - >> compared to the >> requirements - you usually increase significance of some features >> for technical or orther reasons. We must make sure that >> the protocol >> definition process has this level of freedom. >> >> Therefore, I suggest to extend my initially suggested paragraph >> >> "Many requirements in this document are not explicitly >> stated as IPFIX protocol requirements, but as requirements >> for the metering process, the exporting process, or for >> other traffic measurement components. However, every >> requirement that needs support from the IPFIX protocol >> MUST be covered by the IPFIX protocol specification >> independent of the significance of the requirement, >> which can be MANDATOYRY, RECOMMENDED, or OPTIONAL. >> Note that the protocol specification itself also assigns >> significance attributes to its features, such that a >> protocol implementation does not necessarily need to >> implement all features." >> >> by appending this sentence or a similar one: >> >> "However, the significance assigned to a protocol feature >> MUST NOT be lower than the significance of the corresponding >> requirements." >> >> Cheers, >> >> Juergen >> >> >> > This statement must be put in the very beginning of the req draft. >> > >> > The last sentence is not necessary in my opinion because >> its pretty obvious >> > that the protocol draft and the other drafts will use more (and more >> > detailed) reqs. >> > >> > Then there is no need to change the requirements to MUST. >> > >> > Cheers, >> > >> > Sebastian >> > >> >>> I agree to say its a MUST for the protocol but not all exporters >> >>> and collectors must support it. >> >> >> >> >> >> Agreed. >> >> >> >>> But i do *not* agree at all that in general anonymization and >> >>> confidentiality is not needed in a LAN. Anonymization can >> be required >> >>> in a LAN and even in a LAN confidentiality may be needed because i >> >>> don't want everybody who has access to my LAN to have access to my >> >>> IPFIX data. >> >> >> >> >> >> Also agreed. >> >> >> >> Best wishes, >> >> >> >> Juergen >> >> >> >>> Cheers, >> >>> >> >>> Sebastian >> >>> >> >>> Juergen Quittek wrote: >> >>> >> >>>> Tal, Reinaldo, and Benoit, >> >>>> >> >>>> Do we all have the following in mind? >> >>>> >> >>>> Anonymization and confidentiality MUST be covered by the >> >>>> protocol specification. But the protocol specifiction >> >>>> specification defines it as a MAY feature (anonymization) >> >>>> or SHOULD feature (confidentiality) for IPFIX protocol >> >>>> implementations. >> >>>> >> >>>> I guess this is what the IESG reviewers were referring to. >> >>>> >> >>>> What is probably missing in the requirements document is a >> >>>> statement like the following: >> >>>> >> >>>> Many requirements in this document are not explicitly >> >>>> stated as IPFIX protocol requirements, but as requirements >> >>>> for the metering process, the exporting process, or for >> >>>> other traffic measurement components. However, every >> >>>> requirement that needs support from the IPFIX protocol >> >>>> MUST be covered by the IPFIX protocol specification >> >>>> independent of the significance of the requirement, >> >>>> which can be MANDATOYRY, RECOMMENDED, or OPTIONAL. >> >>>> Note that the protocol specification itself also assigns >> >>>> significance attributes to its features, such that a >> >>>> protocol implementation does not necessarily need to >> >>>> implement all features. >> >>>> >> >>>> Any comment on this? >> >>>> >> >>>> Juergen >> >>>> >> >>>> >> >>>> -- Tal Givoly wrote on 26 May 2003 09:42 -0700: >> >>>> >> >>>>> Juergen, >> >>>>> >> >>>>> I agree with Benoit - it doesn't make sense to make >> confidentiality and >> >>>>> anonymization mandatory. In many deployments, there could be >> >>>>> physical and >> >>>>> other security measures that addresses these issues. >> >>>>> >> >>>>> Tal >> >>>>> >> >>>>> -----Original Message----- >> >>>>> From: majordomo listserver >> [mailto:majordomo@mil.doit.wisc.edu]On >> >>>>> Behalf >> >>>>> Of Benoit Claise >> >>>>> Sent: Monday, May 26, 2003 8:02 AM >> >>>>> To: Juergen Quittek >> >>>>> Cc: ipfix@net.doit.wisc.edu >> >>>>> Subject: Re: [ipfix] IPFIX requirements - final changes >> >>>>> >> >>>>> >> >>>>> Juergen, >> >>>>> >> >>>>>> >> ============================================================== >> ========== >> >>>>>> >> >>>>>> 12. Confidentiality from SHOULD to MUST >> >>>>>> >> ============================================================== >> ========== >> >>>>>> >> >>>>>> Problem Description: >> >>>>>> >> -------------------------------------------------------------- >> ---------- >> >>>>>> >> >>>>>> Requirements on the ipfix implementation - the >> document is long and I >> >>>>>> wonder if >> >>>>>> the working group really meant the protocol's >> confidentiality and >> >>>>>> anonymization >> >>>>>> features to be so optional - SHOULD confidentiality, MAY >> >>>>>> anonymization. >> >>>>>> Just for implementation. >> >>>>>> >> ============================================================== >> ========== >> >>>>>> >> >>>>>> Suggested solution: >> >>>>>> >> -------------------------------------------------------------- >> ---------- >> >>>>>> >> >>>>>> change confidentiality requirement in section 6.3.3 >> >>>>>> from SHOULD to MUST >> >>>>>> >> ============================================================== >> ========== >> >>>>>> >> >>>>>> Status: solution to be agreed on >> >>>>>> >> ============================================================== >> ========== >> >>>>>> >> >>>>> >> >>>>> >> >>>>> >> >>>>> I'm just wondering if a MUST is not a little bit extreme. >> >>>>> The draft would become: >> >>>>> "Confidentiality of flow specific data transferred >> from an exporting >> >>>>> process to a collecting process MUST be ensured." >> >>>>> We all agree that over the Internet confidentiality is >> compulsory but >> >>>>> this sentence would mean that even in a private LAN or >> environment, we >> >>>>> must have confidentiality to be IPFIX compliant. And we >> know that, with >> >>>>> the software based encryption, the throughput would be >> lowered. Ok, >> >>>>> then >> >>>>> we should have an crypto engine in hardware on the >> exporter to export >> >>>>> the numerous flow export packets but there is a price to it. >> >>>>> My point is that, as Confidentiality is not needed in >> all cases, why >> >>>>> make it a MUST? >> >>>>> >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> >> ============================================================== >> ========== >> >>>>>> >> >>>>>> 13. Anonymization from MAY to MUST >> >>>>>> >> ============================================================== >> ========== >> >>>>>> >> >>>>>> Problem Description: >> >>>>>> >> -------------------------------------------------------------- >> ---------- >> >>>>>> >> >>>>>> Requirements on the ipfix implementation - the >> document is long and I >> >>>>>> wonder if >> >>>>>> the working group really meant the protocol's >> confidentiality and >> >>>>>> anonymization >> >>>>>> features to be so optional - SHOULD confidentiality, MAY >> >>>>>> anonymization. >> >>>>>> Just for implementation. >> >>>>>> >> ============================================================== >> ========== >> >>>>>> >> >>>>>> Suggested solution: >> >>>>>> >> -------------------------------------------------------------- >> ---------- >> >>>>>> >> >>>>>> change anonymization requirement in section 6.7 from >> MAY to MUST >> >>>>>> >> ============================================================== >> ========== >> >>>>>> >> >>>>>> Status: solution to be agreed on >> >>>>>> >> ============================================================== >> ========== >> >>>>>> >> >>>>> >> >>>>> >> >>>>> >> >>>>> Again, I have the same type of comments. Anonymization >> is not required >> >>>>> in all cases. So why make it a MUST? >> >>>>> >> >>>>> Regards, Benoit. >> >>>>> >> >>>>> >> >>>>> >> >>>>> -- >> >>>>> Help mailto:majordomo@net.doit.wisc.edu and say "help" in >> >>>>> message >> >>>>> body >> >>>>> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >> >>>>> "unsubscribe ipfix" in message body >> >>>>> Archive http://ipfix.doit.wisc.edu/archive/ >> >>>>> >> >>>> >> >>>> >> >>>> >> >>>> -- >> >>>> Help mailto:majordomo@net.doit.wisc.edu and say >> "help" in message >> >>>> body >> >>>> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >> >>>> "unsubscribe ipfix" in message body >> >>>> Archive http://ipfix.doit.wisc.edu/archive/ >> >>>> >> >>> >> >>> >> >>> -- >> >>> Sebastian Zander E-mail: >> >>> zander@fokus.fraunhofer.de >> >>> Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 >> >>> Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 >> >>> D-10589 Berlin, Germany >> >>> www.fokus.fraunhofer.de/usr/sebastian.zander >> >>> >> >>> >> >>> >> >>> >> >> >> >> >> >> >> > >> > >> > -- >> > Sebastian Zander E-mail: >> zander@fokus.fraunhofer.de >> > Fraunhofer FOKUS / METEOR Tel: +49-30-3463-7287 >> > Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 >> > D-10589 Berlin, Germany >> www.fokus.fraunhofer.de/usr/sebastian.zander >> > >> > >> > >> > >> >> >> >> -- >> Help mailto:majordomo@net.doit.wisc.edu and say "help" >> in message body >> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >> "unsubscribe ipfix" in message body >> Archive http://ipfix.doit.wisc.edu/archive/ >> -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri May 30 20:20:49 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA04945 for ; Fri, 30 May 2003 20:20:49 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19LttA-0005Pn-00 for ipfix-list@mil.doit.wisc.edu; Fri, 30 May 2003 19:06:24 -0500 Received: from mailhost2.auckland.ac.nz ([130.216.1.4]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19Ltt8-0005Pe-00 for ipfix@net.doit.wisc.edu; Fri, 30 May 2003 19:06:22 -0500 Received: from mailhost.auckland.ac.nz (IDENT:mirapoint@mailhost [130.216.191.61]) by mailhost2.auckland.ac.nz (8.12.9/8.12.9/8.12.9-ua) with ESMTP id h4V06J8W019719; Sat, 31 May 2003 12:06:19 +1200 (NZST) Received: from hotlava.auckland.ac.nz (hotlava.auckland.ac.nz [130.216.191.123]) by mailhost.auckland.ac.nz (Mirapoint Messaging Server MOS 3.3.5-GR) with ESMTP id AQP33023; Sat, 31 May 2003 12:06:18 +1200 (NZST) Received: (from apache@localhost) by hotlava.auckland.ac.nz (8.11.6/8.11.6) id h4V06Io19495; Sat, 31 May 2003 12:06:18 +1200 X-Authentication-Warning: hotlava.auckland.ac.nz: apache set sender to n.brownlee@auckland.ac.nz using -f Received: from dyn41.caida.org (dyn41.caida.org [192.172.226.41]) by hotlava.auckland.ac.nz (Horde) with HTTP for ; Sat, 31 May 2003 12:06:18 +1200 Message-ID: <1054339578.309efd3a4eaf2@hotlava.auckland.ac.nz> Date: Sat, 31 May 2003 12:06:18 +1200 From: Nevil Brownlee To: Luca Deri Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] nFlow References: <3ECE367C.30608@ntop.org> In-Reply-To: <3ECE367C.30608@ntop.org> MIME-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Internet Messaging Program (IMP) 4.0-cvs X-Originating-IP: 192.172.226.41 Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hello Luca: > I have decided to define a new flow format named nFlow (see - - - - - - > I would like to know whether: > - we could work together so that IPFIX and nFlow could merge (I am open > to change it) First, you're very welcome to contribute to the IPFIX WG. I guess you know that the group's work is documented in its mailing list archive, i.e. http://ipfx.doit.wisc.edu/list/ipfix/archive/ You also need to read the WG charter, which provides fairly tight limits on what the group can actually work on. At the last IETF meeting the WG reached consensus on using NetFlow v9 as the starting point for the IPFIX protocol. Now we're working on a set of four new Internet Drafts which will document the complete IPIFIX system as follows: Applicability: Tanja Zseby, Reinaldo Penno Explains what IPFIX is about, and how it can be used, gives brief overview of other documents. [RFC 2721 is a reasonable example of an AS RFC] Architecture: Ganesh Sadasivan, Nevil Brownlee Describes (in full detail) the IPFIX system, defines terms used, including exactly what we mean by an 'IPFIX flow.' Information Model: Paul Calato, Jeff Meyer, Juergen Quittek Explains how IPFIX flow data is made up, i.e. what data elements may be part of a flow, what they mean, how they are represented, and how they are combined into a flow's data structure. Protocol: Mark Fulmer, Paul Calato, Benoit Claise, Reinaldo Penno Specifies how IPFIX flow data and control information is carried (via a congestion-aware transport protocol) from IPFIX exporter to IPFIX collector. Once the initial versions of these are published (real soon now ...), you'll be able to decide where your interests fit - my guess is that you'll want to contribute to the Information Model and Protocol drafts. That means reading the drafts and commenting on them, and posting paragraphs of text to replace or improve things in the drafts. > - I could show up at the next IETF meeting in Vienna and present nFlow No problem with that; we've often had short (~10-minute) presentations on topics related to network measurement. Please email me a paragraph or two giving an outline of your talk. Cheers, Nevil ----------------------------------------------------------------------- Nevil Brownlee Director, Technology Development Phone: +64 9 373 7599 x88941 ITSS, The University of Auckland FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand ------------------------------------------------- This mail sent through University of Auckland http://www.auckland.ac.nz/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/