Subject: [ipfix] Update available: draft-ietf-ipfix-info-01.txt

From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" <[email-masked]>
Date: Tue, 12 Aug 2003 20:46:03 -0700

Hi,

  A revised version of the IPFIX Information Model has been submitted.

  While this is getting uploaded, the document (and other artifacts) may be
found at the following location:

    - IETF formatted .txt file is available at:
 
http://www.ipdr.org/documents/ipfix/infomodel/draft-ietf-ipfix-info-01.txt

    - The HTML formatted and source XML documents are also available:
 
http://www.ipdr.org/documents/ipfix/infomodel/draft-ietf-ipfix-info-01.html
 
http://www.ipdr.org/documents/ipfix/infomodel/draft-ietf-ipfix-info-01.xml

    - The Change History for the -01 draft is at:
         http://www.ipdr.org/documents/ipfix/infomodel/changehistory.txt

    - The README document describing the artifacts from this process and
with
      pointers to historical documents is available at:

         http://www.ipdr.org/documents/ipfix/infomodel/README.html

Regards,

  Jeff Meyer

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: Benoit Claise <[email-masked]>
Date: Wed, 13 Aug 2003 14:33:09 +0200

Jeff,

>http://www.ipdr.org/documents/ipfix/infomodel/draft-ietf-ipfix-info-01.txt
One quick comment, from the Vienna meeting minutes
http://ipfix.doit.wisc.edu/IETF57/minutes.txt
        Juergen mentioned that the the definitions named with the "ipdr" 
prefix should change.

Regards, Benoit.

---

From: Stewart Bryant <[email-masked]>
Date: Wed, 13 Aug 2003 17:17:25 +0100

Jeff

Some comments on <draft-ietf-ipfix-info-01>:

             Information Model for IP Flow Information Export
                         draft-ietf-ipfix-info-01

SB> An (IETF) information model shows the relationship between abstract
SB> information objects. I don't think that there are any we need
SB> to be concerned about here. This is surely a data model.

<snip>

2. Scope

    This document defines an information model for the IP Flow
    Information eXport (IPFIX) protocol. The model consists of of a set
    of information elements, each one defining a single attribute of a
    flow. For each individual attribute, the semantics is clearly
    specified and a data type is assigned to it.

SB> Surely this is a data model that should be shared from day one
SB> with PSAMP. That being so perhaps PSAMP should be referenced
SB> here.

<snip>

4. Type Space

SB> I think that you need another type, the unsigned variant integer.
SB> If you look at the netflow V9 draft you will see that this is
SB> used extensively, but in the current version is not explicitly
SB> called out. Look for the integer-like objects of length N.
SB> I believe that the next version of the draft will provide more
SB> detail on how to interpret numeric objects of length N.
SB>
SB> Unsigned variant integers are used where the precision of the
SB> object is implementation dependent. This has the advantage
SB> that we can have large counters on fast interfaces, but do
SB> not need to waste cache space in the exporter, nor do we need
SB> to waste export bandwidth when a smaller counter will suffice.
SB> There are other objects that can also usefully use this,
SB> for example AS numbers.
SB>
SB> Ganesh Sadasivan and I and worked on this and our conclusion
SB> was that you need to define an unsigned variant integer
SB> as follows:
SB>
SB> type varUnsignedInt
SB>    if (element.length == 1)
SB>          type = unsignedByte
SB>    else if (element.length == 2)
SB>         type = unsignedShort
SB>    else if (element.length <= 4)
SB>         type = unsignedInt
SB>    else if (element.length <= 8)
SB>         type = usignedLong
SB>   else type = undefined
SB>
SB> I show where these need to be used in the comments that follow.
SB>
SB> Ganesh also noted that in a 32 bit machine unsignedInt &
SB> unsignedLong are used used interchangeably. It might therefore be
SB> clearer if you used the terms uint8_t, uint16_t, uint32_t and uint64_t
SB> to make things more explicit.
SB>
SB>

<snip>

4.9 boolean

    The type "boolean" represents the values for binary logic. (i.e.
    "true/false" or "1/0").

SB> We are never going to export a data unit of less than 8 bits
SB> and normally we will just export some flag field from a packet
SB> so I wonder if this serves any purpose other than completeness.

<snip>

6. Flow Attributes

<snip>

6.5 protocolIdentifier

    Description:

    Protocol number identified in the IP packet.

    In the Internet Protocol version 4 (IPv4) [RFC791] there is a field,
    called "Protocol", to identify the next level protocol.  This is an 8
    bit field.  In Internet Protocol version 6 (IPv6) [RFC1883] this
    field is called the "Next Header" field.

    These numbers are administered by IANA.

    Type:  The protocolIdentifier element is of type int.

SB> Surely it should be unsignedByte to match the definition in both
SB> RFC791 and RFC1883?

    Reference:  Additional information on this element can be found at
    http://www.iana.org/assignments/protocol-numbers.

    Field Id:  4

<snip>

6.8 ingressPort

    Description:

    The ifIndex where the packets for the flow are being received.
    ifIndex is defined by RFC 2233.

    Type:  The ingressPort element is of type unsignedShort.

    Field Id:  10

SB> This should be an unsigned variant integer, because some systems
SB> will have more than 64K ports.

6.9 egressPort

    Description:

    The ifIndex where the packets for the flow are exiting. ifIndex is
    defined by RFC 2233.

    Type:  The egressPort element is of type unsignedShort.

SB> This should be an unsigned variant integer, because some systems
SB> will have more than 64K ports.

    Field Id:  14

6.10 packetCount

    Description:

    Contains the count of packets sent and received associated with the
    identified flow.

    The packet count can be for packets received (towards source) or
    packets sent (towards destination) or both (bi-directional flow).

    The packet count can be a running counter and is the count from the
    beginning of the flow establishment.

    The packet count can be a delta counter and is the count since the
    last report for this flow.

    Type:  The packetCount element is of type int.

SB> This cannot be a signed number, -1 packets has no meaning. In
SB> any case it needs to be an unsigned variant integer, because
SB> some interfaces will need 32 bit counters and some will need
SB> 64 bit counters.

    Units:  The unit of measure is  packets.

    Field Id:  2

6.11 byteCount

    Description:

    Contains the count of octets sent and received associated with the
    identified flow.

    The byte count can be for bytes received (towards source) or bytes
    sent (towards destination) or both (bi-directional flow).

    The byte count can be a running counter and is the count from the
    beginning of the flow establishment.

    The byte count can be a delta counter and is the count since the last
    report for this flow.

    Type:  The byteCount element is of type int.

SB> This cannot be a signed number, -1 bytes has no meaning. In
SB> any case it needs to be an unsigned variant integer, because
SB> some interfaces will need 32 bit counters and some will need
SB> 64 bit counters.

    Units:  The unit of measure is  bytes.

    Field Id:  1

6.12 classOfService

    Description:

    The class of service associated with a flow.

    Class of Service Received

    Class of Service Transmitted

    1. IPv4, CoS value is defined by ToS in RFC 791 2. IPv6, CoS value is
    defined by Traffic Class in RFC 2460 3. MPLS, CoS value is defined by
    Exp in RFC 3032 4. VLAN, CoS value is defined by user_priority in
    IEEE802.1q[802.1q] and IEEE 802.1p[802.1p]

    Type:  The classOfService element is of type byte.

SB> This should be unsignedByte

    Field Id:  5

6.13 flowLabel

    Description:

    The Flow Label information element contains the IPV6 Flow Label
    information as defined by RFC 2460.

    Type:  The flowLabel element is of type int.

SB> This cannot be an int. It might be an unsignedInt, but that
SB> wastes a byte. It could be an unsigned variant integer of length
SB> 3, or it could be a hexbyte of length 3.

    Field Id:  31

6.14 flowCreationTime

    Description:

    The timestamp of the first packet of the flow.  (Ed. note:  current
    NFv9 protocol uses sysuptime vs. direct time.  Not interesting from
    an info model perspective, an artifact (and an annoying one from a
    consumer perspective) of the protocol implementation details. How to
    address?)

    Type:  The flowCreationTime element is of type dateTime.

    Field Id:  21

SB> For consistency with Netflow v9 this should be 22 (FIRST_SWITCHED)

6.15 flowEndTime

    Description:

    The timestamp of the last packet of the flow.  (Ed. note:  current
    NFv9 protocol uses sysuptime vs. direct time.  Not interesting from
    an info model perspective, an artifact (and an annoying one from a
    consumer perspective) of the protocol implementation details. How to
    address?)

    Type:  The flowEndTime element is of type dateTime.

    Field Id:  22
SB> For consistency with Netflow v9 this should be 21 (LAST_SWITCHED)

6.16 sourceAS

    Description:

    The Autonomous System (AS) numbers for the source address associated
    with a flow. Autonomous System (AS) number is defined by RFC 1930 and
    RFC 1771 (BGP-4):

    Type:  The sourceAS element is of type int.

SB> This cannot be an int, it must be an unsignedInt. However the change
SB> from unsignedShort to unsignedInt is still a draft and in any case
SB> much of the world will remain unsignedShort for a long time. I think
SB> that this needs to be an unsigned variant integer. I think that some
SB> netflow v9 systems export this as an unsignedShort.

    Field Id:  16

6.17 destinationAS

    Description:

    The Autonomous System (AS) numbers for the destination address
    associated wit a flow. Autonomous System (AS) number is defined by
    RFC 1930 and RFC 1771 (BGP-4).

    Type:  The destinationAS element is of type int.

SB> This cannot be an int, it must be an unsignedInt. However the change
SB> from unsignedShort to unsignedInt is still a draft and in any case
SB> much of the world will remain unsignedShort for a long time. I think
SB> that this needs to be an unsigned variant integer. I think that some
SB> netflow v9 systems export this as an unsignedShort.

    Field Id:  17

6.18 nextHopAS

    Description:

    The Autonomous System (AS) numbers for the next hop IP. Autonomous
    System (AS) number is defined by RFC 1930 and RFC 1771 (BGP-4).

    Type:  The nextHopAS element is of type int.

SB> This cannot be an int, it must be an unsignedInt. However the change
SB> from unsignedShort to unsignedInt is still a draft and in any case
SB> much of the world will remain unsignedShort for a long time. I think
SB> that this needs to be an unsigned variant integer.

    Field Id:  -1

6.19 tcpControlBits

    Description:

    The TCP control bits seen for this flow. Note a 0 value for each bit
    only indicates that the flag was not detected (i.e. it may have
    occurred but was not detected by the reporting CCE). TCP Control Bits
    are defined by RFC 793.

    Type:  The tcpControlBits element is of type int.

SB> RFC 793 defines this as a set of flags that ocupy a byte. Making
SB> it a signed object is wrong. It should surely be an unsignedByte
SB> since it can never get to be any longer.

    Field Id:  6

6.22 droppedPacketCount

    Description:

    Contains the count of packets dropped at the observation point
    associated with the identified flow.

    The dropped packet count can be a running counter and is the count
    from the beginning of the flow establishment.

    The dropped packet count can be a delta counter and is the count
    since the last report for this flow.

    Type:  The droppedPacketCount element is of type int.

    Units:  The unit of measure is  packets.

SB> This cannot be a signed number, -1 packets has no meaning. In
SB> any case it needs to be an unsigned variant integer, because
SB> some interfaces will need 32 bit counters and some will need
SB> 64 bit counters.

    Field Id:  -1

6.23 samplingInterval

    Description:

    When using Sampling, the rate at which packets is sampled. For
    example, a value of 100 indicates that one of every hundred packets
    is sampled.

    Type:  The samplingInterval element is of type int.

SB> This must be an unsignedInt, because a negative interval has no meaning.
SB> The netflow V9 informational draft has this as 4 bytes, but I think
SB> that there are some netflow V9 systems that implemented it as an
SB> unsignedShort, so it would useful if we made it an unsigned
SB> variant integer to gain backwards compatibility.

    Field Id:  34

6.24 samplingAlgorithm

    Description:

    The type of algorithm used for sampling data. Currently, the only
    sampling algorithm defined  is: 0x02 packet-sampling

    Type:  The samplingAlgorithm element is of type int.

SB> This must be an unsigned int, although I think that some netflow V9
SB> implementations implemented it as an unsignedShort. Again unsigned
SB> variant integer would fix this.

    Field Id:  35

6.25 flowEndState

    Description:

    The reason the flow has ended. 1. Inactivity timeout 2. End of flow
    detected (e.g. TCP FIN) 3. Forced end ???? 4. Cache full
    [enumerations in IPDR service def schemas are recommended to be of
    form string, w/ integer values (for Compact format) defined via
    annotation]

    Type:  The flowEndState element is of type int.

SB> I am not sure we need 4G states, but in any case it should
SB> be unsigned.

    Field Id:  -1

6.26 droppedByteCount

    Description:

    Contains the count of octets dropped at the observation point
    associated with the identified flow.

    The dropped byte count can be a running counter and is the count from
    the beginning of the flow establishment.

    The byte count can be a delta counter and is the count since the last
    report for this flow.

    Type:  The droppedByteCount element is of type int.

    Units:  The unit of measure is  bytes.

    Field Id:  -1

SB> This cannot be a signed number, -1 bytes has no meaning. In
SB> any case it needs to be an unsigned variant integer, because
SB> some interfaces will need 32 bit counters and some will need
SB> 64 bit counters.

7. The Benefits of a Formal Machine Readable Information Model

SB> This is something of a sales pitch. I think that you will have
SB> to cut this down by the time the draft goes for IESG review

    Appendix A. expresses the IPFIX Information model as an XML-Schema.
    Using a formal and machine readable syntax for the Information model
    enables the creation of IPFIX aware tools which can automatically
    adapt to extensions to the information model, by simply reading
    updated information model specifications.

    The use of XML-Schema as the formal specification language is modeled
    after the techniques employed by the IPDR NDM-U specification.

    The wide availability of XML aware tools and libraries for client
    devices is a primary consideration for this choice.  In particular
    libraries for parsing XML documents are readily available.  Also
    mechanisms such as the Extensible Stylesheet Language (XSL) allow for
    transforming a source XML document into other documents.  This draft
    was initially authored in XML and transformed according to RFC2629.

    It should be noted that the use of XML in exporters, collectors or
    other tools is not mandatory for the deployment of IPFIX.  In
    particular exporting processes do not produce or consume XML as part
    of their operation.  It is expected that IPFIX collectors MAY take
    advantage of the machine readability of the Information Model vs.
    hardcoding their behavior or inventing proprietary means for
    accomodating extensions.

<snip>

Appendix A. IPFIX IPDR Service Definition

    This proposal does not currently address possible IANA implications
    associated with XML Namespace URIs. The use of Namespaces as an
    extension mechanism implies that an IANA registered Namespace URI
    should be available and that directory names below this base URI be
    assigned for relevant IETF specifications. The author is not aware of
    this mechanism today. Alternatively IPDR.org could fulfill this role.
    The sample uses the IPDR.org namespace.

    The normative status of this appendix versus the section "Flow
    Attributes" is a point of discussion.  The "Flow Attributes" section
    is simply machine generated from the formal XML document below.  As
    such using the formal XML document would seem preferable.  However
    historical conventions and IETF's overall level of XML adoption may
    lead to selection of the human readable text in the "Flow Attributes"
    section as being preferable as normative.

SB> The fact that the body is machine generated from this XML does
SB> not rescue you from the fact that the RFC editor will use some
SB> other process to produce the final document that may result in errors.
SB> You will need some means to check the consistency of the two
SB> definitions after than process. In any case as a final backstop
SB> in the event of some unforeseen gremlin creeping into the process
SB> one or the other definitions must be given priority.

Regards

Stewart

---

From: Sebastian Zander <[email-masked]>
Date: Wed, 13 Aug 2003 18:16:50 +0200

Hi Jeff,

i have some comments and questions for you:

section 4.19
	dateTimeUsec is 64bit as on common operating systems i assume.
section 5
	i guess the idea of the unique vendor ids is to allow non unique field ids.
	however this requires protocol support.	
section 6.5
	IPv6 next header is 8 bit as well so it should be byte instead of int.
section 6.8, 6.9
	ifIndex is 32 bit (as far as RFC 2233 is concerned) so it should be (unsigned)int
section 6.10, 6.11, 6.22, 6.26
	shouldn't the counters be unsigned and of type long (wrap around)?
	how does a collector know whether a counter is an absolute or delta counter?
	how does a collector know whether a counter is forward, backward, bidirectional?
section 6.23
	why not unsigned?
section 6.24
	what is the meaning of "packet-sampling"? shouldn't this field describe the type of 		 
sampling?
	can something from PSAMP be referenced here?
	what about sampling algorithm parameters? are they fixed for a samplingAlgorithm
	or communicated as separate fields?
section 6
	There is no IP version attribute defined. is the idea to get the ip protocol
	version by looking at the address attributes? Because the information is mandatory
	in the req draft.

Cheers,

Sebastian

-- 
Sebastian Zander                         E-mail: [email-masked]
Fraunhofer FOKUS / METEOR                Tel: +49-30-3463-7287
Kaiserin-Augusta-Allee 31                Fax: +49-30-3463-8287
D-10589 Berlin, Germany                  www.fokus.fraunhofer.de/usr/sebastian.zander

---

From: Stewart Bryant <[email-masked]>
Date: Wed, 13 Aug 2003 17:58:39 +0100

>     how does a collector know whether a counter is an absolute or delta 

I also noted that in the first set of comments that I sent to Jeff re
the 00 draft, but did not pick it up again in the set I just sent to
the list re the 01 draft.

I think that we need to define the counters as absolute, because that
way they are idenpotent (i.e. it does not matter if you miss a data
record). That also give compatibility with NFv9. If anyone wants a
delta counter it should be a new information element.

>     how does a collector know whether a counter is forward, backward, 

All of the counters defined so far only have any meaning as a forward
counter.

> section 6.23
>     why not unsigned?
> section 6.24
>     what is the meaning of "packet-sampling"? shouldn't this field 
> describe the type of          sampling?

<snip>

Stewart

---

From: Sebastian Zander <[email-masked]>
Date: Thu, 14 Aug 2003 10:25:45 +0200

Stewart Bryant wrote:
>>     however this requires protocol support.   section 6.5

i agree that absolute counters are a must in case of not 100% reliable
transport because otherwise information can be lost forever. however the
data type should then be 64bit (or var int supporting up to 64bit) to
make wrap-arounds more infrequent and the wrap-around issue should be
addressed (don't know if this is already done in the protocol spec).

cheers,

Sebastian

---

From: Stewart Bryant <[email-masked]>
Date: Thu, 14 Aug 2003 09:59:28 +0100

> addressed (don't know if this is already done in the protocol spec).

Actually I disagree with you about 64bit. It should be up to the
implementation to decide the appropriate counter length. In systems
with very large numbers of low bandwidth links, using 64bit counters
at least doubles the memory needed by the exporter and at least doubles
the export bandwidth. The exporter know the characteristics of the system
that it is monitoring the reliability of the transport etc, and is best
placed to determine optimum the length.

Stewart

---

From: Sebastian Zander <[email-masked]>
Date: Thu, 14 Aug 2003 11:24:10 +0200

> placed to determine optimum the length.

I see that my previous statement wasn't too clear. Using counters <64bit for
optimization purposes is fine but there must be the option of having 64bit
counters for high bandwidth links IMO. I'm wondering whether his would double
the overall export bandwidth because the protocol does transmit more information
than just the counters. Do you have some numbers on that?

---

From: "Fulvio Risso" <[email-masked]>
Date: Thu, 14 Aug 2003 12:14:00 +0200

> -----Original Message-----
> From: majordomo listserver [mailto:[email-masked]]On Behalf
> Of Stewart Bryant
> Sent: giovedi 14 agosto 2003 10.59
> To: Sebastian Zander
> Cc: MEYER,JEFFREY D (HP-Cupertino,ex1); [email-masked]'
> Subject: Re: [ipfix] Update available: draft-ietf-ipfix-info-01.txt
>
>
>
>
> Sebastian Zander wrote:
> > Stewart Bryant wrote:
> >
> >>
> >>
> >> Sebastian Zander wrote:
> >>
> >>> Hi Jeff,
> >>>
> implementation to decide the appropriate counter length.

This means that the collector is more complicated, because it must have to
code to handle 64bit counters, 32 bit ones, etc.
I prefer a unique size for counters (which makes the code simpler).
For that, 64bits seem to me more appropriate.

> In systems

... and the collector must handle several counters with different sizes. And
the same for the application (e.g. GUI) that display results. And when we
compare results, the programmer has to take care about these differences.
In my opinion, let's keep things simpler. Less optimized, but simpler.

Cheers,

	fulvio