Subject: [ipfix-protocol] IPFIX protocol draft: list of issues

From: Benoit Claise <[email-masked]>
Date: Fri, 13 Feb 2004 01:05:30 +0100

Dear all,

I will be posting a new version 03 of the protocol draft at the latest during the weekend.
In the mean time, here is a list of issues that have been identified.
	1. all the issues from the previous drafts.
	2. issues from the mailing list.
	3. issues from various discussions. 
	  That's the reasons why there are sometimes proposals from the authors.
I hope I'm covering all of them. If not, feel free to update this list and assign a new number to your issue.

When discussing one issue on the mailing list, please always insert the PROTO-<number> in the Subject: it will ease everybody live.
As always, let's try to discuss one issue at the time.

The goal is not to try to solve all the issue before posting the new draft this weekend ;), but to discuss on the mailing list or at the next IETF meeting.
Note that this list will be updated in the version 03 of the protocol draft.

Issues in the Terminology section

PROTO-1: Is flowset the right term to use?
    - leave as is
    - Record Set
    - Record Array
    - Record Collection
    - Record List
PROTO-2: Some discrepancies between data types, field type and Information
    Element terminology.
    - field type (IPFIX-PROTO) conflicts with field ID (IPFIX-INFO)
    - suggestion: use field type instead of field Id in IPFIX-INFO
    - rename 'type' to 'data type' and 'info elements' to 'fields'
       in IPFIX-INFO
PROTO-3: IP encapsulated packet
    - IP Traffic Flow definition speaks of IP packets
    - Metering Process definitions say: 
      Input to the metering process are packets 
    - We don't want to limit ourselves to IPv4 and IPv6

Issues in the Transport Protocol section

PROTO-4: TCP and UDP are is not yet covered
PROTO-5: Error recovery, for example what to do if a collector receives a message it
    can't decode.
    Per protocol issue, ie TCP reset the session because it's a stream protocol
    and can't recover.
PROTO-6: Section 5.2.3.3, Association: What happens if the Exporter gets no response
    from any Collector?
    I think we should specify a (not-too-aggressive) retry algorithm.
PROTO-7: Dropping data before export: What to do with sequence numbers?
PROTO-8: Why having a stream per SID?
PROTO-9: IPFIX message should have MTU size.
PROTO-10: How to re-establish a lost connection? Procedure per transport protocol?

Issues in the Failover section

PROTO-11: (initial version required) Section needs to be written
    If we tackle reliability/failover a state diagram is needed.

Issues in the Message Layout section

PROTO-12: Do we need the IETF exclusive template flowset format?
    suggested solution:
    - reserve flowset ID 0 & 1 for compatibility with NFv9
    - try to make flowset ID 2 & 3 definition fully compatible with NFv9
    - add note that if you implement ID 2/3 correctly, you can also process ID 0/1.
PROTO-13: How to distinguish IETF field IDs from vendor field IDs
    - Specify method for detecting the difference in section 8.
    - Add a note that there is a common ID space for for field types used in
      data templates and option templates
    - (Editorial) make clear that Section 8.2 also applies to option templates
PROTO-14: Why do we need padding? Should we shift it to MAY?
    limit the size of the padding? Yes
    solution:
    - padding shorter than actual Record Length
    - fill with 0
    - only at end of flowset
    - applies to all flowsets
    - padding is OPTIONAL (MAY) , not RECOMMENDED (SHOULD)

Issues in the IPFIX Message Format section

PROTO-15: Remove Reserved 2 octets in Vendor specific option template flowset
    and add padding at the end
PROTO-16: relationship between several different scopes in one record
PROTO-17: redefine scope values?
    - 1 System
      2 IP interface
      3 observation domain (SID) (preciously called line card)
      4 reserved (previously used for cache)
      5 template
      6 metering process?
      7 flow recording process?
      8 exporting process?
      9 observation point?
PROTO-18: Can we have an optional length of 0 bytes for the scope section
    in the option template?
PROTO-19: Do we really need different templates formats for flows and options?
PROTO-20 Do we really need different record formats for flows and options?

Issues in the Specific Reporting Requirement section

PROTO-21: Do we need to define some mandatory content of the metering process
    statistics option template?
    - Maurizio suggested text on the mailing list
PROTO-22: Exporter ID (ie IP address of exporter)

Issues in the Export Packet "Export Time" Computation and Flow Record Time section

PROTO-23: Finalize the time details

Issues in the Linkage with the Information Model section

PROTO-24: Section 11 "Linkage with the information model" must be completed with
    types used in [IPFIX-INFO]

Issues in the Template Management section

PROTO-25: The section 11 "Template Management" will have to updated
    according to the transport protocol.
    - For example, the point 2 of the section "Template Management".
      Remark: the template management will vary with TCP, SCTP, etc...
      Must have both sections updated: transport updated and template
      management sections (BTW, this is the same for the failover
      section).
  
Issues in the IANA section

PROTO-26: IANA considerations section to be updated: have a look at RFC 2434,
     which sets out guidelines for IANA Considerations.
     Also, searching the RFCs for 'IANA Considerations' brings
     up quite a few RFCs to look at as models.

Issues - Miscellaneous

PROTO-27: Need an example with the Vendor Specified Information Element
PROTO-28: Packet Sampling.  This is mentioned in both the Requirements I-D
     and the AS I-D.  We need to decide how it should be covered in the
     IPFIX drafts.

ACTION

PROTO-29: number all the figures
PROTO-30: Review the requirements draft to see what we miss, once it's an I-RFC

Regards, Benoit.

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: Ganesh Sadasivan <[email-masked]>
Date: Thu, 12 Feb 2004 17:00:02 -0800

Benoit,

PROTO-3: IP encapsulated packet
   - IP Traffic Flow definition speaks of IP packets
   - Metering Process definitions say:      Input to the metering 
process are packets    - We don't want to limit ourselves to IPv4 and IPv6

The architecture spec clearly covers the encapsulated packets in the IP 
Traffic Flow Definition.
Otherwise we are leaving out MPLS packets, tunnel packets from being 
able to be defined
as flows.

       "A 'Flow' is a set of IP packets, *or encapsulated IP packets*,
       passing an Observation Point in the network during a certain time
       interval. All packets belonging to a particular Flow have a set
       of common properties. Each property is defined as the result of
       applying a function to the values of:

         1. One or more header fields of the actual packet, e.g.
            destination IP address, or *fields in the packet's
            encapsulation header,* e.g. label for MPLS, tunnel end-points
            for IP-in-IP or fields in transport header (e.g. destination
            port number), or fields in application header field (e.g.
            RTP header fields [RFC1889])
         2. One or more properties of the packet itself (e.g. packet
            length)
         3. One or more of fields derived from packet treatment (e.g.
            next hop IP address,AS number)"

Is this clear enough to cover all the cases we want?
I agree we have to change the Metering Process definition to
include
      "The Metering Process generates Flow Records. Input to the process
       are IP packets or encapsulated IP packets observed at an
       Observation Point and packet treatment at the Observation Point,
       for example the selected output interface...."

Thanks
Ganesh

---

From: Ganesh Sadasivan <[email-masked]>
Date: Thu, 12 Feb 2004 17:05:43 -0800

This term is used for grouping  IPFIX records with similar structure.
 I suggest we use "Record Group" or "Record Set"

Thanks
Ganesh

---

From: Stewart Bryant <[email-masked]>
Date: Fri, 13 Feb 2004 09:44:41 +0000

How about Record batch?

The only reason the records were collected together for
export was that they were ready for export at a particular
time. You are unlikely to get the same grouping at a later
time. All the other terms imply some more systematic
association.

- Stewart

---

From: Ganesh Sadasivan <[email-masked]>
Date: Fri, 13 Feb 2004 10:19:07 -0800

The current definition is:
The set of Observation Points, which is the ./largest aggregatable set   
of Flow information at the Metering Proces/s is termed an Observation   
Domain. Each Observation Domain presents itself as a unique ID to   the 
Collecting Process for identifying the IPFIX Messages it   generates.   
For example, a router line card composed of several interfaces with   
each interface being an Observation Point.   Every Observation Point is 
associated with an Observation Domain.

The term "/largest aggregatable set   of Flow information at the 
Metering Proces/s" does not seem to be correct.
Reasons:
1. From the wording there is an inherent assumption of 1 metering 
process per Observation Domain (OD). For example
    each Obdervation point within an OD could have a separate metering 
process.
2. Metering Process itself is independent of the Observation Domain. For 
example: If a Metering Process to the entire
    IPFIX device (as its scope), which has multiple OD, then it breaks 
the above definition . Specifically say there are
    2 linecards (each of them being an OD) in a router (IPFIX device). 
Say sampling is configured on all interface in
    LC1 and LC2, the scope of metering process spans the entire IPFIX 
device as compared to the OD.

All that the Observation Domain defines is a unique way to identify with 
the collecting process in combination with the
exporting process. I suggest we re-word the definition to the following:

"The set of Observation Points, which is the largest aggregatable set of 
Flow information at the Exporting Process is termed an Observation 
Domain. Each Observation Domain presents itself as a unique ID to the 
Collecting Process for identifying the IPFIX Messages it generates. For 
example, a router line card composed of several interfaces with each 
interface being an Observation Point. Every Observation Point is 
associated with an Observation Domain."

Please send your comments.

Thanks
Ganesh

---

From: Sebastian Zander <[email-masked]>
Date: Mon, 16 Feb 2004 10:50:12 +0100

Hi Ganesh,

your example makes sense to me. But then the observation domain would
no longer be the "_largest_ aggregatable set" right? Because if there is
one metering process (followed by exporting process) it could aggregate
the information from different ODs. Hence it is a set but not necessarely
the largest aggregatable!?

Cheers,

Sebastian

Ganesh Sadasivan wrote:
> The set of Observation Points, which is the .largest aggregatable set   
> of Flow information at the Metering Process is termed an Observation   
> The term "largest aggregatable set   of Flow information at the Metering 
> Process" does not seem to be correct.
> Reasons:

-- 
Sebastian Zander                         E-mail: [email-masked]
Fraunhofer FOKUS / METEOR                Tel: +49-30-3463-7287
Kaiserin-Augusta-Allee 31                Fax: +49-30-3463-8287
D-10589 Berlin, Germany                  www.fokus.fraunhofer.de/usr/sebastian.zander

---

From: Benoit Claise <[email-masked]>
Date: Mon, 16 Feb 2004 15:22:42 +0100

Hi,

I can agree that the term FlowSet is maybe not the best one.
But I'm not convinced by any of the new proposal: Record Set, Record 
Array, Record Collection, Record List, Record Batch, Record Group, 
Record Set

For example, a (Option) Template FlowSet will not contain any record, 
only a template.
For example, a specific FlowSet might be used in the future for 
something else: Exporter ID? Synchronization? etc...
For example, ...

Being unable to find an appropriate/better word, I would stick to the 
old one.
But, OK, my view is maybe a little bit biased as I'm used to this 
FlowSet term ;) as this is the one used in NetFlow version 9.

Regards, Benoit.

---

From: Ganesh Sadasivan <[email-masked]>
Date: Mon, 16 Feb 2004 19:32:59 -0800

You mean
" A set of Observation Points that present a a unique ID to the 
Collecting Process for identifying the IPFIX Messages it generates is
called an Observation Domain" .

I would agree with this definition.

Thanks
Ganesh

Sebastian Zander wrote:

>> The set of Observation Points, which is the .largest aggregatable 
>> set   of Flow information at the Metering Process is termed an 
>> Observation   Domain. Each Observation Domain presents itself as a 
>> unique ID to   the Collecting Process for identifying the IPFIX 
>> Messages it   generates.   For example, a router line card composed 
>> of several interfaces with   each interface being an Observation 
>> Point.   Every Observation Point is associated with an Observation 
>> Domain.
>>
>> The term "largest aggregatable set   of Flow information at the 
>> Metering Process" does not seem to be correct.
>> Reasons:
>>     each Obdervation point within an OD could have a separate 
>> metering process.
>> 2. Metering Process itself is independent of the Observation Domain. 
>> For example: If a Metering Process to the entire
>>     IPFIX device (as its scope), which has multiple OD, then it 
>> breaks the above definition . Specifically say there are
>>     2 linecards (each of them being an OD) in a router (IPFIX 
>> device). Say sampling is configured on all interface in
>> All that the Observation Domain defines is a unique way to identify 
>> with the collecting process in combination with the
>> "The set of Observation Points, which is the largest aggregatable set 
>> of Flow information at the Exporting Process is termed an Observation

---

From: Nevil Brownlee <[email-masked]>
Date: Tue, 17 Feb 2004 16:55:19 +1300

Hi Benoit, Stewart and Ganesh

I don't like 'Flowset' either.  It's OK for flow data records,
but seems very odd when it refers to template or option records.

I suggest that 'block' would be a better word.
That way we'd have blocks of template records, flow records, etc.

You already have a block structure defined -
each block starts with
    a template ID (your Flowset ID)
and a length (inclusive in bytes) of the block.

The template ID identifies which template defines the structure
for this block, so lets call it the template ID (rather than Flowset ID).

You have special values of template ID (0..255) used to overload
the field, i.e. they tell us that these are defining various kinds
of template - they're definition blocks rather than data blocks.

Of those, template
  0  defines a new template (your 'IETF exclusive' one)
        [how about just calling that a 'standard' template, 
        i.e. one which uses only information elements defined in
        the IPFIX INFO RFC?]

  2  defines a new template in which the element ids are qualified
        by explicitly specifying their Enterprise Number.
        [how about reserving Enterprise 0 for the IPFIX standard
        information elements, just to be complete?]

  1  defines a new options template, i.e. one which uses info
        elements from a set which describe the meter/exporter
        [these need to be defined in IPFIX INFO].

  3  defines a new options template, with enterprise IDs specified
        for each 'option' element.

I find the name 'options template' confusing.  They're really
information elements which describe the meter/exporter.

How about renaming them 'meter templates?'
That way we'd have
    Flow Templates - specifying Flow Records, using info elements defined
                       in terms of the flows

and Meter Templates - specifying Meter Records, using info elements
                       defined in terms of the meter/exporter.

To go with that, we'd have four kinds of blocks:
  Flow Template block
  Flow block
  Meter Template block
  Meter block

Now, rereading the section on your options templates, I'm confused about
their Scope-related fields.  I understand why that's needed, i.e. to
specify which part of the meter a given subset of the options template
relates to.  If I understand it correctly, 

for each scope (1..X) we have a sub-block specifying the elements
we want reported for that scope.  Then

Option Length is the total length of all the (type,length) pairs
for a scope, and

Option Scope Length is the total length for all the scope sub-blocks.

Have I got that right?  If so, your example on page 50 doesn't
seem to be correct.  Or have I misunderstood??

Also, do we actually need an Option Scope field?  Given that we know
the total block length (from the Length field), we can walk along
the scope sub-blocks until we get to the end.
Or, if we do want something to make it easier to parse, how about
having the 'Option Scope Length' give the number of scope sub-blocks
which follow?

Again, looking at the example on page 50, it shows an options
data record with sub-blocks for each scope (as I thought from the
above discussion).  But the 'options data flowset' description
in section 8.5.3 says the order is 'values for each scope' for
each option in the list.  Which order is intended??

Sorry to send such a rambling posting, but I really find the
'options data' sections very confusing indeed.  I think it would
help if each of the defining sections began with a sentence or two
describing the record structure in terms of header and data,
as you do for the IPFIX message layout in section 7.

If you're happy with my 'blocks, headers and sub-blocks' approach
I could try writing an overview description for you ...

One more thing: I like your section 8.2, Field Type format.
Since you've specified it so clearly, how about referring to it in following
sections, e.g. "a template block is a block header followed by an
ordered list of Field Types" ?  As it is you don't actually use
Field Types anywhere, at least I didn't notice it.

Cheers, Nevil

-----------------------------------------------------------------------
   Nevil Brownlee                 Computer Science Department | ITSS
   Phone: +64 9 373 7599 x88941           The University of Auckland
   FAX: +64 9 373 7021      Private Bag 92019, Auckland, New Zealand

-------------------------------------------------
This mail sent through University of Auckland
http://www.auckland.ac.nz/

---

From: Juergen Quittek <[email-masked]>
Date: Tue, 17 Feb 2004 14:56:06 +0100

Dear all,

>     - add note that if you implement ID 2/3 correctly, you can also process ID 0/1.

This issue is about message formats.

Currently we have 6 binary formats for templates and records:

  - an IETF-exclusive flow template format   (flowset ID 0)
  - a vendor-specified flow template format  (flowset ID 2)
  - an IETF-exclusive option template format   (flowset ID 1)
  - a vendor-specified option template format  (flowset ID 3)

  - a data flowset format
  - an options data record format

I think these are far too many for our purpose.
(BTW: Thanks to Lutz Mark from Fraunhofer FOKUS who
initially raised this point at a discussion in January!)

I see no reason why we should have IETF-exclusive template formats.
the vendor-specific may contain IETF-defined fields, they may even
have only IETF-defined fields.  So, if I implement these two templates,
I can define any flow record format or option record format, respectively.
There is nothing gained by also having IETF-specific template formats.

Consequently, my firat proposal is removing flowset ID's 0 and 1 from
the standard.

Now let's have a look at the data formats.

The only difference between both of them is that in the option data
record format the first N values are scope values.  How many scope values
there are is defined by the corresponding template.  Assuming a template
could specify the number of scope values to be zero, then the option
template record format includes the data flowset format.  If we allow
an option data record format to have a scope length of zero, then there
is no need anymore for implementing a separate data record format.

So, my second proposal is merging data flowset format and option data
record format to a single one.  Note that still the template ID clearly
indicates the kind of record (data flowset or option data record).

Then we arrive at two template formats and a single data format:

  - a flow template format    (flowset ID 2)
  - a option template format  (flowset ID 3)

  - a data format

This achievement is a simplification of the protocol definition
and the stack implementation on both sides.  I do not see any
drawback concerning functionality, performance or safety compared
to the current protocol definition!

Once started, I find it hard to stop :-) :

If we accept a small performance degradation, we could also merge
the two template formats.  The only difference between them is that
one contains scope information and the other one does not.  If we
allow the value of field 'option scope length' to be zero, then
we could use just the option template format for all templates.
Please note that the first 16 bits of all templates still contain
the flowset ID with value 2 indicating a data template and value 3
indicating an option template.  There would be no problem with
identifying the kind of template, but all implementations would
have to implement only one template format.

The only drawback of this solution is that all templates would
have a field called 'scope length', which is currently not included
in the data template format.  Then all data template would increase
in size by the size of this field.

Still it looks like a reasonable simplification of format definition.
Please consider that one of the factors influencing the acceptance
of a standard is its complexity.  And I see this third step of
reducing the number of formats as a simplification ending up with
just two formats:

    a single template format
and
    a single data format.

The template flowset indicates with its first 16 bits whether it
is a flow data template or an option data template.

The data flowset indicates with its first 16 bits to which
template it corresponds.

Thanks,

    Juergen
-- 
Juergen Quittek        [email-masked]        Tel: +49 6221 90511-15
NEC Europe Ltd.,       Network Laboratories        Fax: +49 6221 90511-55
Kurfuersten-Anlage 36, 69115 Heidelberg, Germany   http://www.ccrle.nec.de

---

From: Sebastian Zander <[email-masked]>
Date: Tue, 17 Feb 2004 16:14:30 +0100

Hi Juergen,

having only one format does not only reduce the complexity of the
protocol but also adds the notion of scope to flowsets. Currently a
receiver has no way to determine the scope for flowset data.

Furthermore for per-packet data (e.g. psamp) it would be good to have the
ability to link flowsets to flowsets. Because you have data of the form:

flow-key, seq-number, packet-sample, ...

But the flow-key does not change per packet. To avoid the overhead of
repeating the flow-key for each packet one could have 2 flowsets and
link them:

Flowset 1
flow-id, flow-key, ...

Flowset 2
flow-id, seq-number, packet-sample, ...

In the example you can exchange packet-sample by packet-id used for
packet matching (one-way measurement, trajectories, ...)

See http://www.ietf.org/internet-drafts/draft-pohl-pktid-00.txt

Why should that be different from linking option data to flowset data
e.g. via the template scope?

However I still miss a good example for having explicit scope.
In a lot of cases the scope can be easily deduced by knowing the
semantics of the fields (hich you have to know if you want to do
something with the data). But maybe somebody has one?

Let me take this proposal one step further:

Instead of using separate scope IDs we can use fields. So instead
of defining the scope interface and the field interface we can use
the field interface as scope field (or non scope field). The obvious
drawback is that the number space is halved. The advantage is that
only one number space must be administrated and used and semantically
equivalent fields are not defined twice (again we reduce the complexity).

Cheers,

Sebastian

Juergen Quittek wrote:
[...]

---

From: Juergen Quittek <[email-masked]>
Date: Tue, 17 Feb 2004 16:47:58 +0100

Ganesh,

--On 12.02.2004 17:00 h -0800 Ganesh Sadasivan wrote:

>    - Metering Process definitions say:      Input to the metering process are packets    - We don't want to limit ourselves to IPv4 and IPv6
>
> The architecture spec clearly covers the encapsulated packets in the IP Traffic Flow Definition.
> Otherwise we are leaving out MPLS packets, tunnel packets from being able to be defined
> as flows.
>

You are right.  When identifying this issue we looked at the flow definition in the
requirements document, not in the architecture document where the term is extended.
This is fine, because our charter explicitly mentions Sub-IP information to be
reported.

However, we should have a discussion on which Sub-IP technologies
to cover.  But this issues is an information model issues.

Thanks,

    Juergen

---

From: Ganesh Sadasivan <[email-masked]>
Date: Tue, 17 Feb 2004 13:02:37 -0800

Juergen,

> to cover.  But this issues is an information model issues. 

If we state flow as  "a set of IP packets"  does it not exclude Sub-IP 
technologies like
MPLS? However if we define the flow to include encapsulated IP packets,
 the details of which encapsulation can be discussed in the information 
model
(and not the other way round).
 I think making it clear that a flow could include encapsulated IP 
packets in the
defintion is fundamental even in the [IPFIX-REQS]. Otherwise the 
defintion is
partial and is liable to be mis-interpreted.

Thanks
Ganesh

---

From: Juergen Quittek <[email-masked]>
Date: Wed, 18 Feb 2004 09:08:30 +0100

Ganesh,

--On 17.02.2004 13:02 Uhr -0800 Ganesh Sadasivan wrote:

> If we state flow as  "a set of IP packets"  does it not exclude Sub-IP technologies like
>  the details of which encapsulation can be discussed in the information model
>  I think making it clear that a flow could include encapsulated IP packets in the
> defintion is fundamental even in the [IPFIX-REQS]. Otherwise the defintion is

First, the requirements state what must be considered by the protocol.  I see
no fundamental problem with extending it in the protocol definition.

Second, I still prefer the definition used in the requirements document.
Defining a flow by its IP header fields and other properties available on
the IP layer does not exclude reporting properties of the flow that concern
sub-IP layers, for example the ATM PVC used for carrying the flow or the MPLS
label.

Thanks,

    Juergen

---

From: Maurizio Molina <[email-masked]>
Date: Wed, 18 Feb 2004 16:58:50 +0100

>    - Maurizio suggested text on the mailing list 

For reference, the suggested text was on this message:

http://ipfix.doit.wisc.edu/archive/2407.html

Maurizio

---

From: Ganesh Sadasivan <[email-masked]>
Date: Wed, 18 Feb 2004 08:46:43 -0800

Jurgene,

> no fundamental problem with extending it in the protocol definition.

But this is definition for the basic terminology and shouldn't all the 
documents
concur on the defintion of flow?

> Second, I still prefer the definition used in the requirements document. 

Is it just a preference or is there a justification for  not being explicit?

> sub-IP layers, for example the ATM PVC used for carrying the flow or 

I guess there is difference between reporting and using these fields as 
flow keys.
The intention is that sub-ip header fields would be used as flow keys.
The examples that you have taken of ATM PVC and MPLS is definitly not 
obvious
from the flow definition given in [IPFIX-REQ] . In the defintion below, 
there is an
explicit mention of  network, transport and application header. So why 
not include
sub-ip header. It would make the defintion a lot more clearer.

  A flow is defined as a set of IP packets passing an observation point
   in the network during a certain time interval.  All packets belonging
   to a particular flow have a set of common properties.  Each property
   is defined as the result of applying a function to the values of:

     1. /one or more packet header field (e.g. destination IP address),
         transport header field (e.g. destination port number), or
         application header field (e.g. RTP header fields [RFC1889])
/
      2. one or more characteristics of the packet itself (e.g. number
         of MPLS labels, etc...)

      3. one or more of fields derived from packet treatment (e.g. next
         hop IP address, the output interface, etc...)

Thanks
Ganesh

---

From: Nevil Brownlee <[email-masked]>
Date: Thu, 19 Feb 2004 16:57:13 +1300

Hi Juergen et al:

I like your suggestions for simplifying the IPFIX protocol, i.e. we
finish up with something very simple, like this (using 'block' instead
of 'flowset' to describe it):

  IPFIX packet ::=  <header>  <block>+  # one or more blocks

  <block> ::= <data block> | <template block>

  <data block> ::= <template id> <data for flow>+

  <template block> ::= <flow template> |  <option template>
       # I suggest that 'meter' might be a better word than 'option' here

  <flow template> uses template id = 2, <option template> uses id = 3

  And I like the idea of just having the scope fields in all templates.
  Can you give us some detail on how the scope info would be encoded?
  And can we juggle all this so that NetFlow-v9 records will fit into
  it neatly?

Cheers, Nevil

---

From: Juergen Quittek <[email-masked]>
Date: Thu, 19 Feb 2004 11:12:24 +0100

Hi Nevil,

>   <flow template> uses template id = 2, <option template> uses id = 3

Your notation is very clear and explains a lot within a few lines.

>   Can you give us some detail on how the scope info would be encoded?

Why don't we use the same fields as we use for the other fields.
We can add some fields to the info model, such as lineCardNumber,
systemID, portNumber, ifIndex, etc. (if we cannot use the already
defined ones).

This is already supported by a feature of the information model that
is currently hidden in the XML appendix.  Thomas Dietz, co-editor
of the PSMAP info model, introduced an attribute called "applicability"
to each field.  In the current XML version of the IPFIX info model
each field has an applicability attribute that has one of the following
values: "data", "option", "all".  We could add "scope" to this list,
tag scope fields accordingly and show this attribute in the text part
of the info model document.

With this solution, the scope identifiers would be as extensible as
the other fields of the info model and we would not need another number
space to be maintained by IANA.

>   And can we juggle all this so that NetFlow-v9 records will fit into

Full compatibility will be hard.

I think we can be compatible concerning the data blocks, but for
template blocks we might not be fully compatible.

But I am not sure how important this is:

  - The version number of IPFIX messages is 10.
  - The message header is already incompatible with version 9,
    because it does not anymore contain the 'UNIX Secs' field.
  - Typically the major differences between different NetFlow
    versions were are differences in the record formats.

I still see compatibility with NFv9 as a goal,
but not as an essential one.

Thanks,

    Juergen

---

From: Ganesh Sadasivan <[email-masked]>
Date: Wed, 25 Feb 2004 12:49:54 -0800

Since there are no more opinions on this topic, can we take this as the 
final
definition?

"A set of Observation Points that present a a unique ID to the 
Collecting Process for identifying the IPFIX Messages it generates is
called an Observation Domain. For example, a router line card composed 
of several interfaces with each interface being an Observation
Point. Every Observation Point  is associated with an Observation Domain."

Thanks
Ganesh

Ganesh Sadasivan wrote:

>>>     LC1 and LC2, the scope of metering process spans the entire 
>>> IPFIX device as compared to the OD.
>>>
>>>   "The set of Observation Points, which is the largest aggregatable 
>>> set of Flow information at the Exporting Process is termed an 
>>> Messages it generates. For example, a router line card composed of 
>>> several interfaces with each interface being an Observation Point. 
>>> Every Observation Point is associated with an Observation Domain."
>>>

---

From: Ganesh Sadasivan <[email-masked]>
Date: Wed, 25 Feb 2004 12:44:55 -0800

Does anybody else have an opinion either way on the inclusion
of  encapsulated IP packets in the flow defintion?

Thanks
Ganesh

Ganesh Sadasivan wrote:

> I guess there is difference between reporting and using these fields 
> as flow keys.
> from the flow definition given in [IPFIX-REQ] . In the defintion 
> below, there is an

---

From: Sebastian Zander <[email-masked]>
Date: Fri, 27 Feb 2004 19:09:24 +0100

Encapsulated IP packets are still IP packets and IP packets are
encapsulated in some lower protocol anyway. So as far as the
terminology of the req draft is concerned the change below doesn't
buy us anything but makes the definition more confusing IMHO.

I'm not sure what your definition of sub-ip is but when using
arbitrary fields from lower layers as flow keys the defintion of
flow as "set of _IP_ packets" does no longer hold meaning we're
not talking of IP flows anymore. Or is that what you propose?

Cheers,

Sebastian

Ganesh Sadasivan wrote:
>>      1. one or more packet header field (e.g. destination IP address),

---

From: Stewart Bryant <[email-masked]>
Date: Fri, 27 Feb 2004 18:42:46 +0000

> not talking of IP flows anymore. Or is that what you propose?

Within the IETF sub-IP is MPLS, and you definately have
MPLS flows and subflows.

- Stewart

---

From: Ganesh Sadasivan <[email-masked]>
Date: Fri, 27 Feb 2004 12:22:57 -0800

Hi Sebastian,

Here is a sample list of encapsulated packets  (sure there is more).

- GRE
- IP-in-IP
- IPV6 tunnel (original packet IPV6/IPV4)

There is nesting of tunnels possible in some of the cases.

The possibilities are:
- The flow is defined only for the original IP packet which is
encapsulated within the tunnel.
- The flow for an encapsulated packet can be defined only
on the outer IP header and in such cases the L4 information
would not be available.
- One can define flows based on the outer encapsulations or the
original packet. IMHO we should allow this.

- MPLS packets with IP(v4/v6) as L3.
- MPLS packets with non-IP as L3.

The possibilities are:
- We support  MPLS or any other sub-IP as long as the original packet is IP.
-  One can define flow based on just MPLS labels. Does not make much sense
    to me.

But in any case,  the behavior of  encapsulated packets and packets with 
sub-Ip
must be made clear.

Thanks
Ganesh

Sebastian Zander wrote:

>>>>>>> process are packets    - We don't want to limit ourselves to 
>>>>>>> IPv4 and
>>>>>>> IPv6
>>>>>>>
>>>>>>> The architecture spec clearly covers the encapsulated packets in 
>>>>>>> the
>>>>>>> IP Traffic Flow Definition.
>>>>>> requirements document, not in the architecture document where the 
>>>>>> term
>>>>>> is extended.
>>> But this is definition for the basic terminology and shouldn't all 
>>> the documents
>>> The examples that you have taken of ATM PVC and MPLS is definitly 
>>> not obvious
>>> explicit mention of  network, transport and application header. So 
>>> why not include

---

From: [email-masked]
Date: Sat, 28 Feb 2004 15:34:36 +0100

Hi Ganesh,

comments inline

> on the outer IP header and in such cases the L4 information

And it could be defined on some IP header in the middle etc... 
I think that the current definition covers all of them. It would 
be ok for me to put an additional sentence below the defintion
explaining that.

> - MPLS packets with non-IP as L3.

I'm not against it. This could have an impact on the flow definition. 
To be discussed in Seoul I guess...
 
> -  One can define flow based on just MPLS labels. Does not make much sense

Wouldn't it make sense to have information about MPLS flows (as Stewart
said) for TE? I don't know about the amount of non-IP traffic but in 
case it would be a significant amount you can't simply ignored it.

Cheers,

Sebastian

---

From: Juergen Quittek <[email-masked]>
Date: Sun, 29 Feb 2004 11:29:10 +0100

Dear all,

I have a need for clarification:

At an IP device, or better: at a device that handles the header of
an IP packet (whether it arrives encapsulated or not) I have no problem
with considering the sub-IP layers as flow attributes or even part of
the flow key.

But at non-IP devices, for example an ATM or MPLS switch, where
packets/frames/cells are just forwarded based on their label/VPI/VCI
we enter a gray area: If we just look at the labels we need some
higher level/outside information in order to know that there are IP
packets carried with a certain label. Still we do not know if the label
is exclusively used for IP packets, for example when running Ethernet
over MPLS.

So, if at the observation point no IP packet is visible to be observed,
then why should we call this an observed IP flow?

A related issue is IPsec. The encapsulated IP flow is not visible at
the IP router forwarding it.  Hence, is cannot observe it at the observation
point. Hence, we do not consider it an observed IP flow.  This would be
different in a situation where the router holds a copy of the key that is
needed for decoding the packets. If done so and the IP header can be observed,
I would call it an observed IP flow (although this smells like surveillance).

Thanks,

    Juergen

---

From: Juergen Quittek <[email-masked]>
Date: Sun, 29 Feb 2004 11:30:06 +0100

Dear all,

Thanks,

    Juergen