Subject: Re: [ipfix] Option templates issues

From: Mark Fullmer <[email-masked]>
Date: Sun, 11 Jan 2004 20:27:35 -0500

It's common for a router to send flow data to a proxy which will then
fan-out to multiple collectors.  For example one collector may be an IDS
while the other is used for billing.

This is why we need to explicitly send the exporter ID (IP address) 
inside
the IPFIX message.  Relying on the transport source IP address becomes
a problem when proxies are involved.  With NetFlow running over UDP you
could work around this problem with the proxy spoofing the source IP
address to the downstream collectors.  This work around is ugly and
problematic given current best practices drop spoofed sources.  It's not
even practical for IPFIX over TCP or SCTP.

We have a requirement for extensibility.  Tal (I believe) suggested that
reliable fail-over be at least specified to fulfill this requirement.

My concern is that trying to add this extension within the options 
framework
is more complicated than it should be and we need a more generic 
mechanism
for protocol extensions.

Another concern is we probably have field(s) in the fixed header that 
don't
belong there.  An example would be the current unix_secs.  NetFlow v9
omitted unix_nsecs from the header thus breaking sub-second timestamps
that were available in prior NetFlow versions.  This is a static header
field, which can't be changed without changing the protocol version thus
totally incompatible with existing deployments.  We need to be able to
add both information and protocol elements to IPFIX without breaking
collectors.

Slightly unrelated is another need for a HELLO or keep-alive.  Currently
there's no way for a collector to tell the difference between a broken
exporter configuration and a quiet exporter generating no data.  This
is less of an issue for TCP/SCTP transports which will eventually close
due to timeouts, but that's still usually on the order of hours for TCP.

A fixed option is really nothing more than a TLV....We have 3-255 to 
work
with.  If we require that ID's 3-255 all have a Length following the
template ID we can add new features without breaking existing 
collectors.

Just an example to clarify that TLV's are not a new encoding scheme and
already in use by IPFIX just with a different terminology:

>  8.3 Data FlowSet Format
>
>    The format of the Data FlowSet is as follows:
>
>     0                   1                   2                   3
>     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>    |    FlowSet ID = Template ID   |          Length               |
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>    |   Record 1 - Field Value 1    |   Record 1 - Field Value 2    |
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>    |   Record 1 - Field Value 3    |             ...               |
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

FlowSetID == T
Length    == L
<rest>    == V

This holds true for all existing FlowSet definitions.

mark

On Jan 7, 2004, at 2:04 AM, Meyer, Jeffrey D (http://usage.fc.hp.c) 
wrote:

> Mark,
>
>   Not having seen requirements or discussion for "proxies", I'm having 
> a hard time conceptualizing this new requirement.
>
>   Since option template Id's and flow record template Id's are 
> currently dynamically determined, I guess what you are arguing is that 
> options should somehow be fixed.
>
>   Since a "hello" option has not even been defined to date, and your 
> use case seems to be centered around reliable failover, I think there 
> are a lot of tangled concerns going on here.  Could you elaborate a 
> bit more on this scenario?
>
> -- Jeff
>
> -----Original Message-----
> From: Mark Fullmer [mailto:[email-masked]]
> Sent: Tuesday, January 06, 2004 10:36 PM
> To: Meyer, Jeffrey D (http://usage.fc.hp.c)
> Cc: 'Ipfix Wg' (E-mail)
> Subject: Re: [ipfix] Option templates issues
>
>
> Consider a proxy.  If the proxy needs to use an option ID to say
> maintain
> HELLO's for a reliable fail-over mechanism it will have to pick a
> template ID
> that potentially clashes with a future one used by an exporter.  So now
> a proxy needs to track all inbound template ID's, map them to outbound
> ID's,
> and rewrite fields in the messages that refer to the template ID's.
>
> mark
>
> On Dec 23, 2003, at 2:12 PM, Meyer, Jeffrey D (http://usage.fc.hp.c)
> wrote:
>
>> Mark,
>>
>>   Agreed, it is a choice, and both will work, it is a questions of
>> anticipated extensibility.  Fewer fields in the header and most
>> information elements defined in a consistent manner seems to me to
>> address the approach of "Make it as simple as possible and no
>> simpler".
>>
>>   Since the requirement appears to be addressable with a single
>> mechanism, i.e. a flag distinguishing options from flow elements, and
>> reusing the information modeling, I still don't see the benefit in
>> introducing more mechanisms which need to be implemented and tested on
>> both ends.
>>
>> -- Jeff
>>
>> -----Original Message-----
>> From: Mark Fullmer [mailto:[email-masked]]
>> Sent: Tuesday, December 23, 2003 10:11 AM
>> To: Meyer, Jeffrey D (http://usage.fc.hp.c)
>> Cc: [email-masked]; Benoit Claise
>> Subject: Re: [ipfix] Option templates issues
>>
>>
>> There are two issues here
>>
>>    1) How to provide extra information about the flow data and 
>> metering
>>       process.
>>
>>    2) How to allow for protocol extensions.
>>
>> If we add a required "key" or "command" field to the options templates
>> then I think what we have will work given the proper text
>> describing what IE's MAY, SHOULD, and MUST be present in an options
>> template with a specific key.  We may want to even have a separate
>> IE namespace for the options....
>>
>> The protocol extension I'm still of the opinion should be handled
>> with an extensible header.  Examples of protocol extensions could be
>>
>>     Reliable fail-over.
>>
>>     Authentication.
>>
>>     The optional time stamp for the split-time model.
>>
>>     Hooks for proxies.
>>
>>     Future unanticipated extensions.
>>
>> And yes, all of this could be stuffed in an options template but the
>> same
>> argument could be used to do away with the protocol header entirely.
>>
>> mark
>>
>>
>> On Dec 22, 2003, at 11:17 AM, Meyer, Jeffrey D (http://usage.fc.hp.c)
>> wrote:
>>
>>> Hi,
>>>
>>>   Again, I'd point out, that as Benoit indicates, there are likely to
>>> be different things we want to send as "options".
>>>
>>>   Rather than inventing an entirely new model, just make option a
>>> flag on the template and REUSE what we already have for data records.
>>> It's that simple!
>>>
>>> -- Jeff
>>>
>>> -----Original Message-----
>>> From: majordomo listserver [mailto:[email-masked]]On
>>> Behalf
>>> Of Benoit Claise
>>> Sent: Monday, December 22, 2003 4:04 AM
>>> To: Mark Fullmer
>>> Cc: [email-masked]
>>> Subject: Re: [ipfix] Option templates issues
>>>
>>>
>>> Mark,
>>>
>>> [sorry for the delay, i've been absent from the list for a while]
>>>
>>> I understand the concern but let me ask a few questions.
>>> What if the metering process can't "meter" one of them? Ex: flows not
>>> exported due to resource starvation
>>> What if we would like to export some extra data types? Ex: IPv6
>>> packets
>>> and bytes dropped
>>> So what if the METERSTAT struct is slightly modified?
>>>
>>> Ganesh had a good point I think:
>>>     If the group can come up with fixed structs that you mentioned
>>> below,
>>>     then I think what you propose can be adopted.
>>>
>>> Aren't we going to spend a long time to determine what MUST be
>>> exported
>>> in this/those structure(s)?
>>> What you defined in your METERSTAT makes perfectly from a collector
>>> point of view but will not always be possible to collect on the
>>> exporter?
>>> And it does perfectly sense now, but what about in 1 year? So would 
>>> we
>>> have a new data type with a new slightly different structure?
>>>
>>> I'm just wondering what we should say in the protocol draft:
>>>     1. SHOULD export a METERSTAT data type(as described in your
>>> email)?
>>> What if one data type can't be exported, we don't send any metering
>>> stats "message"?
>>>     2. MUST export a metering statistics Options Templates that 
>>> SHOULD
>>> contain lost_flows, lost_flows_pkts, lost_flows_bytes, lost_pkts,
>>> lost_bytes, time.
>>>     3. MUST export a metering statistics Options Templates that MUST
>>> contain X, Y, and SHOULD contain Z
>>>     4. MUST export a metering statistics Options Templates that
>>> contain
>>> at least X, Y
>>>
>>> Actually, I think the flexibility of the Options Template is
>>> benefitial
>>> for any data types.
>>>
>>> Now, this is a different story for the Hello, Failover, etc... types
>>> of
>>> messages that you spoke about, as the goal is totally different!
>>>
>>> Regards, Benoit.
>>>
>>>>
>>>> I'd like to propose an alternative mechanism for sending auxiliary
>>>> data such
>>>> as time synch messages, protocol hellos (needed for fail-over),
>>>> metering
>>>> process loss statistics (ie packets/flows lost due to resource
>>>> starvation),
>>>> etc.
>>>>
>>>> The existing framework allows "other" data to be sent in options
>>>> templates.
>>>> Take for example a metering process stat message.  Lets say the
>>>> metering
>>>> process will periodically report statistics such as
>>>>
>>>>   struct METERSTAT {
>>>>     u_int32_t lost_flows;       /* flows not exported due to 
>>>> resource
>>>> starvation */
>>>>     u_int32_t lost_flows_pkts;  /* packets in the lost flows */
>>>>     u_int32_t lost_flows_bytes; /* bytes in the lost flows */
>>>>     u_int32_t lost_pkts;   /* packets dropped by metering process */
>>>>     u_int32_t lost_bytes;  /* bytes dropped by metering process */
>>>>     time_t    now;         /* when this record was generated */
>>>>   };
>>>>
>>>>   The question becomes how to encode this with an options template.
>>>>
>>>>   Each field in the record could have a type (6 types), then the
>>>> options template
>>>>   would list each of the 6 types.  The problem here is during
>>>> decoding.  What
>>>>   will the collector do if only 5 of these items show up.  The 
>>>> packet
>>>> decode
>>>>   process is also not very straightforward when just data is sent
>>>> (other option
>>>>   fields could be in the same template) because it will have to
>>>> figure
>>>> out
>>>>   what to do based on the field type and there's no way to group
>>>> fields.
>>>
>>>>
>>>>   The other option is to call METERSTAT a type (of length 24).  Now
>>>> the
>>>>   decode process can key off of the type and all the fields will
>>>> always be
>>>>   there because METERSTAT is fixed and defined in the protocol
>>>> document just
>>>>   like other fields like PACKETS, BYTES, PROTOCOL, etc.  The issue I
>>>> have here
>>>>   is why are we bothering with a template/data model for this.  Why
>>>> not just
>>>>   use traditional TLV's.
>>>>
>>>>   So what I'd like to see (I think) is to take one of the existing
>>>> reserved
>>>>   flowset ID's, lets say 3 and use it for generic TLV data 
>>>> encodings.
>>>> Then
>>>>   define structured data in the protocol such as METERSTAT.  IMHO
>>>> this
>>>> allows us
>>>>   to address a number of other issues including how to add HELLO's
>>>> for
>>>> the
>>>>   (optional) reliable failover, how to add aux flags like "this is
>>>> potentially
>>>>   a duplicate PDU because I failed over", etc.
>>>>
>>>>   The TLV encodings would only be used for the low bit-rate protocol
>>>> messages,
>>>>   NOT the high bandwidth flow data.  I'm also not proposing the
>>>> option
>>>> templates
>>>>   go away, just that we use a more traditional method for encoding
>>>> the
>>>> non flow
>>>>   data.
>>>>
>>>> mark
>>>>
>>>>
>>>>
>>>> -- 
>>>> Help        mailto:[email-masked] and say "help" in
>>>> message body
>>>> Unsubscribe mailto:[email-masked] and say
>>>> "unsubscribe ipfix" in message body
>>>> Archive     http://ipfix.doit.wisc.edu/archive/
>>>
>>>
>>>
>>>
>>> --
>>> Help        mailto:[email-masked] and say "help" in
>>> message body
>>> Unsubscribe mailto:[email-masked] and say
>>> "unsubscribe ipfix" in message body
>>> Archive     http://ipfix.doit.wisc.edu/archive/
>>>
>>
>
>
> --
> Help        mailto:[email-masked] and say "help" in 
> message body
> Unsubscribe mailto:[email-masked] and say
> "unsubscribe ipfix" in message body
> Archive     http://ipfix.doit.wisc.edu/archive/
>

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: Mark Fullmer <[email-masked]>
Date: Sun, 11 Jan 2004 20:41:24 -0500

Yes.

On Jan 8, 2004, at 7:18 AM, Benoit Claise wrote:

> Hi,
>
> In order to progress with the protocol draft... is there a consensus 
> for the following concepts?
>
>   1) How to provide extra information about the flow data and metering
>      process.
>
> 1) How to provide extra information about the flow data and metering 
> process.
> Typical example: the meterstats.
> An option template will be used
> The protocol draft will specify what the minimal set of data types is 
> required
> A new data type (referred to ipfixOption by Jeff) would be used to 
> identify specific options templates
> This would help the collector answering the question:
> switch (ipfixOption)
>   case METER_STATS
>     do that.
>   case ...
>     do something else.
>
> Typical example: hello, exporter ID, authentication, etc...
> No options template: an extension to the header will be used.
>
> Regards, Benoit.
>
> Mark,
>
>    1) How to provide extra information about the flow data and metering
>       process.
>
> And it does perfectly sense now, but what about in 1 year? So would we
>     2. MUST export a metering statistics Options Templates that SHOULD
>     u_int32_t lost_flows;       /* flows not exported due to resource
> starvation */
>   will the collector do if only 5 of these items show up.  The packet
> decode
>   flowset ID's, lets say 3 and use it for generic TLV data encodings.
> Then

---

From: Stewart Bryant <[email-masked]>
Date: Mon, 12 Jan 2004 11:32:27 +0000

Mark Fullmer wrote:

> This is why we need to explicitly send the exporter ID (IP address) inside
> the IPFIX message.  
 >
> Relying on the transport source IP address becomes

Will we ever need to run IPFIX though a NAT? ie, can we reply on the
IP address to be globally unique? If so we only need to worry
about the v4/v6 issues with including it in the IPFIX message.
If not then perhaps we need to use a real UID rather than IP address
plus local ID.

> is more complicated than it should be and we need a more generic mechanism
> for protocol extensions.
> 
> Another concern is we probably have field(s) in the fixed header that don't

Mark is right, the whole system would be much more flexible if we stripped
the fixed header down to just the version and length, and then encoded the
rest as a flowset.

> due to timeouts, but that's still usually on the order of hours for TCP.

Won't we see templates from time to time?

> A fixed option is really nothing more than a TLV....We have 3-255 to work
> template ID we can add new features without breaking existing collectors.
> 
> On Jan 7, 2004, at 2:04 AM, Meyer, Jeffrey D (http://usage.fc.hp.c) wrote:
> 
>> Mark,
>>

---

From: Mark Fullmer <[email-masked]>
Date: Mon, 12 Jan 2004 09:36:07 -0500

> If not then perhaps we need to use a real UID rather than IP address

I don't think this matters.  A NAT would not modify this field, it's
there so a collector knows the real IP address of the exporter.  This
is useful in itself as a unique ID or possibly to allow the collector
to query the exporter, say with SNMP.

> Won't we see templates from time to time?

Depends.  If the transport is reliable there is no need to refresh
template state.

--
mark

---

From: Stewart Bryant <[email-masked]>
Date: Mon, 12 Jan 2004 16:06:25 +0000

> to query the exporter, say with SNMP.

But the IP address on the far side of a NAT (which is the one that would be
in the IPFIX message) is not globally unique and therefore you could have
an alias.

> Depends.  If the transport is reliable there is no need to refresh

But if there is a reliable transport it will have some form of hello,
and it's just a question of tuning its timing.

- Stewart

---

From: Mark Fullmer <[email-masked]>
Date: Mon, 12 Jan 2004 11:31:48 -0500

> in the IPFIX message) is not globally unique and therefore you could 

Okay, so if an exporter is behind a NAT this field would not be valid.

I'll leave this at it's a potential problem...

> and it's just a question of tuning its timing.

TCP keepalives are generally two hours and not easily modified.

Back to template retransmissions....The retransmission timer is 
currently
not specified in the protocol, so the collector wouldn't know how to set
a timeout alarm.