Subject: RE: [ipfix] Option templates issues

From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" <[email-masked]>
Date: Wed, 19 Nov 2003 15:06:22 -0500

Mark,

  I think this discussion may be wandering off into hypotheticals versus
specific use cases.

  In the current (pre-v9) netflow implementations, there is no additional
metadata being sent, and this works fine.  So the analogy to HELLO's seems a
bit overstated (i.e. I would picture an exporter only having 1 or 2
different statistics blocks it may send [if any])

  You've cited a specific example of metering statistics which summarize the
behavior of the exporter at intermittent points in time.  This sounds
worthwile, but I would imagine that like the flow data itself, different
vendors may chose to send varying sets of information fields in their
statistics records.

  Scoping issues sound to me like it is just a matter of having the
appropriate identifier transferred as part of the record.  I.e. if I want
statistics on a per interface basis, then the record may be:

  ingressPort
  totalFlows
  lostFlows
  lostPackets
   .. whatever

  I don't really see how this differs from general flow data.  Or why you
wouldn't want to use the same Information modeling (and encoding) techniques
hammered out so far.  In fact it seems almost like a specialized form of
aggregation.  Introducing different models for largely similar information
introduces complexity (and probably inconsistency in the long term).

  So, I'd really like to see as few mechanisms as possible for moving data,
unless we simply can't get by with the set available.  In this case I think
we can.

Regards,

  Jeff Meyer

> -----Original Message-----
> From: majordomo listserver 
> [mailto:[email-masked]]On Behalf
> Of Mark Fullmer
> Sent: Wednesday, November 19, 2003 11:01 AM
> To: [email-masked]
> Subject: [ipfix] Option templates issues
> 
> 
> 
> A few more examples where TLV's seem a more natural approach:
> 
>    The exporter needs to present it's ID, which is usually the source
>    IP address inside the packet so that clients that connect 
> via proxies
>    get the correct exporter ID.  This is an item that needs to be sent
>    once right after connection setup.
> 
>    A proxy might want to indicate to the collector it's 
> inline or maybe 
> that
>    it's doing proxy aggregation.  With TLV's this is easy. 
> With templates
>    the proxy now has to construct a template before sending any data.
>    It has no way of knowing that the template ID will not be used by
>    the collector in the future so now it has to provide template ID
>    mappings.  This could get really ugly if template ID's get encoded
>    in fields later.  Proxies that provide a fanout (receive from the
>    exporter, send to many collectors) are common in existing NetFlow
>    deployments.
> 
>    I'm also looking at how to add the reliable fail-over option which
>    also requires protocol messages that don't benefit from 
> the template
>    style encodings.
> 
> mark
> 
> On Nov 18, 2003, at 8:43 PM, Mark Fullmer wrote:
> 
> >
> > On one level I would agree if the difference was only the 
> namespace, 
> > but
> > options templates also define a scope for the option data fields.
> >
> > Think of protocol messages like a HELLO.  With the 
> template/data model
> > first you have to send a HELLO template then send the HELLO data 
> > message.
> > This just seems a bit silly and potentially complex when 
> you have many
> > many different ways a HELLO could be sent (single options 
> template, vs
> > mixing it with other data).
> >
> > The template model is very useful for the flow data because of the 
> > overhead
> > it saves, I'm just not sure why we would not want to stick to 
> > traditional
> > TLV's with fixed record formats for the other messages.
> >
> > Another way to put this is if we just use the field type, 
> someone is 
> > going
> > to have to write up a "what if" scenario for every bit of 
> option data.
> > For example in the METERSTAT example what if the collector only 
> > receives
> > a template with lost_bytes and no other data.  What does it do?  
> > Ignore it
> > because it's incomplete, do the best it can by adding a 
> local timestamp
> > and assume the other stats are zero?  What if it gets two 
> templates one
> > with lost_bytes and lost_pkts and the other with just a lost_bytes, 
> > what
> > then?
> >
> > mark
> 
> 
> --
> Help        mailto:[email-masked] and say "help" 
> in message body
> Unsubscribe mailto:[email-masked] and say
> "unsubscribe ipfix" in message body
> Archive     http://ipfix.doit.wisc.edu/archive/
> 

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: Mark Fullmer <[email-masked]>
Date: Thu, 20 Nov 2003 15:28:10 -0500

These aren't hypothetical.  A robust collector needs to be able to
deal with anything the exporter throws at it.  There are messages
from the exporter to the collector that have a fixed format, it's
much much easier for the collector to deal with these using TLV's.

There needs to be a well defined standard for certain types of
messages, including loss stats.  Vendors may choose to augment this
with their own messages, including an aggregated flow which has
interface based loss information.

mark