Subject: Re: [ipfix-protocol] IPFIX Implementation Report

From: Benoit Claise <[email-masked]>
Date: Sun, 01 Aug 2004 16:56:46 +0200

Hi Luca,

Thanks for your draft!
Let me comment on it, from a IPFIX protocol specification point of view

You wrote
   "it has the drawback of requiring a different
   template format from IETF-defined fields where a numeric identifier
   is used. This difference requires that both probe and collector
   support both formats, with and without the PEN. It should be
   investigated whether it is feasible to merge both template formats
   (IETF and vendor-specific extensions) into one, so that the
   implementation complexity is reduced. "
Good point, already taken care of in the draft-ietf-ipfix-protocol-04.txt

You wrote
     "Hence using SCTP, template
     sending can be optimized, as it is not really necessary to
     periodically send templates, because it is now possible to detected
     whether the collector has disconnected."
Correct, this is addressed in this draft-ietf-ipfix-protocol-04.txt
However, I put the following action item, to make sure the draft is 
checked for consistency throughout all the sections.
   PROTO-25: The section 11 "Template Management" will have to updated
       according to the transport protocol.
       - For example, the point 2 of the section "Template Management".
         Remark: the template management will vary with TCP, SCTP, 
         etc...

You wrote, as an improvement
     "- As with NetFlow, the IPFIX flow format is fixed. Consider the
     following scenario: a user requires in-depth ICMP flow analysis
     (e.g. ICMP code and type). There are basically two solutions:
     either a probe/collector agree on two different templates (one for
     ICMP and one for non-ICMP flows) or they agree on a common template
     for all protocols with the outcome that some flow fields are empty
     when non-ICMP flows are emitted. The ideal solution to this kind of
     scenario is dynamic flows, where emitted flows can have all the
     template fields or a subset of them according to the flow kind."
What you are referring to is the metering process capabilities, which is 
out of scope of this WG.
In IPFIX, we standardize the IP Flow Information eXport.
BTW, you comment that NetFlow V9 has got fixed flow format; this is not 
quite correct. Depending on the traffic, the templates may change.
But again, this is a Metering Process issue; what is important here is 
that the IPFIX protocol gives you the freedom to do what you want within 
the metering process.

You wrote:
     "Finally it should be explored whether it still makes sense to have
     a flow identifier in the flow header when SCTP is taking care of
     issues such as flow retransmission or duplicate discard."
I'm not too sure which field you are talking about?

You wrote:
     - Limited innovation, if any, with respect to existing technologies
     such as NetFlow.
I would say at it was agreed upon at the time of the protocol selection.
 From the architecture draft:
   "The working group went through an extensive evaluation of the various
   existing protocols that were available, weighing the level of
   compliance with the requirements, and finally selected NetFlow V9
   with minor modification as the basis for the IPFIX protocol"
Now up to you to shoot for IPFIX version 2 when this version is out ;)

You wrote:
     "Lack of configuration and monitoring that can be solved through
     an SNMP MIB."
 From the architecture draft
     "Note that the IPFIX system does not provide for remote configuration
   of an IPFIX device. "

Regards, Benoit.

> Dear all,
> I have written an IPFIX implementation report internet draft available 
> here: http://luca.ntop.org/draft-deri-ipfix-impl-00.txt
>
> Feel free to send me your comments.
>
> Regards, Luca
>

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: Luca Deri <[email-masked]>
Date: Tue, 03 Aug 2004 15:25:02 +0200

Hi Benoit,

> Thanks for your draft!

thanks for all you've done so far for IPFIX. I'm following IPFIX since 
about a year, so I apologize if some of my statements have been 
addressed already. As you can see my implementation was based on draft-3 
whereas now draft-4 is out.

Before I comment your mail, I would like to tell my main concerns in a 
few lines:
- I understand that IPFIX is based on NetFlow v9 but you should somehow 
avoid sentences like "The FlowSet ID value of 0 is reserved for 
backwards compatibility..."/"This definition is backwards compatible 
with Netflow v9". As this is IPFIX v1 I don't see any backward 
compatibility issue (even if IPFIX is based on NF v9) so this has to be 
removed. IMHO this WG should not pay attention with compatibilities with 
commercial protocols as IMHO this is out of scope. This is particular if 
backward compatibility means more complexity.
- IPFIX has been designed to be an efficient way to export flows. Now it 
seems to me that from draft to draft all the process is becoming 
increasingly complex. For instance in draft 4 it is specified that in 
addition to SCTP also SCTP-PR must be supported. This would further 
reduce the number of implementations because it increases complexity 
while not providing real added value. If you don't want to turn IPFIX 
into a protocol for an elite of people (see what happened to SNMPv3) 
then you need to reduce (not increase) the effort required to implement 
the protocol. Other examples are sentences like "The Templates FlowSet 
and Option Template FlowSet MUST be sent on stream zero with full 
reliability."
 - I believe that moving away from UDP has many more implications that 
what you probably have expected. It's not an implementation issue but a 
chance to design a protocol without the legacy imposed by UDP. For 
instance under SCTP I don't really see the need to maintain the same UDP 
header.

In a nutshell: if you want to standardize NetFlow v9 rebranding it into 
IPFIX I'm fine. Instead if you pretend to create a flow protocol baed on 
NFv9 that is also suppose to use lessons learnt in the past 10 years 
(just to make you an example have you read this paper "Building a Better 
NetFlow",http://www.caida.org/outreach/papers/2004/betternetflow/betternetflow.pdf 
?) then I'm not happy at all.

Now back to your questions.

> Good point, already taken care of in the draft-ietf-ipfix-protocol-04.txt

Yes and not, because the PEN (SNMP concept that has nothing to do with 
NF) is still around for vendor-specific fields whereas it's not 
necessary for IETF (you have used the trick of the first bytes to avoid 
that). I'm asking myself: as a flow is associated with a template that 
is coming from a precise probe, why do you need to have such PEN? Namely
- either you remove PEN and leave some field ID for vendor-specific fields
- or you use PEN also for IETF.
In any case you should not increase the work of somebody like me 
(ntop.org) who wants to export a new field type. I feel that using PEN 
will prevent people from adding new fields, or (as happened with past NF 
version) will push people to jeopardize the protocol just to avoid the 
need to play with PENs.

>         Remark: the template management will vary with TCP, SCTP, 

This is a total nightmare for both probe and collector implementors. 
This is because you have too many exceptions as the IPFIX behaviour 
changes according to the transport used. IMHO is SCTP is the default 
protocol then you must optimize IPFIX for SCTP, and not for UDP that's 
now optional. My feeling is that people will use SCTP just for sending 
templates and UDP for exporting flows. In fact as you have seen from my 
report SCTP has the drawback to send ACKs back to the probe that is 
probably not what most people want.

> What you are referring to is the metering process capabilities, which 
> is out of scope of this WG.

I'm saying that many people jeorpardize NetFlow fields (see ICMP 
code/type in many NetFlow v5 flows) and as this is a common pratice I 
believe it should be addressed by IPFIX. I understand that NF bets on 
the concept of flows and templates, but in practice this creates complexity.

> BTW, you comment that NetFlow V9 has got fixed flow format; this is 
> not quite correct. Depending on the traffic, the templates may change.

Right but (1) this is a hack (2) this increases application complexity. 
Instead without templates but with inline fields (see my nFlow proposal) 
that is basically a merge between templates and data, everything is much 
much simpler. In particular when you move from RFC to implementation.

> that the IPFIX protocol gives you the freedom to do what you want 
> within the metering process.
>
> You wrote:

I'm talking about the flow sequence number. Please note that in draft 
for there is a MUST SHOULD (typo I guess) that needs to be fixed ("This 
value MUST SHOULD be used by the Collecting Process to...").

> Now up to you to shoot for IPFIX version 2 when this version is out ;)

Well as v1 is not out yet, I don't see why we whould wait until v2 for 
fixing these issues. Isn't it?

>     "Note that the IPFIX system does not provide for remote configuration

I can accept this for configuration but not for monitoring. In fact you 
should at least remove the options (template) and let a collector be 
informed by a probe via SNMP trap, and let a collector query the probe. 
You somehow use a flow for sending information that has nothing to do 
with flows. I hope you agree with me.

Regards, Luca

> Regards, Benoit.
>
>> Dear all,
>> I have written an IPFIX implementation report internet draft 
>> available here: http://luca.ntop.org/draft-deri-ipfix-impl-00.txt
>>

-- 
Luca Deri <[email-masked]>	http://luca.ntop.org/
Hacker: someone who loves to program and enjoys being
clever about it - Richard Stallman

---

From: Benoit Claise <[email-masked]>
Date: Sat, 07 Aug 2004 07:26:13 +0200

Luca,

> addressed already. As you can see my implementation was based on 
> draft-3 whereas now draft-4 is out.
>
> - I understand that IPFIX is based on NetFlow v9 but you should 
> somehow avoid sentences like "The FlowSet ID value of 0 is reserved 
> for backwards compatibility..."/"This definition is backwards 
> compatible with Netflow v9". As this is IPFIX v1 I don't see any 
> backward compatibility issue (even if IPFIX is based on NF v9) so this 
> has to be removed. IMHO this WG should not pay attention with 
> compatibilities with commercial protocols as IMHO this is out of 
> scope. This is particular if backward compatibility means more 
> complexity.

Corrected in the next version of the draft.

> - IPFIX has been designed to be an efficient way to export flows. Now 
> it seems to me that from draft to draft all the process is becoming 
> then you need to reduce (not increase) the effort required to 
> implement the protocol. Other examples are sentences like "The 
> Templates FlowSet and Option Template FlowSet MUST be sent on stream 
> zero with full reliability."
> what you probably have expected. 

The IETF is mandating moving away from UDP, not me.

> It's not an implementation issue but a chance to design a protocol 
> without the legacy imposed by UDP. For instance under SCTP I don't 
> really see the need to maintain the same UDP header.
>
> In a nutshell: if you want to standardize NetFlow v9 rebranding it 
> into IPFIX I'm fine. Instead if you pretend to create a flow protocol 
> baed on NFv9 that is also suppose to use lessons learnt in the past 10 
> years (just to make you an example have you read this paper "Building 
> a Better 

Yes, I read it and it concerns the metering process, which we don't 
standardize in IPFIX.

> necessary for IETF (you have used the trick of the first bytes to 
> avoid that). I'm asking myself: as a flow is associated with a 
> template that is coming from a precise probe, why do you need to have 
> such PEN? Namely
> will prevent people from adding new fields, or (as happened with past 
> NF version) will push people to jeopardize the protocol just to avoid 
> the need to play with PENs.
>
>> You wrote
> now optional. 

will be improved in the next version of the draft.

> My feeling is that people will use SCTP just for sending templates and 
> UDP for exporting flows. In fact as you have seen from my report SCTP 
> has the drawback to send ACKs back to the probe that is probably not 
> what most people want.
>
> Right but (1) this is a hack 

this has nothing to do with a hack.

> (2) this increases application complexity. Instead without templates 
> but with inline fields (see my nFlow proposal) that is basically a 
> merge between templates and data, everything is much much simpler. In 
> particular when you move from RFC to implementation.
>
>> But again, this is a Metering Process issue; what is important here 
>> is that the IPFIX protocol gives you the freedom to do what you want 
> for there is a MUST SHOULD (typo I guess) that needs to be fixed 
> ("This value MUST SHOULD be used by the Collecting Process to...").

corrected.

regards, Benoit

> I can accept this for configuration but not for monitoring. In fact 
> you should at least remove the options (template) and let a collector 
> be informed by a probe via SNMP trap, and let a collector query the 
> probe. You somehow use a flow for sending information that has nothing 
> to do with flows. I hope you agree with me.
>
> Regards, Luca
>
>> Regards, Benoit.
>>
>>> Dear all,