Subject: Re: [IPFIX] DRAFT IPFIX minutes from IETF 80

From: Lothar Braun <[email-masked]>
Date: Fri, 1 Apr 2011 10:04:53 +0200

Hi Gerhard,

On Mar 31, 2011, at 9:58 PM, Gerhard Muenz wrote:
>>> Brian Trammell presented a report on the 'DEMONS IPFIX
>>> Interoperability Test,' held in Prague, 24-25 March.  This was the
>>> fourth such event for IPFIX, with eight implementations (4
>>> exporters, 3 collectors) being tested.  Interoperation was complete
>>> for UDP, less so for TCP; for SCTP it was dependent on the SCTP
>>> implementations used.  Quite a few issues came to light (see the
>>> slides), many were fixed during the event. Several people commented
>>> on SCTP implementation issues, suggesting that perhaps "template
>>> handling is needlessly complicated in the (IPFIX) protocol."
>> 
>> The guidance regarding the complexity of template handling comes from
>> the fact that nobody implemented template withdrawal or template
>> number reuse, not from SCTP implementation issues.
> 
> That does not seem to be true. Vermont sends Template withdrawal messages and reuses Template IDs if you trigger an appropriate reconfiguration at runtime. Maybe this was just not tested at the interop event.

Oh, I haven't been aware that this has actually been implemented. Hence, it wasn't tested at the Interop.

Best regards,
  Lothar

--
Lothar Braun
Chair for Network Architectures and Services (I8)
Department of Informatics
Technische Universität München
Boltzmannstr. 3, 85748 Garching bei München, Germany
Phone:  +49 89 289-18010       Fax: +49 89 289-18033
E-mail: [email-masked]

---

From: Paul Aitken <[email-masked]>
Date: Fri, 01 Apr 2011 09:37:05 +0100

Gerhard,

> Vermont sends Template withdrawal messages and reuses Template IDs if 
> you trigger an appropriate reconfiguration at runtime.

Is this is transport dependant? Or do you not export over UDP?

Obviously it's a bad idea to reuse template IDs when exporting over UDP.

So 5101 says:

    If UDP is selected as the transport protocol, the Template Withdraw
    Messages MUST NOT be used, as this method is inefficient due to the
    unreliable nature of UDP.

Of course "Withdraw" should be "Withdrawal" (here, and in three other 
places). Time for another errata.

P.

---

From: Gerhard Muenz <[email-masked]>
Date: Fri, 01 Apr 2011 11:15:39 +0200

Paul,

 On Fri, 01 Apr 2011 09:37:05 +0100, Paul Aitken wrote:
> Gerhard,
>
>> Vermont sends Template withdrawal messages and reuses Template IDs 
>> if you trigger an appropriate reconfiguration at runtime.
>

 As defined in the standard, Vermont only sends Withdrawal Messages if 
 SCTP is transport protocol, not with UDP (Vermont does not support TCP 
 on the Exporter side).

> places). Time for another errata.

 Good catch. Maybe this is the reason why I'm always unsure whether to 
 say Withdraw or Withdrawal :)

 Regards,
 Gerhard

---

From: Paul Aitken <[email-masked]>
Date: Fri, 01 Apr 2011 10:33:46 +0100

Gerhard,

> SCTP is transport protocol, not with UDP (Vermont does not support TCP 

Right. However, my question was whether Vermont re-uses Template IDs 
only for SCTP and not for UDP.
It seems you'd need special case code for that.
What benefits do you see in re-using template IDs rather than taking a 
new ID?

> say Withdraw or Withdrawal :)

Apparently "withdraw" is the verb ("withdraw a Template") whereas 
"withdrawal" is a noun ("Template withdrawal Message").

P.

---

From: Brian Trammell <[email-masked]>
Date: Fri, 1 Apr 2011 11:34:56 +0200

hi, Lothar, Gerhard,

That would explain it. :)

ripfix, at least, should be able to collect this in the release sitting on the floor of my home directory (again, once I squash some UDP bugs); we can set up an ad hoc interop test at some point in the near future.

Cheers,

Brian

---

From: Lothar Braun <[email-masked]>
Date: Fri, 1 Apr 2011 11:45:28 +0200

Hi Paul,

On Apr 1, 2011, at 11:33 AM, Paul Aitken wrote:
>>>> Vermont sends Template withdrawal messages and reuses Template IDs if you trigger an appropriate reconfiguration at runtime.
>>> 
>> As defined in the standard, Vermont only sends Withdrawal Messages if SCTP is transport protocol, not with UDP (Vermont does not support TCP on the Exporter side).
> 
> Right. However, my question was whether Vermont re-uses Template IDs only for SCTP and not for UDP.
> What benefits do you see in re-using template IDs rather than taking a new ID?

Template IDs in Vermont are configured by the user, and not chosen automatically at run time by the probe. Vermont therefore tries to reuse Template IDs on reconfiguration if the same ID is configured in the new config file.

Reconfiguring a SCTP Exporter will trigger a Template Withdrawal Message, reconfiguring an UDP Exporter will result in the creation of a new Transport Session.

Best regards,
  Lothar

---

From: Paul Aitken <[email-masked]>
Date: Fri, 01 Apr 2011 12:08:13 +0100

Lothar,

>  Template IDs in Vermont are configured by the user, and not chosen
>  automatically at run time by the probe.

I'm surprised, since I consider Template IDs to be an internal part of 
the protocol which should be invisible to users.

I see a potential danger: that users come to expect a certain template 
ID = specific content.

eg, suppose Vermont exports Template ID #260 which contains IPv6 fields, 
so they configure their collector to expect #260 = IPv6.
However, other exporters use different (and perhaps unpredictable) 
Template IDs when sending the same content.
eg, the next available ID when the template is created.

They might wonder how to make those exporters send template #260.
They might even point to the IPFIX RFCs which show specific template IDs 
in the examples...

I think this would be against the spirit of IPFIX, though I can't point 
to anything specific in the RFCs.

>  Vermont therefore tries to reuse Template IDs on reconfiguration if
>  the same ID is configured in the new config file.
>
>  Reconfiguring a SCTP Exporter will trigger a Template Withdrawal
>  Message, reconfiguring an UDP Exporter will result in the creation
>  of a new Transport Session.

ie, a new port will be selected?

Thanks,
P.

---

From: Brian Trammell <[email-masked]>
Date: Fri, 1 Apr 2011 13:20:15 +0200

Hi, Paul, all, 

You can reuse templates in UDP after they expire.

Theoretically.

I would suggest in 5101bis (assuming there is such a thing) that we remove the prohibition on sending withdrawals on UDP. Agreed that withdrawals are useless on UDP -- templates simply get replaced anyway, and expire according to rules involving the phase of the moon and the casting of runes.  The CP can't do anything sensible anyway if it gets a withdrawal on UDP, so it might as well try to follow the same rules as everywhere else.

Indeed, I think it's probably possible to define new set of template management rules that 1. interoperates with 5101 (aside from CP resets, which are a bad idea anyway) and 2. allows a single universal policy without regard to transport. I'll put some thought into this.

Best regards,

Brian

On Apr 1, 2011, at 10:37 AM, Paul Aitken wrote:

> Of course "Withdraw" should be "Withdrawal" (here, and in three other places). Time for another errata.
> 
> P.

---

From: Gerhard Muenz <[email-masked]>
Date: Fri, 01 Apr 2011 13:22:24 +0200

Hi Paul, Lothar

 On Fri, 1 Apr 2011 11:45:28 +0200, Lothar Braun wrote:
>>>>> Vermont sends Template withdrawal messages and reuses Template 
>>>>> IDs if you trigger an appropriate reconfiguration at runtime.
>>>>
>>> As defined in the standard, Vermont only sends Withdrawal Messages 
>>> if SCTP is transport protocol, not with UDP (Vermont does not support 
>>> TCP on the Exporter side).
>>
> automatically at run time by the probe. Vermont therefore tries to
> reuse Template IDs on reconfiguration if the same ID is configured in
> the new config file.

 Vermont's behavior is a bit more comlex than that :)
 1) The user is not required to configure a Template ID (although he was 
 some time ago).
 2) If the user configures a Template ID which does not conflict with 
 any another Template exported by the same Exporting Process, then 
 Vermont will use this configured Template ID. Otherwise, the Exporting 
 Process overwrites the configured ID with another ID in order to resolve 
 the conflict.
 
> Message, reconfiguring an UDP Exporter will result in the creation of
> a new Transport Session.

 If I remember correctly Vermont tries to be polite to the Collector by 
 sending Withdrawal messages before shutting down a SCTP Transport 
 Session (although this is not required by the standard).

 Regards,
 Gerhard

---

From: Lothar Braun <[email-masked]>
Date: Fri, 1 Apr 2011 13:40:38 +0200

On Apr 1, 2011, at 1:22 PM, Gerhard Muenz wrote:

> 1) The user is not required to configure a Template ID (although he was some time ago).
> 2) If the user configures a Template ID which does not conflict with any another Template exported by the same Exporting Process, then Vermont will use this configured Template ID. Otherwise, the Exporting Process overwrites the configured ID with another ID in order to resolve the conflict.
> If I remember correctly Vermont tries to be polite to the Collector by sending Withdrawal messages before shutting down a SCTP Transport Session (although this is not required by the standard).

Yes, vermont will send a Withdrawal message before shutting down the connection.

Best regards,
  Lothar

---

From: Benoit Claise <[email-masked]>
Date: Mon, 04 Apr 2011 10:27:08 +0200

Nevil,

Reviewing my note, I have one comment
>
> Hi all:
>
> Here are my draft minutes for Tuesday's meeting; they include my
> summary of the alternatives from our 'Draft Standards?' discussion.
>
> Please send me any corrections, changes, etc as soon as possible.
>
> Minutes of the IPFIX meeting at IETF 80
> About 36 people present
> Scribes: Cyndi Mills & Nevil Brownlee
>
> Nevil Brownlee presented current WG document status.  Three documents
> completed since IETF 79 (Export-per-SCTP-stream, Mediators Framework
> and Anonymisation Support), one with IESG (Structured Data).  We have
> received comments on the IPFIX Configuration Model draft from the YANG
> Doctors, and have been carefully considered.  Juergen will do its
> write-up.  The PSAMP MIB has been revised to use the UnsiUnsigned64TC
> and Float64TC Textual Conventions from other MIBs.  Nevil will do its
> write-up.  The remaining work item, Flow Selection, is under review,
> and will be discussed further on the IPFIX list.
>
> fourth such event for IPFIX, with eight implementations (4 exporters,
> 3 collectors) being tested.  Interoperation was complete for UDP,
> less so for TCP; for SCTP it was dependent on the SCTP
> slides), many were fixed during the event.
> Several people commented on SCTP implementation issues, suggesting
> that perhaps "template handling is needlessly complicated in the
> (IPFIX) protocol."  Dan Romascanu (our AD) asked for an Interoperation
> Report, Brian says he has one written as an Internet Draft.
>
> Lothar Braun presented Recommendations for Implementing IPFIX over
> DTLS/UDP.  DTLS is mandatory for IPFIX over UDP and SCTP, but using
> it is difficult because IPFIX traffic is unidirectional, but DTLS
> requires shared state.  Discussion centred on IPFIX's need for a
> heartbeat to detect collector failures, and whether IPFIX should do
> its own heartbeat.  Lothar's recommendations could fit in a revision
> of IPFIX Implementation Guidelines.
>
> Juergen lead a discussion on whether we should work on moving some of
> the IPFIX standards from Proposed to Draft.  Dan explained that to
> do so any changes would need to be editorial, not technical.  There
> was considerable discussion, the main points being:
> 1. There are many errata for 5101 and 5102, it would be good to have a
>    new draft that does that, along with some more explanatory
>    (editorial) text where needed
> 2. If we move 5101 and 5102 to Draft, any changes - however small -
>    would need to be a new version of the IPFIX protocol.  Doing that
>    could lead to confusion among IPFIX implementors and users
> 3. An alternative approach would be to work on a new draft (which
>    implemented small changes that did not affect interoperation, for
>    example adding detail where there are gaps in 5101) as a Standards
>    Track successor to 5101.  Once that had been published as an RFC
>    for some time, we could work on moving it to Draft; that should be
>    possible in a reasonably short time.
> 4. Other things that could be considered: IPFIX heartbeat provision
>    (we need to consider how long it will take TSVWG to complete the
>    DTLS Heartbeat Extension), change canonical transport to TCP, ... ?
> 5. Another possibility is to make a new "all about IPFIX" document;
>    there was only weak consensus for this
> The meeting reached consensus for (3, rather than 2), we will discuss
> this further on the IPFIX list.
>
> Five drafts were presented as candidates (in addition to the 'Standards
> upgrade') for an IPFIX re-chartering.
>
> Brian Trammell presented 'IPFIX Intermediate Aggregation,' this drew
> strong consensus as a new WG item.
>
> Brian presented the 'IE Doctors' draft, pointing out that this draft
> "lays out the ground rules for developing new IPFIX Information
> Elements, and clarifies how the IE Registry process works."  Michelle
> Cotton (IANA) commented that other working groups, e.g. DNS, have
> similar processes to those in this draft; we need to be clear about
> whether we're proposing "approval by IE-Doctors," or changes to
> "expert review" (which we have now).  Dan commented that to set up a
> team of IE Doctors, we need AD approval, and must keep IESG informed.
> Paul Aitken asked (via jabber), whether an IE could be reviewed and
> not made public until the product is shipped?  Dan replied "we have a
> body of experience to say that this should be an exception and not the
> rule."  There was clear consensus for adopting this as a WG item,
> with one person expressing strong dissent.  We will discuss this
> further on the list - the issues here are
> a. Should we develop an 'IE Guidelines' draft?
> b. Do we want to have an 'IE Doctors' team (with IESG overview),
>    an expanded group of IE Expert reviewers, or what?
Maybe this is an obvious one, but a conclusion from the discussion is 
that IANA should be point of contact for request, as opposed to IE doctors.
And the IE doctors should be called for review, when necessary.

Regards, Benoit.

>
> Benoit Claise presented the 'IPFIX Mediation protocol' draft, now at
> version -03.  There was clear consensus for adopting this.
>
> Benoit presented 'Exporting MIB variables using IPFIX.'  A spirited
> discussion of how Odis should be referred to in the IPFIX protocol.
> There was stronger consensus for this than against it.  Again,
> discussion of this will continue on the list.
>
> Benoit presented 'Exporting Application Information,' prompting
> considerable discussion.  Steven Campbell commented that
> "vendor-specific labels for layer-7 mapping is difficult. However, the
> way to discover layer 7 (behavioral, DPI) could be standardized."
> Benoit said he wasn't proposing this as a WG item, however anyone
> interested should continue discussing this topic on the list.
>
> The meeting finished at 1459.
>

---

From: Benoit Claise <[email-masked]>
Date: Mon, 04 Apr 2011 10:14:08 +0200

Hi,
> I would suggest in 5101bis (assuming there is such a thing) that we remove the prohibition on sending withdrawals on UDP. Agreed that withdrawals are useless on UDP -- templates simply get replaced…
Since it simplifies the code (similar code for the three transport 
protocols), while having no disadvantages (withdrawals on UDP doesn't do 
any good or bad), I agree.
This goes in the same direction as the point number 3 in 
http://www.ietf.org/mail-archive/web/ipfix/current/msg05809.html

Regards, Benoit.
> _______________________________________________
> IPFIX mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ipfix