Subject: Re: [IPFIX] Promotion of Enterprise-Specific IEs to IANA IEs

From: Benoit Claise <[email-masked]>
Date: Tue, 18 Sep 2012 15:33:25 +0200

Dear all,

[catching up with emails after my vacation]

I believe that "IE equivalence options template" (solution 2) sounds 
like a nice idea but will not work in real live.
In a perfect world, all the EPs would export this "IE equivalence 
options template" and the CP could ONLY rely on this mechanism.
We don't live in a perfect world, so not all EPs will export this 
options template. This implies that the CPs need anyway a plan B: hard 
coding the mapping, potentially received from IANA.

This is the exact same example as RFC5610. It's not implemented by all 
EPs (because we don't live in a perfect world), so the CPs must 
sometimes hard code the information, and they can rely on RFC5610 only.

Now, I would like to hear from the collectors people on the list:
- Would the "IE equivalence options template" be an important 
improvements? Warning: assuming that only some EPs might implement this 
feature!
- What about the fact that the collectors don't update the registry 
frequently from IANA, as an argument for the solution (2) below?
Do you update from IANA? Yes/No? how frequently?

Regards, Benoit.

> Greetings, all,
>
> To close the discussion from Vancouver, we have two proposals under consideration for handling the transition of enterprise-specific IEs used for testing, research, or other pre-standardization purposes to IANA IEs:
>
> (1) The addition of an Enterprise-specific Reference column to the IANA registry, which would include PEN(s) and IE number(s) which have a description compatible with the IANA-registered IE; this is covered in the present revision of the IE-DOCTORS draft. The advantage of this is it provides a central registry; however, it presumes a model in which CPs are either frequently updated or periodically retrieve new registration information from IANA, and requires all users of ESIE codepoints replaced with an IANA codepoint to register that information with IANA.
>
> (2) The definition of an IE equivalence options template, which would define the equivalence of a set of IANA and enterprise-specific IEs. This would be sent by EPs which had updated an IE from an older enterprise-specific codepoint to a new IANA codepoint; this would be covered in a new draft which Paul Aitken has (I think) already started work on. This has the advantage that equivalence is not dependent on the IANA registry. Benoit noted that this required the EP to (i) send additional information on session startup and (ii) to be updated both to use the new IANA codepoint as well as to send this additional information: a CP would not know an old ESIE was equivalent to a newer IANA IE if the EP it was receiving data from had not been updated.
>
> As I see it, there are four possible ways forward:
>
> (a) Support both methods: Leave approach (1) in IE-DOCTORS, develop a draft describing approach (2).
>
> (b) Support only the IANA method: leave approach (1) in IE-DOCTORS.
>
> (c) Support only the Options method: remove approach (1) from IE-DOCTORS, develop a draft describing approach (2).
>
> (d) Defer the question and leave ESIE promotion an open issue: remove approach (1) from IE-DOCTORS with no present decision on a draft about approach (2).
>
> I suppose I'm in favor of option (a), since each approach has its own use cases, as long as we're clear in both IE-DOCTORS and the equivalence options draft about the applicability of each approach.
>
> Thoughts?
>
> Cheers,
>
> Brian
> _______________________________________________
> IPFIX mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ipfix
>
>

---

From: Paul Aitken <[email-masked]>
Date: Tue, 18 Sep 2012 14:48:41 +0100

Benoit,

0. Your argument that "we don't live in a perfect world" doesn't hold in 
this case. Although the world may be imperfect, IPFIX implementers MUST 
follow the defined standards. I can't simply put the wrong export 
version, or use the wrong set ID, or the wrong field sizes, or send NFv5 
to your IPFIX collector, and simply claim "but we don't live in a 
perfect world!".

1. If the equivalence options template is mandated as the IPFIX 
mechanism, then EPs MUST implement it to be IPFIX compliant, perfect 
world or not.

1a. Whereas, if we give a choice of two mechanisms, some may choose one, 
some may choose the other, and we'll have only ourselves to blame for 
the ensuing mess.

2. Regardless of what collector experts say on this list, we know that 
some enterprises lock down their network configuration and spend a lot 
of time testing updates before rolling them out in their networks. 
Allowing a device to update on the fly using an unverified third-party 
configuration (even if it is from IANA) would be entirely unacceptable.

P.

On 18/09/12 14:33, Benoit Claise wrote:
> improvements? Warning: assuming that only some EPs might implement 
> this feature!
>> To close the discussion from Vancouver, we have two proposals under 
>> consideration for handling the transition of enterprise-specific IEs 
>> used for testing, research, or other pre-standardization purposes to 
>> IANA IEs:
>>
>> (1) The addition of an Enterprise-specific Reference column to the 
>> IANA registry, which would include PEN(s) and IE number(s) which have 
>> a description compatible with the IANA-registered IE; this is covered 
>> in the present revision of the IE-DOCTORS draft. The advantage of 
>> this is it provides a central registry; however, it presumes a model 
>> in which CPs are either frequently updated or periodically retrieve 
>> new registration information from IANA, and requires all users of 
>> ESIE codepoints replaced with an IANA codepoint to register that 
>> information with IANA.
>>
>> (2) The definition of an IE equivalence options template, which would 
>> define the equivalence of a set of IANA and enterprise-specific IEs. 
>> This would be sent by EPs which had updated an IE from an older 
>> enterprise-specific codepoint to a new IANA codepoint; this would be 
>> covered in a new draft which Paul Aitken has (I think) already 
>> started work on. This has the advantage that equivalence is not 
>> dependent on the IANA registry. Benoit noted that this required the 
>> EP to (i) send additional information on session startup and (ii) to 
>> be updated both to use the new IANA codepoint as well as to send this 
>> additional information: a CP would not know an old ESIE was 
>> equivalent to a newer IANA IE if the EP it was receiving data from 
>> had not been updated.
>>
>> (c) Support only the Options method: remove approach (1) from 
>> IE-DOCTORS, develop a draft describing approach (2).
>>
>> (d) Defer the question and leave ESIE promotion an open issue: remove 
>> approach (1) from IE-DOCTORS with no present decision on a draft 
>> about approach (2).
>>
>> I suppose I'm in favor of option (a), since each approach has its own 
>> use cases, as long as we're clear in both IE-DOCTORS and the 
>> equivalence options draft about the applicability of each approach.
>>
>> Thoughts?
>>
>> Cheers,
>>
>> Brian

---

From: Benoit Claise <[email-masked]>
Date: Tue, 18 Sep 2012 16:23:12 +0200

Hi Paul,
> Benoit,
>
> 0. Your argument that "we don't live in a perfect world" doesn't hold 
> in this case. Although the world may be imperfect, IPFIX implementers 
> MUST follow the defined standards. I can't simply put the wrong export 
> version, or use the wrong set ID, or the wrong field sizes, or send 
> NFv5 to your IPFIX collector, and simply claim "but we don't live in a 
> perfect world!".
There is a big difference here between your examples above and what we 
discuss here.
Your examples deal with the basis IPFIX protocol: RFC5101 or RFC5101bis.
The "IE equivalence options template" would be an extension, exactly 
like RFC5610. RFC 5610 is Standards Track
 From RFC 5610:

    an Exporting Process exporting
    data using Templates containing enterprise-specific Information
    Elements SHOULD export an Information Element type record for each
    enterprise-specific Information Element it exports.

So it's standards track, and it's a SHOULD. However, is it implemented?
Vendors might implement the base IPFIX, but the 
customers/vendors/collectors take a decision for each extension.
> mechanism, then EPs MUST implement it to be IPFIX compliant, perfect 
Even if this new RFC contains a "MUST", this doesn't change anything to 
the logic.
It's not because the IETF produces Standards Track RFCs with "MUST's" 
that they are implemented.
>
> 1a. Whereas, if we give a choice of two mechanisms, some may choose 
> one, some may choose the other, and we'll have only ourselves to blame 
> for the ensuing mess.
No disagreement on this one.
> configuration (even if it is from IANA) would be entirely unacceptable.
I propose to let the collector people answer what (un)acceptable.

Regards, Benoit.
>> options template. This implies that the CPs need anyway a plan B: 
>> hard coding the mapping, potentially received from IANA.
>>
>> This is the exact same example as RFC5610. It's not implemented by 
>> all EPs (because we don't live in a perfect world), so the CPs must 
>>> IANA registry, which would include PEN(s) and IE number(s) which 
>>> have a description compatible with the IANA-registered IE; this is 
>>> covered in the present revision of the IE-DOCTORS draft. The 
>>> advantage of this is it provides a central registry; however, it 
>>> presumes a model in which CPs are either frequently updated or 
>>> periodically retrieve new registration information from IANA, and 
>>> requires all users of ESIE codepoints replaced with an IANA 
>>> codepoint to register that information with IANA.
>>>
>>> (2) The definition of an IE equivalence options template, which 
>>> would define the equivalence of a set of IANA and 
>>> enterprise-specific IEs. This would be sent by EPs which had updated 
>>> an IE from an older enterprise-specific codepoint to a new IANA 
>>> codepoint; this would be covered in a new draft which Paul Aitken 
>>> has (I think) already started work on. This has the advantage that 
>>> equivalence is not dependent on the IANA registry. Benoit noted that 
>>> this required the EP to (i) send additional information on session 
>>> startup and (ii) to be updated both to use the new IANA codepoint as 
>>> well as to send this additional information: a CP would not know an 
>>> old ESIE was equivalent to a newer IANA IE if the EP it was 
>>> receiving data from had not been updated.
>>>
>>> (d) Defer the question and leave ESIE promotion an open issue: 
>>> remove approach (1) from IE-DOCTORS with no present decision on a 
>>> draft about approach (2).
>>>
>>> I suppose I'm in favor of option (a), since each approach has its 
>>> own use cases, as long as we're clear in both IE-DOCTORS and the

---

From: Andrew Feren <[email-masked]>
Date: Thu, 20 Sep 2012 16:14:41 -0400

> coding the mapping, potentially received from IANA.

The argument that this will not work in real life confuses me.  The base 
protocol doesn't have a required way to share IE information 
(equivalence or other).  Plan A (not plan B) is, therefore, hard coded 
IE information.  Plan B is any other option.  If an EP can give me 
useful information about an IE it is in my (and my customers') interest 
to use it for that EP even if not all EPs will send that information.  
Are you proposing that there should never be a plan B?

> sometimes hard code the information, and they can rely on RFC5610 only.

Exactly!  (I read the end of that sentence as "they can[not] rely on 
RFC5610 only".  I assume that was just a typo :-)

> improvements? Warning: assuming that only some EPs might implement 

I have seen no need for IE equivalence yet, but let's assume the need is 
on the horizon.  In that case I think Brian left out option 3 in his 
original email.  Extend 5610.

At IETF 84 Chris Inacio presented an idea to export a URI to XML ( 
http://recordings.conf.meetecho.com/Recordings/watch.jsp?recording=IET84_IPFIX&chapter=part_10). 
Brian further suggested it made sense to extend 5610 to do this.  It 
seems like IE equivalence would fit within that work.  This way we can 
have a plan A and plan B instead of an alphabet soup of plans.

I do see a large and looming need for sharing vendor IE information.  
Especially for security and policy information as in Chris's example.

> Do you update from IANA? Yes/No? how frequently?

Yes I update from IANA.  I do this by importing the XML from the IANA 
site.  Currently this is done statically prior to each new release, but 
I anticipate the ability to update installed collectors in the near future.

>> equivalence options draft about the applicability of each approach.

I if you replace "IE equivalence options template" with "extend 5610" 
then I support (a) too.

-Andrew

---

From: Benoit Claise <[email-masked]>
Date: Mon, 24 Sep 2012 11:43:47 +0200

Andrew,

Thanks for your feedback.
See in line.
> The argument that this will not work in real life confuses me. The 
> base protocol doesn't have a required way to share IE information 
> IE information. 
Yes.
> Plan B is any other option.  If an EP can give me useful information 
> about an IE it is in my (and my customers') interest to use it for 
> that EP even if not all EPs will send that information.  Are you 
> proposing that there should never be a plan B?
No. I'm advocating that, if the plan B is the "IE equivalence options 
template", that will not work.
> RFC5610 only".  I assume that was just a typo :-)
Yes, a typo. "And they can not] rely on RFC5610 only". Thanks for the 
correction.

>> improvement? Warning: assuming that only some EPs might implement 
>> this feature!
>
> I have seen no need for IE equivalence yet, but let's assume the need 
> is on the horizon.  In that case I think Brian left out option 3 in 
> his original email.  Extend 5610.
>

This solution makes more way more sense, for the following reasons:
1. The URI would get the entire list of IEs in XML format, similar to IANA
2. All enterprise-specific IEs for the specific vendor would be included
3. The URI could include some more metadata, if needed.
      For example, the mapping between the mapping between 
entreprise-specific and IANA.
4. If a single router from a specific vendor sends that URI, the 
collector receives all the mappings.

The point 4 is my primary argument against "IE equivalence options 
template".
- "IE equivalence options template" must be sent from _all _the 
exporters (because different exporters support different sets of IEs) 
for the collector to rely on the mechanism. And we know there are 
different platforms, with different software versions, even from a 
single vendor.
- Sending the URI only needs to be sent from a single exporter (from 
that vendor) and the collector gets all the required information. Note 
that the collector could even be hard coded the URI in the collector, as 
this should not change.

Regards, Benoit (as a contributor)

> site.  Currently this is done statically prior to each new release, 
> but I anticipate the ability to update installed collectors in the 
> near future.
>
>
>>
>> Regards, Benoit.
>>
>>> Greetings, all,
>>>

---

From: Paul Aitken <[email-masked]>
Date: Mon, 24 Sep 2012 13:16:01 +0100

Benoit,

> different platforms, with different software versions, even from a 

That's not quite correct.

If each box must send it's own unique equivalence option, then each box 
would require a unique URI for the collector to obtain the correct 
mapping for that device alone.

Since you clearly see that a single URI is sufficient for all devices 
from a vendor, then a single option template from a single device is 
also sufficient for all devices from a vendor, since the option would 
contain the exact same information as the URI.

> that vendor) and the collector gets all the required information.

Similarly for the equivalence option.

Consider the mechanism versus the content: there are two mechanisms 
(option versus URI), while the underlying content is the same.

> Note that the collector could even be hard coded the URI in the 
> collector, as this should not change.

Consider how the URI mechanism would handle versioning. ie, an 
enterprise-specific IE changes from A to B to C. If the latest URI says 
A is B and B is C (as it must), then older EPs from before the B to C 
change, which truly do export B, may not be interpreted correctly.

Whereas with the equivalence option mechanism, an older device could 
export the "A to B" equivalence without the "B to C" equivalence.

P.

---

From: Andrew Feren <[email-masked]>
Date: Mon, 24 Sep 2012 09:04:00 -0400

Hi Paul,

On 09/24/2012 08:16 AM, Paul Aitken wrote:
> If each box must send it's own unique equivalence option, then each 
> box would require a unique URI for the collector to obtain the correct 
Benoit can correct me if I am not understanding him correctly, but I 
think he is envisioning something a bit broader than just the mapping 
needed for a single device.  I think what Benoit has in mind is a URI to 
a vendor maintained IANA like registry.  This registry would have info 
about all of that vendors information elements.

> enterprise-specific IE changes from A to B to C. If the latest URI 
> says A is B and B is C (as it must), then older EPs from before the B 
> to C change, which truly do export B, may not be interpreted correctly.
>

First I thought the idea was to declare the "equivalence" of a vendor IE 
to some now standard IE.  I suppose a large vendor might have to IEs 
(say A and C) that are both equivalent to some new standard IE B.  Maybe 
A and C each only ever exported a subset of the values now standard for 
B, whatever.  If I can say that A is equivalent to B and I can say B is 
equivalent to C then I better also be able to say that A is equivalent 
to C.... Ahhh.

OK as I type this out I think I see where you are going.  Let's be more 
specific with my above example

A exports values (1 and 2)
C exports values (3 and 4)
B exports values (1, 2, 3, 4)

saying A is equivalent to C is a bit bizarre.  Maybe equivalence is the 
wrong word.  Anyone have a better suggestion?

I think this is a pretty compelling argument for each vendor maintaining 
a single registry.  I'm not really interested in versioning beyond 
getting the latest (most complete definition). Maintaining a per 
exporter registry seems like a lot of work with not a lot of upside.

I'll give this some more thought though.

-Andrew

---

From: Paul Aitken <[email-masked]>
Date: Mon, 24 Sep 2012 20:50:01 +0100

Andrew,

The idea was that the equivalence option could map any IEspec to any 
other IEspec. Although it'd be commonly used to map enterprise specific 
to IANA standard, it could also map enterprise to enterprise, and even 
IANA standard to enterprise specific, or IANA standard to IANA standard. 
In fact it could also map between MIB / enterprise / IANA.

Regarding A/B/C below, imagine that enterprise specific IE A exports 
uint8 values 1 and 2. Later, enterprise specific IE B increases the size 
to uint16 and adds new values 0x123 and 0x45678, so A isn't exported 
anymore. Then B is replaced by the standardised version, C - so neither 
A nor B are required anymore.

However, older devices still export B. As an older device, the 
equivalence option coming from this device wouldn't know about C, so 
it'd just report that A has been replaced by B.

Whereas the URI mechanism would report that A is now B and B is now C, 
without any way to determine when (in code / release terms) the 
equivalence happened - so collectors might only expect C, which would be 
incorrect since older devices could still be exporting B (or indeed, A).

P.

On 24/09/12 14:04, Andrew Feren wrote:
>>> exporters (because different exporters support different sets of 
>>> IEs) for the collector to rely on the mechanism. And we know there 
>>> are different platforms, with different software versions, even from 
>>> a single vendor.
>>
>> box would require a unique URI for the collector to obtain the 
>> correct mapping for that device alone.
>>
> needed for a single device.  I think what Benoit has in mind is a URI 
> to a vendor maintained IANA like registry.  This registry would have 
> info about all of that vendors information elements.
>
> First I thought the idea was to declare the "equivalence" of a vendor 
> IE to some now standard IE.  I suppose a large vendor might have to 
> IEs (say A and C) that are both equivalent to some new standard IE B.  
> Maybe A and C each only ever exported a subset of the values now 
> standard for B, whatever.  If I can say that A is equivalent to B and 
> I can say B is equivalent to C then I better also be able to say that 
> A is equivalent to C.... Ahhh.
>
> OK as I type this out I think I see where you are going.  Let's be 
> more specific with my above example
>
> saying A is equivalent to C is a bit bizarre.  Maybe equivalence is 
> the wrong word.  Anyone have a better suggestion?
>
> I think this is a pretty compelling argument for each vendor 
> maintaining a single registry.  I'm not really interested in 
> versioning beyond getting the latest (most complete definition). 
> Maintaining a per exporter registry seems like a lot of work with not 
> a lot of upside.
>

---

From: Benoit Claise <[email-masked]>
Date: Mon, 24 Sep 2012 23:58:34 +0200

Hi,
> info about all of that vendors information elements.
Exactly.
>> (option versus URI), while the underlying content is the same.
Not quite.
The "IE equivalence options template" is for the situation when one 
exporter upgraded his software, and is aware of one new IE mapping (due 
to the software upgrade).
This "IE equivalence options template" is basically telling: "mister 
collector, you were receiving enterprise-specific IE X from me, and now 
I'm sending you the IANA IE Y: the two are equivalent"

The "IE equivalence options template"  has the context of the exporter 
only, while the URI solution contains a URI to a vendor maintained IANA 
like registry
>> export the "A to B" equivalence without the "B to C" equivalence.
I'm not convinced this is a real use case.
> Maintaining a per exporter registry seems like a lot of work with not 
I share the same view.

---

From: Paul Aitken <[email-masked]>
Date: Mon, 24 Sep 2012 23:18:53 +0100

Benoit,

> exporter upgraded his software, and is aware of one new IE mapping 
> (due to the software upgrade).
> collector, you were receiving enterprise-specific IE X from me, and 
> now I'm sending you the IANA IE Y: the two are equivalent"
>
> only, while the URI solution contains a URI to a vendor maintained 
> IANA like registry

Unless the new code can determine exactly what the previous version was, 
and knows exactly which IEs that version exported, it can't export such 
a specific equivalence option.

So the normal situation would be for the new code to export a table with 
all the equivalences which it knows.

Also from an implementation perspective, it's easier to build the 
equivalence table into every new product and export the whole table, 
rather than determine the correct subset for the old code versus current 
code.

A simply collector could simply merge all the equivalence options into 
an equivalence superset in order to know all possible equivalences in 
the network. ie there's no need to maintain device-specific equivalences.

P.

---

From: Benoit Claise <[email-masked]>
Date: Tue, 25 Sep 2012 00:26:01 +0200

On 25/09/2012 00:18, Paul Aitken wrote:
>> The "IE equivalence options template"  has the context of the 
>> exporter only, while the URI solution contains a URI to a vendor 
>> maintained IANA like registry
>
> Unless the new code can determine exactly what the previous version 
> was, and knows exactly which IEs that version exported, it can't 
> export such a specific equivalence option.
Good! It was the next point I wanted to address....
>
> So the normal situation would be for the new code to export a table 
> with all the equivalences which it knows.
>
> rather than determine the correct subset for the old code versus 
> current code.
>
Instead, just point the collector to the URI with all the information. 
That's way easier for everybody: the exporters and the collector.

---

From: Paul Aitken <[email-masked]>
Date: Mon, 24 Sep 2012 23:36:28 +0100

Benoit,

> That's way easier for everybody: the exporters and the collector.

My constant point is that there's no way that network operators will 
allow this.

Network configuration is locked down, and only changeable during 
pre-planned maintenance windows. At such times, only known-good and 
pre-tested configurations are rolled out.

The equivalence option can be tested in isolation in a lab, and the 
configuration proven. There's no external influence upon the configuration.

Whereas the URI mechanism cannot be tested in isolation because it's 
dependent upon the URI content, which could be modified at any time 
without notice.

Also, this introduces a new attack vector upon the collector - ie, by 
feeding it malformed or incorrect data through the URI.

So from a network operator point of view, I wouldn't buy that.

P.