Subject: [IPFIX] Application of IPFIX to new application spaces.

From: Stewart Bryant <[email-masked]>
Date: Thu, 02 Jan 2014 13:54:47 +0000

Whilst I accept that IPFIX is pretty much complete in terms of
addressing it's original goals as a network management
protocol, I have some concerns about it's completeness
as an  observation reporting protocol. I think these need
to be addressed, before declaring completion of the work
programme as a whole.

(In my own time) I am looking at the definition of a protocol
to observe and report the radio noise floor. The purpose
is to map the rise in the radio noise floor due to RF pollution
from domestic and industrial digital systems such as
plasma televisions, and power line (digital) transmission
systems. IPFIX is a good protocol for such mass observations,
and has been used successfully in the radio propagation
monitoring system pskreporter.

In doing this I ran into two problems, firstly there are no
experimental IE types, and thus I cannot prototype the
application without either "camping" on a set of IEs, or
waiting until I am allocated a PEN. For reasons that will
be clear from other work I have done in the IETF, I dislike
the idea of "camping" on code-points. This makes it clear
to me that we need a small, but not trivial set of
experimental IEs that are intended for prototyping but
explicitly excluded from use in production systems.
This needs to be a reasonable number since a new application
space might need a fair number of IEs to be practical.
Thus I think that we need to either allocate some of the
base protocol IEs to experimental, or to have IANA
allocate a PEN specifically to experimental use which
would allow experimentation in new applications spaces
without the need to formally allocate IEs in either the
base IE space, or the PEN space of the organization
(or person) conducting the experiment.

The second problem that I see is the lack of a public registry
for non-network managements applications. Specifically
I am going to need "callsign", "maidenhead locator",
"frequency", "noise power", maybe "field strength", "receiver type"
etc. Now some of those may be of general use (frequency)
but most of them would be cruft in the base protocol IE set
which is set up for networking applications. On the other hand,
I know of at least one PEN space that will have a number of the
terms I need already defined, although these are by
definition private. With the current IE registration structure
the absence of a public registry inevitably means the
redefinition of other than mainstream/network management
terms across a number of private registries. I thus think
it would be useful to introduce a PEN + registry for
"other applications" or introduce the concept of a series
of application specific registries with appropriate  PENs to
identify the application space rather than  the private
enterprise space.

- Stewart

---

From: Brian Trammell <[email-masked]>
Date: Fri, 3 Jan 2014 10:39:22 +0100

hi Stewart,

Wanted to reply briefly to two points here inline (will reply in more detail later):

On 02 Jan 2014, at 14:54, Stewart Bryant <[email-masked]> wrote:

> waiting until I am allocated a PEN.

I’m a little puzzled about the hesitation about getting a PEN; indeed, in my experience, had you applied for a PEN for this purpose when you sent this message initially you’d have one by now. There is a little overhead in managing PEN space for different experiments (see https://trammell.ch/pen-35566 for the range registry used for my own experimentation) but this is orders of magnitude less work than designing and implementing the IEs in the first place.

I could see this being a problem if one were trying to run many experiments within the same very large organization with undefined or draconian processes for reserving an enterprise-specific IE. But PEN space is 2^32-1 large and we haven;t even exhausted the first block of 2^16, so “get a new PEN and label it useful for a given experiment” is probably an acceptable way out of this quandary.

> For reasons that will

I agree in principle that allocating a single “IPFIX Experimentation” PEN would be one solution to this problem, but it would have to be fairly tightly scoped (only for experimental use among EPs and CPs implemented and deployed by a single entity within the scope of a single experiment; MUST be logged as an error by CPs in production use). And I’d be very concerned that we were basically inviting people to camp on a whole new code space — requiring experiments to use their own PEN space at least keeps experimental IEs that “leak” into production from colliding with each other, provided that the PEN owner manages their own space competently.

> identify the application space rather than  the private

This seems like a very good idea. I’ll have to think about it some more.

Thanks, cheers,

Brian

> __________________________________________
> IPFIX mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ipfix

---

From: Stewart Bryant <[email-masked]>
Date: Fri, 03 Jan 2014 12:06:30 +0000

> I’m a little puzzled about the hesitation about getting a PEN; indeed, in my experience, had you applied for a PEN for this purpose when you sent this message initially you’d have one by now. There…
I brought up the concept of ownership of PENs by individuals
in a couple of IESG discussions on IPFIX, and the opinion that
I get (as I recall from the APPs ADs) is that PENs were never
intended to be assigned to individuals together with surprise
that any had been allocated to individuals. Hence my initial
reluctance. I know that there are some other owned by individuals,
and I  have requested one for the purpose of trying out this
application of the protocol. However I think the concern is
that it would not scale if every individual experimenter
requested their own, much less one per experiment.

> I could see this being a problem if one were trying to run many experiments within the same very large organization with undefined or draconian processes for reserving an enterprise-specific IE. Bu…
I think the question is one of whether or not that is best practice.
Sure the space is large, but so was the IPv4 Address space when it
was created :)
> I agree in principle that allocating a single “IPFIX Experimentation” PEN would be one solution to this problem, but it would have to be fairly tightly scoped (only for experimental use among EPs a…
Text of the following format needs to be associated with the
space:

Code points in the experimental range MUST be used according to the
guidelines ofRFC 3692  <http://tools.ietf.org/html/rfc3692>  [RFC3692  <http://tools.ietf.org/html/rfc3692>].

> This seems like a very good idea. I’ll have to think about it some more.
Thanks

Stewart

---

From: Brian Trammell <[email-masked]>
Date: Fri, 3 Jan 2014 18:54:39 +0100

> requested their own, much less one per experiment.

It’s probably well outside the scope of IANA’s competence to rule on the differences between individuals, sole proprietorships, single member LLCs/GmbHs, C/S corporations, AGs/ABs/SAs, unorganized collectives of software developers, and so on. Without that competence, it’s pretty difficult to give them guidelines as to what to allow and what to deny in terms of individual registrations. So FCFS to anyone who can control an email address it is.

If you’re doing experiments with IPFIX (or, indeed, SNMP) as an individual or academic at an organization without a PEN or where it’s difficult to get “official” IPFIX IE number space within that PEN, getting your own is the only way to go.

Of course, I’m probably the worst abuser of this property, having requested the first PEN (29305) for an RFC (5103) for the single-record biflow hack. 
> Sure the space is large, but so was the IPv4 Address space when it

I share your concern in theory. But practically I cannot imagine a world in which the number of IPFIX experiments (plus normal SMI “enterprises”) is anywhere near on the order of addressable locations in the IPv4 number space. We’re talking tens of PENs, hundreds if we are successful in expanding the IPFIX information model to be the One True Way the Internet of Things talks about itself.

I realize I’ve just made a “nobody needs more than 64k autonomous systems” statement. And I completely agree that this is somewhat outside the original intention of the PEN registry at its creation time, but compare this to the difference between the domain name space as it was conceived and as it’s (ab)used today, and I think what I’m proposing is a relatively minor violation, and is indeed not really all that different from the PENs-for-application-areas proposal (which I quite like as well).
> guidelines of RFC 3692 [RFC3692].
Yep, that’s good and unambiguous, but it still does not address the issue of what happens after the termination of the experiment.
>>> definition private. 
s/private/not necessarily public/ — there is nothing keeping an operator of a PEN registry from publishing that registry. Indeed, the intention of RFC 5610 was to allow such operators to publish such registries _inline_ to allow polymorphic collectors / file readers to load such registries at runtime.
>>> With the current IE registration structure

As I noted above, there’s already precedent for this: PEN 29305 for RFC 5103.

We could certainly define a few “extended application” areas, assign them PENs, and allocate them via IANA; we’d just need to add a PEN column to the registry, and update 7102. Deciding on an initial list might take a while though.

Cheers,

Brian

---

From: Paul Aitken <[email-masked]>
Date: Sat, 04 Jan 2014 04:57:38 +0000

Stewart, Brian,

> I agree in principle that allocating a single "IPFIX Experimentation" PEN would be one solution to this problem, but it would have to be fairly tightly scoped (only for experimental use among EPs and CPs implemented and deployed by a single entity within the scope of a single experiment; MUST be logged as an error by CPs in production use). And I'd be very concerned that we were basically inviting people to camp on a whole new code space --- requiring experiments to use their own PEN space at least keeps experimental IEs that "leak" into production from colliding with each other, provided that the PEN owner manages their own space competently.

+1

I was about to say the same. While the "Experimentation PEN" idea sounds 
good, there are practical dangers:

With my netflow-police hat on, I allocate cisco's NFv9 and IPFIX 
enterprise-specific IDs. I've had to reserve blocks of NFv9 IDs which 
have been used by third parties (external to cisco) in released code 
without telling us (a collector partner was good enough to inform us 
about the potential clash). And in the past we've allocated our own 
experimental IPFIX IDs which were meant to be replaced with IANA IDs 
before release, but got overlooked.

To avoid any issues, I'd rather see experiments done in private PEN space.

P.

> This seems like a very good idea. I'll have to think about it some more.
>
> Thanks, cheers,
>
> Brian
>
> _______________________________________________
> IPFIX mailing list
> [email-masked]

---

From: Andrew Feren <[email-masked]>
Date: Wed, 8 Jan 2014 15:28:08 -0500

Hi Stewart, Brian, all,

Comments inline

On 01/03/2014 12:54 PM, Brian Trammell wrote:
>
> On 03 Jan 2014, at 13:06, Stewart Bryant <[email-masked]
> <mailto:[email-masked]>> wrote:
>
> It's probably well outside the scope of IANA's competence to rule on
> the differences between individuals, sole proprietorships, single
> member LLCs/GmbHs, C/S corporations, AGs/ABs/SAs, unorganized
> collectives of software developers, and so on. Without that
> competence, it's pretty difficult to give them guidelines as to what
> to allow and what to deny in terms of individual registrations. So
> FCFS to anyone who can control an email address it is.
>
> If you're doing experiments with IPFIX (or, indeed, SNMP) as an
> individual or academic at an organization without a PEN or where it's
> difficult to get "official" IPFIX IE number space within that PEN,
> getting your own is the only way to go.

+1

I assume that the point of the experiments is to eventually move beyond
the experiment.  At some point a PEN will likely be needed for the IEs
that proved useful in the experiment.  Might as well get the PEN and
start using it.

>
> Of course, I'm probably the worst abuser of this property, having
> requested the first PEN (29305) for an RFC (5103) for the
> single-record biflow hack. 
>>> I could see this being a problem if one were trying to run many experiments within the same very large organization with undefined or draconian processes for reserving an enterprise-specific IE. But PEN space is 2^32-1 large and we haven;t even exhausted the first block of 2^16, so "get a new PEN and label it useful for a given experiment" is probably an acceptable way out of this quandary.
> I share your concern in theory. But practically I cannot imagine a
> world in which the number of IPFIX experiments (plus normal SMI
> "enterprises") is anywhere near on the order of addressable locations
> in the IPv4 number space. We're talking tens of PENs, hundreds if we
> are successful in expanding the IPFIX information model to be the One
> True Way the Internet of Things talks about itself.
>
> I realize I've just made a "nobody needs more than 64k autonomous
> systems" statement. And I completely agree that this is somewhat
> outside the original intention of the PEN registry at its creation
> time, but compare this to the difference between the domain name space
> as it was conceived and as it's (ab)used today, and I think what I'm
> proposing is a relatively minor violation, and is indeed not really
> all that different from the PENs-for-application-areas proposal (which
> I quite like as well).
>>>> For reasons that will
>> guidelines of RFC 3692 <http://tools.ietf.org/html/rfc3692> [RFC3692 <http://tools.ietf.org/html/rfc3692>].
> Yep, that's good and unambiguous, but it still does not address the
> issue of what happens after the termination of the experiment.
> s/private/not necessarily public/ --- there is nothing keeping an
> operator of a PEN registry from publishing that registry. Indeed, the
> intention of RFC 5610 was to allow such operators to publish such
> registries _inline_ to allow polymorphic collectors / file readers to
> load such registries at runtime.

In fact many PEN/IE registries have in fact been published in one form
or an other, but I sure would love a central location with a parseable
format.  This is not the first time that a desire for a central
enterprise registry has been voiced
(http://www.ietf.org/mail-archive/web/ipfix/current/msg05825.html).

Obviously the owner of each PEN should decide which elements they want
to publish information about and when.  Aside from making my life as a
collector developer easier I think there are other advantages to a
central registry.  One such advantage would be exposing IEs for which
there is significant interest and which are therefore potential
candidates to be standardized.

I have built my own version of a central registry as I encounter new
exports.  Making a quick scan of the DB I see, for example, about 5
different IEs each for HTTP urls, HTTP return codes, and HTTP host.

> As I noted above, there's already precedent for this: PEN 29305 for
> RFC 5103.
>
> We could certainly define a few "extended application" areas, assign
> them PENs, and allocate them via IANA; we'd just need to add a PEN
> column to the registry, and update 7102. Deciding on an initial list
> might take a while though.

The idea of application PENs is interesting, but I'm not sure I see a
need at this point.  Besides, multiple application registries feels a
bit like the IE categories that existed in 5102 and no one cared to
carry forward to 7012.  As noted at the time (
http://www.ietf.org/mail-archive/web/ipfix/current/msg06621.html) as
applications move up and down layers IEs would need to be redefined.  I
would expect similar categorization issues trying to decide what
application an IE belongs to and very little upside to doing the
categorization.

A single catch all PEN for "other applications" is a little better, but
I don't really see this as much different than just having a registry
where PEN/IE details can be registered/shared.  An advantage of a
registry with various PENs would be that an experimental PEN/IE could be
"promoted" (shared) by adding it to the registry once the experimenter
is ready.

-Andrew

---

From: Paul Aitken <[email-masked]>
Date: Wed, 08 Jan 2014 23:07:38 +0000

PEN/IE registri

> I assume that the point of the experiments is to eventually move 
> beyond the experiment.  At some point a PEN will likely be needed for 
> the IEs that proved useful in the experiment.  Might as well get the 
> PEN and start using it.

Or, at some point the experiemental IEs should be standardised through 
IANA / IE-doctors.

> different IEs each for HTTP urls, HTTP return codes, and HTTP host.

The problem is that once you have your own PEN, there's no incentive to 
transition your private IEs into standard IEs.

A PEN/IE registry could be a good way for IE-doctors to discover useful 
/ duplicate IEs, and create standard versions for general consumption.

P.

---

From: Benoit Claise <[email-masked]>
Date: Mon, 20 Jan 2014 12:02:37 +0100

Stewart,
> (or person) conducting the experiment.
I don't believe that allocating experimental IEs is a good idea.
First you need one per type. Unless you use RFC 5610, but if you do some 
experiments, then you most probably don't have RFC 5610. Then, as you 
said, you need multiple ones. How many? would 5 be enough, 10, 50, 100?

If you don't want to "waste" PEN for individual, which I understand, 
then why don't you request an experimental PEN.
Not an IPFIX experimental PEN, just an experimental PEN. This could be 
used for IPFIX, MIB, etc...
This would actually be better than individual PENs, somehow forcing 
people to go with a proper registration.

> identify the application space rather than  the private
It's true that IPFIX became a generic push mechanism.
We tried this in the past with the groups in 
http://tools.ietf.org/search/rfc5102#section-5
This was removed in RFC 7102.

Regards, Benoit
> .

---

From: Benoit Claise <[email-masked]>
Date: Mon, 20 Jan 2014 12:35:37 +0100

On 8/01/2014 21:28, Andrew Feren wrote:
>> time, but compare this to the difference between the domain name 
>> space as it was conceived and as it's (ab)used today, and I think 
>> what I'm proposing is a relatively minor violation, and is indeed not 
>> really all that different from the PENs-for-application-areas 
>> proposal (which I quite like as well).
>>>>> For reasons that will
I fully understand the need from a collector point of view, and broadly 
for the community interest point of view, but practically, I don't see 
this working. Along the same line of easing the collector live, there is 
RFC 5610, but I don't see the love to implement it.
> applications move up and down layers IEs would need to be redefined.  
> I would expect similar categorization issues trying to decide what 
Agreed.
What could make sense is to specify a templateRecordName, as a string, 
to have a free-form template name/application/whatever you want.

Regards, Benoit
>
> A single catch all PEN for "other applications" is a little better, 
> but I don't really see this as much different than just having a 
> registry where PEN/IE details can be registered/shared. An advantage 
> of a registry with various PENs would be that an experimental PEN/IE 
> could be "promoted" (shared) by adding it to the registry once the 
> experimenter is ready.
>
> -Andrew
>
>>
>> Cheers,
>>
>> Brian
>>
>>

---

From: Benoit Claise <[email-masked]>
Date: Mon, 20 Jan 2014 12:36:41 +0100

On 4/01/2014 05:57, Paul Aitken wrote:
> I was about to say the same. While the "Experimentation PEN" idea 
> sounds good, there are practical dangers:
>
+1

Regards, B.