Subject: [ipfix] Observations on Selection Process and Requirements

From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" <[email-masked]>
Date: Mon, 10 Mar 2003 21:17:26 -0500

Hi,

  As the advocate for Streaming IPDR, which represents the collected
  work of most vendors in the mediation and billing space, and as
  a member of HP's Software Business Unit, a leading provider of
  network management systems, I would like to reemphasize my belief
  that Streaming IPDR, or a version representing some compromise
  among the other IPFIX stakeholders, utilizing many of the concepts
  of IPDR, is the correct choice, for addressing such a protocol.
  
  Note, I have not seen an offer of compromise from other 
  advocates at the mailing list level to date.  I'm listening...

  The first paragraph of the IPFIX WG's charter states:
  
   "...there is a need in industry and the Internet research community for 
    IP devices such as routers to export flow information in a standard 
    way to external systems such as mediation systems, accounting/billing 
    systems, and network management systems to facilitate services 
    such as Internet research, measurement, accounting, and billing."
    
  And one of the key goals for Feb. '03 is listed as:
  
   "Select IPFIX protocol, revise Architecture and Data Model drafts."

  After rereading the various submissions, the IPFIX requirements draft 
  and other relevant RFC's, I'd like to share a few observations which
  should at the least produce some relevant conversation.

  I apologize for the length of this memo, but I'd encourage people
  to slog through it...

  1.  The IPFIX protocol operates as an accounting protocol.  I notice
  that the current requirements draft does not cite the following 
  important RFC's:
  
    RFC2975 - Introduction to Accounting Management
    RFC2924 - Accounting Attributes and Record Formats
    
    It should.  My bad for coming in past "last call".  They're still
  relevant even if not cited.  If folks believe that IPFIX is defining
  something other than an accounting protocol, this should be a
  point of discussion.

  2.  The problem being dealt with is complex, and as such decomposing
  the problem, "divide and conquer" or "separation of concerns" plays
  an important role.
  
    I believe that the requirements draft does a good job of this in
defining
  functional areas such as the "observation point", "metering process"
  "exporter" and "collector".
  
    However, to the extent that these areas can be discussed separately,
  they should be.  Otherwise unnecessary complexity creeps in from the
  "tangling" of concerns which can be treated separately.
  
    I've noticed a lot of discussion around the behavior of the metering
  process, which is interesting, yet should be irrelevant to the
  selection of a protocol.

  3.  The use of UDP as a transport for flow information is widely used
  today.  Despite the statement in section 6.3.1 of the IPFIX requirements
  document that the protocol MUST  be congestion aware, it is likely that 
  the "backpressure" associated with congestion aware protocols will put 
  an unacceptable burden on SOME common observation points in SOME 
  situations. 
  
    This seems to be an item which has been avoided because of the IESG
  mandate on congestion aware protocols, however it seems quite relevant
  to this particular domain.  Cisco has just recently provided a mapping 
  of their proposed NFv9 to SCTP, but at the same time left the door open 
  to use variants of SCTP which continue to allow them to "fire
  and forget".  There is probably a meaningful technical issue here, 
  which should not be ignored.
  
    I.e. although from a collector/reliablity perspective, TCP is much more 
  desirable,  especially given the relative immaturity and expense of SCTP 
  implementations on general purpose platforms, there seems to be a desire 
  to avoid TCP on the part of a major player in the flow observation 
  equipment space.  This should be explored!

  4.  The use of the flow information does not just stop at the 
  collector.  This information may be used by a variety of applications.
  As such some of the ideas put forth in RFC2924 are worth considering:
  
    (Nevil, as a coauthor of RFC2924, please feel free to speak up!)
    
  - Sec. 9.4.1 "Service Independence" - talks about the benefit of
     separation of concerns.  And cites SNMP as a key example of 
     having an independent "Service Definition Model".
     
     IPDR provides just that.  And rather than using the cumbersome
    ASN.1 mechanism for this, IPDR uses XML Schema as the means to
    create independent service defintions.
    
     Given the availability of tools and reference material around
    XML, I see no compelling argument in favor of any other structured
    information model for DESCRIBING the content.  PLEASE avoid the
    confusion that IPDR's use of XML for describing the service 
    definition (or "Information Model") requires XML format for 
    the actual transfer of flow information. IPDR defines a very
    compact, template-based, binary means of encoding.
    
   - Sec. 9.4.4 "Service Namespace Management" - this articulates the
    benefit of allowing independent vendor extension, some candidates
    have a very limited means of addressing this.  IPDR uses XML 
    Namespaces to allow for independent vendor extension, and puts
    all namespaces on equal footing.
    
   - Sec. 10 "Encodings" - describes the potential benefit of supporting
    multiple encodings.  IPDR does just that.  For high volume exchange
    a binary format is defined; for other applications, e.g. presentation
    or human editing, an XML format is defined.  IPDR provides a lossless
    and unambiguous mapping between the two.
    
   - Sec. 9.1 "Record Format vs. Protocol" - describes the benefit of
    separating the encoding model from the transport.  So that a variety
    of transports, including batch file oriented transports can be 
    employed.  IPDR does just this.

 5. Terminology.  The requirements draft talks of an Information Model,
  a Data Model and a Transport.  The distinction between the words 
  "Information" and "Data" is probably subject to debate.  Rather than
  sticking with these terms, I'd recommend the IPDR terminology which
  is "Service Definition" and "Encoding".  Or "Information Model" and
  "Encoding".

In conclusion, the apparent 5 way tie between the various protocol
candidates appears to be the result of the current framing of the 
requirements.

The recent changes to the requirements seems unlikely to change 
the outcome of the current scoring model in any material way.

The recent dialog around "reliability" on the mailing list, may in
fact be more related to the ignored issue cited in observation 
#3 than anyone has explicitly acknowledged.  It may imply that some
compromise around the use of UDP may be in order.  Note that there
is also an extensive discussion of reliability in section 2.1 of
RFC 2975.

I would however point out that if some of the aspects from RFC's
2975 and 2924 were considered, and the requirements were either
broken down around these terms, or scored on a 1-5 vs. 0-1 basis
that breaking the tie would likely be easier.

As someone who has had to architect and lead development of a system 
which accomodates the current "Tower of Babel" in the accounting 
space, I can certainly say that creating yet another narrow accounting 
format is NOT a good thing for the industry.  This is why I've been 
very active in evolving IPDR over the past three and a half years.
To define a clean, open and extensible separation between the Service
Definition, encoding and transport.  This allows for a single Service
definition model to drive current and future encodings and transports.
And IPFIX is an ideal candidate to take advantage of this.

Regards,

  Jeff Meyer
  Chief Architect
  HP OpenView Internet Usage Manager
  IPDR Advocate
  http://www.hp.com/usage
  http://www.ipdr.org/documents/ipfix

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: Nevil Brownlee <[email-masked]>
Date: Wed, 12 Mar 2003 08:54:50 +1300

Hello Jeff:

Some comments in response to your 'Observations on Selection Process ..'

<wg chair hat ON>
Your point 3, use of UDP transport.  As Allison pointed out, the use
of congestion-aware transport for new standards-track protocols is an
IETF consensus, not an IESG mandate.

Of course people who implement new applications may choose, for
whatever reason, to use transport which is not congestion-aware.
That's fine, but it won't conform to the standard.  Or to put it
another way, to be standards-compliant, a vendor has to give customers
the ability to use congestion-aware transport.

Having said that, this is a rathole which the WG spent a lot of time
exploring in the first half of last year - we're not going to go there 
again.
</wg chair hat ON>

Now your other points:

<wg chair hat OFF, i.e. these are my own opinions>

I agree that RFCs 2975 and 2924 are relevant.  They came from the
AAA WG, which was strongly focussed on Accounting.  Their end product
is DIAMETER, which is designed to be an all-inclusive protocol for
all three A's.  So if we want a one-true-faith Accounting protocol,
we should all use DIAMETER.

Alas, reality isn't like that.  A lot of the complexity in DIAMETER
comes from its need to handle Authentication and Authorisation across
multiple provider realms, so Accounting is only a subset of DIAMETER.

Again, there are lots of network operators (not to mention researchers)
using LFAP, NetFlow, RTFM, etc. to gather flow data from routers and
switches, who do it to gain understanding of how their network is
behaving, and who probably have no interest in using that data to
recover costs.  IPFIX started in this context, with the notion of
setting a single standard for exporting flow data from routers and
switches.  That's only part of making a 'universal accounting protocol,'
but it would nonetheless be a very useful step forward.  And one which,
to me at least, seems achievable.

Your remarks about the Data Model are interesting.  In writing the
'attributes' part of RFC 2924 I was trying to make a list of all the
attributes in use or defined at the time, and looking at extensible ways
to encode them.  Although it would be nice to have everyone use a
single description/encoding method, I'd be happy to settle for two
or three methods, with the user choosing the one which is most suitable
for any particular situation.

To sum up: although it would be nice to have IPFIX become the one
method everyone uses for all accounting-related flow data export,
that seems a distant goal right now.

Since IP mediation vendors are coping now with the variety of flow
export systems in use, concentrating on something which only handles
export from routers and switches would be a fine goal for IPFIX.
That way, AAA servers could become the logical clearing-house, taking
flow data from routers and switches, and passing it - along with
user identification, etc. - to IP mediation (i.e. billing) systems.

</wg chair hat OFF>

See you in San Francisco!

Cheers, Nevil

-----------------------------------------------------------------------
   Nevil Brownlee                   Director, Technology Development
   Phone: +64 9 373 7599 x88941     ITSS, The University of Auckland
   FAX: +64 9 373 7021      Private Bag 92019, Auckland, New Zealand

-------------------------------------------------
This mail sent through University of Auckland
http://www.auckland.ac.nz/

---

From: "Randall Stewart (cisco)" <[email-masked]>
Date: Fri, 14 Mar 2003 11:57:45 -0600

Jeff:

A couple of quick comments to some of what you said :>

>  which should not be ignored.

Yes there is a technical issue.. and I think the PR-SCTP draft removes the
concern and thus is why NFv9 has a mapping to SCTP and specifically will
 use PR-SCTP's features...

>  desirable,  especially given the relative immaturity and expense of SCTP 
I am a bit curious about these comments above...

There have so far been 5 SCTP-bakeoffs (including connect-a-thon last 
year)... and
all of these went well. There are a huge number of implementations if I 
remember
some of the major ones (my aplogies if I miss any .. there are a lot 
more ...):

1) Linux has an implementation (it will be standard in 2.6 and is 
available now in 2.5)
2) IBM AIX had a killer version at the last bakeoff.
3) The BSD's have the version that I wrote and can be found in the KAME 
release.
4) HP has interop'd a version
5) Compac (now I guess HP :->) has shown up with a version
6) Sun has a version..
and a huge number of telco mfngr's  (Nokia, Erricison and Seimens).. to
name a few...  and there were lots more as well from smaller companies...
the big ones stick in my mind :-D

the last bakeoff had close to 20 participants.. and every single 
implementation interop'd fine...

So, since it is available free... and is in LOT of implementations I am not
sure what you mean by the "expense" of SCTP?

And as far as maturity goes.. well.. as of the last bakeoff the fact 
that every implementation
was able to interop with every other one is to me a sign that it IS 
maturing quite nicely...

As far as PR-SCTP.. I know from the BSD work it was an addtion of about 
20-40 lines of code
and I think the same is true for the IOS version of SCTP...

So I am a bit baffeled by your statements... especially when PR-SCTP 
will allow
an ipfix device to be able to send completely reliable and partially 
reliable data over
the same congestion controlled association... a very desireable thing in 
my view..

R

-- 
Randall R. Stewart
ITD
Cisco Systems Inc.
[email-masked] 815-342-5222(cell) or 815-477-2127(office)

---

From: "Peter Ludemann" <[email-masked]>
Date: Fri, 14 Mar 2003 12:34:55 -0800

Randall Stewart wrote Friday, March 14, 2003 9:58 AM:

> There have so far been 5 SCTP-bakeoffs (including connect-a-thon
> last year)... and all of these went well. There are a huge
> number of implementations if I remember some of
> the major ones (my apologies if I miss any .. there are a lot more ...):

The latest list of implementations (Oct 2002) seems to be:
http://www.sctp.org/archive/0123.html and
http://www.sctp.org/archive/0129.html ... Is there anything else?

Are there any implementations of SCTP that I can use from Java without
wrapping in JNI? (That is, the way I can use TCP and UDP without wrapping in
JNI.) For Java, I could only find http://www.sctp.org/archive/0282.html ,
which recommended using JNI.

- peter