Subject: RE: [ipfix] Forming consensus on IPFIX default protocol

From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" <[email-masked]>
Date: Wed, 19 Nov 2003 21:25:49 -0500

Alex,

  SCTP SHOULD be a kernel space protocol.  Obviously since you can bind to
raw IP sockets you COULD and there is code out there to put it in user
space, with all the associated penalties.

  The only user space implementation I could actually find working links to
is at:  http://www.sctp.de/sctp-download.html

  Note that sigtran.org, links to sctp.org claiming a user space "reference"
implementation there.  Could only find kernel implementations on the
download page :-(

  Buried in SCTP.de's documentation is the following statement:

    ... Currently this is implemented via function callbacks within 
    one program (i.e. the ULP has functions registered at the very \
    beginning of the program, that are to be called if some notification 
    is due).  Right now, there is only one ULP instance. In the 
    future the communication between SCTP and the ULP shall be done
    with a locally bound UDP socket, that will be used to register a ULP
    instance with the SCTP instance.

     Thus it will be possible to asynchronously call functions of the 
    SCTP protocol engine from a process that sends data on a local 
    UDP socket, and is passed the results and other notification 
    messages back over that socket.

  So, at least this implementation (the only user space I could find),
says they only support one process.  They say they might support 
multiple processes in the future, but by having a single SCTP instance 
(i.e. ONE user space process) turn around and dispatch local UDP 
(i.e. more data copies) to additional processes.

  This seems like a disadvantage to me which is addressed by having
this layer 4 protocol in the kernel, so that dispatching can be
done directly to processes based on SCTP bound port, as opposed
to all IP traffic bound to the SCTP IP protocol ID going up one 
raw socket to one consumer.  Thus saving an unnecessary data copy
for each transmission and receipt.

  Maybe I'm missing something, but your arguments, that it "just
isn't so" ring rather hollow when I try to find concrete vs. anectdotal
evidence.  Give me a URL and some actual description of how these 
user space issues are avoided and maybe I can be convinced....
If your answer is "we've addressed it" but the source is not 
available, then I rest my case.

  So, can we move past the bogus arguments that SCTP is somehow "just as
easy" to use today as TCP, then maybe we can focus on how to word the
protocol spec so that SCTP is encouraged, but TCP and UDP are allowed.

-- Jeff

> -----Original Message-----
> From: Alex Audu [mailto:[email-masked]]
> Sent: Wednesday, November 19, 2003 2:05 PM
> To: MEYER,JEFFREY D (HP-Cupertino,ex1)
> Cc: 'Danny McPherson'; ipfix wg
> Subject: Re: [ipfix] Forming consensus on IPFIX default protocol
> 
> 
> I know I am thousands of e-mails behind. I appologize for that. But I
> wish people would stop making this lame argument that "SCTP is kernel
> space protocol".  It is just totally bogus! And to say there is
> only one user space implementation but it is not 
> multiprocessing is just
> incredibly disingenuous as well.
> 
> I voted for SCTP because I know it is a better protocol. That is from
> experience. I have articulated my reasons (all technical) on 
> this list. And
> if we can get past the emotional argument that CISCO may be pushing
> SCTP, you'll agree that SCTP is a better choice than TCP for IPFIX.
> It should be pointed out that SCTP's design was a group 
> affort involving
> people from Motorola, Cisco, Siemens, Nortel, Ericsson, Telocrdia and
> other organizations.
> 
> I hope we can get this behind us and move on.
> 
> Regards,
> Alex.
> 
> 
> "MEYER,JEFFREY D (HP-Cupertino,ex1)" wrote:
> 
> > Danny,
> >
> >   Thanks for the additional information.  Sorry I couldn't 
> personally attend
> > the IETF, but dollars for travel are tight in our neck of the woods.
> >
> >   I certainly agree with your sentiment around UDP.  
> Regardless of IETF
> > policy, I suspect this will dominate for some time.
> >
> >   The one concern is the "implement what you like aspect".  
> The decision
> > dictates the mandatory support of SCTP, which is fine if 
> you're running on
> > Linux, but is much more problematic on other platforms.  
> There are key
> > differences between deploying kernel space and user space 
> implementations.
> > And although a user space implementation of SCTP is readily 
> available it
> > effectively allows only one process to use SCTP, because 
> all IP traffic
> > which is targeted at the SCTP IP protocol id is targeted to 
> a single user
> > space process.
> >
> >   As a product developer who needs to address customer's 
> desires to run on
> > Linux, HP-UX, Solaris and Windows, this mandatory aspect 
> raises serious
> > concerns, this is doubled when we are talking about Java 
> based collectors.
> >
> >   In an ideal world all platforms would have SCTP out of 
> the box and Java
> > would have a nice object oriented class of sockets which 
> supports it.  This
> > is not the world today, nor do I expect it to be the world 
> for some time.
> >
> >   If we're all willing to wink and agree that we're really 
> just going to use
> > UDP for the forseeable future, and that the choice of SCTP is really
> > targeted at some distant nirvana-esque future, then I guess 
> it doesn't
> > matter what is chosen.  This seems to be the status of 
> Diameter today
> > (although the wink is around TCP vs. SCTP).
> >
> >   If however we want to deal with the brutal reality of the 
> playing field
> > today and define a protocol which can be readily realized 
> on any number of
> > platforms, then the only choice (given IETF constraints) is 
> TCP.  As Stuart
> > Smally says, "Denial ain't just a river in Egypt."
> >
> > Regards,
> >
> >   Jeff Meyer
> >
> > -----Original Message-----
> > From: majordomo listserver 
[mailto:[email-masked]]On Behalf
> Of Danny McPherson
> Sent: Wednesday, November 12, 2003 9:34 PM
> To: ipfix wg
> Subject: Re: [ipfix] Forming consensus on IPFIX default protocol
>
> I saw lots of folks raise their hand when the "How many
> folks would like to see SCTP be the default" (~twice as many
> as opposed to TCP) question was asked and a great number
> of those were NOT Cisco employees -- not that it's even a
> relevant argument here as we're all individuals!
>
> I've never liked the hand-raising thing much anyways, and
> I'd prefer "humming" or nothing at all in the meeting, but
> nonetheless...
>
> As a large consumer of flow information in my day job, I
> suspect UDP will be all that matters in the near term (for
> a number of presumably obvious reasons), and when folks are
> ready to make a change SCTP does have some appealing
> attributes.
>
> And of course, you're always welcome to implement anything
> you'd like...
>
> -danny
>
> On Nov 12, 2003, at 6:09 PM, MEYER,JEFFREY D (HP-Cupertino,ex1) wrote:
>
> > Nevil,
> >
> >   I guess I haven't heard anyone other than Cisco strongly supporting
> > SCTP,
> > but
> > then again that was my impression for NFv9 vs. the other candidates.  I
> > guess
> > will just sit back and let John Chambers do the driving...
>
> --
> Help        mailto:[email-masked] and say "help" in message
> body
> Unsubscribe mailto:[email-masked] and say
> "unsubscribe ipfix" in message body
> Archive     http://ipfix.doit.wisc.edu/archive/
>
> --
> Help        mailto:[email-masked] and say "help" in message
body
> Unsubscribe mailto:[email-masked] and say
> "unsubscribe ipfix" in message body
> Archive     http://ipfix.doit.wisc.edu/archive/

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: "Randall Stewart (cisco)" <[email-masked]>
Date: Thu, 20 Nov 2003 06:01:02 -0600

>implementation there.  Could only find kernel implementations on the
Jeff:

Yeah, thats true... its there but you have to know the name of the file :-0

The reason mainly because:

a) the user space implementation had fallen out of current standards (its
    not up to the Implementors-Guide v 10 like the kernel one).

b) It needed signifigant work to bring in a couple of the extension

c) We thought our efforts best focused on a good solid kernel 
implementation.

Now you can get the 4.5 release of the user space version on a cd when you
buy the book on SCTP.. but the copy in the book does not properly support
PR-SCTP or the other extension ADD-IP.

As to your other post.. if you can get that by the IESG I am fine with it.

>(i.e. more data copies) to additional processes.

If you do go get the book (SCTP a reference guide) you would find a base 
SCTP
implementation that supports MULTIPLE processes. It does this by trading off
having only one process (a daemon) handle the I/O to the kernel. The 
consequence of
this is that data goes in and out of the kernel twice... It does work.. 
but there is a cost...

>for each transmission and receipt.

exactly.. its a trade off.. but if you bind multiple raw sockets its 
even worse since
then every listener sees every SCTP packet... thats bad too :-0

>protocol spec so that SCTP is encouraged, but TCP and UDP are allowed.

If you have a kernel space implementation (which HP/Sun/Linux and BSD 
do) and it fully
supports the socket api (which HP's is soon to  I understand) then you can
easily move things from TCP or UDP to SCTP. Examples of this are:

1) I moved a 1.1 version of mozilla to use only SCTP with changing two lines
    of code. Considering that Mozilla's compressed source image is about 
50+ Megabytes
    not to bad.

2) I moved a version of apache 2.x to use BOTH SCTP and TCP with about 
100 lines of
    code. Again with such a large set of source.. not bad for making the 
engine support
   both TCP and SCTP.

3) I have also moved ftp/ftpd and inetd all to support SCTP with very 
little changes (we are
    talking a few lines of code).

4) Peter Lei moved the complete ssh suite over SCTP with a few lines of 
change.

So I do disagree with you that moving to SCTP is tough.. it just is not...

But I do, as I said above, agree, your idea for transport consensus 
looks like a
decent compromise to me ... but I have doubts as to if the IESG will 
agree :-0

R

-- 
Randall R. Stewart
ITD
Cisco Systems Inc.
[email-masked] 815-342-5222(cell) or 815-477-2127(office)