Subject: RE: [ipfix] Congestion management

From: "Carter Bullard" <[email-masked]>
Date: Fri, 10 Oct 2003 01:55:06 -0400

Hey Peter,
   You have to incur SCTP-PR functions regardless if you
use them or not.  The draft is very clear on this fact.
SCTP-PR doesn't modify the wire protocol, so -PR doesn't
reduce any overhead states or messages.  It simply modifies
the API and some sender semantics.  And, of course, you don't
get some required features such as congestion detection,
without turning on some rather constrained behavior.  For
instance you can't do any kind of loss detection with
SCTP-PR unless you use in order delivery (no "U" bits set).
That can't be too efficient.  And the header is huge, what
is it? a common header, a chunk header and a data
header, we're talking, what, 32 bytes?

One problem I see is that IPFIX will use IPSec for its security.
Without RFC 3554, "On the Use of Stream Control Transmission
Protocol (SCTP) with Ipsec", which is not widely implemented,
IPSec isn't compatible with SCTP features like multi-homing,
basically making the fail-over mechanisms unuseable, and
of course NAT doesn't work with multi-homing.

I'm sure you feel that SCTP-PR is the most efficient mechanism
for transporting bulk records at 100 Mbps across a single hop
dedicated link, but does anyone have any real experience to
back this position?

Carter

-----Original Message-----
From: Peter Lei [mailto:[email-masked]]
Sent: Friday, October 10, 2003 12:06 AM
To: [email-masked]
Cc: [email-masked]; 'David Moore'; 'Nevil Brownlee'
Subject: Re: [ipfix] Congestion management

Carter Bullard wrote:
> Hey David,
>    SCTP-PR is partial reliable, not unreliable.  You
> can't seem to turn off the failure detection and
> protection capabilities of normal reliable SCTP, nor can
> you turn off the congestion avoidance logic of SCTP, so
> it is definitely more complex and more expensive compared
> to say RTP/UDP, and probably TCP.

PR-SCTP can definitely be "unreliable" if you so choose;
have the service definition be "try this only once please"
and the stack will only trasmit each send() call once.

If the transport choice is to be "congestion controlled",
why would NOT do congestion avoidance?!  The feedback loop
is exactly what allows SCTP to have TCP-friendly congetion
control.

I see failure detection as a benefit, not a detriment...
It certainly would be nice to know that the transport has
"gone away" so I can maybe failover to an alternate
collector.  Plus, I don't want to send packets needlessly
if the collector has somehow crashed (e.g. might cause
congestion?)

>    The key is to find a transition technology for the
> 1000's of existing IP flow monitors that are purely
> UDP based today.  A simple transitional technology
> should be appropriate, and I think RTP/UDP is a good
> one, especially since the IESG has historically accepted
> it as passing the congestion awareness criteria.
>
>    I'm personally concerned that the only mentioned
> transport candidates are connection-oriented and unicast
> only.  There are serious advantages to connection-less
> transports and multicast support, and I'm interested in
> ensuring that IPFIX can explore those advantages.  Neither
> SCTP-PR or TCP can support multicast, so I'm an advocate
> of a three transport model.
 >
>    But Nevil is right, the real issue is what should be the
> default transport for IPFIX, and here it seems to me that
> SCTP-PR should NOT be the default, although there is enough
> consensus for it to be a supported transport.  TCP seems to
> be the only real candidate for the default, simply from an
> interoperability standpoint.  If you've only got one shot
> at attaching to an IP flow probe, are you going to use SCTP-PR
> or TCP, if the probe could support either, but not both?
> I think TCP will be the higher probability choice, which
> would make it the default, by default.

I disagree... picking TCP as the default tranport for reliable
data then requires you to use RTP/UDP for the unreliable data
(unless you want to send it ALL over TCP)...

SCTP w/PR-SCTP extension allows both (over the same association
even, if desired) and IMO seems to provide all the transport
services that IPFIX requires.

regards,
--peter

> Carter
>
>
>
>
>>-----Original Message-----
>>From: David Moore [mailto:[email-masked]]
>>Sent: Thursday, October 09, 2003 7:37 PM
>>To: Carter Bullard
>>Cc: 'Nevil Brownlee'; [email-masked]
>>Subject: Re: [ipfix] Congestion management
>>
>>
>>On Thu, Oct 09, 2003 at 07:04:39PM -0400, Carter Bullard wrote:
>>
>>>   When you're dealing with 1M+ records per second, the
>>>concepts of record loss detection and selective retransmission
>>>are just not practical.  With these types of monitors, if
>>>there is packet loss, there really isn't any way that the probes
>>>can slow down, so, like RFC 3497, the concept would have to be
>>>to have some type of feedback signal so the probe can stop.
>>>So RTP/RTCP sounds good.  I'd rather IPFIX send the message,
>>>but the concept of IPFIX being congestion aware itself seems
>>>to have dropped through a fissure somewhere.
>>
>>With SCTP-PR, afaik, allows you to tune whether you want any
>>particular pieces of sent data to be reliable or unreliable.  By
>>setting every piece to be unreliable, then you have UDP in terms
>>of reliability.  However, SCTP-PR has the feedback signal which
>>already provides TCP-friendly congestion avoidance.
>>
>>Is there some performance cost to SCTP-PR when all data is marked
>>unreliable that you feel is too high, to require building a different
>>feedback mechanism on top of UDP to also have congestion avoidance?
>>
>>I understand the desire for a UDP transport which has no congestion
>>avoidance.  But if the goal is something UDP-like (unreliable) but
>>which does have congestion avoidance, IPFIX should use something
>>already a standard (or about to be a standard) rather than inventing
>>its own.
>>
>>
>>-- david
>>
>
>
>
>
>
> --
> Help        mailto:[email-masked] and say "help" in message
body
> Unsubscribe mailto:[email-masked] and say
> "unsubscribe ipfix" in message body
> Archive     http://ipfix.doit.wisc.edu/archive/
>

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: "Carter Bullard" <[email-masked]>
Date: Fri, 10 Oct 2003 10:28:45 -0400

Gentle people,
   Just a follow up on SCTP-PR available implementations.
There were only 3 entities that had SCTP-PR implementations
at the 5th annual bakeoff, Sept 2002, Cisco, Temple and
Uni Essen.

   There was suppose to be the 6th annual bakeoff in June,
2003, but the sctp.org mailing list ends abruptly in May,
2003.  Does anyone know where the mailing list went, and/or
the results of the bakeoff, specifically who else has
implemented SCTP-PR?

Carter

---

From: "Randall Stewart (cisco)" <[email-masked]>
Date: Sat, 11 Oct 2003 19:25:36 -0500

Carter Bullard wrote:

>reduce any overhead states or messages. 
>
Well I think you need to re-read the draft. It extends
the wire protocol and it WILL remove data from being
retransmitted in cases where thing are being abandonded... which
will result in less overhead on the network.

> It simply modifies
>the API and some sender semantics.
>
No, it does far more than this.. Carter please go read the current
draft :>

>  And, of course, you don't
Huh? how do you reach this conclusion? Having the U bit
set makes NO differenct to loss detection. No matter if a
Data chunk is ordered or unordered you will detect loss.

>header, we're talking, what, 32 bytes?
Take a look again ploease.. its 4 bytes larger than TCP period.. And in many
cases it ends up being smaller (since most TCP implementations turn on
time-stamps by default).

I really think you need to spend some time reading RFC2960 and the 
PR-SCTP draft... if
2960 is to heavy to read you can also try my book on SCTP ... a bit easier
to read and it covers all the gory details too ;->

>of course NAT doesn't work with multi-homing.
If you are behind NATs then you need to do the singly homed thing.. this 
does
work.. its just you lose the benefits of multi-homing .. we actually run 
this
as a test at the bakeoff...aka getting through a NAT

>dedicated link, but does anyone have any real experience to
Well we are talking of things beyond a single link network. Otherwise why
are we standardizing it in the IETF? The IETF standardizes things that
run across the Big-I not things that are in your local LAN..

R

-- 
Randall R. Stewart
ITD
Cisco Systems Inc.
[email-masked] 815-342-5222(cell) or 815-477-2127(office)

---

From: "Randall Stewart (cisco)" <[email-masked]>
Date: Sat, 11 Oct 2003 19:29:32 -0500

>at the 5th annual bakeoff, Sept 2002, Cisco, Temple and
But that was 2002 .. the bakeoff at the U-of-Deleware (2003 June) had
Cisco, BSD/Temple, Uni-Essen and The HP-UX stack.
Also we had some testing with our linux friends .. but they just were
not quite done.. I would imagine they will be at the next bakeoff..

Adax wanted to test there PR-SCTP but could not attend due to
a conflict .. most other implementations that were at the June bakeoff
said they would be testing PR-SCTP in the 2004 bakeoff..

R

---

From: Peter Lei <[email-masked]>
Date: Mon, 13 Oct 2003 09:48:49 -0500

> header, we're talking, what, 32 bytes?

I'm not sure what you are getting at here... of course the
functions have to be there in order for you to get the
functionality...

Please re-read the draft... PR-SCTP does indeed extend the
on-the-wire protocol; it is NOT just an API change.

The setting of the "U" bits does NOT "disable" loss detection;
the whole association is congestion controlled.  And that is
the point.  I'd like to have IPFIX and provide reliable and
un-reliable service while being congestion controlled; and I
can do this with one association between a probe and collector.
This is why IMO PR-SCTP fits so nicely.

> of course NAT doesn't work with multi-homing.

Yes, without RFC 3554 one cannot do multi-homed SCTP w/IPsec
efficiently.  I would argue this is no different than the TCP
case today, i.e. you need multiple associations.  However, at
least SCTP gives you the *possibility* of doing this with one
association once IPSec implementations catchup w/3554.

> dedicated link, but does anyone have any real experience to

Well, it is probably not the "most efficient"... but this isn't
the general case we're designing IPFIX for, are we?  If so, we
can forget the NAT arguments.  And yes, we definitely have done
high rate bulk data transfers with SCTP/PR-SCTP.

regards,
--peter