Subject: RE: [ipfix] STCP as a default transport not

From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" <[email-masked]>
Date: Thu, 9 Oct 2003 11:03:09 -0700

Benoit,
 
  In answer to your question "So why have TCP at all?"
 
  I'll repeat, ready availability of implementations on common platforms.
 
  The cost of packaging and certifying a collector solution on TCP vs. SCTP
is going to be SIGNIFICANTLY less expensive.  Because SCTP belongs in kernel
vs. user space, this is not going to just magically become ubiquitous.
 
  I guess if your goal is to limit collectors to C implementations on Linux
or BSD then one could argue that "we have it now".  However if Windows or
Java are targets for collection platforms, you're kinda stuck at the moment.
I don't think these are insignificant considerations.
 
-- Jeff

-----Original Message-----
From: Benoit Claise [mailto:[email-masked]]
Sent: Thursday, October 09, 2003 9:15 AM
To: MEYER,JEFFREY D (HP-Cupertino,ex1)
Cc: [email-masked]'; [email-masked]; [email-masked];
'Reinaldo Penno'; [email-masked]; [email-masked]
Subject: Re: [ipfix] STCP as a default transport not

Jeff,

I would like to comment on your recommendations.

Hi,

  I guess there is a concern of whether we are approaching this from

the "optimize for the usual case" or "optimize for the worst case".

  The former is usually preferred.

  The general concern seems to be appropriate "capacity" on the 

collector side so that little to no back pressure occurs on the

sender.

  UDP's fire and forget model makes this trivial, but its lack of

congestion aware behavior is apparently a no no.  From the discussion

it sounds like PR_SCTP has similar properties on overflow, where

things just get tossed, like the behavior of the input UDP socket

buffer when the collector is falling behind.

  However, in a mode where the collector is sufficiently endowed

to consume over TCP, which would hopefully be the majority of

situations, taking advantage of TCP's ubiquity and reliability

properties makes a lot of sense to me.

  I'm still dubious of whether disallowing UDP (which happens to 

be the only current concrete binding of NFv9) in favor of SCTP-PR

is really the best approach.

  Personally I'd recommend:

     o bindings over TCP for reliable communication while accomodating for 

       some failover mechanism if the collector becomes congested or

disabled; 

But if there is no congestion, I assume that SCTP_PR would do the same good
job as TCP in term of reliability.
And we know that TCP would have too big memory requirements in case of
congestion. Hence you propose to failover mechanism.
So why have TCP at all?

     o bindings of UDP for best effort (i.e. no exporter back pressure) in

       configurations where exporter and collector are coresident and

congestion 

       aware behavior is not a concern; 

We tried very hard this path. It's a NO from the area director, even for
coresident exporter/collector.

     o binding of PR_SCTP where congestion aware behavior and best effort

are 

       desired.

So I conclude that there is only solution left that makes sense! PR_SCTP.

Regards, Benoit.

Regards,

  Jeff Meyer

-----Original Message-----

From: Stewart Bryant [ mailto:[email-masked] <mailto:[email-masked]>
]

Sent: Wednesday, October 08, 2003 6:09 AM

To:  [email-masked] <mailto:[email-masked]> 

Cc:  [email-masked] <mailto:[email-masked]> ; 'Reinaldo
Penno'; 'MEYER,JEFFREY D

(HP-Cupertino,ex1)';  [email-masked]
<mailto:[email-masked]> ;

[email-masked] <mailto:[email-masked]> 

Subject: Re: [ipfix] STCP as a default transport not

Carter Bullard wrote:

Hey Stewart,

We should spend some time thinking about the behavior we want in the

exporter when the network is congested, perhaps due to an

attack. PR-SCTP

gives us the option of running a best effort data collection

to gleen what

is going on, when a TCP based exporter would otherwise 

collapse due to

backlog on the exporter.

This is a real engineering issue to talk about.  Issues in 

producer/consumer

dynamics are real problems for IPFIX, especially when the 

flow record

source is generating 1M records per second, which my probes 

are capable

of doing.  In my experience, the reader/collector not keeping up is

never a network congestion problem,  it is almost always a 

reader/collector

resource problem, slow disks, not enough memory, competing 

processes,

whatever.  

The critical issue is which side of TCP does it occur. If TCP 

has accepted

the data then the collector application had better do some 

intelligent.

My concern is with the exporter or probe having a backlog for 

any reason.

It happens in SMP multi-threaded environments where the flow

generator and its collector are on the same machine, it 

happens when the

consumer is on the other end of a dedicated ATM, hyper-channel or

infiniband links.  Its not an issue of reliable vs 

unreliable transport,

its more an issue of what does the probe do when it finds 

it has 1M records

that aren't being consumed, and another 1M records are 

coming very soon.

On this point we absolutely agree.

Does it store them, drop them with an indication?  

I think that we have to assume that it does not have the storage.

 From the PR_SCTP draft:

    One example of partially reliable service is specified in this

    document, namely a "timed reliability" service.  This 

service allows

    the service user to indicate a limit on the duration of 

time that the

    sender should try to transmit/retransmit the message (this is a

    natural extension of the "lifetime" parameter already in the base

    protocol).

This seems to have the right properties

Does it proactively seek

an alternate association, the same association or does it 

wait for the

multiple readers to come back when they are ready.  Are 

there going to be

multiple readers?

Usually the monitor is being attacked, or its generating 

twice the load

because something else is being attacked in the network.  

No congestion,

just a lot of records.  

Anything that means that the exporter has a bunch of data wedged in

its output queues waiting for an ACK looks like congestion to 

the exporter.

I don't think PR-SCTP touches these problems.

If you want the record generator to have a proactive role in

failover and recover, maybe RTP/RTCP would be a better choice.

Explicit feedback control channel provides enough to decide if

it should fail over.

As I recall RTP requires the application to define its own approach to

congestion, and we do have to have a congestion strategy. It would

be better to use a congestion aware transport that has a build-in

strategy to bail-out if there is a problem, and PR-SCTP has just those

properties.

Stewart

Carter

Stewart

--

Help         mailto:[email-masked]
<mailto:[email-masked]>  and say "help"

in message body

Unsubscribe  mailto:[email-masked]
<mailto:[email-masked]>  and say

"unsubscribe ipfix" in message body

Archive      http://ipfix.doit.wisc.edu/archive/
<http://ipfix.doit.wisc.edu/archive/> 

      

--

Help         mailto:[email-masked]
<mailto:[email-masked]>  and say "help" in message body

Unsubscribe  mailto:[email-masked]
<mailto:[email-masked]>  and say

"unsubscribe ipfix" in message body

Archive      http://ipfix.doit.wisc.edu/archive/
<http://ipfix.doit.wisc.edu/archive/>

---

From: Randy Bush <[email-masked]>
Date: Thu, 9 Oct 2003 11:20:05 -0700

it is interesting that folk seem to have enough free time on
their hands to be willing to go around the same discussion
for the Nth time.

randy

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: "Randall Stewart (cisco)" <[email-masked]>
Date: Sat, 11 Oct 2003 19:02:25 -0500

MEYER,JEFFREY D (HP-Cupertino,ex1) wrote:

>   The cost of packaging and certifying a collector solution on TCP vs. 
> SCTP is going to be SIGNIFICANTLY less expensive.  Because SCTP 
> belongs in kernel vs. user space, this is not going to just magically 
> become ubiquitous.
>  
>   I guess if your goal is to limit collectors to C implementations on 
> Linux or BSD then one could argue that "we have it now".  However if 
> Windows or Java are targets for collection platforms, you're kinda 
> stuck at the moment.  I don't think these are insignificant 
> considerations.
>

Jeff:

SCTP is available on

HP-UX
Solaris
Linux
BSD
IOS
and even Windows..

At the last bake-off for SCTP HP-UX's stack demonstrated PR-SCTP testing 
against
the U-of-Essen, BSD and Cisco-IOS

Linux's version were pre-liminary and not tested until next bakeoff (at 
least that is my
understanding).. The windoz implementations were a bit behind.. Sun was 
not present
at this bakeoff.. so I don't know there status..

Bottom line is its out there.. its being used in a LOT of telephone 
signalling apps and
it is NOT more expensive..

R

>     *From:* Benoit Claise [mailto:[email-masked]]
>     *Sent:* Thursday, October 09, 2003 9:15 AM
>     *To:* MEYER,JEFFREY D (HP-Cupertino,ex1)
>     *Cc:* [email-masked]'; [email-masked];
>     [email-masked]; 'Reinaldo Penno';
>     [email-masked]; [email-masked]
>     *Subject:* Re: [ipfix] STCP as a default transport not
>
>     Jeff,
>
>     But if there is no congestion, I assume that SCTP_PR would do the
>     same good job as TCP in term of reliability.
>     And we know that TCP would have too big memory requirements in
>     case of congestion. Hence you propose to failover mechanism.
>     So why have TCP at all?
>
>     We tried very hard this path. It's a NO from the area director,
>     even for coresident exporter/collector.
>
>>>From: Stewart Bryant [mailto:[email-masked]]
>>>To: [email-masked]
>>>Cc: [email-masked]; 'Reinaldo Penno'; 'MEYER,JEFFREY D
>>>(HP-Cupertino,ex1)'; [email-masked];
>>>[email-masked]

-- 
Randall R. Stewart
ITD
Cisco Systems Inc.
[email-masked] 815-342-5222(cell) or 815-477-2127(office)