Subject: RE: [ipfix] Congestion management

From: "Carter Bullard" <[email-masked]>
Date: Mon, 13 Oct 2003 10:59:12 -0400

Hey Randall,
So please lets focus on one issue at a time.  Your draft states:

4.4 Usage Notes
   Detecting missing data in a PR-SCTP stream is useful for some
   applications (e.g. Fiber channel or SCSI over IP). With PR-SCTP this
   becomes possible - the upper layer simply needs to examine the stream
   sequence number of the arrived user messages of that stream to detect
   any missing data.  Note, this detection only works when all the
   messages on that stream are sent in order, i.e., the "U" bit is not
   set.

Seems like this is referring to receiver side data loss detection at
the application level, which I believe is important (a requirement?)
of IPFIXs.  Seems pretty definitive, and an issue important enough to
get its own section, so I expect it to be a true statement.

So, is in order message delivery a requirement for receiver side data
loss detection using PR-SCTP?

Carter

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: Nevil Brownlee <[email-masked]>
Date: Wed, 22 Oct 2003 13:06:34 +1300

Forwarded message from Jon Peterson ......................

Carter, all,

> Well, when you have the other Transport WG AD co-authoring
> a draft that uses RTP/UDP to transport SMPTE video
> at 1.485 Gbps with the intent that it be used over
> "the Big-I", and the IESG approves it in 2003 (RFC 3497),
> it is very difficult for me to accept the notion that
> the IESG has a mandatory requirement that new IP
> applications must use congestion friendly transport.

To start with (RFC3497):

   Since
   congestion control is not possible for SMPTE 292M over RTP flows, use
   of the payload SHOULD be narrowly limited to suitably connected
   network endpoints, or to networks where QoS guarantees are available.

One could argue that this should have been a MUST, but the message is clear
- this RTP profile is intended for use in constrained (non-public-Internet)
environments. This is not intended to be used over "the Big-I", as you state
above. Okay?

Secondly, no, there is no absolute IESG requirement that every new Internet
protocol MUST NOT use UDP. That would be a self-defeating policy - clearly
that are places where UDP can usefully be employed today. This sort of
decision needs to be made on a case-by-case basis. It was made when the
IPFIX charter was approved by the IESG. For RTP, a different decision was
made.

Why? Because things like HDTV streaming video transmission have certain
requirements and constraints that make RTP an appropriate underlying
protocol. "Current" video information should not be retransmitted if it is
lost in network transit - by the time lost packets could be detected and
retransmitted, they would no longer be current, and the receiver would
ultimately rather get the latest information than stale data from three
seconds ago. This is why TCP is unsuitable for real-time protocol
applications like RFC3497 - reliability is actually bad for HDTV.

> Unless of course RTP/UDP is considered by the IESG to
> be congestion friendly.  (I didn't think RTP was friendly,
> just aware, so I possibly erred when I suggested the
> IESG was only concerned about awareness).

You're right - RTP/UDP is not considered "congestion friendly". Though over
the long term, we intend that RTP will run over DCCP rather than UDP, which
will provide the necessary congestion control properties to applications
like streaming HDTV without incurring any of the undesired behavior of TCP.

It's not at all clear to me that IPFIX protocols would actually be broken if
they ran over TCP or SCTP. On the contrary, the IPFIX applicability document
singles out gathering accounting information as one use of IPFIX data - that
strikes me as an environment in which reliability is likely to be important.
It also seems that IPFIX protocols are intended for use on "the Big-I",
unlike RFC3497.

> If an IESG mandatory requirement for congestion friendly
> transport actually existed and RTP/UDP wasn't a congestion
> friendly transport, the Transport WG AD that co-authored
> the draft surely would have known there was a problem, but
> of course if she missed the meeting, the other Transport
> WG AD would have reminded her, the chair of the WG and
> possibly the authors of the draft of the problem.  Of
> course if somehow these mechanisms failed, the draft
> surely would not have been approved and promoted to Standards
> Track RFC by the IESG itself.  Seems like a pretty big
> booo-booo for so many checks and balances to have failed on
> an IP application that runs at
>
>   1.485 GIGABITS per second
>
> minimum.  Obviously, RTP/UDP is congestion friendly, but
> given the possibility that it is not, then I'm sure that
> the IESG has begun the process to retract the RFC, now
> that this potential violation of the IESG mandatory
> congestion friendly transport requirement has been
> identified.
>

Hopefully the misunderstandings that motivate this argument are obvious from
the above.

> I don't know, maybe there isn't really a requirement.

The requirement that I see is in IPFIX's charter, which was created with the
oversight of the IESG, and which states:

   The protocol must run over an IETF approved congestion-aware transport
   protocol such as TCP or SCTP.

It is likely that the IESG will hold this working group to that, yes.

Jon Peterson
NeuStar, Inc.
(Transport Co-AD)

----- End forwarded message -----

-----------------------------------------------------------------------
   Nevil Brownlee                   Director, Technology Development
   Phone: +64 9 373 7599 x88941     ITSS, The University of Auckland
   FAX: +64 9 373 7021      Private Bag 92019, Auckland, New Zealand

-------------------------------------------------
This mail sent through University of Auckland
http://www.auckland.ac.nz/

Subject: Re: [ipfix] Congestion management
From: "Peterson, Jon" <[email-masked]>
To: "[email-masked]'" <[email-masked]>
Date: Sun, 19 Oct 2003 20:21:26 -0400

Carter, all,

> Well, when you have the other Transport WG AD co-authoring
> a draft that uses RTP/UDP to transport SMPTE video
> at 1.485 Gbps with the intent that it be used over
> "the Big-I", and the IESG approves it in 2003 (RFC 3497),
> it is very difficult for me to accept the notion that
> the IESG has a mandatory requirement that new IP
> applications must use congestion friendly transport.

To start with (RFC3497):

   Since
   congestion control is not possible for SMPTE 292M over RTP flows, use
   of the payload SHOULD be narrowly limited to suitably connected
   network endpoints, or to networks where QoS guarantees are available.

One could argue that this should have been a MUST, but the message is clear
- this RTP profile is intended for use in constrained (non-public-Internet)
environments. This is not intended to be used over "the Big-I", as you state
above. Okay?

Secondly, no, there is no absolute IESG requirement that every new Internet
protocol MUST NOT use UDP. That would be a self-defeating policy - clearly
that are places where UDP can usefully be employed today. This sort of
decision needs to be made on a case-by-case basis. It was made when the
IPFIX charter was approved by the IESG. For RTP, a different decision was
made.

Why? Because things like HDTV streaming video transmission have certain
requirements and constraints that make RTP an appropriate underlying
protocol. "Current" video information should not be retransmitted if it is
lost in network transit - by the time lost packets could be detected and
retransmitted, they would no longer be current, and the receiver would
ultimately rather get the latest information than stale data from three
seconds ago. This is why TCP is unsuitable for real-time protocol
applications like RFC3497 - reliability is actually bad for HDTV.

> Unless of course RTP/UDP is considered by the IESG to
> be congestion friendly.  (I didn't think RTP was friendly,
> just aware, so I possibly erred when I suggested the
> IESG was only concerned about awareness).

You're right - RTP/UDP is not considered "congestion friendly". Though over
the long term, we intend that RTP will run over DCCP rather than UDP, which
will provide the necessary congestion control properties to applications
like streaming HDTV without incurring any of the undesired behavior of TCP.

It's not at all clear to me that IPFIX protocols would actually be broken if
they ran over TCP or SCTP. On the contrary, the IPFIX applicability document
singles out gathering accounting information as one use of IPFIX data - that
strikes me as an environment in which reliability is likely to be important.
It also seems that IPFIX protocols are intended for use on "the Big-I",
unlike RFC3497. 

> If an IESG mandatory requirement for congestion friendly
> transport actually existed and RTP/UDP wasn't a congestion
> friendly transport, the Transport WG AD that co-authored
> the draft surely would have known there was a problem, but
> of course if she missed the meeting, the other Transport
> WG AD would have reminded her, the chair of the WG and
> possibly the authors of the draft of the problem.  Of
> course if somehow these mechanisms failed, the draft
> surely would not have been approved and promoted to Standards
> Track RFC by the IESG itself.  Seems like a pretty big
> booo-booo for so many checks and balances to have failed on
> an IP application that runs at
>
>   1.485 GIGABITS per second
>
> minimum.  Obviously, RTP/UDP is congestion friendly, but
> given the possibility that it is not, then I'm sure that
> the IESG has begun the process to retract the RFC, now
> that this potential violation of the IESG mandatory
> congestion friendly transport requirement has been
> identified.
>

Hopefully the misunderstandings that motivate this argument are obvious from
the above.

> I don't know, maybe there isn't really a requirement.

The requirement that I see is in IPFIX's charter, which was created with the
oversight of the IESG, and which states:

   The protocol must run over an IETF approved congestion-aware transport
   protocol such as TCP or SCTP.

It is likely that the IESG will hold this working group to that, yes.

Jon Peterson
NeuStar, Inc.
(Transport Co-AD)

---

From: "Carter Bullard" <[email-masked]>
Date: Tue, 21 Oct 2003 21:16:09 -0400

Hey Nevil,
The second paragraph in the introduction of RFC 3497 states:

   The use of a dedicated serial interconnect is appropriate in a
   studio environment, but it is desirable to leverage the widespread
   availability of high bandwidth IP connectivity to allow efficient
   wide area delivery of SMPTE 292M content.  Accordingly, this memo
   defines an RTP payload format for SMPTE 292M format video.

While they do indicate throughout the RFC that this strategy is
not congestion controlled, the above sentence describing the
intent is very clear.  They state unambiguously, in paragraph 3,
that if you use this in public networks, there should be resource
reservation and enhanced quality of service.    Interesting
that the RFC doesn't mention DCCP as a remedy for the lack of
congestion control.  Must not have been heavy on the minds of
the Transport AD's or the IESG at the time the RFC was written.

Carter

> -----Original Message-----
> From: majordomo listserver
> [mailto:[email-masked]] On Behalf Of Nevil Brownlee
> Sent: Tuesday, October 21, 2003 8:07 PM
> To: [email-masked]
> Subject: [ipfix] Re: Congestion management
>
>
>
>

---

From: Nevil Brownlee <[email-masked]>
Date: Thu, 23 Oct 2003 10:34:41 +1300

----- Forwarded message from [email-masked] -----
    Date: Wed, 22 Oct 2003 16:42:24 -0400
    From: "Peterson, Jon" <[email-masked]>
Reply-To: "Peterson, Jon" <[email-masked]>
 Subject: FW: [ipfix] Re: Congestion management
      To: "[email-masked]'" <[email-masked]>

Pls fwd to ipfix. Thanks again!

- J

----

Carter,

The text that you cite here is non-normative (unlike the text that I cited)
and it is contextualized by the following paragraph, which says that the
"use of this payload format should be tightly controlled and limited to
private networks".  It also mentions resource reservation, which I would say
essentially entails a virtual, tightly-controlled network, yes. Your
contention that this document encourages the use of this 1.5G payload format
on the "Big-I" Internet simply is not substantiated by the document.

In any event, I don't think any of this argues that the IESG will not hold
IPFIX to its charter.

To your second point, while it might seem interesting that RFC3497 doesn't
mention DCCP, I'd note that the string 'UDP' doesn't appear in the RFC
either. This RFC merely defines a payload type for RTP, which will work over
DCCP or UDP as necessary.

Jon Peterson
NeuStar, Inc.

------- start of forwarded message -------
From: "Carter Bullard" <[email-masked]>
To: "'Nevil Brownlee'" <[email-masked]>,
	<[email-masked]>
Subject: RE: [ipfix] Re: Congestion management
Date: Tue, 21 Oct 2003 21:16:09 -0400

Hey Nevil,

Carter

---

From: "Carter Bullard" <[email-masked]>
Date: Thu, 23 Oct 2003 21:50:04 -0400

Hey Jon,
   You know, RFC 2119 doesn't have the word "normative" in
it anywhere.  I must be missing something.  Well, its clear that
the discussion has come to the point where the AD declares
that the IESG is in control and its heart is pure, thus bringing
to a close yet another insightful technical discussion.  Sorry we
didn't get to discuss why the IESG thinks IPFIX MUST (evidently,
the first normative keyword) have congestion control, and 1.485
Gbps SMPTE video SHOULD.  Really, 1.485 Gbps minimum.  The single
most demanding application stream in the history of the IETF, and
it SHOULD consider congestion control, but IPFIX where 95% of
implementations today have directly connected collection, is a MUST.

   It really is a mystery.

Carter

> From: Peterson, Jon [mailto:[email-masked]]
> Sent: Thursday, October 23, 2003 3:16 PM
> To: [email-masked]'; 'Nevil Brownlee'; [email-masked]
> > Hey Jon,
> >    The text I provided was the complete 2nd paragraph
> > from the introduction, that states the motivation for
> > the RFC.  It doesn't get any more "normative" than
> > that.
>
> By "normative" here I refer to the use of RFC2119 keywords,
> which actually
> define qualia for conformance with a specification.
>
> [snip]
> >    The point is, and you have confirmed it now twice,
> > that there is no IESG moratorium restricting WG's
> > from using transports that are not congestion aware.
> > This is unfortunate, as the IPFIX charter was developed
> > with this basic myth as its foundation.
> >
>
> Well, you seem to be under some sort of misapprehension that
> the IESG has
> moratoriums, or policies, or something. The IESG is a
> collection of people
> who hold individual positions on particular charters or
> documents, not a
> group that is subservient to a rigid canon of pre-established
> policies.
>
> What I have confirmed twice is that IPFIX will be held to its
> charter. If
> the IPFIX group produces a protocol that does not conform to
> the constraints
> identified in its charter, that protocol will not pass the IESG. This
> constraint was placed on IPFIX at the time of its inception
> because the IESG
> believed that there was a material risk in deploying such a
> protocol without
> congestion control. The charter does not contain this
> constraint because of
> a "myth" that no protocols are allowed to run over UDP - it
> was because of a
> reasoned decision that IPFIX in particular needed something
> safer. One size
> does not fit all.
>
> >    The result will be that the IPFIX protocol will not be
> > designed to meet the needs of the existing IP flow record
> > community, and I believe that it is safe to say that no existing
> > commercial router will be able to support the default IPFIX
> > protocol.
>
> And that would be regrettable. Understand, though, that the IESG hears
> arguments of this form all of the time, especially from
> participants who
> want to avoid building security into their protocols. Security and
> congestion control can make a protocol more heavyweight,
> sure, and it is
> frequently argued that by requiring such things, we risk
> irrelevance. By not
> requring things like security and congestion control, we risk
> releasing
> low-quality protocols that can damage the networks or
> implementations of our
> participants. On balance, I think it's clear which risk we
> should tolerate
> and which we should not.
>
> >
> > I was hoping for a better outcome.
>
> Sorry.
>
> Jon Peterson
> NeuStar, Inc.
>
> >
> > Carter
> >
> >
>