Subject: [rtcweb] #13: Transport of DATA_CHANNEL_OPEN

From: "rtcweb issue tracker" <[email-masked]>
Date: Thu, 11 Apr 2013 23:04:00 -0000

#13: Transport of DATA_CHANNEL_OPEN

 Comment on draft-jesup-rtcweb-data-protocol:

 All DATA_CHANNEL_OPEN messages MUST be sent reliably and in-order.

 [BA] The DATA_CHANNEL_OPEN message is not acknowledged at the application
 layer.  Therefore all that reliability and in-order delivery provides
 (within the SCTP implementation) is the message was *received*, which
 doesn't tell you if the message was acted on or not.  If you do support
 app-layer acknowledgement to address this, then you don't need transport
 reliability.

-- 
-------------------------------------+-------------------------------------
 Reporter:                           |      Owner:  draft-jesup-rtcweb-
  [email-masked]          |  [email-masked]
     Type:  defect                   |     Status:  new
 Priority:  major                    |  Milestone:  milestone1
Component:  data-channel             |    Version:  1.0
 Severity:  Candidate WG Document    |   Keywords:
-------------------------------------+-------------------------------------

Ticket URL: <http://trac.tools.ietf.org/wg/rtcweb/trac/ticket/13>
rtcweb <http://tools.ietf.org/rtcweb/>

---

From: Randell Jesup <[email-masked]>
Date: Tue, 16 Apr 2013 01:38:52 -0400

>   app-layer acknowledgement to address this, then you don't need transport

I disagree (though I suppose you don't *need* in-order, but it's a good 
idea).

Reliable delivery means that you won't have a problem of "Open message 
gets lost, all data packets get queued at receiver but never 
delivered".  Unless you have some meta-channel to tell the receiver that 
an open is on it's way, or add something to say " I have data but no 
Open", you want it to automatically retry to get the Open message 
through and unblock the channel.  This isn't about the sending-side 
application knowing the Open was received (in fact, this draft gives no 
way for the JS app to know that).

In theory because of the queue-on-data-before-open, you don't *need* 
in-order (and it only matters anyways if the channel is marked as 
in-order; otherwise it has no effect).  But I also see no reason for it 
not to be in-order in that case.

-- 
Randell Jesup
[email-masked]

---

From: Martin Thomson <[email-masked]>
Date: Tue, 16 Apr 2013 09:11:47 -0700

> I disagree (though I suppose you don't *need* in-order, but it's a good

In order delivery is not going to change anything.  It's neither good
nor bad, it's straight up useless.  It's the ordering of the next
packet that matters.  If that is marked for out of order delivery, it
can be delivered to the application (in this case, I'm talking about
the browser) before the open message.  Thus, the browser can (and
will) receive messages prior to getting an open.

The only safe assumption it can make at this point is that the channel
is configured for out of order delivery.  Better to be silent about in
order delivery and leave that for buffering.

I do agree with you about reliability.  If you care about this message
enough to have it sent, then you probably want retransmission.

---

From: Peter Thatcher <[email-masked]>
Date: Wed, 17 Apr 2013 08:34:34 -0700

A reminder:  If the JS specifically sets the SID when creating a data
channel, then it can receive data without getting an OPEN message.

I agree with Martin and Randell that the OPEN message must be sent
reliably.

I agree with Martin about in-order-ness and that the real question, if I
can put it in my own words, is: What does the browser do when it receives
data before an OPEN message for a SID that is not currently configured to
receive data (ie JS has not called create channel for that SID and we've
not received an OPEN for that SID)?

No matter what we specify as the correct behavior, the browser still has to
deal with this scenario, at the very least because of possibly erroneous or
even malicious senders.

It sounds like the answer appears to be "buffer until the OPEN".  But there
has to be a limit to how much we buffer.  What if the remote peer
accidentally or maliciously sends us 1GB of data and never sends an OPEN
message?  Then what does the browser do?

I think at that point, we either have to drop data or fire an event up to
JS letting it know.  I would suggest that after some buffer limit N, the
browser may fire .ondatachannel even if an OPEN message is not received,
and then deliver buffered data through the channel given in that event.
 The only alternative seems to be to just drop the data without telling the
JS.  Are there are any other alternatives I haven't thought of?

On Tue, Apr 16, 2013 at 9:11 AM, Martin Thomson <[email-masked]>wrote:

> _______________________________________________
> rtcweb mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rtcweb
>

---

From: Martin Thomson <[email-masked]>
Date: Wed, 17 Apr 2013 11:46:39 -0700

On 17 April 2013 08:34, Peter Thatcher <[email-masked]> wrote:
> What does the browser do when it receives data
> before an OPEN message for a SID that is not currently configured to receive
> data (ie JS has not called create channel for that SID and we've not
> received an OPEN for that SID)?

> channel, then it can receive data without getting an OPEN message.

The only reasonable thing to do is to buffer data with an upper bound
on time, size and number of messages.  When any of those limits are
hit, then you need to do something.

> I think at that point, we either have to drop data or fire an event up to JS
> letting it know.  I would suggest that after some buffer limit N, the
> browser may fire .ondatachannel even if an OPEN message is not received, and
> then deliver buffered data through the channel given in that event.  The
> only alternative seems to be to just drop the data without telling the JS.
> Are there are any other alternatives I haven't thought of?

I see the same two options: create a data channel with some default or
inferred properties and fire events for the arrived messages, or
complain loudly and chuck the messages away.

I like the former, but we've violated expectations about consistency
of channel properties.  And this time its our (as a browser, that is)
fault, we were asked to do something and we failed to do it.  Packet
loss happens and we didn't allow for it.

---

From: Harald Alvestrand <[email-masked]>
Date: Wed, 17 Apr 2013 23:58:17 +0200

I think I (on behalf of applications I'll be writing) am in the camp of 
"If I asked to be told what the incoming data channel was (by not 
registering to handle data without an OPEN), and didn't get told what it 
was, I don't want the data".

Perhaps provide a few buffers to handle packet reordering.
But beyond that: Throw the data away.

---

From: Martin Thomson <[email-masked]>
Date: Wed, 17 Apr 2013 15:01:52 -0700

On 17 April 2013 14:58, Harald Alvestrand <[email-masked]> wrote:
> I think I (on behalf of applications I'll be writing) am in the camp of "If
> I asked to be told what the incoming data channel was (by not registering to
> handle data without an OPEN), and didn't get told what it was, I don't want
> the data".
>

What if the data is ordered?  Couldn't the data you actually get
depend on the data that got discarded?

---

From: Jim Barnett <[email-masked]>
Date: Wed, 17 Apr 2013 22:12:03 +0000

I agree.  Here's another way to look at it:  if you were in charge of security, would you allow installation of a browser that would quietly accept large amounts of data without notifying anyone/anything?

- Jim

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Harald Alvestrand
Sent: Wednesday, April 17, 2013 5:58 PM
To: [email-masked]
Subject: Re: [rtcweb] #13: Transport of DATA_CHANNEL_OPEN

I think I (on behalf of applications I'll be writing) am in the camp of "If I asked to be told what the incoming data channel was (by not registering to handle data without an OPEN), and didn't get told what it was, I don't want the data".

On 04/17/2013 08:46 PM, Martin Thomson wrote:
>> What does the browser do when it receives data before an OPEN message 
>> for a SID that is not currently configured to receive data (ie JS has 
>> not called create channel for that SID and we've not received an OPEN 
>> for that SID)?
>> I think at that point, we either have to drop data or fire an event 
>> up to JS letting it know.  I would suggest that after some buffer 
>> limit N, the browser may fire .ondatachannel even if an OPEN message 
>> is not received, and then deliver buffered data through the channel 
>> given in that event.  The only alternative seems to be to just drop the data without telling the JS.

---

From: Harald Alvestrand <[email-masked]>
Date: Thu, 18 Apr 2013 09:01:46 +0200

> depend on the data that got discarded?
If the data is required to be delivered in-order, it can't be delivered 
before the (in-order) OPEN packet I'm waiting for anyway, so the problem 
cannot occur.

The problem can occur if data can be sent reliably, but without ordering.

I'm willing to consider the option of saying that "throw the data away" 
means "report to the sender that this data channel closed with an 
error". It's even reasonably correct; what the sender expected to happen 
did not happen.

(The receiver will never know that a data channel was attempted. It 
never opened.)

---

From: Martin Thomson <[email-masked]>
Date: Thu, 18 Apr 2013 09:03:59 -0700

> before the (in-order) OPEN packet I'm waiting for anyway, so the problem

Of course.

> means "report to the sender that this data channel closed with an error".
> It's even reasonably correct; what the sender expected to happen did not
> happen.
>
> (The receiver will never know that a data channel was attempted. It never
> opened.)

In that case we'd have to reset the stream.  Maybe our SCTP experts
can advise on whether we could make that look like something other
than a normal close.

---

From: Harald Alvestrand <[email-masked]>
Date: Thu, 18 Apr 2013 18:51:11 +0200

> can advise on whether we could make that look like something other

Yep, "reset the stream" is probably the right terminology.

Sounds good to me.

---

From: Martin Thomson <[email-masked]>
Date: Thu, 18 Apr 2013 10:00:00 -0700

> Yep, "reset the stream" is probably the right terminology.

Note that I was using a very specific terminology, not making shit up
(I know, it's hard to tell sometimes).  c.f.
http://tools.ietf.org/html/rfc6525#section-4.2

Unfortunately, the "Incoming SSN Reset Request Parameter" doesn't
include a way to signal that a particular reset is somehow different
from a normal close.  Therefore, unless you look at the SACK chunks -
which is probably not possible above the SCTP layer - you can't tell
the difference between:
 - I got your data and I'm done with this channel now
 - The OPEN message got lost for long enough that I ran out of
buffering time or space, so this channel is borked; nothing was
delivered

This is where I think that Bernard was going with this.  It's not that simple.

I really don't want to get back to the point where we require
application-level (read: browser-level) acknowledgement of OPEN
messages to address this corner case.  But we have to acknowledge that
it's a problem and deal with it somehow.  (HTTP/2.0 has an error
message that it uses for this sort of case, maybe that could be a
solution, assuming that the stream ID on the reverse path is
available.)

--Martin

---

From: Michael Tuexen <[email-masked]>
Date: Thu, 18 Apr 2013 20:23:03 +0200

> Yep, "reset the stream" is probably the right terminology.
I think just dropping messages without any indication that something went wrong should
be avoided. Closing the channel by resetting the streams is an option.

Best regards
Michael

---

From: Michael Tuexen <[email-masked]>
Date: Thu, 18 Apr 2013 20:30:39 +0200

> buffering time or space, so this channel is borked; nothing was
Absolutely correct!
> This is where I think that Bernard was going with this.  It's not that simple.

I think you are looking for some kind of "reliable opening a datachannel".
That was provided by the three way handshake. But the working group decided
that this kind of setup is not required...
> solution, assuming that the stream ID on the reverse path is
I think the OPEN can only be sent if both streams (incoming / outgoing) are
are available. I suggested an Error message already several times. Also
for other things like reception of unsupported message types, unexpected messages,
message formatting issues, ... There are several error condition. I think
the alternative suggested was to reset the streams. However, for the peer there
is no way to figure out why the data channel was closed if something was lost.

Best regards
Michael

---

From: Michael Tuexen <[email-masked]>
Date: Thu, 18 Apr 2013 20:44:26 +0200

> buffering time or space, so this channel is borked; nothing was
What brings up a question, since I'm not really familiar with the JS API.

If I have a data channel dc and call close() on the data channel. Are
incoming messages still delivered, i.e. does onmessage still fire?

Best regards
Michael

---

From: Martin Thomson <[email-masked]>
Date: Thu, 18 Apr 2013 12:35:49 -0700

> incoming messages still delivered, i.e. does onmessage still fire?

My expectation is that any pending onmessage events would be suppressed.

---

From: Martin Thomson <[email-masked]>
Date: Thu, 18 Apr 2013 12:40:35 -0700

> that this kind of setup is not required...

I certainly don't want this.  If you recall, I advocated for no OPEN
message at all.

> are available.

Yes, that would at least make it possible to indicate an error (except
if the error is that an OPEN is received with no available return
stream ;)

> I suggested an Error message already several times. Also
> message formatting issues, ...

Those at least are application-layer error conditions.  No need to add
anything at the OPEN/ERROR layer for those.  Applications can provide
their own feedback path for those.

> There are several error condition. I think

Exactly the point: not knowing would be bad because it would make data
channels untrustworthy.  Occasionally you would lose a channel and all
the data you sent on it with no way, other than asking your peer, of
knowing what got through.

---

From: Randell Jesup <[email-masked]>
Date: Fri, 19 Apr 2013 12:39:42 -0400

Since we send the Open reliably, barring active attempts to game the 
system with a non-browser, the Open *will* eventually get through unless 
you have 100% (or virtually so) packet loss (and in that case, nothing 
useful, including an error response, is getting through anyways).  So I 
honestly feel it's ok to just buffer all incoming packets while waiting 
for the Open.

No one is going to get a gigabyte of data in without an Open...  A 
non-browser could fake up a session and start sending data without ever 
sending an Open... but flushing the data doesn't actually help you 
against that sort of active DOS (they can just start again, they can 
spread it across thousands of channels, etc, etc), and there are FAR 
better DOS methods - all this would do is burn some CPU and some memory.

-- 
Randell Jesup
[email-masked]

---

From: Martin Thomson <[email-masked]>
Date: Fri, 19 Apr 2013 11:11:04 -0700

On 19 April 2013 09:39, Randell Jesup <[email-masked]> wrote:
> the Open *will* eventually get through unless you have
> 100% (or virtually so) packet loss

I'm going to pretend you didn't say that.  If you want to talk odds,
that's fine, but I think that you'll find that this sort of error is
far more likely than you realize.  We're talking the probability of
incoming data exceeding a given threshold prior to an open being
delivered.  After all, unless you have 0% loss, the *possible* maximum
amount of data is infinite.  Though large numbers might be of
relatively low probability on an individual basis, operating at scale
you are going to encounter surprising spikes.

> I honestly feel it's ok to just buffer all incoming packets while waiting for the Open.

That's not a warm fuzzy that I share.

> sending an Open... but flushing the data doesn't actually help you against
> that sort of active DOS (they can just start again, they can spread it
> across thousands of channels, etc, etc), and there are FAR better DOS
> methods - all this would do is burn some CPU and some memory.

I think that would be a mistake.  This isn't about denial of service,
it's about genuine usage cases that encounter errors.  The receiver
can't use the receive window to apply back pressure if they are
reading from the stream to look for the open message, so you end up
with an unbounded amount of data.  The amount of data will scale with
bandwidth delay product.  A long, fat pipe might burn more CPU and
memory than you are willing to tolerate.

Then it comes down to what experience you want to provide to the
unfortunates who encounter this problem.

---

From: Peter Thatcher <[email-masked]>
Date: Fri, 19 Apr 2013 12:12:50 -0700

I like all the discussion, but I feel like we need to get back to the
question and what options we have.  The question: "what does the browser do
with unexpected data (before an open of an unregistered sid)?"

1. Buffer forever without limits:  Randell thinks it's OK to buffer forever
without limits.  Martin disagrees.  I disagree (I agree with Martin).
2. Buffer with limits, and then:
  a.  Hand an error to JS saying "got some data for a data channel, but an
OPEN never came" WITHOUT providing the data to JS: Harald likes this.  I'm
OK with this.
  b.  Hand an error to JS saying "got some data for a data channel, but an
OPEN never came" WITH providing the data: I like this better, since I don't
see a reason not to give JS the data.
  c.  Fire .ondatachannel: This is what I was was originally thinking, but
I understand the downsides mentioned, and would be happy with (b) instead.

Right now, it seems like 2a or 2b is our best options, mixed with resetting
the stream.  In other words, if I'm a browser, I'd do something like:

function handle_data(sid, data) {
  if (is_open_message(data)) {
    var info = get_stream_info(data)
    var buffered_data = clear_buffer(sid)
    fire_ondatachannel(sid, info)
    fire_ondata(sid, buffered_data)
  } else if (has_known_stream(sid)) {
    fire_ondata(sid, data)
  } else if (buffer_has_space(sid)) {
    add_to_buffer(sid, data)
  } else {
    var buffered_data = clear_buffer(sid)
    fire_unknownchannelerror_in_js(sid, buffered_data)
    reset_stream(sid)
  }
}

Are there other options I'm missing or didn't understand?

On Fri, Apr 19, 2013 at 11:11 AM, Martin Thomson
<[email-masked]>wrote:

> against

---

From: Bernard Aboba <[email-masked]>
Date: Fri, 19 Apr 2013 17:32:04 -0700

Peter Thatcher said: 

I like all the discussion, but I feel like we need to get back to the question and what options we have.  The question: "what does the browser do with unexpected data (before an open of an unregistered sid)?"

1. Buffer forever without limits:  Randell thinks it's OK to buffer forever without limits.  Martin disagrees.  I disagree (I agree with Martin).
[BA] This seems like a bad idea.  If the receiver is expecting an OPEN then it shouldn't allow an unlimited amount of (ordered!) data to be received before getting one.  If the receiver buffers a large amount of data and passes it to the  JS even if the application indicated that an OPEN was required, this might encourage implementations to bypass the OPEN, since receivers are so tolerant about not getting it even if the application has indicated it is required.   2. Buffer with limits, and then:
  a.  Hand an error to JS saying "got some data for a data channel, but an OPEN never came" WITHOUT providing the data to JS: Harald likes this.  I'm OK with this.
[BA] One desirable aspect of this is that it (combined with a stream reset) limits the potential for attacks or implementation problems to affect applications. 
  b.  Hand an error to JS saying "got some data for a data channel, but an OPEN never came" WITH providing the data: I like this better, since I don't see a reason not to give JS the data.  
[BA]  For the reasons described above, I think we could end up regretting this option.

---

From: Randell Jesup <[email-masked]>
Date: Fri, 19 Apr 2013 23:18:50 -0400

On 4/19/2013 8:32 PM, Bernard Aboba wrote:
> question and what options we have.  The question: "what does the 
> browser do with unexpected data (before an open of an unregistered sid)?"
>
> 1. Buffer forever without limits:  Randell thinks it's OK to buffer 
> forever without limits.  Martin disagrees.  I disagree (I agree with 
> Martin).
>
> [BA] This seems like a bad idea.  If the receiver is expecting an OPEN 
> then it shouldn't allow an unlimited amount of (ordered!) data to be 
> received before getting one.

This case only applies in an un-ordered channel I believe, if we send 
the Open as reliable-in-order.  Ordered data must be delivered in-order 
(!), so the Open must be delivered before any following ordered data.  
(This is why I said the Open *should* be sent in-order; it removes the 
need to buffer in an ordered channel.)

>  If the receiver buffers a large amount of data and passes it to the 
>  JS even if the application indicated that an OPEN was required, this 
> might encourage implementations to bypass the OPEN, since receivers 
> are so tolerant about not getting it even if the application has 
> indicated it is required.

Well, if we do decide to timeout or size-out buffering of unordered 
early data, I wouldn't do so quickly.  My example for this would be a 
data-only application opening an channel just as it's fading out of 
WiFi/cell coverage, and hasn't picked up the new one yet (going through 
a tunnel, dead spot in the building, walk down to the basement, etc).

I would far rather the transaction 'freeze' and resume when a new 
connection is picked up (and ICE restart brings the connections alive 
again), than to have an arbitrary timer cause my app/game/etc to fail in 
a very very hard-to-test way.  In practical terms, it would be Very hard 
for significant data to end up buffered.  The application can deal with 
timeouts of the entire connection; I don't want to drop the data if the 
app has decided to stay in suspension waiting for a connection to 
resume.  After all, what is gained by dropping the data?  Don't say 
"knowledge that the channel failed to open" or "the connection is 
borked" - this isn't a good way to find out about total connection loss, 
and given we don't have timeouts on opens, I see no reason to insert 
them in an edge case for open for no useful reason.  If you want opens 
to timeout, that should be part of how Open works (partial-reliable with 
a max time), not indirectly-in-some-edge-cases like this.

>   a.  Hand an error to JS saying "got some data for a data channel, 
> but an OPEN never came" WITHOUT providing the data to JS: Harald likes 
> this.  I'm OK with this.
>
> [BA] One desirable aspect of this is that it (combined with a stream 
> reset) limits the potential for attacks or implementation problems to 
> affect applications.

Painful for implementers to test, and virtually impossible for 
application authors to test, even if they realize it's a possibility.

>   b.  Hand an error to JS saying "got some data for a data channel, 
> but an OPEN never came" WITH providing the data: I like this better, 
> since I don't see a reason not to give JS the data.
>

I agree, though for different reasons than Bernard.

-- 
Randell Jesup
[email-masked]

---

From: Bernard Aboba <[email-masked]>
Date: Sat, 20 Apr 2013 10:15:35 -0700

Randell said:

    "Well, if we do decide to timeout or size-out buffering of unordered
    early data, I wouldn't do so quickly.  My example for this would be
    a data-only application opening an channel just as it's fading out
    of WiFi/cell coverage... If you want opens to timeout, that should be part    of how Open works (partial-reliable with a max time), not    indirectly-in-some-edge-cases like this." [BA] I agree that it makes sense to be explicit about the transport properties desired for OPEN. There is a tradeoff between a timeout/size-out of unordered early data and the amount of "reliability" that the OPEN will really have.    For example, if the timeout is more than one and less than 3 RTOinitial, then you are effectively only allowing for the OPEN to be retransmitted once; between 3 and 7 RTOinitial, twice; between 7 and 15 RTOinitial, three times, etc.  In terms of buffering, there are similar implications.  The amount of buffer allocated will determine how many times the OPEN can be retransmitted (or the amount of time that the endpoint can be out of coverage in the case you gave).  To be resilient, I wouldn't recommend a max time of less than 15 seconds, and one could argue for 30 seconds or more (e.g. the potential effect of routing transients).

---

From: Bernard Aboba <[email-masked]>
Date: Sat, 20 Apr 2013 10:24:25 -0700

(retransmitted to fix a cut-and-paste error).  Randell said:

    "Well, if we do decide to timeout or size-out buffering of unordered
    early data, I wouldn't do so quickly.  My example for this would be
    a data-only application opening an channel just as it's fading out
    of WiFi/cell coverage... If you want opens to timeout, that should be part    of how Open works (partial-reliable with a max time), not    indirectly-in-some-edge-cases like this."
 
[BA] I agree that it makes sense to be explicit about the transport properties desired for OPEN. There is a tradeoff between a timeout/size-out of unordered early data and the amount of "reliability" that the OPEN will really have.    For example, if the timeout is more than one and less than 3 RTOinitial, then you are effectively only allowing for the OPEN to be retransmitted once; between 3 and 7 RTOinitial, twice; between 7 and 15 RTOinitial, three times, etc.  The timeout also limits the amount of time that the endpoint can be out of coverage in the case you gave. In terms of buffering, there are similar implications.  The amount of buffer allocated will determine how many times the OPEN can be retransmitted.  To be resilient, I wouldn't recommend a max time of less than 15 seconds, and one could argue for 30 seconds or more (e.g. the potential effect of routing transients).

---

From: Michael Tuexen <[email-masked]>
Date: Sat, 20 Apr 2013 21:34:33 +0200

> My expectation is that any pending onmessage events would be suppressed.
And how do you close a datachannel without message loss?
I would prefer that if a reliable data channel gets set up, data gets
transferred and it is closed without any error indication, there is
no message loss.
I think this requires that onmessage fires in the CLOSING state.

Best regards
Michael

---

From: Michael Tuexen <[email-masked]>
Date: Sat, 20 Apr 2013 21:38:26 +0200

On Apr 19, 2013, at 6:39 PM, Randell Jesup wrote:

> Since we send the Open reliably, barring active attempts to game the system with a non-browser, the Open *will* eventually get through unless you have 100% (or virtually so) packet loss (and in that case, nothing useful, including an error response, is getting through anyways).  So I honestly feel it's ok to just buffer all incoming packets while waiting for the Open.
I'm not sure what you want to protect against:
1. Network conditions resulting in loss of the OPEN_REQ
2. The peer attacking you and violating the protocol spec.

If you consider 1., SCTP will get the message through or fail the SCTP association eventually.
In the BSD stack we even limit the number of retransmissions of a particular chunk. If that
number is reached, the association is failed. We used this to protect against buggy peers
or path MTU problems.

Best regards
Michael
> 
> No one is going to get a gigabyte of data in without an Open...  A non-browser could fake up a session and start sending data without ever sending an Open... but flushing the data doesn't actually help you against that sort of active DOS (they can just start again, they can spread it across thousands of channels, etc, etc), and there are FAR better DOS methods - all this would do is burn some CPU and some memory.
> 
> -- 
> Randell Jesup
> [email-masked]
>

---

From: Michael Tuexen <[email-masked]>
Date: Sat, 20 Apr 2013 21:41:56 +0200

> memory than you are willing to tolerate.
If you are thinking about a sender violating the spec, then you are right.
If the sender is following the spec, then SCTP will get the OPEN_REQ through.
The amount of data is limited by the send buffer size, since all data sent
after the OPEN_REQ is only SACKed and needs to be buffered (unless you use
NR_SACKs).

Best regards
Michael

---

From: Michael Tuexen <[email-masked]>
Date: Sat, 20 Apr 2013 21:45:53 +0200

On Apr 20, 2013, at 2:32 AM, Bernard Aboba wrote:

> [BA] This seems like a bad idea.  If the receiver is expecting an OPEN then it shouldn't allow an unlimited amount of (ordered!) data to be received before getting one.  If the receiver buffers a large amount of data and passes it 
If the peer is sending ordered data and the first received message is not an OPEN, this is a violation
of the spec and I would terminate the SCTP association.
> to the  JS even if the application indicated that an OPEN was required, this might encourage implementations to bypass the OPEN, since receivers are so tolerant about not getting it even if the application has indicated it is required.  
>

---

From: Michael Tuexen <[email-masked]>
Date: Sat, 20 Apr 2013 21:48:45 +0200

On Apr 20, 2013, at 7:24 PM, Bernard Aboba wrote:

> (retransmitted to fix a cut-and-paste error). 
>  
> Randell said:
> 
> "Well, if we do decide to timeout or size-out buffering of unordered early data, I wouldn't do so quickly.  My example for this would be a data-only application opening an channel just as it's fading out of WiFi/cell coverage... If you want opens to timeout, that should be part of how Open works (partial-reliable with a max time), not indirectly-in-some-edge-cases like this."
>  
SCTP provides already mechanisms to determine that the connectivity os lost and the SCTP
association is terminated. The parameter to control this is the number of consecutive timer-based
retransmissions. Why not just make use of these mechanisms.

Best regards
Michael

---

From: Harald Alvestrand <[email-masked]>
Date: Sun, 21 Apr 2013 02:34:37 +0200

On 04/20/2013 05:18 AM, Randell Jesup wrote:
>> [BA] This seems like a bad idea.  If the receiver is expecting an 
>> OPEN then it shouldn't allow an unlimited amount of (ordered!) data 
>> to be received before getting one.
>
> the Open as reliable-in-order.  Ordered data must be delivered 
> in-order (!), so the Open must be delivered before any following 
> ordered data.  (This is why I said the Open *should* be sent in-order; 
> it removes the need to buffer in an ordered channel.)
>
> WiFi/cell coverage, and hasn't picked up the new one yet (going 
> through a tunnel, dead spot in the building, walk down to the 
> basement, etc).
>
> again), than to have an arbitrary timer cause my app/game/etc to fail 
> in a very very hard-to-test way.  In practical terms, it would be Very 
> hard for significant data to end up buffered.

File transfer protocol: OPEN, send data unordered with in-band file 
block positions, close.
If the OPEN packet gets lost, data will accumulate mighty fast.

>   The application can deal with timeouts of the entire connection; I 
> don't want to drop the data if the app has decided to stay in 
> suspension waiting for a connection to resume.  After all, what is 
> gained by dropping the data?  Don't say "knowledge that the channel 
> failed to open" or "the connection is borked" - this isn't a good way 
> to find out about total connection loss, and given we don't have 
> timeouts on opens, I see no reason to insert them in an edge case for 
> open for no useful reason.  If you want opens to timeout, that should 
> be part of how Open works (partial-reliable with a max time), not 
> indirectly-in-some-edge-cases like this.

Partial-reliable with max time for Open might be a very good idea.
>> but an OPEN never came" WITHOUT providing the data to JS: Harald 
>> likes this.  I'm OK with this.
>>

Hmm. I wonder if this is really very easy to test: Tell the sending app 
you're not going to use OPEN, and tell the receiving app you're going to 
use OPEN. Send data.

---

From: Michael Tuexen <[email-masked]>
Date: Sun, 21 Apr 2013 12:26:39 +0200

On Apr 21, 2013, at 2:34 AM, Harald Alvestrand wrote:

>>> [BA] This seems like a bad idea.  If the receiver is expecting an OPEN then it shouldn't allow an unlimited amount of (ordered!) data to be received before getting one.
>> 
>> This case only applies in an un-ordered channel I believe, if we send the Open as reliable-in-order.  Ordered data must be delivered in-order (!), so the Open must be delivered before any following ordered data.  (This is why I said the Open *should* be sent in-order; it removes the need to buffer in an ordered channel.)
>> 
>>>  If the receiver buffers a large amount of data and passes it to the  JS even if the application indicated that an OPEN was required, this might encourage implementations to bypass the OPEN, since receivers are so tolerant about not getting it even if the application has indicated it is required.  
>> 
>> Well, if we do decide to timeout or size-out buffering of unordered early data, I wouldn't do so quickly.  My example for this would be a data-only application opening an channel just as it's fading out of WiFi/cell coverage, and hasn't picked up the new one yet (going through a tunnel, dead spot in the building, walk down to the basement, etc).  
>> 
>> I would far rather the transaction 'freeze' and resume when a new connection is picked up (and ICE restart brings the connections alive again), than to have an arbitrary timer cause my app/game/etc to fail in a very very hard-to-test way.  In practical terms, it would be Very hard for significant data to end up buffered.
> 
> File transfer protocol: OPEN, send data unordered with in-band file block positions, close.
If it gets dropped on the wire and transmission continues, it gets retransmitted roughly after an RTT.
The amount of data is limited by the socket buffer size of the peer.
> 
>>   The application can deal with timeouts of the entire connection; I don't want to drop the data if the app has decided to stay in suspension waiting for a connection to resume.  After all, what is gained by dropping the data?  Don't say "knowledge that the channel failed to open" or "the connection is borked" - this isn't a good way to find out about total connection loss, and given we don't have timeouts on opens, I see no reason to insert them in an edge case for open for no useful reason.  If you want opens to timeout, that should be part of how Open works (partial-reliable with a max time), not indirectly-in-some-edge-cases like this.
> 
Which means you can't rely on getting an Open... Why not send the open request reliable and ordered. Send
all following user data ordered (no matter if the data channel is ordered or not) until the first message
of the peer is received.
This limits the amount of data to the local socket buffer size. You might have suboptimal performance
until a data message flows in the backwards direction, but that is the price for protection.

Using your example from above, I guess the receiver of the file sends requests for the block positions,
so it wouldn't hurt.

Best regards
Michael
>> Painful for implementers to test, and virtually impossible for application authors to test, even if they realize it's a possibility.
> 
> Hmm. I wonder if this is really very easy to test: Tell the sending app you're not going to use OPEN, and tell the receiving app you're going to use OPEN. Send data.
> 
> 
>>

---

From: Matthew Kaufman <[email-masked]>
Date: Sun, 21 Apr 2013 22:21:22 -0700

>   All DATA_CHANNEL_OPEN messages MUST be sent reliably and in-order.

Ok, I've read the document and the whole discussion.

Why do we need DATA_CHANNEL_OPEN messages?

I can't understand the value of any of the things in there except 
possibly the "Label" field, which is allowed to be empty anyway(?)

Matthew Kaufman

---

From: Peter Thatcher <[email-masked]>
Date: Tue, 23 Apr 2013 11:17:23 -0700

Based on feedback, it sounds like we're basically narrowing this down to
"buffer for a while and then fire an error locally without giving the data
to JS, and close the stream so the remote side knows".  That sounds pretty
good to me.

On Fri, Apr 19, 2013 at 12:12 PM, Peter Thatcher <[email-masked]>wrote:

> Right now, it seems like 2a or 2b is our best options, mixed with
> resetting the stream.  In other words, if I'm a browser, I'd do something
> like:
>
> On Fri, Apr 19, 2013 at 11:11 AM, Martin Thomson <[email-masked]
> > wrote:
>

---

From: Bernard Aboba <[email-masked]>
Date: Tue, 23 Apr 2013 20:25:56 -0700

There is still the need to define "a while".    IMHO, for "reliable" transport of the OPEN that could survive a routing transient, we are talking about roughly 30 seconds or 5 re-transmissions.   If we assume that only the OPEN is lost (no losses in the data),  required buffer = Timeout * min(RWIN, CWIN)/RTT .  This may not be small number. 
Example: Timeout = 30 seconds, min (CWIN, RWIN) = 16 KB, RTT = 50 ms,  buffer = 30 second * 16 KB/50 ms = 9.6 MB

From: [email-masked]
Date: Tue, 23 Apr 2013 11:17:23 -0700
To: [email-masked]
CC: [email-masked]; [email-masked]
Subject: Re: [rtcweb] #13: Transport of DATA_CHANNEL_OPEN

Based on feedback, it sounds like we're basically narrowing this down to "buffer for a while and then fire an error locally without giving the data to JS, and close the stream so the remote side knows".  That sounds pretty good to me.

On Fri, Apr 19, 2013 at 12:12 PM, Peter Thatcher <[email-masked]> wrote:

1. Buffer forever without limits:  Randell thinks it's OK to buffer forever without limits.  Martin disagrees.  I disagree (I agree with Martin).2. Buffer with limits, and then:
  a.  Hand an error to JS saying "got some data for a data channel, but an OPEN never came" WITHOUT providing the data to JS: Harald likes this.  I'm OK with this.  b.  Hand an error to JS saying "got some data for a data channel, but an OPEN never came" WITH providing the data: I like this better, since I don't see a reason not to give JS the data.  

  c.  Fire .ondatachannel: This is what I was was originally thinking, but I understand the downsides mentioned, and would be happy with (b) instead. 

Right now, it seems like 2a or 2b is our best options, mixed with resetting the stream.  In other words, if I'm a browser, I'd do something like:

function handle_data(sid, data) {
  if (is_open_message(data)) {    var info = get_stream_info(data)    var buffered_data = clear_buffer(sid)    fire_ondatachannel(sid, info)

    fire_ondata(sid, buffered_data)
  } else if (has_known_stream(sid)) {    fire_ondata(sid, data)  } else if (buffer_has_space(sid)) {    add_to_buffer(sid, data)  } else {
    var buffered_data = clear_buffer(sid)    fire_unknownchannelerror_in_js(sid, buffered_data)    reset_stream(sid)  }
}

On Fri, Apr 19, 2013 at 11:11 AM, Martin Thomson <[email-masked]> wrote:

rtcweb mailing list

[email-masked]

---

From: Michael Tuexen <[email-masked]>
Date: Wed, 24 Apr 2013 08:19:10 +0200

> Based on feedback, it sounds like we're basically narrowing this down to "buffer for a while and then fire an error locally without giving the data to JS, and close the stream so the remote side kn…
Does the remote side should get any indication that the sent messages are dropped and not
being delivered?

Best regards
Michael

---

From: Michael Tuexen <[email-masked]>
Date: Wed, 24 Apr 2013 08:24:10 +0200

> Example: Timeout = 30 seconds, min (CWIN, RWIN) = 16 KB, RTT = 50 ms,  buffer = 30 second * 16 KB/50 ms = 9.6 MB
I guess you are considering the case where the OPEN message was dropped by the network multiple times...

The sender and receiver normally bound the memory for processing data (socket buffers). So
your above only applies if the send buffer is larger than 9.6 MB. Ofter, the send and
receiver buffer are the same and reflect the RWIN, so you are considering on the receiver
side a 16 KB buffer, on the sender side a 10 MB buffer.

The sender can not delete message sent after the OPEN message, since they can be reneged at the
SCTP layer.

Best regards
Michael
> _______________________________________________ rtcweb mailing list [email-masked]://www.ietf.org/mailman/listinfo/rtcweb

---

From: Bernard Aboba <[email-masked]>
Date: Wed, 24 Apr 2013 07:03:28 -0700

Michael said:
> I guess you are considering the case where the OPEN message was dropped by the network multiple times... [BA] Right.  That's the case in which the details of the OPEN transport makes a difference.
 
> your above only applies if the send buffer is larger than 9.6 MB.  [BA] The assumption is that the receiving application processes the incoming data, so that the RWIN does not decrease.  If the data is sent unordered and unreliable, and is acknowledged by SACK chunks (e.g. only the OPEN is lost), the sender can send this much data (assuming that it has that much to send).   > Often, the send and receiver buffer are the same and reflect the RWIN, so you are considering on the receiver side a 16 KB buffer, on the sender side a 10 MB buffer. [BA] No.  You can have 16 KB on both sides and the sender can end up sending this much, as long as the data isn't lost and the receiving application keeps processing it.  Because of that the RWIN does not limit the sender.

---

From: Matthew Kaufman <[email-masked]>
Date: Wed, 24 Apr 2013 08:19:10 -0700

> possibly the "Label" field, which is allowed to be empty anyway(?)

Today's additional messages have failed to answer my question above. If 
there's no answer, then I propose we get rid of it.

Matthew Kaufman

---

From: Michael Tuexen <[email-masked]>
Date: Wed, 24 Apr 2013 19:17:00 +0200

On Apr 24, 2013, at 4:03 PM, Bernard Aboba wrote:

> [BA] Right.  That's the case in which the details of the OPEN transport makes a difference.
>  
> > your above only applies if the send buffer is larger than 9.6 MB. 
>  
> [BA] The assumption is that the receiving application processes the incoming data, so that the RWIN does not decrease.  If the data is sent unordered and unreliable, and is acknowledged by SACK chunks (e.g. only the OPEN is lost), the sender can send this much data (assuming that it has that much to send).  
That is right for the receiver. But the sender has to buffer the sent messages until they
are acknowledged by the cumack, not only by SACK gat reports. So if the OPEN message
was lost, it is not acked and no TSN after it can be cumacked, only gap-acked. That
is why the sender can't remove it from its send buffer. So the limiting point is
the send buffer size of the sender, not a buffer on the receiver side.
>  
> > Often, the send and receiver buffer are the same and reflect the RWIN, so you are considering on the receiver side a 16 KB buffer, on the sender side a 10 MB buffer.
>  
> [BA] No.  You can have 16 KB on both sides and the sender can end up sending this much, as long as the data isn't lost and the receiving application keeps processing it.  Because of that the RWIN does not limit the sender. 
No. If the send buffer is 16 KB, the sender can only hold up to 16 KB for retransmissions. It has
to buffer TSNs, even if they are gap acked since they can be reneged.
The only way around this is to use the NR-SACK extension (which allows to gap ack
TSN's indicating that they won't be reneged), but this is not an RFC yet...

Best regards
Michael

---

From: Peter Thatcher <[email-masked]>
Date: Wed, 24 Apr 2013 17:29:31 -0700

If you propose getting rid of it, you still need to answer the question:
 what does the browser do with incoming data for an SID that hasn't been
registered by the receiver?  It's almost the same question.

On Wed, Apr 24, 2013 at 8:19 AM, Matthew Kaufman <[email-masked]> wrote:

>>>   Comment on draft-jesup-rtcweb-data-**protocol:
>>>
> ______________________________**_________________
> rtcweb mailing list
> [email-masked]
> https://www.ietf.org/mailman/**listinfo/rtcweb<https://www.ietf.org/mailman/listinfo/rtcweb>
>

---

From: "Matthew Kaufman (SKYPE)" <[email-masked]>
Date: Thu, 25 Apr 2013 05:28:08 +0000

It is a much simpler question as there’s only a couple of options left… how about we throw it on the floor? That seems safest to me.

I’d still love to hear why it is “needed” though.

Matthew Kaufman

From: [email-masked] [mailto:[email-masked]] On Behalf Of Peter Thatcher
Sent: Wednesday, April 24, 2013 5:30 PM
To: Matthew Kaufman
Cc: <[email-masked]>
Subject: Re: [rtcweb] #13: Transport of DATA_CHANNEL_OPEN

If you propose getting rid of it, you still need to answer the question:  what does the browser do with incoming data for an SID that hasn't been registered by the receiver?  It's almost the same question.

On Wed, Apr 24, 2013 at 8:19 AM, Matthew Kaufman <[email-masked]<mailto:[email-masked]>> wrote:
On 4/21/2013 10:21 PM, Matthew Kaufman wrote:
On 4/11/2013 4:04 PM, rtcweb issue tracker wrote:
#13: Transport of DATA_CHANNEL_OPEN

I can't understand the value of any of the things in there except possibly the "Label" field, which is allowed to be empty anyway(?)

Today's additional messages have failed to answer my question above. If there's no answer, then I propose we get rid of it.

Matthew Kaufman
_______________________________________________
rtcweb mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/rtcweb

---

From: "Cullen Jennings (fluffy)" <[email-masked]>
Date: Thu, 25 Apr 2013 21:10:04 +0000

So with my co-chair hat on here… 

It seems we have been around the need for OPEN several times and have come to consensus on it in the past. Can someone please:

1) summarize the arguments that in the past that lead us to think we need OPEN 

2) sketch out the range of possible solutions to deal with unexpected data before the OPEN 

Cullen

---

From: Randell Jesup <[email-masked]>
Date: Thu, 25 Apr 2013 17:42:58 -0400

> 1) summarize the arguments that in the past that lead us to think we need OPEN

The Open message has some important and useful properties:

1) It's easier to work with.  JS isn't an ideal language for writing 
network protocols, especially for non-network-programmers (i.e. a large 
portion of the expected developer community).  Open makes it really easy 
for a developer to get the type of bidirectional stream them want with 
little fuss, and in a manner that maps easily to APIs they're already 
used to (e.g. WebSockets).  In many cases the label field will avoid the 
JS programmer having to build their own mini-protocol to identify their 
channels (and this is especially painful for them if it's an unreliable 
channel).

2) It makes it possible to have different applications exchange data, by 
having an IANA-registered protocol name, like WebSockets (and there was 
strong agreement on it's being needed for that at Atlanta.)  With 
external-negotiation-only, it would be very hard for different apps to 
interoperate, since they'd need to agree on negotiation protocols as 
well, which are likely to be highly divergent between apps.

3) The Open message, being in-channel, reliable and in-order, makes the 
issue that was the genesis of this thread (what to do with data that 
arrives unexpectedly) simpler.  With Open, only degenerate cases can 
cause more than a relatively small amount of data to be buffered.  With 
external notification, the external negotiation channel can fail (or be 
very slow), or the app can have a bug and fail to install the negotiated 
values, leading to larger or unbounded buffering requirements - or you 
punt the problem to the application by delivering the data, but the 
application is facing the same conundrum of buffer it or throw it away.

> 2) sketch out the range of possible solutions to deal with unexpected data before the OPEN

The issue exists regardless of whether Open is used or external 
negotiation (and in fact is much simpler for Open).

*tl;dr: *I'm ok with any setting of maximum sizes and/or times that 
would not adversely impact temporary buffering for normal cases with 
Open of unordered channels.  This is option C below.  I also would be ok 
with B, but I realize others may not be.

Regardless of supporting Open or not, any external negotiation of 
dynamic channels must use one of these:

A) a 2-or-3-way handshake so the sender knows the receiver is ready to 
receive the data on the specified channel before sending it, or

B) unbounded buffering of data if the external negotiation messages are 
delayed (again, what we're discussing in this thread), or

C) bounded buffering of data (bounded by time, size or both), with data 
being dropped and the channel closed if the limits are exceeded, or

D) deliver unexpected data to the application, which will do .... I 
don't know what with it.

For (D), the application will likely drop it on the floor (leading to 
hard-to-test-for problems if the channel is later configured by Open or 
external negotiation), or buffer it waiting for the channel to open.

Supporting Open or not has little bearing on these scenarios -- and in 
fact, since Open is in-channel, reliable, and ordered, it reduces the 
problem set (when Open is used) to only unordered channels (in ordered 
channels Open will always be first).

Buffering unexpected data on channels (options B or C) is useful. It 
means that in the external negotiation case, one side asking for a new 
channel to open by some private means doesn't need to wait to start 
sending data on that channel.  Note that with external negotiation 
(possibly on a non p2p path, like via signaling), the receiver might not 
know what to install for a short while, especially if there's a routing 
issue or server issue (not a problem that happens with Open).

Since normal Open cases have very little chance of triggering this 
problem (triggering buffering), some arbitrary size limit seems 
reasonable (option C).  Often in network protocols there are small 
buffers (4, 16, 64KB).  I prefer a larger value of say 256KB so that 
apps using external negotiation can just send largish data immediately - 
and note: actually buffering data is still an unlikely occurrence even 
in most external negotiation cases.  If people want to bikeshed on the 
buffersize or timeout, that's fine. ;-)

Also, external negotiation is the only case where more than a trivial 
amount of data can "pile up" in the buffer waiting for the receiving 
side to finish it's side of the negotiation (i.e. if your external 
negotiation channel fails or the app has a brain fart).

-- 
Randell Jesup
[email-masked]

---

From: Harald Alvestrand <[email-masked]>
Date: Fri, 26 Apr 2013 07:30:42 +0200

On 04/25/2013 07:28 AM, Matthew Kaufman (SKYPE) wrote:
>
> It is a much simpler question as there's only a couple of options 
> left... how about we throw it on the floor? That seems safest to me.
>
> I'd still love to hear why it is "needed" though.
>

The fact that Matthew doesn't want to accept the arguments for why it is 
needed doesn't seem relevant to me.

I think people have said enough times that they see value in the label 
concept, and they want some way to carry it. The OPEN message is a 
simple way to fill that need (and also makes some other aspects of setup 
simpler than they would otherwise be).

---

From: Matthew Kaufman <[email-masked]>
Date: Fri, 26 Apr 2013 06:53:15 -0700

On 4/25/2013 2:42 PM, Randell Jesup wrote:
>> It seems we have been around the need for OPEN several times and have 
>> come to consensus on it in the past. Can someone please:
>>
> network protocols, especially for non-network-programmers (i.e. a 
> large portion of the expected developer community).  Open makes it 
> really easy for a developer to get the type of bidirectional stream 
> them want with little fuss, and in a manner that maps easily to APIs 
> they're already used to (e.g. WebSockets).  In many cases the label 
> field will avoid the JS programmer having to build their own 
> mini-protocol to identify their channels (and this is especially 
> painful for them if it's an unreliable channel).

Easier how? The initiating side needs to create their object without the 
benefit of the contents of an open message. If a JS developer isn't 
smart enough to set up the channel parameters they want at the 
initiating end, then there's nothing we can do for them. There's then 
several ways forward after that, including "the other side does the same 
thing to create their end" (my preference) and "the parameters are 
transported using the existing SDP O/A mechanism to the far end" 
(generally how the WG solves this kind of problem).

As far as I can tell, the one useful thing in the OPEN message is the 
label (as I pointed out in my very first inquiry about why anyone 
thought it was needed), and yet the label is allowed to be null, so in 
that case how can it be helpful?

>
> 2) It makes it possible to have different applications exchange data, 
> by having an IANA-registered protocol name, like WebSockets (and there 
> was strong agreement on it's being needed for that at Atlanta.)  With 

If we went with "negotiated via the SDP O/A mechanism" then we could go 
to our favorite SDP-defining WG and have a negotiation protocol that is 
standardized, so not divergent at all between apps.

>
> 3) The Open message, being in-channel, reliable and in-order, makes 
> the issue that was the genesis of this thread (what to do with data 
> that arrives unexpectedly) simpler.

I disagree. It increases the number of possible states... there's cases 
where the OPEN message arrives but the application doesn't want to be 
receiving that data, and the cases where the OPEN message doesn't arrive 
but the application does want the data that is already arriving. Both of 
those states don't exist if there's no OPEN message.

> With Open, only degenerate cases can cause more than a relatively 
> small amount of data to be buffered.

Sure, but those degenerate cases occur many times per day on the real 
Internet.

> With external notification, the external negotiation channel can fail 
> (or be very slow), or the app can have a bug and fail to install the 
> negotiated values, leading to larger or unbounded buffering 
> requirements - or you punt the problem to the application by 
> delivering the data, but the application is facing the same conundrum 
> of buffer it or throw it away.
>
> Open of unordered channels.  This is option C below.  I also would be 
> ok with B, but I realize others may not be.
>
> B) unbounded buffering of data if the external negotiation messages 
> are delayed (again, what we're discussing in this thread), or
>
> C) bounded buffering of data (bounded by time, size or both), with 
> data being dropped and the channel closed if the limits are exceeded, or
>
> hard-to-test-for problems if the channel is later configured by Open 
> or external negotiation), or buffer it waiting for the channel to open.
>
> Supporting Open or not has little bearing on these scenarios

So if that's true, then why do we need it?

> -- and in fact, since Open is in-channel, reliable, and ordered, it 
> reduces the problem set (when Open is used) to only unordered channels 
> (in ordered channels Open will always be first).
>
> sending data on that channel.

Of course this is different from the TCP model, where the buffering 
happens at the *sender* until the handshake is complete.

The problem here is that we have an existing mechanism inside of SCTP 
and then we're adding another layer on top of that which is neither 
inside SCTP (where it probably belongs) or under the application 
developer's control (where I'd like to see it, if the data transport 
protocol can't have it), and this layer conflicts with what both of 
those might be doing.

> Note that with external negotiation (possibly on a non p2p path, like 
> via signaling), the receiver might not know what to install for a 
> short while, especially if there's a routing issue or server issue 
> (not a problem that happens with Open).

Which is fine if the sender doesn't start sending (like TCP). Or if the 
transport protocol itself handles the delivery of the channel metadata. 
(Which is what SCTP, or whatever transport we choose for RTCWEB, should 
do) then you just handle the label and whatever other metadata the 
receiver would like to know at that layer.

> apps using external negotiation can just send largish data immediately 
> - and note: actually buffering data is still an unlikely occurrence 
> even in most external negotiation cases.  If people want to bikeshed 
> on the buffersize or timeout, that's fine. ;-)
>

Another great argument for having this negotiation happen within the 
transport itself.

Matthew Kaufman

---

From: Matthew Kaufman <[email-masked]>
Date: Fri, 26 Apr 2013 06:56:02 -0700

On 4/25/2013 10:30 PM, Harald Alvestrand wrote:
> The fact that Matthew doesn't want to accept the arguments for why it 
> is needed doesn't seem relevant to me.

Until Randell's message arrived a few hours before yours, there was no 
concise summary of why it might be needed.

Never mind that your personal view as to whether or not to just ignore 
my technical arguments offhand because they're sent by me is hardly 
relevant to the rest of the working group.

> simple way to fill that need (and also makes some other aspects of 
> setup simpler than they would otherwise be).
>
>

And as I pointed out in my original message on this topic, which until 
Randell's message was met with silence, the only thing that made sense 
to me was the label field, and yet the label can be null, so might also 
be useless.

Matthew Kaufman

---

From: Randell Jesup <[email-masked]>
Date: Fri, 26 Apr 2013 13:36:06 -0400

On 4/26/2013 9:53 AM, Matthew Kaufman wrote:
>>> It seems we have been around the need for OPEN several times and 
>>> have come to consensus on it in the past. Can someone please:
>>>
> Easier how? The initiating side needs to create their object without 
> the benefit of the contents of an open message. If a JS developer 
> isn't smart enough to set up the channel parameters they want at the 
> several ways forward after that, including "the other side does the 
> same thing to create their end" (my preference) and "the parameters 
> are transported using the existing SDP O/A mechanism to the far end" 

It's helpful if you need it (if you're opening a number of channels, 
such as one per participant in a conference).  Protocol is also helpful 
(note it's in the dictionary and is also optional); both for 
cross-application channels and for within an app to indicate what 
handling logic the channel should feed.

// pseudo-codey:
function called_from_ondatachanne(event) {
   channel = event.channel;
   if (channel.protocol == "file transfer") { channel.onmessage = 
handle_incoming_file; }
   else if (channel.protocol == "chat") {chat = new 
chat_instance(channel.label); channel.onmessage = chat.incoming; }
}

>> by having an IANA-registered protocol name, like WebSockets (and 
>> there was strong agreement on it's being needed for that at 
>> Atlanta.)  With external-negotiation-only, it would be very hard for 
>> different apps to interoperate, since they'd need to agree on 
>> negotiation protocols as well, which are likely to be highly 
>> divergent between apps.
>
> If we went with "negotiated via the SDP O/A mechanism" then we could 
> go to our favorite SDP-defining WG and have a negotiation protocol 
> that is standardized, so not divergent at all between apps.

We also don't have one that's divergent by using Open (and we leave the 
option of externally negotiating or pre-defining channels). And SDP is 
at least one signaling-RTT to complete, plus a bunch of logic to handle 
matching everything up (what if channels disappear in the offer or 
answer, or in an offer or answer they change properties? What if the 
signaling channel is slow or unavailable any more (server rebooted, 
server has network issues, etc)?

I'm not saying there aren't answers/solutions to all these.  But those 
answers and solutions would need to be speced out or at least 
understood, and the code to handle all of them is likely to be similar 
in scope (though quite different in detail).  And the less done in SDP, 
the better IMHO; I have no wish to increase our reliance on SDP.

> I disagree. It increases the number of possible states... there's 
> cases where the OPEN message arrives but the application doesn't want 
> to be receiving that data, and the cases where the OPEN message 
> doesn't arrive but the application does want the data that is already 
> arriving. Both of those states don't exist if there's no OPEN message.

My point was you need to decide what to do with that data regardless of 
if the channel gets created with a delayed OPEN or with a delayed 
external negotiation.  And with OPEN, those cases don't even arise for 
ordered channels.

> Sure, but those degenerate cases occur many times per day on the real 

I doubt that.  The degenerate cases for Open require that all the open 
packets get lost, while a sizable amount of non-open packets get through 
- and after a time, SCTP will fail the entire association if reliable 
data isn't getting through.  Plus, as I indicated, we can also have 
limits on time and/or amount of data buffered.

Also: nothing *requires* that an application send data immediately on 
onopen.  An application can institute it's own handshake trivially:
function my_ondatachannel(event) { 
event.channel.send(ok_to_start_sending); ... }

>> A) a 2-or-3-way handshake so the sender knows the receiver is ready 
>> to receive the data on the specified channel before sending it, or
>>

This was a discussion of "what to do with data that arrives unexpectedly 
(i.e. before Open, or before the application installed the result of an 
external negotiation); this was the original point of the thread before 
you decided to question the selection of Open as a message.

The caveat at the top was "external negotiation of dynamic channels must 
do one of these" regardless of whether Open is supported.  The options 
for use with Open are the same, but the amount of data that can be 
buffered is smaller (since with external negotiation there are no limits 
on the amount that could attempt to be queued -- the same applies SDP if 
you allow the sender to send data before the negotiation is complete; 
SDP is effectively a form of external negotiation).

>> reduces the problem set (when Open is used) to only unordered 
>> channels (in ordered channels Open will always be first).
>>
>> means that in the external negotiation case, one side asking for a 
>> new channel to open by some private means doesn't need to wait to 
>> start sending data on that channel.
>

Ok, but this isn't TCP.  And handshakes (and the resultant delay before 
onopen fired) were eliminated in order to respond to the requests of 
multiple people, giving us a 0-RTT declarative protocol where you can 
send immediately, which will help apps that need to exchange data 
quickly at the start of a connection.

> protocol can't have it), and this layer conflicts with what both of 

If you're referring to Open, you can totally ignore Open and not use it 
in your datachannels, and use all external negotiation or pre-defined 
channels.  If you want, you can just pre-define N open channels that the 
other side can then use to send data to you, and nothing will ever be 
buffered, and you can handle it all yourself.

> Which is fine if the sender doesn't start sending (like TCP). Or if 
> the transport protocol itself handles the delivery of the channel 
> metadata. (Which is what SCTP, or whatever transport we choose for 
> RTCWEB, should do) then you just handle the label and whatever other 
> metadata the receiver would like to know at that layer.

I.e. data streams from RTMFP, I take it? ;-)

>> apps using external negotiation can just send largish data 
>> immediately - and note: actually buffering data is still an unlikely 
>> occurrence even in most external negotiation cases.  If people want 
>> to bikeshed on the buffersize or timeout, that's fine. ;-)
>>

If you want to argue for replacing SCTP as the base layer - we made that 
decision a good long while ago.  And there's a good, maintained, 
BSD-licensed implementation - and the authors of that are working with 
us here in designing a DataChannel protocol to ride on top of it.  If 
you want to argue that the layer on top of SCTP should be handshaked 
once again (as it was until people objected) that's up to you to make 
the case for.

IMO:
I'd like to avoid continuing to circle through the range of all possible 
protocols one at a time, repeating every few months.  I'll assert we can 
bikeshed on this until we all retire, and we'll never have something 
that makes everyone entirely happy (unless most people get tired and 
stop caring).  All solutions have tradeoffs; none is perfect for all 
use-cases and users.  We arrived at a proposal that had general 
agreement in the room to move forward with per the consensus call, and 
I'm preparing a WG draft in response to that.  Some may like it more and 
some less, but there was agreement that people could work with it.

-- 
Randell Jesup
[email-masked]

---

From: Cullen Jennings <[email-masked]>
Date: Fri, 26 Apr 2013 11:57:33 -0600

Thank you summarizing all this. 

On Apr 25, 2013, at 3:42 PM, Randell Jesup <[email-masked]> wrote:

> 1) It's easier to work with.  JS isn't an ideal language for writing network protocols, especially for non-network-programmers (i.e. a large portion of the expected developer community).  Open makes it really easy for a developer to get the type of bidirectional stream them want with little fuss, and in a manner that maps easily to APIs they're already used to (e.g. WebSockets).  In many cases the label field will avoid the JS programmer having to build their own mini-protocol to identify their channels (and this is especially painful for them if it's an unreliable channel).
> 
> 2) It makes it possible to have different applications exchange data, by having an IANA-registered protocol name, like WebSockets (and there was strong agreement on it's being needed for that at Atlanta.)  With external-negotiation-only, it would be very hard for different apps to interoperate, since they'd need to agree on negotiation protocols as well, which are likely to be highly divergent between apps.
> 
> 3) The Open message, being in-channel, reliable and in-order, makes the issue that was the genesis of this thread (what to do with data that arrives unexpectedly) simpler.  With Open, only degenerate cases can cause more than a relatively small amount of data to be buffered.  With external notification, the external negotiation channel can fail (or be very slow), or the app can have a bug and fail to install the negotiated values, leading to larger or unbounded buffering requirements - or you punt the problem to the application by delivering the data, but the application is facing the same conundrum of buffer it or throw it away.
> 
> The issue exists regardless of whether Open is used or external negotiation (and in fact is much simpler for Open).
> 
> *tl;dr: *I'm ok with any setting of maximum sizes and/or times that would not adversely impact temporary buffering for normal cases with Open of unordered channels.  This is option C below.  I also would be ok with B, but I realize others may not be.
> 
> Regardless of supporting Open or not, any external negotiation of dynamic channels must use one of these:
> 
> A) a 2-or-3-way handshake so the sender knows the receiver is ready to receive the data on the specified channel before sending it, or
> 
> B) unbounded buffering of data if the external negotiation messages are delayed (again, what we're discussing in this thread), or
> 
> C) bounded buffering of data (bounded by time, size or both), with data being dropped and the channel closed if the limits are exceeded, or
> 
> D) deliver unexpected data to the application, which will do .... I don't know what with it.
> 
> For (D), the application will likely drop it on the floor (leading to hard-to-test-for problems if the channel is later configured by Open or external negotiation), or buffer it waiting for the channel to open.
> 
> Supporting Open or not has little bearing on these scenarios -- and in fact, since Open is in-channel, reliable, and ordered, it reduces the problem set (when Open is used) to only unordered channels (in ordered channels Open will always be first).
> 
> Buffering unexpected data on channels (options B or C) is useful. It means that in the external negotiation case, one side asking for a new channel to open by some private means doesn't need to wait to start sending data on that channel.  Note that with external negotiation (possibly on a non p2p path, like via signaling), the receiver might not know what to install for a short while, especially if there's a routing issue or server issue (not a problem that happens with Open).
> 
> Since normal Open cases have very little chance of triggering this problem (triggering buffering), some arbitrary size limit seems reasonable (option C).  Often in network protocols there are small buffers (4, 16, 64KB).  I prefer a larger value of say 256KB so that apps using external negotiation can just send largish data immediately - and note: actually buffering data is still an unlikely occurrence even in most external negotiation cases.  If people want to bikeshed on the buffersize or timeout, that's fine. ;-)
> 
> Also, external negotiation is the only case where more than a trivial amount of data can "pile up" in the buffer waiting for the receiving side to finish it's side of the negotiation (i.e. if your external negotiation channel fails or the app has a brain fart).
> 
> 
> -- 
> Randell Jesup
> [email-masked]
>

---

From: "Ejzak, Richard P (Richard)" <[email-masked]>
Date: Mon, 29 Apr 2013 14:11:38 +0000

After remaining neutral during most of this discussion about the need for OPEN, I've finally come to the conclusion that Matthew is right.  The only thing the OPEN provides is the label, which is just an opaque string with no meaning to the browser or any intermediaries.  It only has application significance.  If this is the case, why not just use stream id?  What can you possibly do with label that you can't do with pre-assigned stream ids (this is a serious question to Randell since it is fundamental to whether we need OPEN)?  Since label only has meaning when both browsers run the same application, what does it matter if you agree on a set of opaque strings to correspond to various functions or just agree on a set of stream ids for these functions?  It doesn't change the complexity of the code and it doesn't even change the readability of the code if you just enumerate the reserved stream ids and "label" them in the code.  

You can even reserve stream ids in odd/even pairs so that either end can initiate them.  You can reserve blocks of stream ids if you want multiple instances of the same type of DC.  I don't see any significant limitation here.

Getting rid of OPEN eliminates any issues around their handling in the unordered case.

Getting rid of OPEN does not preclude use of new subprotocols as needed and does not preclude use of any other signaling to negotiate use of DataChannels between heterogeneous applications.  OPEN does not help in this case anyway.

> On 4/25/2013 12:36 PM, Randell Jesup wrote:
> >>> So with my co-chair hat on here...
> >>>
> >>> 1) summarize the arguments that in the past that lead us to think
> we
> >>> need OPEN
> >>
> (note it's in the dictionary and is also optional); both for cross-
> application channels and for within an app to indicate what handling
> logic the channel should feed.
> 
> // pseudo-codey:
> chat_instance(channel.label); channel.onmessage = chat.incoming; } }
> 
> >
> >>

---

From: Bernard Aboba <[email-masked]>
Date: Mon, 29 Apr 2013 09:40:52 -0700

I would also note that the maximum number of streams can be indicated in the association negotiation.  So if there is concern about unexpected data channels and associated data, that mechanism can be used. If the application knows how many streams it will need, might it not be wise to indicate this in association setup?

---

From: Randell Jesup <[email-masked]>
Date: Mon, 29 Apr 2013 14:56:59 -0700

> I would also note that the maximum number of streams can be indicated in the association negotiation.  So if there is concern about unexpected data channels and associated data, that mechanism can…

The number of streams in the SDP is the initial number of streams (with 
a suggested default of 16).  That's entirely an optimization to avoid 
the application needing to immediately need to renegotiate a larger 
number, or to have a large number of never-used channels if the default 
was set way too high.

If streams are needed past the number currently allocated, an increase 
will be requested and the createDataChannel queued.  (Note: this applies 
to pre-negotiated channels using streams past the currently allocated 
number as well).

-- 
Randell Jesup
[email-masked]

---

From: Bernard Aboba <[email-masked]>
Date: Mon, 29 Apr 2013 15:29:36 -0700

Right. What I was trying to say is that an application has control over how many data channels it can need to handle. 

On Apr 29, 2013, at 2:57 PM, "Randell Jesup" <[email-masked]> wrote:

> The number of streams in the SDP is the initial number of streams (with a suggested default of 16).  That's entirely an optimization to avoid the application needing to immediately need to renegotiate a larger number, or to have a large number of never-used channels if the default was set way too high.
> 
> If streams are needed past the number currently allocated, an increase will be requested and the createDataChannel queued.  (Note: this applies to pre-negotiated channels using streams past the currently allocated number as well).
> 
> -- 
> Randell Jesup
> [email-masked]
>

---

From: Martin Thomson <[email-masked]>
Date: Mon, 29 Apr 2013 15:58:23 -0700

On 29 April 2013 14:56, Randell Jesup <[email-masked]> wrote:
> The number of streams in the SDP is the initial number of streams (with a
> suggested default of 16).  That's entirely an optimization to avoid the
> application needing to immediately need to renegotiate a larger number, or
> to have a large number of never-used channels if the default was set way too
> high.
>
> If streams are needed past the number currently allocated, an increase will
> be requested and the createDataChannel queued.  (Note: this applies to
> pre-negotiated channels using streams past the currently allocated number as
> well).

Actually, I'm not sure where this has been discussed, but there is an
assumption of SDP use that doesn't fit my model for SDP.

That is, when I use SDP, I describe what I can do, i.e., my limits.
That establishes an operating envelope within which I can change
behavior.  When I negotiate 16 streams, I had assumed that this would
be a limit and that I might negotiate fewer than that initially and be
open to attempts to open more.  That is, up to but not exceeding 16.
If this is something that doesn't make sense to negotiate in SDP, then
let's not do that.

As another issue, have we ever discussed the SCTP profile that we are
expected to implement?  Are we expected to have and use every SCTP
feature ever defined or just a specific subset?  Where is the right
place to capture that?  Does this have ramifications for SDP, or shall
we let the SCTP extension mechanisms handle the feature negotiation?

---

From: Randell Jesup <[email-masked]>
Date: Mon, 29 Apr 2013 16:34:01 -0700

> After remaining neutral during most of this discussion about the need for OPEN, I've finally come to the conclusion that Matthew is right.  The only thing the OPEN provides is the label, which is j…

ok, multi-point response:

Please review the proposed JS API & dictionary posted here and the W3 
list a while ago (4 weeks?)

1) Label isn't the only data transmitted in the Open message. Protocol 
(which is particularly important to inter-operating apps, though can 
still be useful for homogenous apps), and also having the channel 
parameters be symmetric.  Channels don't have to be symmetric, but if 
they aren't then the receiver of the initial data would need to know by 
some external method what the reverse channel parameters should be, and 
then do operations to set it to the appropriate transfer mode.

2) labels can specify some easy tags for the channel.  For example, in a 
"Hangout", if the server opens up a private chat channel on behalf of 
another participant, it might do "pc.createDataChannel("Tom", { 
protocol: "application/funkychat"});".    The server might also open one 
with a label "Sarah", etc.  I also envisioned (in this mailing list) an 
FTP-like file transfer setup where the label is the filename being 
transferred.  (Think dragging a directory of files onto your chat 
session to transfer to the other person.)

That said, you can always push anything signaled in Open into an initial 
message in an ordered reliable channel.  When it's NOT ordered and 
reliable, then it gets more complex for the application to get initial 
setup done for simple uses (retries to handle packet loss, 
in-application acks, out-of-order delivery, etc, just to set up the 
channel, or run a separate reliable negotiation channel (more likely), 
which would need to be at least 1RTT to avoid races with trying to use 
the unreliable channel before it's set up, etc.  Yes, applications can 
deal with all this, but some applications will want a simple JS API and 
simple setup.  And it will be tough to get right in a 0-RTT use for 
non-reliable channels.

3) I proposed something similar to your proposal a year ago, and 
discussion and comments from people expecting to use this convinced me 
to adapt the protocol proposal to include label (in particular comments 
from Justin Uberti, but there were others).   Perhaps some of them can 
chime in, as I can only do so well at describing how they intend or hope 
to use label.  The other major piece, protocol, was added at the strong 
request from the people in the room at IETF Atlanta.

Matthew's proposal is effectively little different than Martin's 
proposal from IETF Orlando (or Matthew's assumed proposal, as his 
comments aren't exactly an alternative proposal and some of the comments 
imply a different model with an RTMFP-like initial packet (i.e. 
Open....).  Per the minutes and earlier comments here, the agreement was 
to adopt the Open draft, and not proceed with either of the two 
alternatives given.  I'll leave how to resolve this issue to the chairs, 
and until told otherwise will continue to work on turning the personal 
draft into a WG document.  (And re-iterate my comments from before - I 
think we have a solution here that everyone can work with, even if it's 
not everyone's first choice.)

-- 
Randell Jesup
[email-masked]

---

From: "Ejzak, Richard P (Richard)" <[email-masked]>
Date: Tue, 30 Apr 2013 15:57:22 +0000

> On 4/29/2013 6:34 PM, Randell Jesup wrote:
> > After remaining neutral during most of this discussion about the need
> for OPEN, I've finally come to the conclusion that Matthew is right.
> The only thing the OPEN provides is the label, which is just an opaque
> string with no meaning to the browser or any intermediaries.  It only
> has application significance.  If this is the case, why not just use
> stream id?  What can you possibly do with label that you can't do with
> pre-assigned stream ids (this is a serious question to Randell since it
> is fundamental to whether we need OPEN)?  Since label only has meaning
> when both browsers run the same application, what does it matter if you
> agree on a set of opaque strings to correspond to various functions or
> just agree on a set of stream ids for these functions?  It doesn't
> change the complexity of the code and it doesn't even change the
> readability of the code if you just enumerate the reserved stream ids
> and "label" them in the code.
> 

For homogeneous apps, the choice of channel parameters can as easily be tied to the stream id as the label.   

The SCTP protocol id field should indicate the subprotocol being carried in the data channel, so I don't see the need to separately announce this. 

For non-homogeneous apps, there is no basis to negotiate anything until some standard subprotocols (beyond "raw data") are defined.  In some cases the subprotocol will dictate the channel parameters to be used (or will at least severely constrain the options), so defaults could be assumed in simple cases.  In most more complex cases you will need a real negotiation mechanism (instead of OPEN) since most protocols require negotiation of additional parameters that can't be expressed or negotiated using OPEN (e.g., the t.140 cps).

> 
> 2) labels can specify some easy tags for the channel.  For example, in
> a "Hangout", if the server opens up a private chat channel on behalf of

This might be useful for some homogeneous apps, but the repeated use of the word "might" in your text implies that this is not a necessity and that there are reasonable alternatives.

My real point is just that the justification for OPEN is weak and we could easily do without it (assuming the proper handling of the protocol id).  Please consider this as a friendly proposal to enhance/simplify the current DataChannel proposal that should be considered if there is enough support.

> 
> That said, you can always push anything signaled in Open into an
> initial message in an ordered reliable channel.  When it's NOT ordered
> and reliable, then it gets more complex for the application to get
> initial setup done for simple uses (retries to handle packet loss, in-
> application acks, out-of-order delivery, etc, just to set up the
> chime in, as I can only do so well at describing how they intend or
> hope to use label.  The other major piece, protocol, was added at the
> strong request from the people in the room at IETF Atlanta.
> 
> comments aren't exactly an alternative proposal and some of the
> comments imply a different model with an RTMFP-like initial packet
> (i.e.
> Open....).  Per the minutes and earlier comments here, the agreement
> was to adopt the Open draft, and not proceed with either of the two
> alternatives given.  I'll leave how to resolve this issue to the
> chairs, and until told otherwise will continue to work on turning the
> personal draft into a WG document.  (And re-iterate my comments from
> before - I think we have a solution here that everyone can work with,
> even if it's not everyone's first choice.)
> 
> 
> 
> --
> Randell Jesup
> [email-masked]
> 
>