Subject: Re: [rtcweb] RTCWEB Data Channel: Usage of PPID for protocol multiplexing

From: Christer Holmberg <[email-masked]>
Date: Thu, 6 Feb 2014 07:56:51 +0000

Hi,

> As discussed in the last IETF meeting, this PPID based fragmentation and reassembly will be removed from the ID.

I assume the associated PPID values will also be un-registered with IANA?

Regards,

Christer

---

From: Michael Tuexen <[email-masked]>
Date: Thu, 6 Feb 2014 09:25:35 +0100

> I assume the associated PPID values will also be un-registered with IANA?
I guess they will stay... They are supported by Firefox and PPIDs are a pretty
cheap resource. They are from a pool of 32-bit entities. That is why
we registered the values before the RFCs get published.

Best regards
Michael

---

From: Christer Holmberg <[email-masked]>
Date: Thu, 6 Feb 2014 08:36:01 +0000

Hi,

> I guess they will stay... They are supported by Firefox and PPIDs are a pretty cheap resource. They are from a pool of 32-bit entities. That is why we registered the values before the RFCs get published.

So, does that mean my WebRTC entity has to support them, in case the remote peer is on Firefox?

Also, if they are removed from the data channel draft, what reference will be used in the IANA table?

Regards,

Christer

---

From: Michael Tuexen <[email-masked]>
Date: Thu, 6 Feb 2014 09:45:51 +0100

> So, does that mean my WebRTC entity has to support them, in case the remote peer is on Firefox?
I don't think so. Not sure what the plan for Firefox is, but I expect Firefox to change the
code as soon as the final solution is specified.
> Also, if they are removed from the data channel draft, what reference will be used in the IANA table?
I think it will stay. For PPIDs you even don't have to have an RFC. You can even register PPIDs
as a person.

---

From: "Makaraju, Maridi Raju (Raju)" <[email-masked]>
Date: Thu, 6 Feb 2014 11:45:46 +0000

> > I guess they will stay... They are supported by Firefox and PPIDs are a
> pretty cheap resource. They are from a pool of 32-bit entities. That is why

[Raju] Per my understanding some of the PPIDs are still needed.
The send() API for data channel comes in 4 flavors:
1.    void send (DOMString data);
2.    void send (Blob data);
3.    void send (ArrayBuffer data);
4.    void send (ArrayBufferView data);

1 maps to text data
2,3,4 maps to binary data

When one end sends this data one would expect that the other end gets an onmessage() callback with same (if other end is also javascript) or similar(if other end is a native client) data structure. I believe this is achieved by using PPIDs.
We need at least 2 PPIDs: text and binary. So, existing "DOM String last" (=51) and "Binary Data Last"(=53) can be used.
To facilitate interworking with native clients I don't think there is a need to represent all sub-data types of send() in PPIDs as native client on the other end using C/C++ won't know what "ArrayBufferView" is?!
Also, I think "DOMString" PPID is too specific to Javascript, instead it should probably have a generic name like "UTF-16 String". The send API can still use DOMString as this is Javascript API.

-Raju

---

From: Christer Holmberg <[email-masked]>
Date: Thu, 6 Feb 2014 11:50:41 +0000

Hi,

I agree that PPID values are still needed, and my question was not about removing all of them - only the ones used with application-level fragmentation.

Regards,

Christer

-----Original Message-----
From: Makaraju, Maridi Raju (Raju) [mailto:[email-masked]] 
Sent: 6. helmikuuta 2014 13:46
To: Christer Holmberg; Michael Tuexen
Cc: [email-masked]
Subject: RE: [rtcweb] RTCWEB Data Channel: Usage of PPID for protocol multiplexing

> >>> As discussed in the last IETF meeting, this PPID based 
> >>> fragmentation and
> reassembly will be removed from the ID.
> >>
> pretty cheap resource. They are from a pool of 32-bit entities. That 
> is why we registered the values before the RFCs get published.
> 

1 maps to text data

-Raju

---

From: Paul Kyzivat <[email-masked]>
Date: Thu, 06 Feb 2014 11:26:13 -0500

> I think it will stay. For PPIDs you even don't have to have an RFC. You can even register PPIDs

I hope that if values are retained simply for backward compatibility 
they are explicitly marked as deprecated.

	Thanks,
	Paul

---

From: Michael Tuexen <[email-masked]>
Date: Thu, 6 Feb 2014 20:17:40 +0100

> 2,3,4 maps to binary data
What is still be needed:
WebRTC Control: Used for the DATA_CHANNEL_OPEN and DATA_CHANNEL_ACK message
DOMString Last: Used for 1.
Binary Data Last: Used for 2, 3, 4.

The PPIDs 'Binary Data Partial' and 'DOMString Partial' won't be used anymore.
These were specific to the PPID based fragmentation and reassembly.
> To facilitate interworking with native clients I don't think there is a need to represent all sub-data types of send() in PPIDs as native client on the other end using C/C++ won't know what "ArrayB…
At least there are no PPIDs for them right now.
> Also, I think "DOMString" PPID is too specific to Javascript, instead it should probably have a generic name like "UTF-16 String". The send API can still use DOMString as this is Javascript API.
Any comments from others?

Best regards
Michael

---

From: Michael Tuexen <[email-masked]>
Date: Thu, 6 Feb 2014 20:30:42 +0100

On Feb 6, 2014, at 5:26 PM, Paul Kyzivat <[email-masked]> wrote:

> I hope that if values are retained simply for backward compatibility they are explicitly marked as deprecated.
My plan was just not to defined what they are used for...

Best regards
Michael
> 
> 	Thanks,
> 	Paul
> 
> _______________________________________________
> rtcweb mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rtcweb
>

---

From: Christer Holmberg <[email-masked]>
Date: Thu, 6 Feb 2014 19:35:22 +0000

Hi,

> Any comments from others?

I agree that it would be good to use something more generic (e.g. UTF-16 String), as not all entities using this PPID value will be JavaScript based.

Regards,

Christer

---

From: Christer Holmberg <[email-masked]>
Date: Thu, 6 Feb 2014 19:37:38 +0000

Hi,

> My plan was just not to defined what they are used for...

I would like to have some explanation text. Having registered values without any kind of explanation is not a good approach, in my opinion.

Regards,

Christer

---

From: Michael Tuexen <[email-masked]>
Date: Thu, 6 Feb 2014 21:30:51 +0100

> I would like to have some explanation text. Having registered values without any kind of explanation is not a good approach, in my opinion.
The reason why I registered them is that they are used by Firefox and there was
the possibility that the PPID based fragmentation and reassembly got
standardised. I wanted to avoid the case that Firefox uses some values
and other protocols register these values. That would have been pretty
bad. Since they are a very cheap resource, I just went ahead and "saved"
the values.

Best regards
Michael

---

From: Christer Holmberg <[email-masked]>
Date: Thu, 6 Feb 2014 20:35:13 +0000

Hi,

> bad. Since they are a very cheap resource, I just went ahead and "saved"

That is a all good. But, when people wonder 5 years from now, I don't want them to have to look for this e-mail in oder to get an explanation :)

Regards,

Christer

---

From: Michael Tuexen <[email-masked]>
Date: Thu, 6 Feb 2014 21:47:10 +0100

> That is a all good. But, when people wonder 5 years from now, I don't want them to have to look for this e-mail in oder to get an explanation :)
Understood. But do you think they walk through the IANA registry? I would expect them
to read RFCs and follow the procedures described there.

Best regards
Michael

---

From: Harald Alvestrand <[email-masked]>
Date: Fri, 07 Feb 2014 00:18:07 +0100

> Any comments from others?

Note: The WebSockets protocol defines the transferred strings as UTF-8.
http://tools.ietf.org/html/rfc6455#section-5.6

As far as I can tell, we've always intended to follow that example.

The fact that Javascript implementations currently choose to represent
text strings as UTF-16 at their API is sad, but not an argument for
sending that particular text representation over the wire, or reflecting
the name in the API.

-- 
Surveillance is pervasive. Go Dark.

---

From: Harald Alvestrand <[email-masked]>
Date: Fri, 07 Feb 2014 00:19:55 +0100

> to read RFCs and follow the procedures described there.

Updating the registry to read "Obsolete; was used by <draft name>"
should satisfy all comers.

People do read registries.

---

From: Christer Holmberg <[email-masked]>
Date: Fri, 7 Feb 2014 07:27:47 +0000

>>>> The reason why I registered them is that they are used by Firefox 
>>>> and there was the possibility that the PPID based fragmentation and 
>>>> reassembly got standardised. I wanted to avoid the case that Firefox 
>>>> uses some values and other protocols register these values. That 
>>>> would have been pretty bad. Since they are a very cheap resource, I just went ahead and "saved"
>>>> the values.
>>> That is a all good. But, when people wonder 5 years from now, I don't 
>>> want them to have to look for this e-mail in oder to get an 
>>> explanation :)
>> Understood. But do you think they walk through the IANA registry? I 
>> would expect them to read RFCs and follow the procedures described there.
>

+ 1

Regards,

Christer

---

From: "Makaraju, Maridi Raju (Raju)" <[email-masked]>
Date: Fri, 7 Feb 2014 22:01:44 +0000

> >> Also, I think "DOMString" PPID is too specific to Javascript, instead it
> should probably have a generic name like "UTF-16 String". The send API can
> still use DOMString as this is Javascript API.

[Raju] I agree that using UTF-8 is desired and more appropriate! Then, should the PPID be changed from "DOMString" to "UTF-8"? Javascript based apps have to use some library to do the conversion of DOMString/UTF-16 to UTF-8. Alternatively, browsers can do this conversion under the APIs (send and onmessage) before sending and after receiving UTF-8 PPID data.
Without such conversion webrtc interworking between browsers and native clients will be problematic (basically, will not work).

Another option, which is more flexible, is to define PPIDs for different encodings like "UTF-8", "UTF-16" (or even "base64" for binary to text instead of using "binary" directly); then pass this encoding information to send() and onmessage() calls, which will use these PPIDs. Passing encoding information might be implicit (like Javavscript DOMString) in most languages by the type of the argument to send. This way it is upto the application to deal with the encoding conversions as it see fits. 

The latter approach is best suitable for interworking between clients of similar type (web or native); but it is bit painful (to do conversions) for clients of different types.

I am wondering what kind of API calls native WebRTC API stacks (e.g. Google Chrome's http://www.webrtc.org/webrtc-native-code-package) provide for data channel send/onmessage ()? UTF-16 strings? Or UTF-8?

-Raju

---

From: Martin Thomson <[email-masked]>
Date: Fri, 7 Feb 2014 15:45:24 -0800

I don't think that we need any complication here.  It's a string.

Strings are UTF-8 on the wire.

Strings are UTF-16 (mostly) in JavaScript.

Anything else would generate pain.

---

From: Christer Holmberg <[email-masked]>
Date: Sat, 8 Feb 2014 00:01:14 +0000

+ 1

Regards,

Christer

________________________________________
From: rtcweb [[email-masked]] on behalf of Martin Thomson [[email-masked]]
Sent: Saturday, 08 February 2014 1:45 AM
To: Makaraju, Maridi Raju (Raju)
Cc: [email-masked]
Subject: Re: [rtcweb] PPID, UTF-16 and DOMString (Re: RTCWEB Data Channel: Usage of PPID for protocol multiplexing)

---

From: "Makaraju, Maridi Raju (Raju)" <[email-masked]>
Date: Sat, 8 Feb 2014 01:14:52 +0000

That's good. Glad to see that no one liked the complex alternate option... yet! :-)
I just put it there for completeness and some discussion.
With "utf-8 on wire", which I too think is the right approach, there is some spec work todo:
1. Define IANA utf-8 ppid. Creating a new one is better instead of replacing existing DOMString as DOMString may be used by browsers for sometime to come and also to have backward compatibility/interworking.
2. Data channel core spec (http://tools.ietf.org/html/draft-ietf-rtcweb-data-channel-06#section-6.6) need to specify explictly the wire format is utf-8.

Then browsers need to do the conversion from utf-16 DOMstring to utf-8 before writing on wire also in the reverse direction. 

-Raju

> From: Christer Holmberg [mailto:[email-masked]]
> Sent: Friday, February 07, 2014 6:01 PM
> To: Martin Thomson; Makaraju, Maridi Raju (Raju)
> Cc: [email-masked]
> Subject: RE: [rtcweb] PPID, UTF-16 and DOMString (Re: RTCWEB Data Channel:
> Usage of PPID for protocol multiplexing)
> 
> 
> + 1
> 
> Regards,
> 
> Christer
> 
> > [Raju] I agree that using UTF-8 is desired and more appropriate! Then,
> should the PPID be changed from "DOMString" to "UTF-8"? Javascript based
> apps have to use some library to do the conversion of DOMString/UTF-16 to
> UTF-8. Alternatively, browsers can do this conversion under the APIs (send
> and onmessage) before sending and after receiving UTF-8 PPID data.
> > Without such conversion webrtc interworking between browsers and native
> clients will be problematic (basically, will not work).
> >
> > Another option, which is more flexible, is to define PPIDs for different
> encodings like "UTF-8", "UTF-16" (or even "base64" for binary to text
> instead of using "binary" directly); then pass this encoding information to
> send() and onmessage() calls, which will use these PPIDs. Passing encoding
> information might be implicit (like Javavscript DOMString) in most languages
> by the type of the argument to send. This way it is upto the application to
> deal with the encoding conversions as it see fits.
> >
> > The latter approach is best suitable for interworking between clients of
> similar type (web or native); but it is bit painful (to do conversions) for
> clients of different types.
> >
> > I am wondering what kind of API calls native WebRTC API stacks (e.g.
> Google Chrome's http://www.webrtc.org/webrtc-native-code-package) provide
> for data channel send/onmessage ()? UTF-16 strings? Or UTF-8?
> >
> > -Raju
> >

---

From: Michael Tuexen <[email-masked]>
Date: Sat, 8 Feb 2014 10:16:49 +0100

> 2. Data channel core spec (http://tools.ietf.org/html/draft-ietf-rtcweb-data-channel-06#section-6.6) need to specify explictly the wire format is utf-8.
Anu objections?

If not, I'll put corresponding text into the spec.

Best regards
Michael

---

From: Christer Holmberg <[email-masked]>
Date: Sat, 8 Feb 2014 09:23:12 +0000

Hi,

I guess there is a good reason why the current PPID values have been registered to begin with, because normally any IANA registrations take place prior to RFC publication.

But, I have no problem with defining a new value for utf-8, and keep the current values in the registry - as long as it is clear that they are deprecated, and kept in the registry only for "historical" reasons (according to the suggestion by Harald).

Regards,

Christer

Sent from my Sony Ericsson Xperia arc S

"Makaraju, Maridi Raju (Raju)" <[email-masked]> wrote:

-Raju

---

From: Christer Holmberg <[email-masked]>
Date: Sat, 8 Feb 2014 09:26:03 +0000

Hi,

We could have an informative note somewhere.in the draft, pointing out the format used by JavaScript, if people think it would be useful.

Regards,

Christer

Michael Tuexen <[email-masked]> wrote:

Anu objections?

Best regards
Michael

---

From: Michael Tuexen <[email-masked]>
Date: Sat, 8 Feb 2014 11:00:54 +0100

> We could have an informative note somewhere.in the draft, pointing out the format used by JavaScript, if people think it would be useful.
Hmm. I don't want to repeat what is in
http://heycam.github.io/webidl/#idl-DOMString
I like the idea to specify UTF-8 using a new PPID and deprecate the usage of
the current PPID for DOMString.

Best regards
Michael

---

From: "Makaraju, Maridi Raju (Raju)" <[email-masked]>
Date: Sat, 8 Feb 2014 13:45:05 +0000

> We could have an informative note somewhere.in the draft, pointing out the
> format used by JavaScript, if people think it would be useful.

[Raju] The format used by Javascript is already documented in http://dev.w3.org/2011/webrtc/editor/webrtc.html#idl-def-RTCSessionDescription
via various API calls:
    void send (DOMString data);
    void send (Blob data);
    void send (ArrayBuffer data);
    void send (ArrayBufferView data);

As long as browser+javascript is used by both webrtc clients the receiving gets these formats in onmessage () callback (NOTE: please see exceptions below).
So, what goes/comes on wire is controlled by browser to give the application suitable format. 
If one side is a non-browser/javascript webrtc client then the wire format really matters, for which the Data Channel Core spec will have a note (per discussion and agreement).

The spec is already assuming, though not explictly, onwire UTF-8 format while doing "bufferedAmount" calculations. This is evident from the note:

"bufferedAmount of type unsigned long, readonly
The bufferedAmount attribute must return the number of bytes of application data (UTF-8 text and binary data) that have been queued using send() but that, as of the last time the event loop started executing a task, had not yet been transmitted to the network."

May be an additional explicit note in the above text about "onwire format is UTF-8" makes it very clear!?

Coming back to http://dev.w3.org/2011/webrtc/editor/webrtc.html#idl-def-RTCSessionDescription :
As shown above the API has send() API which takes Blob or ArrayBuffer or ArrayBufferView as arguments. Now since there will only be one "binary" PPID defined, all these maps to it. Assuming browser on the other end, how does it knows which type of data type to give in the onmessage() callback? Is there any expectation on the other end it gets same data type? I guess not! Other end may always default to using "Blob". In either case, we need a note here about the data type to be expected in onmessage(), which may always be "Blob"/binary or "DOMString"/text?!

-Raju

---

From: Christer Holmberg <[email-masked]>
Date: Sat, 8 Feb 2014 16:00:02 +0000

Hi,

Without commenting on specifics: we shall NOT have a mechanism where one side needs to know whether the remote peer is a browswer or not. The data channel mechanism itself shall be generic.

Regards,

Christer

________________________________________
From: Makaraju, Maridi Raju (Raju) [[email-masked]]
Sent: Saturday, 08 February 2014 3:45 PM
To: Christer Holmberg; Michael Tuexen
Cc: Martin Thomson; [email-masked]
Subject: RE: [rtcweb] PPID, UTF-16 and DOMString (Re: RTCWEB Data Channel: Usage of PPID for protocol multiplexing)

-Raju

---

From: "Makaraju, Maridi Raju (Raju)" <[email-masked]>
Date: Sat, 8 Feb 2014 16:38:37 +0000

Hi Christer, et. al.,
I completely agree and that's the intention of having just two PPIDs, one for text and one for binary. However, it would be good to have the clarifications that I mentioned in my earlier reply.
I have a general comment on some of the non-browser/javascript webrtc specs assuming, unnecessarily, browser+javascript as the client, which leads to some confusion like utf-8 vs DOMString confusion. It would be nice if these specs make it generic to apply all webrtc clients: either browser, native or hybrid. In future I will point these places as I review the relevant specs. But it would be great if authors keep this mind.

Regards
-Raju

----- Reply message -----
From: "Christer Holmberg" <[email-masked]>
Date: Sat, Feb 8, 2014 10:00 am
Subject: [rtcweb] PPID, UTF-16 and DOMString (Re: RTCWEB Data Channel: Usage of PPID for protocol multiplexing)
To: "Makaraju, Maridi Raju (Raju)" <[email-masked]>, "Michael Tuexen" <[email-masked]>
Cc: "[email-masked]" <[email-masked]>

Hi,

Regards,

Christer

-Raju

---

From: Randell Jesup <[email-masked]>
Date: Sat, 08 Feb 2014 12:24:24 -0800

> Anything else would generate pain.

(Back online while traveling; been off mostly due to extended power 
outage in Philadelphia suburbs)

100% agree.  And currently that's what we do in Firefox at the 
JS->protocol interface:

void
nsDOMDataChannel::Send(const nsAString& aData, ErrorResult& aRv)
{
   NS_ConvertUTF16toUTF8 msgString(aData);
   Send(nullptr, msgString, msgString.Length(), false, aRv);
}

The JS interface matches WebSockets Send().  The wireline interface is 
(and should be) UTF8; any registrations should be updated to make this 
clear.

-- 
Randell Jesup
[email-masked]

---

From: Michael Tuexen <[email-masked]>
Date: Sat, 8 Feb 2014 21:35:33 +0100

On Feb 8, 2014, at 9:24 PM, Randell Jesup <[email-masked]> wrote:

> (Back online while traveling; been off mostly due to extended power outage in Philadelphia suburbs)
> 
> 100% agree.  And currently that's what we do in Firefox at the JS->protocol interface:
> 
> void
> The JS interface matches WebSockets Send().  The wireline interface is (and should be) UTF8; any registrations should be updated to make this clear.
Great. I'll update the spec. Let's change the name of the PPID...

Best regards
Michael

---

From: Paul Kyzivat <[email-masked]>
Date: Sun, 09 Feb 2014 13:37:57 -0500

> People do read registries.

Yes. In some cases the registry is how you discover what RFC to read.

Maybe five years from now, somebody debugging some app gets one of these 
values, and doesn't know what it means. It won't be mentioned in any of 
the current RFCs he has read, because none of them use it.

So he ends up looking in the registry to find where it is defined.

	Thanks,
	Paul

---

From: Harald Alvestrand <[email-masked]>
Date: Mon, 10 Feb 2014 05:56:52 +0100

> 1. Define IANA utf-8 ppid. Creating a new one is better instead of replacing existing DOMString as DOMString may be used by browsers for sometime to come and also to have backward compatibility/int…

Have you ever seen UTF-16 on the wire in the DataChannel protocol?

> 2. Data channel core spec (http://tools.ietf.org/html/draft-ietf-rtcweb-data-channel-06#section-6.6) need to specify explictly the wire format is utf-8.

Fully agreed.

---

From: "Makaraju, Maridi Raju (Raju)" <[email-masked]>
Date: Mon, 10 Feb 2014 11:08:07 +0000

> > 1. Define IANA utf-8 ppid. Creating a new one is better instead of
> replacing existing DOMString as DOMString may be used by browsers for
> sometime to come and also to have backward compatibility/interworking.
> 

[Raju] I have used websockets and no, I did not see UTF-16 on wire. Further, after going thru websocket API spec http://www.w3.org/TR/2011/WD-websockets-20110419/ , which does have same send(DOMString) API call, I realized that browsers are very well doing this conversion already in other areas. Later, Randell Jesup also comfirmed this for Firefox. So, no concerns on this. Thanks!

-Raju