Subject: [rtcweb] Proposed Plan for Usage of SDP and RTP - Lower level API minus SDP

From: Robin Raymond <[email-masked]>
Date: Wed, 06 Mar 2013 18:45:09 -0500

Do we need SDP "blob" format in the exchange in the first place? All media
can be done without SDP given an intelligent stream API. An API already
exists to create these streams (albeit somewhat lacking if we remove the
SDP 'blob'). This API helps "simplify" creating this blob for later
exchange. But the blob is truly not needed. Each side could in fact create
the desired streams, pass in the appropriate media information such as
codecs and ICE candidates and chose the socket pair to multiplex upon.
Yes, it's a bit more low level but it certainly can be done (and cleanly).

https://datatracker.ietf.org/doc/draft-jennings-rtcweb-plan/

Nothing wrong with the draft in an SDP/SIP mindset but I'm going to take
it from a totally different non-SDP angle. I have to say, the ideas
presented are very good. I appreciate FEC, and synchronizing streams is
cool. But SDP isn¹t needed to do it. Let me as the programmer worry about
how to manage streams and the features on the streams and associations
between the streams via an API only.

Point 4, 5 and 6 in the specification all have to do with the complexities
of having to describe the intentions of mixing in SDP. So no comment
beyond ³don¹t use SDP².

As for 7.1  ³this is because the sender choses the SSRC²  only true
because we are forced to use SDP and the assumptions is that it¹s SIP. We
could have the receiver dictate what the sender should use in advance of
any media. In our case, we establish in advance what we want from all
parties before even ³ringing² the other party. We do not have SSRC
collisions as we reversed the scenario allowing the receiver to pick the
expected SSRC. Coordinating the streams is a problem with SIP because of
how they do forking/conferencing. This specification forces this issue on
those not using SIP. If SIP has problems with streams arriving early to
their stateful offer/answer then let them worry about ³how² they intend to
match the streams at a higher SDP layer and get this draft out of the
RTCWEB track on the SIP track. To be clear, the proposal seems entirely
reasonable and intelligent for SIP/SDP. But it¹s way to SIP centric for
general use.

On that note, what I do need in the API is an ability to dictate the SSRC
when I create an RTP stream for sending (should I care to do that).

7.2 Multiple render

Again this is an issue of SIP/SDP. We can control the SSRCs to split them
out to allow multiplexing easily on the same RTP ports with multiple
parties/sources. If given the primitives to control the streams just, this
specification could be used to dictate how to negotiate issues in their
space.

7.2.1 I¹m feeling the pain. How about just giving me an API where I can
indicate what streams are FEC associated.

7.3 Give me API to give crypto keys to RTP layer. Let me handle the
fingerprint and security myself beyond that.

8. Let's just say politely that I would not want to be the developer
assigned to programming around all this stuff.

Again, a perfect illustration why I don¹t want SDP.

Media is complicated for good reason as there are many untold use cases.
The entire IETF/W3C discussion around video constraints illustrates some
of the complexities and competing desires for just one single media type.
If we tie ourselves to SDP we are limiting ourselves big time, and some of
the cool future stuff will be horribly hampered by it.

My issues with SDP can be summarized as:

- unneeded - much too high level an API
- arcane format - legacy and problematic
- offer/answer
- incompatibilities
- lack of API contact
- doesn't truly solve goal of interoperability with legacy systems (eg.
SIP)

Regret that I did not have time for feedback earlier. As you can tell, I
am not at all happy with where we sit today wrt requiring SDP. IMHO we
need a lower level API if we are going to insist on using SDP.

You can read my entire (long) rant against SDP here
http://blog.webrtc.is/2013/03/06/sdp-the-webrtc-boat-anchor/

---

From: Hadriel Kaplan <[email-masked]>
Date: Thu, 7 Mar 2013 16:50:45 +0000

I knew there was a reason for posting an email for the archive machine to store for eternity... see this:
http://www.ietf.org/mail-archive/web/rtcweb/current/msg01256.html

-hadriel

On Mar 6, 2013, at 6:45 PM, Robin Raymond <[email-masked]> wrote:

> _______________________________________________
> rtcweb mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rtcweb

---

From: Martin Thomson <[email-masked]>
Date: Thu, 7 Mar 2013 09:43:12 -0800

Obviously I (and my employer) agree with these sentiments
wholeheartedly.  Both Robin and Hadriel.

That doesn't change the fact that a number of people are highly
motivated to protect their investment in SDP offer/answer.  For those
people, the pain that causes everyone else is clearly far less
important than the pain they feel at this moment.  So here we are.

---

From: Mary Barnes <[email-masked]>
Date: Thu, 7 Mar 2013 11:56:32 -0600

> important than the pain they feel at this moment.  So here we are.

[MB] I originally thought that either approach could work.  I did see
the value that folks saw in using SDP offer/answer. But after sitting
through the interim meeting last month, I am very much of the mindset
that using SDP O/A is a bad idea.   I think many of us thought that
using the SDP blob would help with interoperability with "legacy" SIP
endpoints.  I don't see that now at all.  I think we will end up with
a very fragile solution that will be very difficult to extend/modify
later if we continue down the SDP O/A path.
[/MB]

---

From: Peter Thatcher <[email-masked]>
Date: Thu, 7 Mar 2013 10:23:48 -0800

Hadriel,

That email is great.  Thank you for preserving it and for bringing it
to our attention.  I am especially sympathetic to the argument that
it's undesirable to have the W3C tied to MMUSIC for all time.

---

From: Peter Thatcher <[email-masked]>
Date: Thu, 7 Mar 2013 10:33:44 -0800

A question for all of you in the "please don't make us use SDP as an
API forever" crowd (and I include myself): Would it be acceptable to
you to have an intermediate step where we keep createOffeer,
setRemoteDescription and setLocalDescription as-is, but allow a JSON
argument?  It seems to be that such a thing could provide all the same
low-level of control as any other setup of methods, but may be much
more likely to be accepted by the group as a whole.  And, it would
still be a lot more pleasant for application developers, and leave
more flexibility for a future where low-level methods might be added.

As both an application developer and a browser implementor, I think a
good SessionDescription JSON format would be easy to implement,
pleasant to use, a small incremental step from what we currently have,
and would relieve the standards body of so much fighting over what the
SDP should look like.

I know it wouldn't be exactly what you're looking for, but I think it
would be achievable and much better than what we have.

If you're interested in such a thing, let me know.  I might be able to
provide something a little more concrete idea as to what such a format
could be.

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 7 Mar 2013 10:42:09 -0800

> later if we continue down the SDP O/A path.
Hasn't the WG already been asked this question not once but
twice.

1. In Taipei:
http://www.ietf.org/proceedings/82/minutes/rtcweb.txt

  Cullen - we need to advise w3c on setting up a media stream.
  - low-level API - browser implementors were not interested.
  - mid-level - browser implements SDP negotiation
  - high-level - browser implements a signal protocol (jingle)

  Hardie - if you agree the state machine should be based on Offer/Answer,
raise
  your hand. Count: 31, 3 in jabber room.

  If you do not agree, raise your hand.
  Count: 4

  Bernard - how can you ask this if you assume ICE, you need Offer/Answer.

  Cullen - could rewrite ICE.

  If there is not enough info, raise your hand
  Count:  5

  Hardie - Rough consensus in the room and jabber that we will base the
state machine on SDP OA.

2. When we picked JSEP, which has SDP at its heart.

And that's just the IETF WG. W3C *also* had a consensus call on
exactly this point:
http://lists.w3.org/Archives/Public/public-webrtc/2012Sep/0098.html.

The consensus was for "Alternative 1":

  1) Continue with a design based on the PeerConnection object, using SDP
  as part of the API, roughly in the style of the current API description.

This happened less than 6 months ago and it wasn't a close thing.

Moreover, there are two shipping--and indeed
interopable!--implementations (Chrome and Firefox) based on the
PeerConnection/3264 OA design style (I'll get to Peter's comment about
JSON in a separate message).

Now, none of this is to say that the WG can't reconsider and come to a
new decision, but I think that at given the number of times this point
has been argued and reargued, the bar to that reconsideration has to
be fairly high (and needs to be jointly taken in IETF and W3C)
especially since this essentially amounts to a request to burn the
entire W3C API and start over.

-Ekr

---

From: Roman Shpount <[email-masked]>
Date: Thu, 7 Mar 2013 13:44:26 -0500

I very strongly agree. I think having an API independent from SDP will
allow the group to concentrate on the actual features instead of arguing
about the useless blob syntax. I was initially proponent of Offer/Answer
but once I started using the actual API, I saw offer/answer presenting
nothing but problems. I actually see current WebRTC SDP offer/answer
implementation as a hindrance to SIP interop. It provides a much more
coarse API then typically present in SIP client. That in turns forces
WebRTC implementer to make decision for a developer that create interop
problems (like no way to switch send codecs without changing transport
parameters).
_____________
Roman Shpount

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 7 Mar 2013 10:45:58 -0800

> would be achievable and much better than what we have.

II don't understand how this changes anything meaningful. The point
of using SDP isn't the crufty line-based encoding, it's being committed
to the SDP offer/answer semantics. So, when you talk about having
JSON, you're talking about one of two things:

1. Having a format which is semantically equivalent to SDP.
2. Having a format which is semantically distinct (or perhaps a superset)
of SDP.

In case 1, I don't see what this bought you, other than programming
convenience. It's not like it would be difficult to build code to
mechanically
dissect the SDP encoding.

In case 2, you have forked SDP, and largely obviated the point of
using SDP in the first place.

-Ekr

---

From: Mary Barnes <[email-masked]>
Date: Thu, 7 Mar 2013 12:51:00 -0600

> Hasn't the WG already been asked this question not once but
[MB] Yes.  And, some of us have changed our positions based upon the
challenges that the group is facing in getting the current approach
specified and agreed.  I don't disagree that it is not a good thing
that this is being discussed yet again.  [/MB]

---

From: Erik Lagerway <[email-masked]>
Date: Thu, 7 Mar 2013 11:43:43 -0800

+1

We feel that a lower level API that does not include SDP, and a layer built
on top (perhaps in a JS library) to manage SDP + offer/answer would solve
this issue for everyone. That would ensure legacy compatibility and give us
future stuff we would know we are going to need.

http://lists.w3.org/Archives/Public/public-webrtc/2012Feb/0198.html <- btw,
this went nowhere in the W3C so it seems the buck stops here

/erik

*Erik Lagerway <http://ca.linkedin.com/in/lagerway> |
*Hookflash<http://hookflash.com>
* | 1 (855) Hookflash ext. 2 | Twitter <http://twitter.com/elagerway> | WebRTC
Blog <http://webrtc.is> *
****

---

From: Peter Thatcher <[email-masked]>
Date: Thu, 7 Mar 2013 12:56:05 -0800

Having a JSON-based SessionDescription that's a 1:1 mapping to SDP
semantically seems like a bad idea to me.  We want something better
than SDP, not just a different syntax for SDP.  Trying to map all of
SDP would be a near-impossible task anyway.

I think you're combining two separate questions:
a.  Should the Browser<->JS API have SessionDescriptions at all, or
should it have just low-level methods?
b.  Should the SessionDescription be representable as SDP or as
something else or both?

So far, we have two camps that combine the two questions:
Pro-SDP camp: methods=createOffer/setLocal/setRemote   and  format=SDP
Anti-SDP camp: methods=low-level methods   and   format=none

I'm suggesting that there might value in a third possibility:
Compromise: methods=createOffer/setLocal/setRemote  and  format=JSON

The Pro-SDP camp will tell the Anti-SDP camp, "don't blow up the
world".  The Anti-SDP camp will tell the Pro-SDP camp, "SDP is a
horrible API surface".  I think they are both right.   And I hope that
an alternative format for a SessionDescription would be a sufficiently
good compromise.

I don't think of it as a fork of SDP, but rather as a much better way
to describe sessions than SDP.  I've personally worked with 6
different kinds of session descriptions (SDP, Jingle, libjingle
internals, and a few different Google internal representations), and
SDP is by far the worst, not just in syntax but in semantics also.  I
think a JSON one could be much better syntax and semantics, without
quite "blowing up the world".

At least, that's my hope.

---

From: Silvia Pfeiffer <[email-masked]>
Date: Fri, 8 Mar 2013 08:09:48 +1100

> that this is being discussed yet again.  [/MB]

[Gotta love the triple negation!]

Why can't we have it both ways?

Maintain the current way to get the raw SDP using createOffer, but then
provide an interface to change that offer before setLocalDescription.

Even CISCO provides such an API:
http://www.cisco.com/en/US/docs/voice_ip_comm/cucm/sip_tn/8_5_1/4-sdp_api.html
(I think we can do a better one than this, but it's a reference point).

Cheers,
Silvia.

---

From: Peter Thatcher <[email-masked]>
Date: Thu, 7 Mar 2013 13:12:08 -0800

You can already do that by "munging" the SDP.  It's just not very
pleasant to do.

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Thu, 7 Mar 2013 22:14:48 +0100

> SDP would be a near-impossible task anyway.

An SDP is so "flexible" that it can include:

- multiple m video lines.
- with multiple SSRC each one.
- may be bundle for audio and video.
- may be separate ICE candidates for audio and each different "m" video.
- and so on. Too much "flexibility".

It's more than difficut to expect that a JS custom code can consider
all those possibilities. That's IMHO the main problem of SDP and not
so much the SDP format itself (which can be managed via a good JS
API).

And the worst of SDP IMGO: having to send the *entire* SDP when any
minimal change is done, so the receive must do magic to realize of the
exact change without "resetting" all.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Thu, 7 Mar 2013 22:17:10 +0100

> Anti-SDP camp: methods=low-level methods   and   format=none

What does "format=none" mean? There should be some kind of "format" to
tell the remote peer our video codec preferences, ICE candidates and
so, am I right? This is not about passing a URL to the remote.

That's exactly what SDP attempts to achieve but SDP is too flexible
and unmanageable.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Peter Thatcher <[email-masked]>
Date: Thu, 7 Mar 2013 13:17:15 -0800

That's a very good point.  I think any replacement SessionDescription
ought to have a clear "update" mechanism built in.

---

From: Silvia Pfeiffer <[email-masked]>
Date: Fri, 8 Mar 2013 08:22:42 +1100

Agreed, but it's also not sufficient. SDP is not "programmer friendly"
enough because it has too many details that are protocol-details only and
it's too hard to see the semantic bits in SDP and ignore the rest.

For example: the programmer wants to say - I want to get this video
resolution, this audio bitrate & channels, I want to use this camera and
this microphone for this call. Having to manipulate SDP directly for this
is a programmer's nightmare.

Silvia.

On Fri, Mar 8, 2013 at 8:12 AM, Peter Thatcher <[email-masked]> wrote:

> >> > On Thu, Mar 7, 2013 at 9:56 AM, Mary Barnes <
> [email-masked]>
> >> > wrote:
> >> >>

---

From: Peter Thatcher <[email-masked]>
Date: Thu, 7 Mar 2013 13:25:40 -0800

So, would you like a SessionDescription that is easy to read and
manipulate, such as a JSON object?

On Thu, Mar 7, 2013 at 1:22 PM, Silvia Pfeiffer
<[email-masked]> wrote:
> this microphone for this call. Having to manipulate SDP directly for this is
> a programmer's nightmare.
>
> Silvia.
>
>
>> >> > On Thu, Mar 7, 2013 at 9:56 AM, Mary Barnes
>> >> > <[email-masked]>
>> >> > wrote:
>> >> >>

---

From: Peter Thatcher <[email-masked]>
Date: Thu, 7 Mar 2013 13:26:33 -0800

So, would you like a SessionDescription to send to the other side that
is flexible and manageable, such as a JSON object?

---

From: Silvia Pfeiffer <[email-masked]>
Date: Fri, 8 Mar 2013 08:30:52 +1100

If your suggestion is only to convert the SDP to JSON, then that's a first
step, but not the full story, because the complexity is just represented in
a different format. I'm looking for something that helps programmers
achieve their goals more easily.

JSON is a first step, because it's a more readable data format. However, it
doesn't help me, to e.g. list the cameras on offer and remove one, or
change the offered video resolution. That needs manipulation functions.

On Fri, Mar 8, 2013 at 8:25 AM, Peter Thatcher <[email-masked]> wrote:

> >> > On Fri, Mar 8, 2013 at 5:51 AM, Mary Barnes <

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Thu, 7 Mar 2013 22:33:18 +0100

> is flexible and manageable, such as a JSON object?

If I have to receive the *entire* JSON / rawSDP /
whateverSessionDescription again with minimal changes just for
enabling/disabling video, I don't like it since it means that I must
"compare" the current SD with the new one.

I don't care whether the format is raw SDP, JSON or whatever. What I
would like is that I can establish an audio/video session with a peer
and, at some time, I can send a *minimal* SD "command" / "RPC" to say
the remote "Hi I am dissabling the video right now".

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Peter Thatcher <[email-masked]>
Date: Thu, 7 Mar 2013 13:33:40 -0800

A JSON format doesn't have to be "SDP in a different format".  It can
be much better.  It can expose an easy way to change the video
resolution to send, or a way to add/remove tracks easily.

Listing cameras, however, is a whole different topic.

On Thu, Mar 7, 2013 at 1:30 PM, Silvia Pfeiffer
<[email-masked]> wrote:
> a different format. I'm looking for something that helps programmers achieve
> their goals more easily.
>
> doesn't help me, to e.g. list the cameras on offer and remove one, or change
> the offered video resolution. That needs manipulation functions.
>
>
>
>> >> > On Fri, Mar 8, 2013 at 5:51 AM, Mary Barnes

---

From: Peter Thatcher <[email-masked]>
Date: Thu, 7 Mar 2013 13:37:15 -0800

Having a minimal SessionDescription command and then a
"SessionDescription update message" that says "I'm adding new video
now" or "I'm disabling the video
right now" absolutely can and should be a part of a better
SessionDescription format.  I think that's a good idea and very
doable.

---

From: Roman Shpount <[email-masked]>
Date: Thu, 7 Mar 2013 16:46:38 -0500

I do not think redefining session description is the best approach. Having
access to direct API methods that control send codecs, send media types,
resolution, etc  is easier to understand and to program. I think even
simply exposing the API from Voice and Video engines in current WebRTC
implementation would be a more usable API then the current offer/answer SDP
based one.
_____________
Roman Shpount

---

From: Matthew Kaufman <[email-masked]>
Date: Thu, 07 Mar 2013 13:48:44 -0800

On 3/7/2013 1:46 PM, Roman Shpount wrote:
> I do not think redefining session description is the best approach. 
> Having access to direct API methods that control send codecs, send 
> media types, resolution, etc  is easier to understand and to program. 
> I think even simply exposing the API from Voice and Video engines in 
> current WebRTC implementation would be a more usable API then the 
> current offer/answer SDP based one.

There is an existing proposal published by my employer which is along 
these lines. It might be helpful if the folks who are just now realizing 
what the issues are with dealing with both the complexity of the format 
of SDP as well as the embedded offer-answer semantics could review that 
and see how closely it matches their needs, rather than starting from a 
clean sheet of paper.

Matthew Kaufman

---

From: Peter Thatcher <[email-masked]>
Date: Thu, 7 Mar 2013 13:58:11 -0800

To help keep track of all the needs mentioned, I made a Google Doc
that tracks them:

https://docs.google.com/document/d/1hXxZoLFMmypWlkQR8LmbYYD_uZnQS0FWaYItSDB9v-s/edit

If you're interested in a better SessionDescription that offers
low-level control in a readable and programmable way, without quite
"blowing up the world" with major changes to the API, please take a
look and leave comments.  I'll try to keep it up to date with what's
on the mailing list, but it's hard to keep track of everything :).

On Thu, Mar 7, 2013 at 1:46 PM, Roman Shpount <[email-masked]> wrote:
> resolution, etc  is easier to understand and to program. I think even simply
> exposing the API from Voice and Video engines in current WebRTC

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Thu, 7 Mar 2013 23:09:41 +0100

> on the mailing list, but it's hard to keep track of everything :).

Hi, I've made the following proposal in ~10 minutes, so probably it
lacks some requried features, but I hope the idea behind it is clear
enough:

http://www.gitpaste.com/paste/1423/

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Silvia Pfeiffer <[email-masked]>
Date: Fri, 8 Mar 2013 10:01:56 +1100

On Fri, Mar 8, 2013 at 8:48 AM, Matthew Kaufman <[email-masked]> wrote:

>> Having access to direct API methods that control send codecs, send media
>> types, resolution, etc  is easier to understand and to program. I think
>> even simply exposing the API from Voice and Video engines in current WebRTC
> what the issues are with dealing with both the complexity of the format of
> SDP as well as the embedded offer-answer semantics could review that and
> see how closely it matches their needs, rather than starting from a clean
> sheet of paper.
>

I assume you're referring to:
http://www.w3.org/TR/2012/WD-webrtc-20120821/#rtcsdptype  ?

I like some of the approaches taken in this proposal.
I like that there are actual objects for RTCSdpType and
RTCSessionDescription .
There are, however, no functions to manipulate a SDP yet.

I assume that the API discussion has to happen in the W3C and not here? To
be honest, I am not overly fussed as to what format the browser deals with
- SDP or JSON - though the latter is a more natural browser format. I do,
however, care what the Web developer has to deal with.

Cheers,
Silvia.

---

From: Matthew Kaufman <[email-masked]>
Date: Thu, 07 Mar 2013 15:10:22 -0800

On 3/7/2013 3:01 PM, Silvia Pfeiffer wrote:
> On Fri, Mar 8, 2013 at 8:48 AM, Matthew Kaufman <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>         I do not think redefining session description is the best
>         approach. Having access to direct API methods that control
>         send codecs, send media types, resolution, etc  is easier to
>         understand and to program. I think even simply exposing the
>         API from Voice and Video engines in current WebRTC
>         implementation would be a more usable API then the current
>         offer/answer SDP based one.
>
>
>     There is an existing proposal published by my employer which is
>     along these lines. It might be helpful if the folks who are just
>     now realizing what the issues are with dealing with both the
>     complexity of the format of SDP as well as the embedded
>     offer-answer semantics could review that and see how closely it
>     matches their needs, rather than starting from a clean sheet of paper.
>
>

No, I was referring to 
http://html5labs.interoperabilitybridges.com/cu-rtc-web/cu-rtc-web.htm

> I assume that the API discussion has to happen in the W3C and not 
> here? To be honest, I am not overly fussed as to what format the 
> browser deals with - SDP or JSON - though the latter is a more natural 
> browser format. I do, however, care what the Web developer has to deal 
> with.

The web developer currently is stuck with SDP *and* the SDP offer/answer 
semantics *plus* whatever gets added in MMUSIC to SDP in order to 
actually do everything that's needed (bundle, stream identification, 
etc.)... assuming that the changes to SDP can be made while keeping them 
compatible with everything else that already uses SDP.

Matthew Kaufman

---

From: Silvia Pfeiffer <[email-masked]>
Date: Fri, 8 Mar 2013 10:52:45 +1100

On Fri, Mar 8, 2013 at 10:10 AM, Matthew Kaufman <[email-masked]> wrote:

> On Fri, Mar 8, 2013 at 8:48 AM, Matthew Kaufman <[email-masked]>wrote:
>

I like many features of this proposal and I would love to see it a good
part of it integrated with the existing API.

> semantics *plus* whatever gets added in MMUSIC to SDP in order to actually
> do everything that's needed (bundle, stream identification, etc.)...
> assuming that the changes to SDP can be made while keeping them compatible
> with everything else that already uses SDP.
>

So, in your opinion what should change in the SDP offer/answer semantic to
allow for a better API for Web developers along the lines of the above
linked proposal?

Silvia.

---

From: "Timothy B. Terriberry" <[email-masked]>
Date: Thu, 07 Mar 2013 16:28:56 -0800

Silvia Pfeiffer wrote:
> this microphone for this call. Having to manipulate SDP directly for
> this is a programmer's nightmare.

I would like to point out that the currently proposed W3C APIs do not 
require (or even allow) SDP manipulation for _any_ of this, with the 
possible exception of audio bitrate (which, as we've long discussed, 
should actually be adapted in real-time based on available bandwidth as 
determined by congestion control, with some app-provided way to set 
priorities that will _not_ be based on SDP... the number in SDP merely 
defines the limits of what's possible).

---

From: Silvia Pfeiffer <[email-masked]>
Date: Fri, 8 Mar 2013 13:56:16 +1100

On Fri, Mar 8, 2013 at 11:28 AM, Timothy B. Terriberry <
[email-masked]> wrote:

> possible exception of audio bitrate (which, as we've long discussed, should
> actually be adapted in real-time based on available bandwidth as determined
> by congestion control, with some app-provided way to set priorities that
> will _not_ be based on SDP... the number in SDP merely defines the limits
> of what's possible).

Is there any way for the Web developer to influence the negotiation of,
say, the video resolution? For example, if the video is displayed in a
160x120 video element, then it makes no sense to receive anything larger
than that resolution. I was under the impression that this would have to be
done through manipulating SDP?

Silvia.

---

From: "Timothy B. Terriberry" <[email-masked]>
Date: Thu, 07 Mar 2013 19:05:13 -0800

Silvia Pfeiffer wrote:
> than that resolution. I was under the impression that this would have to
> be done through manipulating SDP?

Again, that would be done via constraints:
http://tools.ietf.org/html/draft-alvestrand-constraints-resolution-00#section-4

See Section 3.1 for examples involving gUM... the draft doesn't have any 
examples, but these same constraints could be applied to incoming 
streams using the settings API currently under discussion in the W3C. 
It's perhaps an open question whether or not a <video> tag should 
propagate back such constraints in the absence of any explicitly set 
ones... but that's a question for the W3C.

---

From: Harald Alvestrand <[email-masked]>
Date: Fri, 08 Mar 2013 07:01:20 +0100

On 03/08/2013 03:56 AM, Silvia Pfeiffer wrote:
> On Fri, Mar 8, 2013 at 11:28 AM, Timothy B. Terriberry 
> <[email-masked] <mailto:[email-masked]>> wrote:
>
>     Silvia Pfeiffer wrote:
>
>     I would like to point out that the currently proposed W3C APIs do
>     not require (or even allow) SDP manipulation for _any_ of this,
>     with the possible exception of audio bitrate (which, as we've long
>     discussed, should actually be adapted in real-time based on
>     available bandwidth as determined by congestion control, with some
>     app-provided way to set priorities that will _not_ be based on
>     SDP... the number in SDP merely defines the limits of what's
>     possible).
>
>
> Is there any way for the Web developer to influence the negotiation 
> of, say, the video resolution? For example, if the video is displayed 
> in a 160x120 video element, then it makes no sense to receive anything 
> larger than that resolution. I was under the impression that this 
> would have to be done through manipulating SDP?

See draft-alvestrand-constraints-resolution for 3 ways of doing this, 2 
of which have no SDP manipulation.

I think SDP manipulation is the messiest of the 3; the first one (out of 
band messaging) can be used on Chrome today when creating video streams. 
(We won't implement changing of constraints until we have an agreed-upon 
API; there are enough other things that need doing that we can wait 
until there's text in the spec before starting to implement it.)

---

From: Stefan Håkansson LK <[email-masked]>
Date: Fri, 8 Mar 2013 10:28:44 +0100

> Yes, it's a bit more low level but it certainly can be done (and cleanly).

I think we are mixing up two different things in the discussion. One is 
"what info to the two endpoints exchange to be able to set up 
connection(s) and send and receive audio, video and data" and the other 
is "what APIs should we use to control the settings".

For the first part, the assumption has so far been SDP. I think we've 
all realized now that the legacy of SDP gives us quite a bit of 
headache. I think that splitting up signaling about connections (ICE 
candidates) from signaling for media stuff (codecs, how different ssrc's 
correlate to MediaStreamTrack's, ...) would simplify things a lot. But 
on the other hand, it seems to me that we're really close to nailing 
those details. I'm pretty sure that switching to some other format would 
cost more in time than we gain.

Also, remember that those session descriptions come with a flag saying 
"sdp". This is intentional, it gives us the freedom to change format 
without affecting the application in the future (as long as the 
application can follow the normal createOffer/setLocal etc. procedures).

When it comes to the API, that discussion is really for the W3C webrtc 
WG. But as has been said by Tim and Harald already, the session 
descriptions should never be changed in normal cases - there should be 
(and are) other APIs for setting up how a MediaStreamTrack is 
transported over the network, resolutions, etc. What is set in those 
APIs will be reflected in the SDP created by "createOffer".

Stefan

---

From: Peter Thatcher <[email-masked]>
Date: Fri, 8 Mar 2013 05:50:17 -0800

One the question of "what info do the two endpoints exchange to be
able to set up connection(s) and send and receive audio, video and
data", my  assumption has *not* been SDP, but rather that it has been
left to the javascript application to decide what info to exhange.
I've heard over and over at WebRTC meeting and conferences that
"signalling is not defined; that's up to the javascript application".
I completely agree with that and think we should stick to it.  In
Google Talk/Hangouts, for example, we use Jingle, not SDP.  SDP only
serves as a javascript API surface, not as a format to exchange with
the remote side.

As an application developer I do not want to exchange SDP.  I only use
it as an API surface.  And it looks like I'm not the only application
developer that sees it this way:
- http://s.phono.com/releases/0.6/jquery.phono.js
- https://github.com/lukeweber/webrtc-jingle-client
- https://code.google.com/p/openfire-candy/source/browse/trunk/plugin/candy/plugins/voicebridge/webrtc-jingle.js
- https://github.com/mweibel/sdpToJingle

Again, for me, and many others it appears, SDP only serves as a
javascript API surface.

On Fri, Mar 8, 2013 at 1:28 AM, Stefan Håkansson LK
<[email-masked]> wrote:
> "what info to the two endpoints exchange to be able to set up connection(s)
> and send and receive audio, video and data" and the other is "what APIs
> should we use to control the settings".
>
> For the first part, the assumption has so far been SDP. I think we've all
> realized now that the legacy of SDP gives us quite a bit of headache. I
> think that splitting up signaling about connections (ICE candidates) from
> signaling for media stuff (codecs, how different ssrc's correlate to
> MediaStreamTrack's, ...) would simplify things a lot. But on the other hand,
> it seems to me that we're really close to nailing those details. I'm pretty
> sure that switching to some other format would cost more in time than we
> gain.
>
> "sdp". This is intentional, it gives us the freedom to change format without
> affecting the application in the future (as long as the application can
> follow the normal createOffer/setLocal etc. procedures).
>
> When it comes to the API, that discussion is really for the W3C webrtc WG.
> But as has been said by Tim and Harald already, the session descriptions
> should never be changed in normal cases - there should be (and are) other
> APIs for setting up how a MediaStreamTrack is transported over the network,
> resolutions, etc. What is set in those APIs will be reflected in the SDP
> created by "createOffer".
>
> Stefan
>
>
>>

---

From: Peter Thatcher <[email-masked]>
Date: Fri, 8 Mar 2013 06:01:45 -0800

Stefan, that is a very good point about the W3C.  Should Robin share
his input with the W3C WG rather than this one?

---

From: Stefan Håkansson LK <[email-masked]>
Date: Fri, 8 Mar 2013 15:04:40 +0100

> "signalling is not defined; that's up to the javascript application".

I agree to the above - signaling is entirely up to the application. I'm 
sorry if my message gave another impression. But we need to define the 
format of the "blobs" (if we call them that) that browser A produces so 
that they can be plumbed into browser B and things work. So far the 
assumption has been that those blobs are SDP descriptions, and my point 
is that changing that to something else may cost us more than we gain.

---

From: Stefan Håkansson LK <[email-masked]>
Date: Fri, 8 Mar 2013 15:05:49 +0100

> his input with the W3C WG rather than this one?

He should at least send his input there as well.

Stefan

---

From: Peter Thatcher <[email-masked]>
Date: Fri, 8 Mar 2013 06:07:44 -0800

Is the blob format defined by the IETF or the W3C?  If the W3C decided
not to use SDP as the blob, or not to use blobs at all, what would
that mean for the IETF?

On Fri, Mar 8, 2013 at 6:04 AM, Stefan Håkansson LK
<[email-masked]> wrote:
> format of the "blobs" (if we call them that) that browser A produces so that
> they can be plumbed into browser B and things work. So far the assumption
> has been that those blobs are SDP descriptions, and my point is that
> changing that to something else may cost us more than we gain.
>
>
>> -
>> https://code.google.com/p/openfire-candy/source/browse/trunk/plugin/candy/plugins/voicebridge/webrtc-jingle.js
>>> connection(s)
>>> without
>>> WG.

---

From: Stefan Håkansson LK <[email-masked]>
Date: Fri, 8 Mar 2013 15:11:22 +0100

On 2013-03-08 15:07, Peter Thatcher wrote:
> Is the blob format defined by the IETF or the W3C?

It is defined by the IETF currently.

> If the W3C decided

That is a good question, and something that could be discussed if the 
IETF fail to make progress on resolving the issues around the SDP blob 
format (in my personal opinion).

---

From: Peter Thatcher <[email-masked]>
Date: Fri, 8 Mar 2013 06:23:17 -0800

> It is defined by the IETF currently.

But that's only because the W3C chose SDP, which is defined by the
IETF.  The W3C has effectively delegated its API surface to the IETF.
Which blob format to use in the first place (SDP, something else, or
none at all) is still, ultimately, a W3C decision.  Right?

So, should the IETF WG's standing response to Robin and future
developers like him (and I'm sure the same thing will come up many
times for  many years) be "we don't talk about non-SDP around here" or
 should it be "that's a W3C issue.  Please share your input with their
working group"?  The latter seems to me much more friendly.

---

From: "Matthew Kaufman (SKYPE)" <[email-masked]>
Date: Fri, 8 Mar 2013 14:55:26 +0000

Stefan Håkansson LK:
>
> I agree to the above - signaling is entirely up to the application. I'm sorry if my
> message gave another impression. But we need to define the format of the
> "blobs" (if we call them that) that browser A produces so that they can be
> plumbed into browser B and things work. 

No we don't. In fact that is exactly what we should mean when we say "signaling is entirely up to the application".

There is no reason why Browser A must produce something a serialized blob that can be stuffed directly into an API at Browser B. And in fact, doing so means that your two browsers are very likely doing things you don't expect... because instead of your JavaScript application saying "do this" to each, it is saying "do whatever this blob that was generated by code I don't control *means to you* (even if that meaning is subtly different than what it meant to the sender)". And then if you want to influence that you have either the pointy sticks called constraints or editing of the blob as your only remaining avenues of control.

Worse, because this now carries the explicit assumption that the two browsers are negotiating directly between each other's code (and really *not* expecting anything in the middle to mess with those blobs), you need a whole set of semantics on top of this to ensure that the browsers converge on something that works without their JavaScript app helping them. That brings in RFC3264-style Offer/Answer (which is not the only way to do it, but the one that was chosen). This then means that if you instead are trying to use the "blob" interface for direct manipulation, you are fighting against what may be an entirely incompatible state machine inside the browser.

> So far the assumption has been
> that those blobs are SDP descriptions, and my point is that changing that to
> something else may cost us more than we gain.

We need to provide the browser with an *API*. And that API should allow the JavaScript developer to directly influence what is happening. Preferably without the browser fighting its developer by trying to run the offer-answer state machine itself. Getting hung up on being able to pass blobs back and forth is now and has always been the whole problem.

Matthew Kaufman

---

From: "Matthew Kaufman (SKYPE)" <[email-masked]>
Date: Fri, 8 Mar 2013 14:57:30 +0000

Peter Thatcher:
> 
> Is the blob format defined by the IETF or the W3C?  If the W3C decided not to
> use SDP as the blob, or not to use blobs at all, what would that mean for the
> IETF?

The W3C has apparently deferred to the IETF for everything, including what codecs should be mandatory and what the JavaScript API should look like.

Why this is the case, I don't know, but eventually there will be a W3C specification and it will contain a bunch of references to other specs and those either will or will not be acceptable to the W3C WG as a complete API.

Matthew Kaufman

---

From: Harald Alvestrand <[email-masked]>
Date: Fri, 08 Mar 2013 22:25:13 +0100

> The W3C has apparently deferred to the IETF for everything, including what codecs should be mandatory and what the JavaScript API should look like.
The charters of the two WGs are interesting reading. I recommend reading 
them to anyone who wishes to know what the division of responsibilities 
was intended to be.

https://tools.ietf.org/wg/rtcweb/charters

" This work will be done in collaboration with the W3C. The IETF WG will 
produce architecture and requirements for selection and profiling of the 
on the wire protocols. The architecture needs to be coordinated with 
W3C. The IETF WG work will identify state information and events that 
need to be exposed in the APIs as input to W3C. The W3C will be 
responsible for defining APIs to ensure that application developers can 
control the components."

http://www.w3.org/2011/04/webrtc-charter.html

" While the specified API Functions will not constrain implementations 
into supporting a specific profile, they will be compatible with the 
Profile that will be specified by the RTCWeb Working Group"

The last-quoted paragraph was inserted after a discussion that concluded 
that only one organization should have the mandatory-to-implement codec 
discussion, and that the IETF should be the one place.

---

From: Matthew Kaufman <[email-masked]>
Date: Fri, 08 Mar 2013 13:29:56 -0800

On 3/8/2013 1:25 PM, Harald Alvestrand wrote:
> The W3C will be responsible for defining APIs to ensure that 
> application developers can control the components."

Yeah, that's the part that the W3C pretty much pointed at the JSEP draft 
and said "whatever IETF does"... and how you get a blob of SDP and an 
O/A state machine fighting back at you as "the JavaScript API".

Matthew Kaufman

---

From: Justin Uberti <[email-masked]>
Date: Fri, 8 Mar 2013 14:56:50 -0800

I sympathize with the concerns originally posted by Robin, and I am glad to
see this kind of application developer feedback in this forum.

The current model has many compromises, to be sure. But in this WG, we have
been able to resolve many tough decisions, and get to a point where we have
multiple interoperating implementations, spanning desktop and mobile, which
means we must be doing at least something right. So, with apologies to the
Beatles, when you talk about destruction, don't you know that you can count
me out.

Rather, as I look at Robin's comments, which are directed toward
https://datatracker.ietf.org/doc/draft-jennings-rtcweb-plan/, I see them as
an indication that "Plan A" ties us more deeply to SDP and legacy
behaviors. He wants the ability to make his own decisions about how these
things should be handled, including setting the SSRC for a stream, setting
FEC, crypto keys, multiplexing, etc. This is more akin to the "Plan B"
approach, which Chrome currently implements (and in which these things are
possible, albeit through SDP).

I certainly agree that SDP is a terrible thing to have to work with from
within a Javascript application, and as Peter has mentioned, there are a
number of ways in which we could improve that. But that is a question about
the API surface, and we can (and should) deal with this separately in the
W3C.

---

From: Harald Alvestrand <[email-masked]>
Date: Sat, 09 Mar 2013 17:04:40 +0100

As usual, I'm trying to use subject line change in order to achieve some 
separation of concerns...

On 03/07/2013 10:22 PM, Silvia Pfeiffer wrote:
> enough because it has too many details that are protocol-details only 
> and it's too hard to see the semantic bits in SDP and ignore the rest.
>
> resolution, this audio bitrate & channels, I want to use this camera 
> and this microphone for this call. Having to manipulate SDP directly 
> for this is a programmer's nightmare.

I think we've been over exactly those pieces, and our current proposed 
solution is called the Media Stream API and the constraints mechanism - 
and they have exactly nothing to do with SDP, or even with PeerConnection.

I don't think we've got it to be "unproblematic" yet, but also, I don't 
think SDP, JSON or even the offer-answer model is either the problem or 
the solution on this set of functionalities.

Or did I misunderstand something basic?

                Harald

---

From: Peter Thatcher <[email-masked]>
Date: Sat, 9 Mar 2013 21:02:11 -0800

There's a difference between the resolution that you open the camera at and
the resolution you send over the network at.  Does the current constraints
API let you capture at one resolution and send at another?  Also, does it
let you change the send resolution on the fly?

These are important controls that, as far as I know, are not currently
supplied to the application.  They could potentially be supplied by SDP,
new methods, or as you suggest, perhaps even as some kind of constraint.
But I don't think they are currently provided.
On Mar 9, 2013 7:54 PM, "Harald Alvestrand" <[email-masked]> wrote:

> solution is called the Media Stream API and the constraints mechanism - and
> they have exactly nothing to do with SDP, or even with PeerConnection.
>
> think SDP, JSON or even the offer-answer model is either the problem or the
> solution on this set of functionalities.
>
> ______________________________**_________________
> rtcweb mailing list
> [email-masked]
> https://www.ietf.org/mailman/**listinfo/rtcweb<https://www.ietf.org/mailman/listinfo/rtcweb>
>

---

From: Stefan Håkansson LK <[email-masked]>
Date: Sun, 10 Mar 2013 08:01:13 +0100

On 2013-03-10 06:02, Peter Thatcher wrote:
> There's a difference between the resolution that you open the camera at
> and the resolution you send over the network at.  Does the current
> constraints API let you capture at one resolution and send at another?
> Also, does it let you change the send resolution on the fly?
>
> new methods, or as you suggest, perhaps even as some kind of
> constraint.  But I don't think they are currently provided.

They are currently not provided. One reason is that there was a 
comprehensive proposal for how to change - on the fly - settings using 
constraints submitted by Travis last fall [1]. After a lot of discussion 
the editors are currently incorporating a slightly modified version of 
it for the local use. Once we have general agreement on that model it 
can be extended to the cases when media is transmitted over the network.

I did make a proposal to the webrtc WG, [2], which is heavily influenced 
by [1], for how we could add an API surface to individually control 
setting for how media is transported (the focus is on Priority but the 
idea was that is should be extendable) but it was never thoroughly 
discussed. And, there are probably other and better ways to do it :-), 
my main point is that SDP manipulation should not be the way.

As for the specific control of resolution there has also been a 
discussion (and that is also part of [1]) that the display area used by 
e.g. a video element should travel back to the source so that is can 
adjust settings to optimize (even if the web app developer does nothing).

[1] 
https://dvcs.w3.org/hg/dap/raw-file/tip/media-stream-capture/proposals/SettingsAPI_proposal_v6.html
[2] 
http://lists.w3.org/Archives/Public/public-webrtc/2013Jan/att-0005/PrioAPI.pdf

Stefan

>
> On Mar 9, 2013 7:54 PM, "Harald Alvestrand" <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>     As usual, I'm trying to use subject line change in order to achieve
>     some separation of concerns...
>
>         Agreed, but it's also not sufficient. SDP is not "programmer
>         friendly" enough because it has too many details that are
>         protocol-details only and it's too hard to see the semantic bits
>         in SDP and ignore the rest.
>
>         For example: the programmer wants to say - I want to get this
>         video resolution, this audio bitrate & channels, I want to use
>         this camera and this microphone for this call. Having to
>         manipulate SDP directly for this is a programmer's nightmare.
>
>
>     I think we've been over exactly those pieces, and our current
>     proposed solution is called the Media Stream API and the constraints
>     mechanism - and they have exactly nothing to do with SDP, or even
>     with PeerConnection.
>
>     I don't think we've got it to be "unproblematic" yet, but also, I
>     don't think SDP, JSON or even the offer-answer model is either the
>     problem or the solution on this set of functionalities.
>
>     _________________________________________________
>     rtcweb mailing list
>     [email-masked] <mailto:[email-masked]>
>     https://www.ietf.org/mailman/__listinfo/rtcweb
>     <https://www.ietf.org/mailman/listinfo/rtcweb>
>
>
>

---

From: Harald Alvestrand <[email-masked]>
Date: Sun, 10 Mar 2013 15:35:41 +0100

On 03/10/2013 06:02 AM, Peter Thatcher wrote:
>
> There's a difference between the resolution that you open the camera 
> at and the resolution you send over the network at.  Does the current 
> constraints API let you capture at one resolution and send at 
> another?  Also, does it let you change the send resolution on the fly?
>
Peter, you were in Boston, where we spent quite a bit of time on this 
very topic.

The relevant presentation is 
http://www.w3.org/wiki/images/f/fd/Constraints20130406.pdf.

Dan has promised to have the consensus of the discussion represented in 
the Media Capture and Streams spec well before the Media Capture and 
Streams meeting that we're currently scheduling for 2 weeks after the 
IETF (Doodle poll at http://doodle.com/xd7ffpidh6mgn2m8 )

> supplied to the application.  They could potentially be supplied by 
> SDP, new methods, or as you suggest, perhaps even as some kind of 

But we DO have a W3C consensus on which approach to take in providing them.

---

From: Ted Hardie <[email-masked]>
Date: Sun, 10 Mar 2013 14:22:18 -0400

> let you change the send resolution on the fly?
Just to make sure I understand the use case here, is the idea that you
open the camera at one resolution, potentially using that for a
recording or local display, then send a different resolution to the
peer?  I ask because I'm trying to understand how this control would
map to the connection objects--is there always a mediastream track
consuming the "native" resolution of the camera?

regards,

Ted

> supplied to the application.  They could potentially be supplied by SDP, new
> methods, or as you suggest, perhaps even as some kind of constraint.  But I
> don't think they are currently provided.
>

---

From: Martin Thomson <[email-masked]>
Date: Sun, 10 Mar 2013 15:39:47 -0400

On 10 March 2013 14:22, Ted Hardie <[email-masked]> wrote:

> and

The W3C media capture task force has discussed this at some length.  I
suggest that you both consult their archives.  It's all there.

This specific example has been discussed and incorporated into the models.
Unfortunately, the work to incorporate what we agreed to in Boston
regarding this model is not yet complete.

---

From: Silvia Pfeiffer <[email-masked]>
Date: Mon, 11 Mar 2013 08:10:31 +1100

> solution on this set of functionalities.

TBH I just hadn't seen
https://dvcs.w3.org/hg/dap/raw-file/tip/media-stream-capture/proposals/SettingsAPI_proposal_v6.html.
Is it implemented anywhere yet?

Now that I've looked at it, I wonder how that interacts with the SDP
created through createOffer(). If indeed the JS programmer never has to
touch the SDP, I don't see a need to expose it in the API at all, i.e.
setLocalDescription() should not be necessary.

Silvia.

---

From: Suhas Nandakumar <[email-masked]>
Date: Sun, 10 Mar 2013 15:59:07 -0700

I agree .,. JS developer need not have to touch SDP unless it is required ,
which I feel might be needed for few special scenarios.
Constraints, Medie Stream Apis, media capture Apis and Peer Connection Apis
should be able to support majority of use cases without JS developer
needing to manipulate SDP directly.

2 cents

Cheers
Suhas

On Sunday, March 10, 2013, Silvia Pfeiffer wrote:

> On Sun, Mar 10, 2013 at 3:04 AM, Harald Alvestrand <[email-masked]<javascript:_e({}, 'cvml', [email-masked]');>
> > wrote:
>
> https://dvcs.w3.org/hg/dap/raw-file/tip/media-stream-capture/proposals/SettingsAPI_proposal_v6.html. Is it implemented anywhere yet?
>

---

From: Silvia Pfeiffer <[email-masked]>
Date: Mon, 11 Mar 2013 10:02:57 +1100

What are the special scenarios?
(Noting that this is likely the wrong list for this topic...)
Silvia.

On Mon, Mar 11, 2013 at 9:59 AM, Suhas Nandakumar <[email-masked]>wrote:

> I agree .,. JS developer need not have to touch SDP unless it is required
> , which I feel might be needed for few special scenarios.
> Constraints, Medie Stream Apis, media capture Apis and Peer Connection
> Apis should be able to support majority of use cases without JS developer

---

From: Harald Alvestrand <[email-masked]>
Date: Mon, 11 Mar 2013 13:39:30 +0100

> (Noting that this is likely the wrong list for this topic...)

One that I've heard mention is if the remote gateway requires stuff in 
the SDP that the browser knows nothing about. (I've never encountered 
this in practice, but it wouldn't surprise me if it happens).

In that case, the application would be written especially for 
communicating with that remote gateway.

---

From: [email-masked] (Dale R. Worley)
Date: Mon, 11 Mar 2013 12:00:00 -0400 (EDT)

> From: Harald Alvestrand <[email-masked]>
> 

I would be surprised if this happens much.  In the offer/answer arena,
backward compatibility considerations mean that the recipient of SDP
is always prepared to receive *less* information about the media than
it would prefer to receive, and to process such SDP successfully.

Dale