Subject: [rtcweb] A problem with both A and B

From: Emil Ivov <[email-masked]>
Date: Fri, 17 May 2013 19:04:22 +0300

I've mentioned this in a couple of other threads but I think it's
important so I am also posting it separately.

Both plan A and B currently describe semantics that would require O/A
exchanges every time a source is added or removed from a session.

Whether it is about adding a second webcam, or a conference participant
that leaves or joins, both schemes suggest that the remote party should
be notified with an updated SDP.

Plan A deals with this by manipulating m= lines, while plan B does it
with msid-s.

We are nowhere near consensus and I think that this fact alone is a
strong argument in favour of the following:

This is not a problem for rtcweb to solve.

We all have our own preferences for a solution and those come from the
different use cases we are planning on addressing and the different apps
we are planning to build. The promise of WebRTC is that, by building on
the Web paradigm we would provide building blocks which can then be
assembled into specific solutions (rather than providing the solutions
themselves).

In other words: it is not our job to try and implant a signalling
protocol into O/A. Signalling is the job of the application.

Applications alone know the meaning of the stream that came from your
second web cam, or that of the tenth conference participant. They know
which conferencing topology they most care about. They have best
knowledge about what streams they are currently rendering and which ones
they don't need. They know that the names Adam and Justin should be
displayed to describe the streams they are currently getting.

Most importantly however: they are best placed to know how they'd like
to communicate that information to their peers or IF they need to
communicate it at all.

Currently neither Plan A nor Plan B would let the application do its
job. However I do believe Plan B is closer.

If we agree that browsers would give applications the latitude that's
necessary to address these issues with app specific signalling, then the
SDP shouldn't change when they are doing it. Plan B is almost there and
the only remaining problem is its insistence on the pre-announcement of
SSRCs.

If we get rid of that requirement then all we need would be for the W3C
to make sure that the APIs have everything they need to allow apps to
retrieve SSRC information, send it remotely then pass it down to the
browser again when they receive it.

I acknowledge that the simulcasting and FEC cases might be a bit
different, which is normal because they are about the resolution of
transport problems. I still think they could be solved with app-specific
signalling and the proper APIs but in this case it would also be worth
investigating the options of doing this through RTP/RTCP.

Does any of this make any sense?

Emil

-- 
https://jitsi.org

---

From: [email-masked] (Dale R. Worley)
Date: Fri, 17 May 2013 16:18:58 -0400 (EDT)

> From: Emil Ivov <[email-masked]>
> 
> [...]

My understanding (and I'm not tracking everything carefully) is that
this is a bad situation.  I've been accumulating desiderata, and one
that has been on the list for a long time is:

   DES F11  It must be possible to add and remove one way video flows
      within the bundle without requiring an additional offer/answer
      cycle.

Do people think that this is not important?

Dale

---

From: Stefan Håkansson LK <[email-masked]>
Date: Sun, 19 May 2013 11:28:48 +0000

> Does any of this make any sense?

There is a lot of sense AFAICU, but there are two things to point out:
* There should be a standardized way for the receiver to say it can't 
handle (parts of) the media the senders wants to send. Someone proposed 
the max-ssrc draft
* There have been discussions about that there should be a standardized 
way for the receiving app to reject - via an api - parts of what the 
sending app wants to send

offer+answer helps with the above.

---

From: Christer Holmberg <[email-masked]>
Date: Mon, 20 May 2013 06:50:40 +0000

Hi Emil,

I agree that the signaling protocol is outside the scope of this group.

However, as SDP O/A is also used within the API, it becomes an issue for this group :)

Regards,

Christer

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Emil Ivov
Sent: 17. toukokuuta 2013 19:04
To: [email-masked]
Subject: [rtcweb] A problem with both A and B

I've mentioned this in a couple of other threads but I think it's important so I am also posting it separately.

Both plan A and B currently describe semantics that would require O/A exchanges every time a source is added or removed from a session.

Whether it is about adding a second webcam, or a conference participant that leaves or joins, both schemes suggest that the remote party should be notified with an updated SDP.

Plan A deals with this by manipulating m= lines, while plan B does it with msid-s.

We are nowhere near consensus and I think that this fact alone is a strong argument in favour of the following:

We all have our own preferences for a solution and those come from the different use cases we are planning on addressing and the different apps we are planning to build. The promise of WebRTC is that, by building on the Web paradigm we would provide building blocks which can then be assembled into specific solutions (rather than providing the solutions themselves).

In other words: it is not our job to try and implant a signalling protocol into O/A. Signalling is the job of the application.

Applications alone know the meaning of the stream that came from your second web cam, or that of the tenth conference participant. They know which conferencing topology they most care about. They have best knowledge about what streams they are currently rendering and which ones they don't need. They know that the names Adam and Justin should be displayed to describe the streams they are currently getting.

Most importantly however: they are best placed to know how they'd like to communicate that information to their peers or IF they need to communicate it at all.

Currently neither Plan A nor Plan B would let the application do its job. However I do believe Plan B is closer.

If we agree that browsers would give applications the latitude that's necessary to address these issues with app specific signalling, then the SDP shouldn't change when they are doing it. Plan B is almost there and the only remaining problem is its insistence on the pre-announcement of SSRCs.

If we get rid of that requirement then all we need would be for the W3C to make sure that the APIs have everything they need to allow apps to retrieve SSRC information, send it remotely then pass it down to the browser again when they receive it.

I acknowledge that the simulcasting and FEC cases might be a bit different, which is normal because they are about the resolution of transport problems. I still think they could be solved with app-specific signalling and the proper APIs but in this case it would also be worth investigating the options of doing this through RTP/RTCP.

Emil

--
https://jitsi.org
_______________________________________________
rtcweb mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/rtcweb

---

From: Emil Ivov <[email-masked]>
Date: Mon, 20 May 2013 11:29:18 +0300

> handle (parts of) the media the senders wants to send. Someone proposed 

Certainly. I am not arguing against use of max-ssrc. Quite on the contrary.

> sending app wants to send

Well, perhaps we need to have these discussions again. If the browser
can't handle H.264 it makes perfect sense to refuse that in SDP.
Indicating a maximum supported resolution of 720p also seems fine.

However, if the receiving application wants to tell the sending
application that it would like to reject certain streams, that's up to
the applications to signal.

> offer+answer helps with the above.

I think one look at the discussions that are taking place on this list
is enough to see that this is exactly where Offer/Answer isn't helping.

Emil

-- 
https://jitsi.org

---

From: Emil Ivov <[email-masked]>
Date: Mon, 20 May 2013 11:30:00 +0300

Hey Christer,

> However, as SDP O/A is also used within the API, it becomes an issue for this group :)

My point was that such semantics do not belong to SDP O/A in the first
place. They are a matter for application specific signalling.

Emil

-- 
https://jitsi.org

---

From: Kevin Dempsey <[email-masked]>
Date: Mon, 20 May 2013 09:53:44 +0100

I would say that at present only the sending application knows what a
stream is for. The receiving side needs to determine the purpose by
matching the RTP packets to the signalling the applications have done.

In unbundled situations this is easy as the port it arrives on makes it
obvious. In bundled cases, you would have to ensure the receiver knows some
unique property of the RTP *before* it arrives. The best thing would be for
the receiver to pick an identifier which, having been signalled in the
offer to the sender, is included in every RTP packet (in a RTP extension
header).

Relying on SSRC signalling would mean that RTP sent before the SSRC
identity arrives would not be handled consistently with RTP arriving after
the SSRC identity.

---

From: Emil Ivov <[email-masked]>
Date: Mon, 20 May 2013 12:33:58 +0300

On 20.05.13, 11:53, Kevin Dempsey wrote:
> identity arrives would not be handled consistently with RTP arriving
> after the SSRC identity.

If the application is doing the SSRC signalling and if the WebRTC API
provides a way of retrieving it, an app would be able to signal an SSRC
before, together with, or after the SDP from O/A. It will all be up to
exactly what the app needs.

Emil

-- 
https://jitsi.org

---

From: Sergio Garcia Murillo <[email-masked]>
Date: Tue, 21 May 2013 11:36:10 +0200

> place. They are a matter for application specific signalling.

+1

BR
Sergio

---

From: Christer Holmberg <[email-masked]>
Date: Mon, 20 May 2013 10:24:50 +0000

Hi,

Whatever mechanism we choose for demuxing RTP packets, I think it is ok that one has to wait for the SDP answer in order to figure out all information needed (e.g. the SSRC value used by the remote sender), and to possibly drop media that is received before the SDP answer is received.

In theory this could cause clipping problem, but at least my experience is that it's not a problem in reality, the reason being that the SDP answer if often sent (e.g. in a SIP 18x response) well before the remote media.

Regards,

Christer

From: [email-masked] [mailto:[email-masked]] On Behalf Of Kevin Dempsey
Sent: 20. toukokuuta 2013 11:54
To: Emil Ivov
Cc: [email-masked]
Subject: Re: [rtcweb] A problem with both A and B

I would say that at present only the sending application knows what a stream is for. The receiving side needs to determine the purpose by matching the RTP packets to the signalling the applications have done.

In unbundled situations this is easy as the port it arrives on makes it obvious. In bundled cases, you would have to ensure the receiver knows some unique property of the RTP *before* it arrives. The best thing would be for the receiver to pick an identifier which, having been signalled in the offer to the sender, is included in every RTP packet (in a RTP extension header).

Relying on SSRC signalling would mean that RTP sent before the SSRC identity arrives would not be handled consistently with RTP arriving after the SSRC identity.

On 17 May 2013 17:04, Emil Ivov <[email-masked]<mailto:[email-masked]>> wrote:
I've mentioned this in a couple of other threads but I think it's
important so I am also posting it separately.

Emil

--
https://jitsi.org
_______________________________________________
rtcweb mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/rtcweb

---

From: Kevin Dempsey <[email-masked]>
Date: Mon, 20 May 2013 12:12:22 +0100

Since webrtc is using ICE, the initial answer must have arrived before any
media can be sent.

However, that doesn't deal with cases where the SSRC is either unknown or
can change. An example would be where a media server is used to play files
during a call. Each of the file replays can use a different SSRC.

On 20 May 2013 11:24, Christer Holmberg <[email-masked]>wrote:

>  Hi,****
>
> ** **
>
> Whatever mechanism we choose for demuxing RTP packets, I think it is ok
> that one has to wait for the SDP answer in order to figure out all
> information needed (e.g. the SSRC value used by the remote sender), and to
> possibly drop media that is received before the SDP answer is received.***
> *
>
> ** **
>
> In theory this could cause clipping problem, but at least my experience is
> that it’s not a problem in reality, the reason being that the SDP answer if
> often sent (e.g. in a SIP 18x response) well before the remote media.****
>
> ** **
>
> Regards,****
>
> ** **
>
> Christer****
>
> ** **
>
> *From:* [email-masked] [mailto:[email-masked]] *On
> Behalf Of *Kevin Dempsey
> *Sent:* 20. toukokuuta 2013 11:54
> *To:* Emil Ivov
> *Cc:* [email-masked]
> *Subject:* Re: [rtcweb] A problem with both A and B****
>
> ** **
>
> matching the RTP packets to the signalling the applications have done.****
>
> ** **
>
> header).****
> ** **
> the SSRC identity.****
> ** **
> On 17 May 2013 17:04, Emil Ivov <[email-masked]> wrote:****
>
> https://www.ietf.org/mailman/listinfo/rtcweb****
>
> ** **
>

---

From: Christer Holmberg <[email-masked]>
Date: Mon, 20 May 2013 11:31:11 +0000

Hi,

>Since webrtc is using ICE, the initial answer must have arrived before any media can be sent.

True.

However, keep in mind that BUNDLE does currently not require ICE.

>However, that doesn't deal with cases where the SSRC is either unknown or can change. An example would be where a media server is used to play files during a call. Each of the file replays can use a different SSRC. 

I assume that those files, from the receiver's perspective, will come from the same remote IP address as the other media within the call?

Regards,

Christer

On 20 May 2013 11:24, Christer Holmberg <[email-masked]> wrote:
Hi,

 
Regards,
 
Christer

 

Emil

---

From: Kevin Dempsey <[email-masked]>
Date: Mon, 20 May 2013 13:51:48 +0100

>I assume that those files, from the receiver's perspective, will come from
the same remote IP address as the other media within the call?
Yes, the RTP payloads are forwarded intact so all have the same source
IP/port.

On 20 May 2013 12:31, Christer Holmberg <[email-masked]>wrote:

> I assume that those files, from the receiver's perspective, will come from
> the same remote IP address as the other media within the call?
>
> Regards,
>
> Christer
>
>
>
>
> possibly drop media that is received before the SDP answer is received.
>
> that it's not a problem in reality, the reason being that the SDP answer if
> often sent (e.g. in a SIP 18x response) well before the remote media.
>
> Regards,
>
> Christer
>

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 20 May 2013 12:46:52 -0400

> matching the RTP packets to the signalling the applications have done.

ISTM that the legacy cases have depended upon doing this in a simplistic 
way. E.g., the app only supports one audio media stream to be played out 
over one audio device. So it refuses anything other than a single audio 
m-line and then maps all the audio packets it receives, regardless of 
SSRC, to that one device. So it needs minimal signaling.

This has gradually evolved to slightly more complex cases, such as 
audio+video using similar logic. The a=content attribute was added to 
cover simple cases of multiple media of the same type, but still 
assuming a separate m-line for each device.

All of this assumes that there is a very small range of fundamentally 
different applications, so signaling is not used much to distinguish them.

But clearly this is beginning to break down. ISTM the current plan is to 
require that the two ends of a session be running the same logical 
application, and that they then define how to sort out the use of 
multiple media in an application-specific way. This is what both CLUE 
and RTCWEB seem to be doing. In the CLUE case the assumption is that 
initial SIP session setup verifies if the assumption is true, and then 
defines different actions if it is or isn't. In the case of RTCWEB, I 
guess the assumption is that it is probably a single web server 
application controlling both ends. But in cases where that is not true - 
gatewaying to the sip world - I guess this is left for the web server to 
figure out.

I don't know how this is going to work when a Google+ with no Facebook 
account wants to interact with a Facebook user that has no Google+ 
account. ISTM in that case you need separate but equivalent web 
applications on both providers, and somehow they still need to have a 
consistent signaling mechanism between them.

	Thanks,
	Paul

---

From: "Cullen Jennings (fluffy)" <[email-masked]>
Date: Thu, 30 May 2013 17:25:23 +0000

> exactly what the app needs.

Uh, so I thought that is how both plan A and plan B worked. Yes the syntax of the passing the SSRC across the API uses SDP in both cases but the application can do whatever it wants with and provide it any way it wants. What am I missing? I think people long ago agreed stats API would reflect up SSRC and other information to the application as a second path that is not SDP. 

If you are suggesting that a syntax like JSON would be better than SDP syntax to pass the semantic information across the JS API, well that back to comment 22.