Subject: [rtcweb] Rejecting MediaStreamTracks in JSEP

From: Justin Uberti <[email-masked]>
Date: Mon, 3 Feb 2014 11:27:06 -0800

While working on the latest JSEP update, I came across the following
problem:

When a answerer wants to reject a "media stream" in 3264, it sets the m=
line port in the answer to zero. However, when an endpoint wants to reject
a remote MediaStreamTrack without rejecting the m= line entirely, because
it is using the m= line for its own MediaStreamTrack, we don't have a clear
way to express that. While setting a=sendonly in the answer will tell the
other side to stop sending data, this won't kill the remote track. As far
as the remote side is concerned, its track is just 'paused'/'held', and is
still associated with the m= line.

Note that there is no such problem if an endpoint decides to kill its own
local MediaStreamTrack; in the offer it sends to indicate this, in addition
to setting the m= line to a=recvonly, it can also remove the MSID of the
track from the m= line, indicating the track is gone. In other words, the
a=msid attribute serves as the binding of MST to m= line, and its absence
indicates the track is gone.

Therefore, I think we need some similar way for a receiver to do the same
for a remote track. And fortunately, I think we have a surface that can be
used for this, namely the "recv-appId" attribute that has been discussed in
the context of unified-plan and explained in
http://tools.ietf.org/id/draft-even-mmusic-application-token-02.txt and
unified-plan. If an endpoint doesn't want to receive a particular track on
a m= line, it sets the m= line to a=sendonly and also removes any
a=recv-appId.

Below is an example of an audio call between A and B, with
MediaStreamTracks MSTA and MSTB. A initially generates an offer, and then I
show 4 different answer scenarios for various cases.

1) A generates offer with MSTA
m=audio X
a=msid:MSTA
a=recv-appId:1 # use 1 in the header extension when sending MSTA

2a) Normal answer: B adds its own track MSTB, generates answer:
m=audio Y
a=msid:MSTB
a=recv-appId:1

2b) Add+reject answer: B adds its own track MSTB, stops MSTA, generates
answer:
m=audio Y
a=msid:MSB
a=sendonly
# no appid; A can realize that MSA has been rejected

2c) Reject-only answer: B stops MSTA without adding a stream, generates
answer:
m=audio 0
# entire m-line has been rejected

2d) Do-nothing answer: B does nothing, generates answer:
m=audio Y
a=recv-appId:1
a=recvonly

If this seems reasonable, I plan to use this mechanism for handling m= line
allocation in JSEP.

Also, this essentially makes recv-appId the converse of MSID; for
simplicity, I think we should consider unifying these concepts into a
single document.

---

From: Martin Thomson <[email-masked]>
Date: Mon, 3 Feb 2014 12:06:33 -0800

> 2a) Normal answer: B adds its own track MSTB, generates answer:

Potential problems:

recv-appId looks like something that applies to what B is receiving
reject looks a little like does not support recv-appid as well

I think that the merging appid/msid is worth considering.  I think
that gets around these minor issues.

---

From: Christer Holmberg <[email-masked]>
Date: Mon, 3 Feb 2014 20:08:35 +0000

Hi,

Would this be solved by using uni-directional m- lines?

Regards,

Christer

Sent from my Sony Ericsson Xperia arc S

Justin Uberti <[email-masked]> wrote:

While working on the latest JSEP update, I came across the following problem:

When a answerer wants to reject a "media stream" in 3264, it sets the m= line port in the answer to zero. However, when an endpoint wants to reject a remote MediaStreamTrack without rejecting the m= line entirely, because it is using the m= line for its own MediaStreamTrack, we don't have a clear way to express that. While setting a=sendonly in the answer will tell the other side to stop sending data, this won't kill the remote track. As far as the remote side is concerned, its track is just 'paused'/'held', and is still associated with the m= line.

Note that there is no such problem if an endpoint decides to kill its own local MediaStreamTrack; in the offer it sends to indicate this, in addition to setting the m= line to a=recvonly, it can also remove the MSID of the track from the m= line, indicating the track is gone. In other words, the a=msid attribute serves as the binding of MST to m= line, and its absence indicates the track is gone.

Therefore, I think we need some similar way for a receiver to do the same for a remote track. And fortunately, I think we have a surface that can be used for this, namely the "recv-appId" attribute that has been discussed in the context of unified-plan and explained in http://tools.ietf.org/id/draft-even-mmusic-application-token-02.txt and unified-plan. If an endpoint doesn't want to receive a particular track on a m= line, it sets the m= line to a=sendonly and also removes any a=recv-appId.

Below is an example of an audio call between A and B, with MediaStreamTracks MSTA and MSTB. A initially generates an offer, and then I show 4 different answer scenarios for various cases.

2b) Add+reject answer: B adds its own track MSTB, stops MSTA, generates answer:
m=audio Y
a=msid:MSB
a=sendonly
# no appid; A can realize that MSA has been rejected

2c) Reject-only answer: B stops MSTA without adding a stream, generates answer:
m=audio 0
# entire m-line has been rejected

If this seems reasonable, I plan to use this mechanism for handling m= line allocation in JSEP.

Also, this essentially makes recv-appId the converse of MSID; for simplicity, I think we should consider unifying these concepts into a single document.

---

From: Adam Roach <[email-masked]>
Date: Mon, 03 Feb 2014 14:19:44 -0600

On 2/3/14 13:27, Justin Uberti wrote:
> Therefore, I think we need some similar way for a receiver to do the 
> same for a remote track.

Yes.

> And fortunately, I think we have a surface that can be used for this, 
> namely the "recv-appId" attribute that has been discussed in the 
> context of unified-plan and explained in 
> http://tools.ietf.org/id/draft-even-mmusic-application-token-02.txt 
> and unified-plan. If an endpoint doesn't want to receive a particular 
> track on a m= line, it sets the m= line to a=sendonly and also removes 
> any a=recv-appId.

No.

The problem is that you're defining semantics to a behavior that is 
indistinguishable from the behavior exhibited by something that does not 
support app-id: if the thing on the other end isn't using app-id, then 
an attempt to put a stream on hold will be misinterpreted as an attempt 
to reject or remove it.

I agree that we want an SDP operation that differentiates between "don't 
send me this now" and "don't send me this ever" [1]  -- but it needs to 
be explicit. Something more like "a=i-am-rejecting-this-stream", but 
with a less cumbersome name.

/a

____
[1] Along with a Javascript control that allows us to do the same thing, 
but that's a W3C issue.

---

From: Justin Uberti <[email-masked]>
Date: Mon, 3 Feb 2014 12:19:44 -0800

> recv-appId looks like something that applies to what B is receiving

It does, in fact. In B's answer, a=msid indicates what B is sending;
a=recv-appId indicates that B wants to receive the MST from A that is
associated with this m-line.

> reject looks a little like does not support recv-appid as well

True. But if, as you say, we merge appid and msid, then the only possible
ambiguity comes from the legacy no-msid, no-appid case, and even that is
not an issue since all the possible outcomes are separable: full duplex
(m=audio Y), one-way reject (m=audio Y, a=sendonly), full reject (m=audio
0). Of course, we can't know that the one-way reject is final, but the
number of add/remove stream operations done by legacy devices should be
limited.

---

From: Justin Uberti <[email-masked]>
Date: Mon, 3 Feb 2014 12:20:53 -0800

On Mon, Feb 3, 2014 at 12:08 PM, Christer Holmberg <
[email-masked]> wrote:

> Yes, but at the cost of compatibility with legacy. There has been broad
support for bidirectional m= lines each time this has come up.

>  Note that there is no such problem if an endpoint decides to kill its
> own local MediaStreamTrack; in the offer it sends to indicate this, in
> addition to setting the m= line to a=recvonly, it can also remove the MSID
> of the track from the m= line, indicating the track is gone. In other
> words, the a=msid attribute serves as the binding of MST to m= line, and
> its absence indicates the track is gone.
>
> same for a remote track. And fortunately, I think we have a surface that
> can be used for this, namely the "recv-appId" attribute that has been
> discussed in the context of unified-plan and explained in

---

From: Justin Uberti <[email-masked]>
Date: Mon, 3 Feb 2014 12:25:42 -0800

On Mon, Feb 3, 2014 at 12:19 PM, Adam Roach <[email-masked]> wrote:

> for a remote track.
> namely the "recv-appId" attribute that has been discussed in the context of
> unified-plan and explained in
> support app-id: if the thing on the other end isn't using app-id, then an
> attempt to put a stream on hold will be misinterpreted as an attempt to
> reject or remove it.
>

See my response to Martin. Putting a stream on hold will never result in
that being misinterpreted as a reject; on the contrary, without appid,
there is no way to 'fully' reject half of a stream, and, as a problem
confined to pre-unified-plan devices, I think that is probably OK.

> send me this now" and "don't send me this ever" [1]  -- but it needs to be
> explicit. Something more like "a=i-am-rejecting-this-stream", but with a
> less cumbersome name.
>
> It's not clear to me that adding a new attribute would solve any problem
that appid doesn't solve; you still need to deal with the "other end not
using it" issue.

> [1] Along with a Javascript control that allows us to do the same thing,

Agree. But while we're on the subject, I am thinking that MST.stop() ==
"not ever", and RtpReceiver.active = false causes "not now".

---

From: Justin Uberti <[email-masked]>
Date: Mon, 3 Feb 2014 12:28:03 -0800

+mmusic (meant to include the first time around)

---

From: Adam Roach <[email-masked]>
Date: Mon, 03 Feb 2014 14:42:38 -0600

On 2/3/14 14:25, Justin Uberti wrote:
> It's not clear to me that adding a new attribute would solve any 
> problem that appid doesn't solve; you still need to deal with the 
> "other end not using it" issue.

Regardless of whether you think there's a problem in the legacy interop 
case, the big benefit here is that an explicit indicator by definition 
makes the operation explicit, and that in-and-of-itself is a good thing.

The core issue is that you're trying to make the operation implicit; 
your proposal edges awfully close to the DWIM model of design, which we 
know causes problems when we try to add further extensions.

/a

---

From: Justin Uberti <[email-masked]>
Date: Mon, 3 Feb 2014 13:03:22 -0800

On Mon, Feb 3, 2014 at 12:42 PM, Adam Roach <[email-masked]> wrote:

> The core issue is that you're trying to make the operation implicit; your
> proposal edges awfully close to the DWIM model of design, which we know
> causes problems when we try to add further extensions.

Good point, but since we are still defining exactly what appId does, I
think we have the opportunity to make it explicit.

---

From: Adam Roach <[email-masked]>
Date: Mon, 03 Feb 2014 15:10:45 -0600

On 2/3/14 15:03, Justin Uberti wrote:
>
> On Mon, Feb 3, 2014 at 12:42 PM, Adam Roach <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>         It's not clear to me that adding a new attribute would solve
>         any problem that appid doesn't solve; you still need to deal
>         with the "other end not using it" issue.
>
>
>     Regardless of whether you think there's a problem in the legacy
>     interop case, the big benefit here is that an explicit indicator
>     by definition makes the operation explicit, and that
>     in-and-of-itself is a good thing.
>
>     The core issue is that you're trying to make the operation
>     implicit; your proposal edges awfully close to the DWIM model of
>     design, which we know causes problems when we try to add further
>     extensions.
>
>

I think I agree, but this isn't congruent with the proposal you made 
before. Omitting app-id as a means of rejecting a stream is an implicit 
operation, regardless of how well it's documented.

If you want to have something like "a=recv-appId:reject", that would be 
suitably explicit, but I really have a hard time imagining how something 
like that would be preferable to just having a new attribute.

/a

---

From: Justin Uberti <[email-masked]>
Date: Mon, 3 Feb 2014 13:22:00 -0800

> like that would be preferable to just having a new attribute.

I was thinking a=recv-appId:0, which is both explicit, and has parallels
with the zero port form of rejecting a m= line.

---

From: Adam Roach <[email-masked]>
Date: Mon, 03 Feb 2014 16:10:17 -0600

On 2/3/14 15:22, Justin Uberti wrote:
>
> I was thinking a=recv-appId:0, which is both explicit, and has 
> parallels with the zero port form of rejecting a m= line.
>

I have no objection to such an approach.

/a

---

From: Jonathan Lennox <[email-masked]>
Date: Mon, 3 Feb 2014 22:32:17 +0000

On Feb 3, 2014, at 2:27 PM, Justin Uberti <[email-masked]<mailto:[email-masked]>> wrote:

AppID (as I envision it, as the author) and MSID (as I understand it) have rather different scopes, which are visible once sources are sent using more than one packet stream.

For example, an m-line which has both a primary encoded stream (of media) and a redundancy stream (of rtx packets) would have two appid values, but only one msid.  On the receive side, if you wanted to receive both a primary encoded stream and a redundancy stream, you would have two recv-appid values.

As such, appid and msid are solving different (though related) problems, and I don't think unifying them is a good idea.

Perhaps we need a recv-msid, explicitly?

---

From: Justin Uberti <[email-masked]>
Date: Mon, 3 Feb 2014 15:00:26 -0800

On Mon, Feb 3, 2014 at 2:32 PM, Jonathan Lennox <[email-masked]> wrote:

>  On Feb 3, 2014, at 2:27 PM, Justin Uberti <[email-masked]> wrote:
>
>  AppID (as I envision it, as the author) and MSID (as I understand it)
> have rather different scopes, which are visible once sources are sent using
> more than one packet stream.
>
>  For example, an m-line which has both a primary encoded stream (of
> media) and a redundancy stream (of rtx packets) would have two appid
> values, but only one msid.  On the receive side, if you wanted to receive
> both a primary encoded stream and a redundancy stream, you would have two
> recv-appid values.
>
>  As such, appid and msid are solving different (though related) problems,
> and I don't think unifying them is a good idea.
>

That would be fine by me - it would allow a single invention (msid +
recv-msid) to focus on solving the three identified problems with MSTs in
unified-plan:
- correlation of a MediaStreamTrack to m= line in signaling (msid)
- correlation of packets from a MST to m= line ahead of signaling
(recv-msid)
- rejection of an individual MST (recv-msid)

Based on your description, appid seems to go one or two levels down from a
MST to an individual RTP flow, but we can let that develop on its own,
separate from the timetable of webrtc 1.0 and unified plan.

---

From: Justin Uberti <[email-masked]>
Date: Mon, 3 Feb 2014 15:03:36 -0800

+Harald as this affects the MSID doc

On Mon, Feb 3, 2014 at 3:00 PM, Justin Uberti <[email-masked]> wrote:

>
>
>
> On Mon, Feb 3, 2014 at 2:32 PM, Jonathan Lennox <[email-masked]>wrote:
>
>>  As such, appid and msid are solving different (though related)
>> problems, and I don't think unifying them is a good idea.
>>

---

From: "Cullen Jennings (fluffy)" <[email-masked]>
Date: Mon, 3 Feb 2014 23:04:07 +0000

On Feb 3, 2014, at 3:10 PM, Adam Roach <[email-masked]> wrote:

>> I was thinking a=recv-appId:0, which is both explicit, and has parallels with the zero port form of rejecting a m= line.
>> 
> 

+1 with Adam on we need something explicitly somewhere. 

If we do it with some combination msid / app-id - I just want to make sure the behavior is still reasonable with things that don’t low how to signal ssrc.

---

From: Jonathan Lennox <[email-masked]>
Date: Mon, 3 Feb 2014 23:11:46 +0000

On Feb 3, 2014, at 6:00 PM, Justin Uberti <[email-masked]<mailto:[email-masked]>>
 wrote:

On Mon, Feb 3, 2014 at 2:32 PM, Jonathan Lennox <[email-masked]<mailto:[email-masked]>> wrote:
On Feb 3, 2014, at 2:27 PM, Justin Uberti <[email-masked]<mailto:[email-masked]>> wrote:

That would be fine by me - it would allow a single invention (msid + recv-msid) to focus on solving the three identified problems with MSTs in unified-plan:
- correlation of a MediaStreamTrack to m= line in signaling (msid)
- correlation of packets from a MST to m= line ahead of signaling (recv-msid)
- rejection of an individual MST (recv-msid)

Based on your description, appid seems to go one or two levels down from a MST to an individual RTP flow, but we can let that develop on its own, separate from the timetable of webrtc 1.0 and unified plan.

Points 1 and 3 make sense to me, and are I think what msid is for; but I don't think point 2 will work.

I don't think a single, m-line-wide, identifier will work for the correlation-ahead-of-signaling problem, once you have multiple RTP flows in the m-line.  That's why we've designed appid the way we have.

I think appid is the right solution to point 2.  More specifically, the problem appid is solving is better expressed as the correlation of packets from an RTP flow to an m-line; the correlation from the m-line to a MST is a separate step, and is msid's job.

---

From: Justin Uberti <[email-masked]>
Date: Mon, 3 Feb 2014 16:05:15 -0800

On Mon, Feb 3, 2014 at 3:11 PM, Jonathan Lennox <[email-masked]> wrote:

>
>  On Feb 3, 2014, at 6:00 PM, Justin Uberti <[email-masked]>
>  wrote:
>
>  Based on your description, appid seems to go one or two levels down from
> a MST to an individual RTP flow, but we can let that develop on its own,
>  I don't think a single, m-line-wide, identifier will work for the
> correlation-ahead-of-signaling problem, once you have multiple RTP flows in
> the m-line.  That's why we've designed appid the way we have.
>

>  I think appid is the right solution to point 2.  More specifically, the
> problem appid is solving is better expressed as the correlation of packets
> from an RTP flow to an m-line; the correlation from the m-line to a MST is
> a separate step, and is msid's job.
>

Agree that msid should map m-line to MST, but it would seem recv-msid could
map RTP flow to m=line as well as indicate whether the receiver is in fact
interested in receiving the MST associated with said m= line. Note that
recv-msid doesn't specify an actual MSID; its purpose is to indicate that
the receiver wants to receive data from the remote side's MST, and provide
a mechanism for the sender to mark the data accordingly. (Which ends up
seeming a lot like appid.)

Generally, I am hesitant to depend on 3 things if we think we could get by
with only 2.

---

From: Christer Holmberg <[email-masked]>
Date: Tue, 4 Feb 2014 14:13:35 +0000

Hi,

Just for my clarification: none of this would have any impact on transport level features related to RTCP, STUN, etc?

Regards,

Christer

From: rtcweb [mailto:[email-masked]] On Behalf Of Justin Uberti
Sent: 4. helmikuuta 2014 2:05
To: Jonathan Lennox
Cc: Cullen Jennings; Harald Alvestrand; [email-masked]; mmusic
Subject: Re: [rtcweb] Rejecting MediaStreamTracks in JSEP

On Mon, Feb 3, 2014 at 3:11 PM, Jonathan Lennox <[email-masked]<mailto:[email-masked]>> wrote:

Agree that msid should map m-line to MST, but it would seem recv-msid could map RTP flow to m=line as well as indicate whether the receiver is in fact interested in receiving the MST associated with said m= line. Note that recv-msid doesn't specify an actual MSID; its purpose is to indicate that the receiver wants to receive data from the remote side's MST, and provide a mechanism for the sender to mark the data accordingly. (Which ends up seeming a lot like appid.)

Generally, I am hesitant to depend on 3 things if we think we could get by with only 2.

---

From: Jonathan Lennox <[email-masked]>
Date: Tue, 4 Feb 2014 16:13:06 +0000

On Feb 3, 2014, at 7:05 PM, Justin Uberti <[email-masked]<mailto:[email-masked]>>
 wrote:

I understand that, but I'm also hesitant to have one mechanism do two (it seems to me) fairly unrelated things, especially where the scenarios in which the problems it's solving are applicable vary fairly widely.

As far as I can tell, the only use case anyone's identified for MSID is WebRTC, whereas the disambiguation problem occurs (and thus appid and recv-appid are applicable) for anything using Bundle with Unified-like semantics.

---

From: Jonathan Lennox <[email-masked]>
Date: Tue, 4 Feb 2014 16:13:42 +0000

MSID doesn't affect the transport level.  Appid, however, defines an RTCP SDES item, and an RTP header extension.

On Feb 4, 2014, at 9:13 AM, Christer Holmberg <[email-masked]<mailto:[email-masked]>>
 wrote:

Hi,

Regards,

Christer

From: rtcweb [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Justin Uberti
Sent: 4. helmikuuta 2014 2:05
To: Jonathan Lennox
Cc: Cullen Jennings; Harald Alvestrand; [email-masked]<mailto:[email-masked]>; mmusic
Subject: Re: [rtcweb] Rejecting MediaStreamTracks in JSEP

---

From: Magnus Westerlund <[email-masked]>
Date: Thu, 13 Feb 2014 10:05:47 +0100

On 2014-02-04 01:05, Justin Uberti wrote:

> 
> Agree that msid should map m-line to MST, but it would seem recv-msid
> could map RTP flow to m=line as well as indicate whether the receiver is
> in fact interested in receiving the MST associated with said m= line.
> Note that recv-msid doesn't specify an actual MSID; its purpose is to
> indicate that the receiver wants to receive data from the remote side's
> MST, and provide a mechanism for the sender to mark the data
> accordingly. (Which ends up seeming a lot like appid.)
> 

A question here, what are you really trying to reject?

Is it the MST, a particular RTP Stream (Id by SSRC) or the usage of this
media description (m= block)  in a particular direction (i.e. SDP internal)?

I think the solution differs quite a lot depending on what you really
try to accomplish.

cheers

Magnus Westerlund

----------------------------------------------------------------------
Services, Media and Network features, Ericsson Research EAB/TXM
----------------------------------------------------------------------
Ericsson AB                 | Phone  +46 10 7148287
Färögatan 6                 | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden | mailto: [email-masked]
----------------------------------------------------------------------

---

From: Harald Alvestrand <[email-masked]>
Date: Thu, 13 Feb 2014 19:11:47 +0100

What's being asked for is a way to control MediaStreamTracks.

The RTP streams are only incidentally impacted.

I'll put a strawman proposal into -msid- today.