Subject: [rtcweb] Proposal for API mapping for RTP

From: Magnus Westerlund <[email-masked]>
Date: Thu, 7 Nov 2013 10:17:48 -0800

WG,
(As draft-ietf-rtcweb-rtp-usage Editor)

There was discussion yesterday afternoon on the W3C MediaStream to RTP
mapping. This did arrive on some conclusions that I here try to word
into a proposal for everyones review.

A MediaStreamTrack is sent over a source packet stream (one SSRC) and
can have additionally redundancy packet streams (SSRCs). These
redundnacy streams are RTP retranmission streams or FEC.

When multiple MediaStreamTracks have the same Media Source, then the
fact that one has creaeted multiple MediaStreamTracks and added these
through a MediaStream to the PeerConnection is going to result in that
each MediaStreamTrack will have its own source packet stream (one SSRC).
This will be true, even if there are no difference in the configuration
of the MediaStreamTrack. Thus no optimizations in regards to collapsing
or aggregating multiple MediaStreamTrack onto a single source packet
stream (SSRC). This is done for keeping things simple and straightforward.

When it comes to the use of CNAME, an RTCWEB end-point shall within the
context of one origin, i.e. a particular JS creating PeerConnections,
use the same CNAME in all these PeerConnections, for all outgoing
mediaStreamTracks. However, a end-point MUST be capable of receiving
multiple different CNAMEs both within and between different RTP sessions
and PeerConnections. This definition comes from the following observations.

A MediaStream contains multiple MediaStreamTrack, and different
MediaStreams can share the same MediaStreamTrack. MediaStreamTrack
within one MediaStream is synchronized. Thus, at any point the JS
application can create a new MediaStream that includes MediaStreamTrack
from different context. To avoid needing to change all SSRC and put the
SSRCs into a single CNAME at that point, we ensure that this can't happen.

MediaStreamTracks that are being received in one PeerConnection and then
forward by being added to another one will need to be re-synchronzied
into the endpoints outgoing. We discussed and came to the agreement on
Monday that forwarding would need to be done equivalent of decoding and
recoding when forwarding. Implementations may do other things, but must
function equivalent to this.

The above have some forward interoperability properties. If one like to
be able to use multiple CNAMEs that can be added given that W3C API
ensures that you don't cause issue with what CNAME a particular SSRC
belongs to. In addition if one like to optimize the cases where multiple
MediaStreamTrack have a common media source that can be added.

Disagreements, requests for clarifications?

Cheers

Magnus Westerlund

----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone  +46 10 7148287
Färögatan 6                | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden| mailto: [email-masked]
----------------------------------------------------------------------

---

From: Stefan Håkansson LK <[email-masked]>
Date: Sun, 10 Nov 2013 09:32:35 +0000

> (As draft-ietf-rtcweb-rtp-usage Editor)

Magnus,

thanks for this write up. Some comments inline.

> redundnacy streams are RTP retranmission streams or FEC.

Can't there be enhancement packet streams as well (for layered codecs)?

And in some proposals a single MediaStreamTrack can be sent in more than 
one encoding (UA - as opposed to JS handled - handled simulcast) - is 
that covered?

> stream (SSRC). This is done for keeping things simple and straightforward.

I think this is the right way to do it.

> A MediaStream contains multiple MediaStreamTrack, and different

s/contains/can contain/

> within one MediaStream is synchronized. Thus, at any point the JS

"synchronized _at playout_"?
> SSRCs into a single CNAME at that point, we ensure that this can't happen.

By stating that all SSRC's originating from an endpoint shall use the 
same CNAME? (Just for clarification)

> belongs to.

Could you expand a bit? Do you mean that if there is an API to alter 
CNAMEs (and I guess there is: SDP mangling) it must ...do what?

> In addition if one like to optimize the cases where multiple

This is not so simple. E.g. assuming we will support pause/resume at 
some point, that could be quite complex if you pause one 
MediaStreamTrack but not another - both using the same source.

The application can already optimize by only sending it once (and clone 
- if required - at the receiving end).

Stefan

> _______________________________________________
> rtcweb mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rtcweb
>

---

From: Magnus Westerlund <[email-masked]>
Date: Mon, 11 Nov 2013 10:11:15 +0100

Stefan, WG,

Please see inline

> Can't there be enhancement packet streams as well (for layered codecs)?

So the enhancement layers if sent as individual packet streams are
called dependent layers in the current RTP taxonomy draft. Yes, they
could also exist. However, as there are no API support for these at the
current day they haven't been defined. If one uses something like SVC
MST, then clearly additions in how to interpret this must be provided.
If this is a single rooted dependency tree then I think additional SSRCs
for dependencies are quite straightforward. There will need to exist a
way of indicating that SSRCs are dependency streams for a source stream
to make handling work well.

> one encoding (UA - as opposed to JS handled - handled simulcast) - is 

So simulcast was discussed, and that would result in more than one
source stream associated with that MediaStreamTrack. This have similar
requirements that the a receiver understands that this is a simulcast of
the same MediaStreamTrack as another encoding version.

For both simulcast and scalable coding, I am happy to make it clear that
these can result in multiple source packet streams, and further have
additional dependency packet streams.

>> A MediaStream contains multiple MediaStreamTrack, and different

Yes

> "synchronized _at playout_"?

That is not for us in IETF to define. My aim is to enable a receiver to
perform synchronized playout. If it does it or not, I think is up to the
API definition.

> same CNAME? (Just for clarification)

Yes. but I would change this from "stating" to "REQUIRE".

> CNAMEs (and I guess there is: SDP mangling) it must ...do what?

I think this requires an API, and I question if any SDP mangling should
be allowed to change the CNAME. This due to the privacy concerns. If the
JS can set the CNAME, then it can enforce traceability of users or
end-points at RTP/RTCP level.

I think the API we did discuss was to consider when the
MediaStreamTracks in one MediaStream incoming in one PeerConnection gets
added as outgoing in another PeerConnection the original CNAME could be
maintained. However, that requires explicit lock down preventing one to
create a MediaStream with one of these MediaStreamTracks and any other
MediaStreamTrack comming from another CNAME context.

> MediaStreamTrack but not another - both using the same source.

Yes, that was understood in the discussion that such combinations
requires sane unions of settings across multiple MediaStreamTrack. In
your of paused or not, it is quite simple to agree that if any consumer
exists, i.e. not all paused, then it must be delivered.

But, clearly a reasons for the simple rule proposed.

> - if required - at the receiving end).

Good,

Cheers

Magnus Westerlund

---

From: "Lijing (Jessie)" <[email-masked]>
Date: Fri, 22 Nov 2013 03:21:15 +0000

A MediaStream contains multiple MediaStreamTrack, and different MediaStreams can share the same MediaStreamTrack. MediaStreamTrack within one MediaStream is synchronized.

-----Does anywhere define how to describe same MST in different mediastreams in SDP? As I know, the current unified plan defines" One m-line == one MediaStreamTrack", so how can we mark multiple mediastreams in msid attribute?
Allow msid carry more mediastream id?
m=video1
a=msid: ma ta; mb ta

or have multiple msid attribute in one m line?
m=vido1
a=msid: ma ta
a=msid: mb ta

or we should expand other attribute? 

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Magnus Westerlund
Sent: Friday, November 08, 2013 2:18 AM
To: [email-masked]
Subject: [rtcweb] Proposal for API mapping for RTP

WG,

There was discussion yesterday afternoon on the W3C MediaStream to RTP mapping. This did arrive on some conclusions that I here try to word into a proposal for everyones review.

A MediaStreamTrack is sent over a source packet stream (one SSRC) and can have additionally redundancy packet streams (SSRCs). These redundnacy streams are RTP retranmission streams or FEC.

When multiple MediaStreamTracks have the same Media Source, then the fact that one has creaeted multiple MediaStreamTracks and added these through a MediaStream to the PeerConnection is going to result in that each MediaStreamTrack will have its own source packet stream (one SSRC).
This will be true, even if there are no difference in the configuration of the MediaStreamTrack. Thus no optimizations in regards to collapsing or aggregating multiple MediaStreamTrack onto a single source packet stream (SSRC). This is done for keeping things simple and straightforward.

When it comes to the use of CNAME, an RTCWEB end-point shall within the context of one origin, i.e. a particular JS creating PeerConnections, use the same CNAME in all these PeerConnections, for all outgoing mediaStreamTracks. However, a end-point MUST be capable of receiving multiple different CNAMEs both within and between different RTP sessions and PeerConnections. This definition comes from the following observations.

A MediaStream contains multiple MediaStreamTrack, and different MediaStreams can share the same MediaStreamTrack. MediaStreamTrack within one MediaStream is synchronized. Thus, at any point the JS application can create a new MediaStream that includes MediaStreamTrack from different context. To avoid needing to change all SSRC and put the SSRCs into a single CNAME at that point, we ensure that this can't happen.

MediaStreamTracks that are being received in one PeerConnection and then forward by being added to another one will need to be re-synchronzied into the endpoints outgoing. We discussed and came to the agreement on Monday that forwarding would need to be done equivalent of decoding and recoding when forwarding. Implementations may do other things, but must function equivalent to this.

The above have some forward interoperability properties. If one like to be able to use multiple CNAMEs that can be added given that W3C API ensures that you don't cause issue with what CNAME a particular SSRC belongs to. In addition if one like to optimize the cases where multiple MediaStreamTrack have a common media source that can be added.

Cheers

Magnus Westerlund

---

From: Harald Alvestrand <[email-masked]>
Date: Fri, 22 Nov 2013 13:38:52 +0100

> or we should expand other attribute?

My thinking is that the last one is the one to use; I have not defined 
special semantics for semicolons in MSID values, but the spec 
(draft-ietf-mmusic-msid-02) does say that there can be multiple msid 
attributes on a single m-line, and it seems logical to go that route.