Subject: [rtcweb] Proposal for dealing with CNAMEs and MSIDs for synchronization

From: Eric Rescorla <[email-masked]>
Date: Thu, 7 Feb 2013 11:22:18 -0800

Here's what I was trying to say at the microphone.

1. When two MSTracks are in the same MediaStream on the sending side:
 They must generate the same MSID in SDP and the same CNAME in RTP.

2. One the receiving side, any two tracks with the same MSID will appear in
the same MediaStreamTrack.

3. On the receiving side, any two RTP streams with the same CNAME will
be synchronized.

4. There are two ways for MSID and CNAME to be inconsistent.
- If MSID indicates synchronization but different CNAMEs are provided,
  synchronization is not attempted.
- If MSID indicates no synchronization but the same CNAME is used,
  then the tracks shall be synchronized, even though they appear in
  different MediaStreams.

-Ekr

---

From: Jim Barnett <[email-masked]>
Date: Thu, 7 Feb 2013 19:27:03 +0000

On point 2,  do you mean "same MediaStream"?

-          Jim

From: [email-masked] [mailto:[email-masked]] On Behalf Of Eric Rescorla
Sent: Thursday, February 07, 2013 2:22 PM
To: [email-masked]
Subject: [rtcweb] Proposal for dealing with CNAMEs and MSIDs for synchronization

-Ekr

---

From: Stefan Hakansson LK <[email-masked]>
Date: Thu, 7 Feb 2013 20:27:52 +0100

> the same MediaStreamTrack.

Same _MediaStream_?

> 3. On the receiving side, any two RTP streams with the same CNAME will

Yes.

My question is basically: what if the sender creates two MediaStreams 
for which all tracks have local sources (cam's, mike's), sends them to a 
peer, will the RTP streams for both MediaStreams have the same or 
different CNAME?

I argued for that they should have the same.

> _______________________________________________
> rtcweb mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rtcweb
>

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 7 Feb 2013 11:43:09 -0800

>> the same MediaStreamTrack.

Yes. Doh.

> My question is basically: what if the sender creates two MediaStreams for
> which all tracks have local sources (cam's, mike's), sends them to a peer,
> will the RTP streams for both MediaStreams have the same or different CNAME?
>

Yes, I think this is a separate (and fraught) question. :)

-Ekr

>> ______________________________**_________________
>> rtcweb mailing list
>> [email-masked]
>> https://www.ietf.org/mailman/**listinfo/rtcweb<https://www.ietf.org/mailman/listinfo/rtcweb>
>>
>>
> ______________________________**_________________
> rtcweb mailing list
> [email-masked]
> https://www.ietf.org/mailman/**listinfo/rtcweb<https://www.ietf.org/mailman/listinfo/rtcweb>
>

---

From: Stefan Hakansson LK <[email-masked]>
Date: Thu, 7 Feb 2013 20:59:38 +0100

On 02/07/2013 08:43 PM, Eric Rescorla wrote:

>
>     My question is basically: what if the sender creates two
>     MediaStreams for which all tracks have local sources (cam's,
>     mike's), sends them to a peer, will the RTP streams for both
>     MediaStreams have the same or different CNAME?
>

Separate question, but I think the answer should be documented 
(regardless on if it is "same", "different" or "implementers choice").

>         _________________________________________________
>         rtcweb mailing list
>         [email-masked] <mailto:[email-masked]>
>         https://www.ietf.org/mailman/__listinfo/rtcweb
>         <https://www.ietf.org/mailman/listinfo/rtcweb>
>
>
>     _________________________________________________
>     rtcweb mailing list
>     [email-masked] <mailto:[email-masked]>
>     https://www.ietf.org/mailman/__listinfo/rtcweb
>     <https://www.ietf.org/mailman/listinfo/rtcweb>
>
>

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 7 Feb 2013 12:03:22 -0800

On Thu, Feb 7, 2013 at 11:59 AM, Stefan Hakansson LK <
[email-masked]> wrote:

> Separate question, but I think the answer should be documented (regardless
> on if it is "same", "different" or "implementers choice").
>

Agreed.

-Ekr

>>         ______________________________**___________________
>>         rtcweb mailing list
>>         https://www.ietf.org/mailman/_**_listinfo/rtcweb<https://www.ietf.org/mailman/__listinfo/rtcweb>
>>         <https://www.ietf.org/mailman/**listinfo/rtcweb<https://www.ietf.org/mailman/listinfo/rtcweb>
>> >
>>
>>
>>     ______________________________**___________________
>>     rtcweb mailing list
>>     https://www.ietf.org/mailman/_**_listinfo/rtcweb<https://www.ietf.org/mailman/__listinfo/rtcweb>
>>     <https://www.ietf.org/mailman/**listinfo/rtcweb<https://www.ietf.org/mailman/listinfo/rtcweb>
>> >
>>
>>
>>
>

---

From: "Charles Eckel (eckelcu)" <[email-masked]>
Date: Thu, 7 Feb 2013 20:08:49 +0000

> -----Original Message-----
> Sent: Thursday, February 07, 2013 3:03 PM
> To: Stefan Hakansson LK
> Cc: [email-masked]
> Subject: Re: [rtcweb] Proposal for dealing with CNAMEs and MSIDs for
> synchronization
> 
> 
> 
> On Thu, Feb 7, 2013 at 11:59 AM, Stefan Hakansson LK
> <[email-masked]> wrote:
> 
> 

I think SHOULD is the correct thing to document. I can imagine there will be cases in which the sender will be a somewhat distributed app that does not really know that the media source should be synchronized, so it will assign a different CNAME.
The rest of the proposal sounded good as well.

Cheers,
Charles
 
> 		        - If MSID indicates synchronization but different CNAMEs
> are
> 		        provided,
> 		            then the tracks shall be synchronized, even though
> they
> 		        appear in
> 		            different MediaStreams.
> 
> 		        -Ekr
> 
> 
> 
> 
> <https://www.ietf.org/mailman/__listinfo/rtcweb>
> 		        <https://www.ietf.org/mailman/listinfo/rtcweb
> <https://www.ietf.org/mailman/listinfo/rtcweb> >
> 
> 
> <https://www.ietf.org/mailman/__listinfo/rtcweb>
> 		    <https://www.ietf.org/mailman/listinfo/rtcweb
> <https://www.ietf.org/mailman/listinfo/rtcweb> >
> 
> 
> 
> 
>

---

From: Stefan Hakansson LK <[email-masked]>
Date: Thu, 7 Feb 2013 21:19:49 +0100

On 02/07/2013 09:08 PM, Charles Eckel (eckelcu) wrote:
>> -----Original Message----- From: [email-masked]
>> [mailto:[email-masked]] On Behalf Of Eric Rescorla Sent:
>> Thursday, February 07, 2013 3:03 PM To: Stefan Hakansson LK Cc:
>> [email-masked] Subject: Re: [rtcweb] Proposal for dealing with
>> CNAMEs and MSIDs for synchronization
>>
>>
>>
> I think SHOULD is the correct thing to document.
"SHOULD use _same_"?

> I can imagine there
> will be cases in which the sender will be a somewhat distributed app
> that does not really know that the media source should be
> synchronized, so it will assign a different CNAME.

Definitely.

> Cheers, Charles
>> 4. There are two ways for MSID and CNAME to be inconsistent. - If
>> MSID indicates synchronization but different CNAMEs are provided,
>> synchronization is not attempted. - If MSID indicates no
>> synchronization but the same CNAME is used, then the tracks shall
>> be synchronized, even though they appear in different
>> MediaStreams.
>>
>> -Ekr
>>
>>
>>
>>
>> _________________________________________________ rtcweb mailing
>> list [email-masked] <mailto:[email-masked]>
>> _________________________________________________ rtcweb mailing
>> list [email-masked] <mailto:[email-masked]>

---

From: Jonathan Lennox <[email-masked]>
Date: Fri, 8 Feb 2013 10:59:30 -0500

>  They must generate the same MSID in SDP and the same CNAME in RTP.

Note that, as a consequence of this -- if the same MediaStreamTrack is put into two different MediaStreams of the same PeerConnection, the sending side will need to synchronize all the tracks of those two MediaStreams together, and give them all the same CNAME.  (Because synchronization is transitive.)

If Stefan's proposal is accepted (i.e., all the tracks from a single endpoint are synchronized, period), this may come for free, at least for locally-generated tracks.  But if it isn't, this is a subtlety that may not be fully appreciated.

--
Jonathan Lennox
[email-masked]

---

From: Harald Alvestrand <[email-masked]>
Date: Fri, 08 Feb 2013 22:37:56 +0100

> The rest of the proposal sounded good as well.
SHOULD often makes me unhappy, because it can lead easily to something 
being true so much of the time, people forget to check for whether it is 
true or not.

I think "all the streams have local sources" is a property that we 
shouldn't require that one end of a PeerConnection reveals to the other end.

(Parenthesis: In the discussion, the question was raised of what it 
meant to send out two tracks in one MediaStream that were not 
originallly synchronized - one local and one remote, for instance. The 
best answer I heard was that they would have to be synchronized 
(re-clocked if necessary) in such a way that when played out at the 
recipient, they would give the same experience as when played out 
locally at the sender - same relative timing for the playout.)

---

From: Stefan Håkansson LK <[email-masked]>
Date: Sat, 9 Feb 2013 14:11:24 +0100

On 2013-02-08 22:37, Harald Alvestrand wrote:
>> I think SHOULD is the correct thing to document. I can imagine there

When you say "streams" above I guess you mean RTP streams (or 
MediaStreamTracks) as it would not make sense to make this a secret when 
they are transported in different MediaStreams but expose it if they are 
in the same.

What info would reveal this ("have local sources")? Is it them having 
the same CNAME?

---

From: Harald Alvestrand <[email-masked]>
Date: Sat, 09 Feb 2013 15:40:10 +0100

On 02/09/2013 02:11 PM, Stefan Håkansson LK wrote:
> MediaStreamTracks) as it would not make sense to make this a secret 
> when they are transported in different MediaStreams but expose it if 
> they are in the same.
When they are in the same MediaStream, they are required to have the 
same CNAME (unless I lost track of the conversation), so it's only when 
they're in different MediaStreams that it matters whether the CNAME is 
the same or not.

Yes.

---

From: Stefan Håkansson LK <[email-masked]>
Date: Sun, 10 Feb 2013 13:47:02 +0100

>> What info would reveal this ("have local sources")? Is it them having

Just to verify that I get all of this correctly:
* There is a privacy issue if the app at the peer can find out if the 
tracks for _different_ MediaStream's have local sources
* This could be found out if the CNAME's are the same and if CNAME's are 
made available to the application
* Therefore we should have a design where tracks belonging to different 
MediaStream's use different CNAME's even if they originate from local 
sources at the same end-point

Did I get that right?

To me this seems far fetched. With this design you would have to select 
between synchronization and privacy, if you care about privacy each 
track would go to a different MediaStream (audio to one MS, video to 
another in the simple "one audio+one video" case) and there would be no 
sync.

If this is indeed a problem, then there must be other ways. A setting 
"privacy=on" on PeerConnection, or do not expose CNAME to the app (do we 
really need to signal it?; and [1] indicates you can use other CNAME's 
than those signaled).

Stefan

[1] http://tools.ietf.org/html/draft-lennox-mmusic-sdp-source-attributes-01

---

From: Harald Alvestrand <[email-masked]>
Date: Sun, 10 Feb 2013 23:36:41 +0100

On 02/10/2013 01:47 PM, Stefan Håkansson LK wrote:
>>>> shouldn't require that one end of a PeerConnection reveals to the 
>>>> other
>>>> end.
>>>
> * This could be found out if the CNAME's are the same and if CNAME's 
> are made available to the application
> * Therefore we should have a design where tracks belonging to 
> different MediaStream's use different CNAME's even if they originate 
> from local sources at the same end-point
>
> Did I get that right?
Points one and two, yes. Point three, no, that's not my thinking.
I see no reason to force different CNAMEs, I just want to permit it.
>
> To me this seems far fetched. With this design you would have to 
> select between synchronization and privacy, if you care about privacy 
> each track would go to a different MediaStream (audio to one MS, video 
> to another in the simple "one audio+one video" case) and there would 
> be no sync.
>
> "privacy=on" on PeerConnection, or do not expose CNAME to the app (do 
> we really need to signal it?; and [1] indicates you can use other 
> CNAME's than those signaled).
What I'd like to have is to have no statement that forces (even at 
SHOULD level) all local sources to have the same CNAME, and perhaps even 
an explicit statement that the sender can use different CNAMEs for local 
sources if it wants to (for whatever reason it wants to do so), as long 
as they are in different tracks.

---

From: Stefan Håkansson LK <[email-masked]>
Date: Mon, 11 Feb 2013 14:10:25 +0100

> as they are in different tracks.

The problem I have with this is that it makes it impossible for someone 
building a service to know whether streams will be synced (and I mean in 
situations when syncing makes sense - not when the video is delayed by 
500ms) or not. This is fully up to the browser - it may be that 
sometimes they are and sometimes not.

I would prefer that all streams used the same CNAME (if it is signaled 
or not is another question). If this is a privacy issue I think we 
should solve that instead of having the browser use, or not use, the 
same CNAMEs at its own will.

---

From: Harald Alvestrand <[email-masked]>
Date: Tue, 12 Feb 2013 08:37:57 +0100

On 02/11/2013 02:10 PM, Stefan Håkansson LK wrote:
> The problem I have with this is that it makes it impossible for 
> someone building a service to know whether streams will be synced (and 
> I mean in situations when syncing makes sense - not when the video is 
> delayed by 500ms) or not. This is fully up to the browser - it may be 
> that sometimes they are and sometimes not.

I don't understand what the problem is here. I may be dumb.
If syncing is important to the guy writing the service, (he can't live 
with them being unsynchronized), he should put them in the same stream.

Under your suggestion, it would be impossible to generate two 
MediaStreams that have local content and are *not* synced - I don't see 
a reason to outlaw that.

The practical issue I can think of would be a source that had its own 
clock, not the system clock, but is otherwise a local source. Having the 
same CNAME would require resynchronizing that source. I don't know if 
any such sources exist in practice, but again, I don't want to outlaw them.

> same CNAMEs at its own will. 

CNAMEs are signaled in RTCP SRs. We don't have the option of leaving 
them out.

---

From: Stefan Håkansson LK <[email-masked]>
Date: Tue, 12 Feb 2013 09:04:09 +0100

> I don't understand what the problem is here. I may be dumb.

Or I am :)

> MediaStreams that have local content and are *not* synced - I don't see

But under your suggestion, it could be synced or not, the app would have 
no way to control or know. It is basically random (and implementation 
dependent). This is my main problem, I would like a consistent behavior 
(secondary is that I would like it to be the same CNAME).

> CNAMEs are signaled in RTCP SRs. We don't have the option of leaving

Exactly, but what goes in the RTCP need not to be made available to the 
application in all cases - why not hide CNAME?

---

From: "Timothy B. Terriberry" <[email-masked]>
Date: Wed, 13 Feb 2013 20:23:26 -0800

> with them being unsynchronized), he should put them in the same stream.

The argument goes that if we leave this unspecified, web authors will 
test against browser A which does X, and not against browser B which 
does Y, and users of browser B will suffer when things don't work like 
they did in browser A.

Personally I'm not convinced this argument applies here. For cases like 
"Do the contents of this table element show up at all when I fail to 
close my <tr>?" it's pretty important, but in reality resynchronization 
is only going to affect things so much... if things get more than a few 
100 ms out of sync on the wire, the experience is going to be terrible 
no matter what you do (and I'd call them "bad" long before that point).

The goal of pixel-perfect rendering for webpages might be reasonable, 
but sample-accurate rendering of real-time media will never be possible 
given the limitations of real-world networks, and whether you're forcing 
synchronization or not, the media will still play.

> any such sources exist in practice, but again, I don't want to outlaw them.

I can certainly think of cases where two local streams with, e.g., the 
same sampling rate would advance at different rates measured against a 
global wallclock. For example, a gUM stream running against the 
microphone's clock vs. a <video> tag playing back against the soundcard 
output clock.

Then AFAICT forcing the use of the same CNAME means we will always add 
some additional latency in the receiver to perform the 
resynchronization. In the above situation, I'm not sure that's always 
the best choice.

---

From: Magnus Westerlund <[email-masked]>
Date: Thu, 14 Feb 2013 10:17:58 +0100

> synchronization or not, the media will still play.

My understanding of the underlying issues why this is important is
discussed in
https://datatracker.ietf.org/doc/draft-burman-rtcweb-mmusic-media-structure/
See Section 4.1.3

Lets start with two media streams from end-point that shares media
sources. Here we encounter another open issue, namely the relation
between media sources, a particular encoding and MediaStreamTrack. To my
knowledge it is not agreed on what should happen if I have two
MediaStreams, each with an audio and video MediaStreamTrack that are the
same media source, i.e. microphone and video camera. Even if no
configuration for encoding has been done, will the peer receive two
copies or are these optimized to send only two RTP streams, one with the
audio, one with the video that is used in both?

If they are then they MUST share the same CNAME in both media streams
all the MediaStreamTracks.

If the audio MediaStreamTrack in each MediaStream actually gets
different encodings, which I think is a waste unless intended due to
encoding requirements, but then we are in fact into Simulcast. IF one
has two different encodings and both are sent, then playing the same
audio source twice without sample accurate synchronization you have just
created an echo effect.

Even if the JS application switches back and forth between the media
streams there could be playback overlapping unless they are in sync.

> microphone's clock vs. a <video> tag playing back against the soundcard

Yes, but that should be dealt with locally. It is after all detectable
by using the global wallclock and can be compensated for. In fact RTCP
synchronization mechanism deals with the relative clock drift of
sampling clocks compared to the senders wall clock by updating the
offset in new RTCP Sender Reports.

This might not be implemented correctly in many implementations, but
that is an implementation bug, not a protocol issue. Same goes for the
local playback with the above characteristics.

> resynchronization. In the above situation, I'm not sure that's always

I think it is important that we are clear on that all media sources that
comes from the same synchronization context must be using the same
CNAME. This implies that media mixing entities track the incoming clocks
and keep the same off-set over multiple RTP streams from the same
synchronization context to maintain synchronization of that context
across the mixing/relaying/video switching.

Yes, it may incur delay, depending on what media operation you are
doing. I think the requirement we are discussing is what the sender MUST
do so that the receiver can ensure synchronization if they want to.

Cheers

Magnus Westerlund

----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone  +46 10 7148287
Färögatan 6                | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden| mailto: [email-masked]
----------------------------------------------------------------------

---

From: "Timothy B. Terriberry" <[email-masked]>
Date: Thu, 14 Feb 2013 05:00:12 -0800

> audio, one with the video that is used in both?

I think you have all of the same problems if you simply have two 
MediaStreams with the same MediaStreamTrack locally, without involving 
RTP at all. In the MediaStream Processing API 
<https://dvcs.w3.org/hg/audio/raw-file/tip/streams/StreamProcessing.html> Section 
2.1, we argued that the best approach is to require that all sinks of a 
MediaStreamTrack advance at the same rate (as otherwise you require 
either multiple decoders or unbounded buffering). I don't think that's 
written down anywhere in either an IETF or W3C spec that's currently active.

Once we have resolved that issue, I think that once you involve 
PeerConnection, the only thing that makes sense is that a series of 
invocations of AddStream() on the sender's PC produces a corresponding 
set of objects on the receiver's PC: if you call AddStream() with two 
MediaStreams that both contain the same MediaStreamTrack, then the 
remote side should create two MediaStreams that also contain the same 
MediaStreamTrack. If a sender wishes to use multiple SSRCs for a single 
track for either layered coding or simulcast or whatever, then following 
draft-alvetrand-mmusic-msid-02, they must have the same value for 
identifier and appdata.

> all the MediaStreamTracks.

I think it is perfectly reasonable to require that all SSRCs that share 
the same msid attributes (i.e., that belong to the same 
MediaStreamTrack) use the same CNAME. That should probably be written 
down somewhere (in the msid draft? in the RTP usage draft? I don't have 
a strong opinion, but the former at least involves fewer cross-document 
references). I think that solves the issue you're raising here.

> do so that the receiver can ensure synchronization if they want to.

Agreed, and I think Harald's answer was that the sender must put the 
tracks it wants synchronized into the same MediaStream, and that the 
browser must then use the same CNAME for any two tracks that belong to 
the same MediaStream (which would have to be applied transitively). I'm 
not sure anyone has disagreed with this. The only question is whether it 
also must use the same CNAME for two different tracks that appear in 
different MediaStreams.

---

From: Magnus Westerlund <[email-masked]>
Date: Thu, 14 Feb 2013 15:41:39 +0100

> written down anywhere in either an IETF or W3C spec that's currently

I don't quite get the example in Section 2.1 in the yellow box. It is
unclear what the URI's represent.

I agree with the sentiment that all copies of the same media resource
will need to be played out in sync also locally. I would like to add
that in the context of different media sources there exist an importance
to have a common clock base and deal with that nominal sampling rates
does not match the actual as measured by the local reference clock.

> draft-alvetrand-mmusic-msid-02, they must have the same value for

I agree that you need to have the corresponding objects on the receivers
side. When it comes to identifiers I get a bit confused here on how the
MSID draft is supposed to work. It switches between MediaStream and
MediaStreamTrack in section 4. Is the WMS semantics providing the
MediaStream IDs or the MediaStreamTrack IDs the SSRC is associated with?
The current text appears to indicate both. Is the ID making it possible
to distinguish between the two types?

> MediaStreamTrack) use the same CNAME. 

MediaStreamTrack or MediaStream?

That should probably be written
> references). I think that solves the issue you're raising here.

At some point we will need to have a mapping between the API concepts
and RTP to be written up and agreed on. We will have to find the most
suitable spot for that.

> not sure anyone has disagreed with this. 

Good, lets start with assuming that this has consensus and that if
anyone objects there is time to do that now.

The only question is whether it
> also must use the same CNAME for two different tracks that appear in

Yes, this is the real question. And I think my answer is it depends.

The issue I have is that different mediaStreams can have different mixes
of tracks, and they can share some or all tracks. From a receivers
perspective it appears necessary to avoid media glitches that a receiver
can determine that a given track is a duplicate of another and ensure
that they are played in sync and on the same time line.

As long as only one encoding and one RTP stream is sent for each media
source then implicit its tracks must have a common CNAME over all the
MediaStreams it occurs in.

Cheers

Magnus Westerlund

---

From: "Timothy B. Terriberry" <[email-masked]>
Date: Thu, 14 Feb 2013 09:35:04 -0800

> to distinguish between the two types?

According to Section 1.1, it is "providing necessary semantic 
information for the association of MediaStreamTracks to MediaStreams...."

IIUC, it is the "identifier" token which indicates the MediaStream, and 
the "appdata" token which indicates the MediaStreamTrack. The text of 
Section 4 says "The value of the msid corresponds to the 'id' attribute 
of a MediaStream," but I think it is supposed to say "msid identifier" 
instead of the whole attribute. It goes on to say "The appdata for a 
WebRTC MediaStreamTrack consists of the 'id' attribute of a 
MediaStreamTrack." It is not clear what happens if an SSRC has two msids 
with different "identifier" tokens but the same "appdata" token.

The thing I think is currently somewhat confusing is the relationship 
between MediaStreamTracks that belong to multiple MediaStreams. In the 
current W3C API, creating a new MediaStream from an existing 
MediaStreamTrack creates a clone of that track, rather than 
incorporating the track by reference. Both tracks would have the same 
underlying "source", but there is currently no representation of a 
source in the MediaStream API, nor any way to tell if two tracks have 
the same source from JS. In the SDP, an SSRC which has multiple msid 
attributes presumably creates multiple MediaStreamTracks with the same 
source.

It is not clear what is supposed to happen if one SSRC has two msid 
attributes, and another SSRC has a single msid which matches one of the 
msid attributes of the first SSRC, but is missing the other msid 
attribute. I'd be happy with almost anything (including "the SDP gets 
rejected as invalid"), but I think we should say something about it.

I also think the msid draft should be clear that just because two msid's 
have the same appdata (indicating MediaStreamTracks with the same "id" 
attribute), they should not necessarily be intended for the same 
MediaStreamTrack object. It currently says that "If two different SSRCs 
have the same value for identifier and appdata, it means that these two 
SSRCs are both intended for the same MediaStreamTrack." It does NOT say 
that if they have different identifiers and the same appdata, they are 
NOT intended for the same MediaStreamTrack object. Even more important 
is the case of a single SSRC with multiple msid's: the draft should make 
clear that this will correspond to multiple MediaStreamTrack JS objects 
(with a common "source"... a property that is currently ill-defined in 
the W3C API), even if the "appdata" token is the same.

> MediaStreamTrack or MediaStream?

I meant MediaStreamTrack. This would be to handle cases like layered 
codecs, FEC, or a source which plans to switch codecs (e.g., to use CN).

> and RTP to be written up and agreed on. We will have to find the most

Section 4 of the msid draft appears to attempt some of that (at least 
w.r.t. SSRCs). I don't think it's anywhere near complete.

> MediaStreams it occurs in.

I think we still want to support simulcast (though possibly not in 
Version 1), but it's an open question how we do so. We might very well 
want the alternatives to be in different MediaStreamTracks with 
different msids (or a receiver would have no way to select among those 
alternatives in JS), so they wouldn't be covered by the "same msid" case 
I suggest above, and would need some other mechanism to signal that they 
originate from the same "source".

However, I think defining CNAME behavior in the simulcast case isn't 
really rtcweb-specific. Right now, as I read 
draft-westerlund-avtcore-rtp-simulcast-01, it requires that related 
encodings be tagged with your proposed SDES item SRCNAME with the same 
value (though that draft only defines a multiple-session approach, and 
you know how well that's likely to go over in this group). I think this 
is a mistake, however, since 
draft-westerlund-avtext-rtcp-sdes-srcname-02 says SRCNAME is "scoped by" 
(which I take to mean "only unique in the context of the same") CNAME. 
What draft-westerlund-avtcore-rtp-simulcast-01 probably wants instead is 
to require the same CNAME:SRCNAME pair. draft-even-clue-rtp-mapping-04 
Section 6.2 (which has an incomplete example of single-session, 
SSRC-multiplexed simulcast) has a similar problem.

---

From: "Cullen Jennings (fluffy)" <[email-masked]>
Date: Thu, 14 Feb 2013 20:36:14 +0000

On Feb 14, 2013, at 2:17 AM, Magnus Westerlund <[email-masked]> wrote:

> CNAME.
 
Agree - I might rephrase that slightly to change 

   	comes from the same synchronization context

to 
        come from the same synchronization context and that are desirable to synchronize 

The issues I am concurred with is a device that is producing both audio and video from a camera and some video from a slide share and may be capable of sending the two such that the receiver can synchronize them but that may not to syncronize them due to the additional latency that would impose on the audio. I want to be able to send the audio and video with same CNAME and the slides with a different CNAME. I was assuming I would do that by putting the Track with the slides in one RTCMediaStream and the other Tracks in a different RTCMediaStream. 

Perhaps the above wording is fine and we just need to be clear on what a  synchronization context is.

---

From: Stefan Håkansson LK <[email-masked]>
Date: Fri, 15 Feb 2013 14:59:34 +0100

> with different "identifier" tokens but the same "appdata" token.

I don't think that could happen. If I read the getUserMedia draft 
correctly, the creation of a new MediaStream from existing 
MediaStreamTracks would result in the new MediaStream containing new 
MediaStreamTrack objects with new id's (they would share source and the 
label could remain though).

But it might be that the Media Capture TF has some additional speccing 
to do here, from the current model it seems very hard to correlate the 
old and new MediaStreamTracks (which one's belong together, i.e. share 
source). You could use label, but perhaps that is not unique enough.

--Stefan

---

From: Harald Alvestrand <[email-masked]>
Date: Sat, 16 Feb 2013 18:05:06 +0100

On 02/14/2013 06:35 PM, Timothy B. Terriberry wrote:
> IIUC, it is the "identifier" token which indicates the MediaStream, 
> and the "appdata" token which indicates the MediaStreamTrack. The text 
> of Section 4 says "The value of the msid corresponds to the 'id' 
> attribute of a MediaStream," but I think it is supposed to say "msid 
> identifier" instead of the whole attribute.

Yep. Thanks for catching this.

> It goes on to say "The appdata for a WebRTC MediaStreamTrack consists 
> of the 'id' attribute of a MediaStreamTrack." It is not clear what 
> happens if an SSRC has two msids with different "identifier" tokens 
> but the same "appdata" token.

I don't think it can happen. It would require having the same 
MediaStreamTrack in two MediaStreams, and I don't think we can do that.

> attributes, and another SSRC has a single msid which matches one of 
> the msid attributes of the first SSRC, but is missing the other msid 

Let me see ... if you mean this:

a=msid: 25 a x
a=msid: 25 b y
a=msid: 50 b w

it means that there are 2 MediaStreams (a and b) both carrying the media 
that's coming on SSRC 25 as a track, and one of these is also carrying 
the media coming on SSRC 50 as a track.
>
> I also think the msid draft should be clear that just because two 
> msid's have the same appdata (indicating MediaStreamTracks with the 
> same "id" attribute), they should not necessarily be intended for the 
> same MediaStreamTrack object. It currently says that "If two different 
> SSRCs have the same value for identifier and appdata, it means that 
> these two SSRCs are both intended for the same MediaStreamTrack." It 
> does NOT say that if they have different identifiers and the same 
> appdata, they are NOT intended for the same MediaStreamTrack object.

That was certainly my intention. But as noted above, I think "same 
appdata" isn't meaningful.

> Even more important is the case of a single SSRC with multiple msid's: 
> the draft should make clear that this will correspond to multiple 
> MediaStreamTrack JS objects (with a common "source"... a property that 
> is currently ill-defined in the W3C API), even if the "appdata" token 
> is the same.

Even with the same source, the data might actually not be the same (if 
it has been transformed, reencoded or otherwise mangled). If it's the 
same data, it seems reasonable to let it be sent once.

> codecs, FEC, or a source which plans to switch codecs (e.g., to use CN).

We already have a requirement for all SSRCs that belong to the same 
MediaStream to have the same CNAME, so  this is just a subset of that - 
that's why I wondered.

> w.r.t. SSRCs). I don't think it's anywhere near complete.

The msid draft tries to explain what the semantics of the identifiers 
is. If there are procedural matters to be documented, I think the JSEP 
draft should do it, since it's the "how do we map onto..." document.

> alternatives in JS), so they wouldn't be covered by the "same msid" 
> case I suggest above, and would need some other mechanism to signal 
> that they originate from the same "source".

I think simulcast fits best as a within-one-track concept; representing 
it as multiple tracks means that the application has to do explicit 
manipulation of it, and that doesn't sound right to me.
It should be one of those "just ask for it, and if you get it, it just 
works" things.
> you know how well that's likely to go over in this group). I think 
> this is a mistake, however, since 
> draft-westerlund-avtext-rtcp-sdes-srcname-02 says SRCNAME is "scoped 
> by" (which I take to mean "only unique in the context of the same") 
> CNAME. What draft-westerlund-avtcore-rtp-simulcast-01 probably wants 
> instead is to require the same CNAME:SRCNAME pair. 
> draft-even-clue-rtp-mapping-04 Section 6.2 (which has an incomplete 
> example of single-session, SSRC-multiplexed simulcast) has a similar 
> problem.

That seems to make sense to me. But "someone else's problem".

---

From: Jim Barnett <[email-masked]>
Date: Sat, 16 Feb 2013 21:36:27 +0000

Harald,
  You say:  " I don't think it can happen. It would require having the same MediaStreamTrack in two MediaStreams, and I don't think we can do that".   When we discussed this in the past, we certainly envisaged allowing the same Track to be in multiple streams.  Why else would MediaStream have an addTrack() method?  If we want to ensure monogamy, don't we have to change this to createTrack()?

- Jim

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Harald Alvestrand
Sent: Saturday, February 16, 2013 12:05 PM
To: [email-masked]
Subject: Re: [rtcweb] Proposal for dealing with CNAMEs and MSIDs for synchronization

On 02/14/2013 06:35 PM, Timothy B. Terriberry wrote:
> Magnus Westerlund wrote:
>> I agree that you need to have the corresponding objects on the 
>> receivers side. When it comes to identifiers I get a bit confused 
>> here on how the MSID draft is supposed to work. It switches between 
>> MediaStream and MediaStreamTrack in section 4. Is the WMS semantics 
>> providing the MediaStream IDs or the MediaStreamTrack IDs the SSRC is associated with?
>> The current text appears to indicate both. Is the ID making it 
>> possible to distinguish between the two types?
>

> but the same "appdata" token.

I don't think it can happen. It would require having the same MediaStreamTrack in two MediaStreams, and I don't think we can do that.

a=msid: 25 a x
a=msid: 25 b y
a=msid: 50 b w

it means that there are 2 MediaStreams (a and b) both carrying the media that's coming on SSRC 25 as a track, and one of these is also carrying the media coming on SSRC 50 as a track.
> appdata, they are NOT intended for the same MediaStreamTrack object.

That was certainly my intention. But as noted above, I think "same appdata" isn't meaningful.

> is currently ill-defined in the W3C API), even if the "appdata" token 

Even with the same source, the data might actually not be the same (if it has been transformed, reencoded or otherwise mangled). If it's the same data, it seems reasonable to let it be sent once.

---

From: Harald Alvestrand <[email-masked]>
Date: Sun, 17 Feb 2013 11:47:54 +0100

>    You say:  " I don't think it can happen. It would require having the same MediaStreamTrack in two MediaStreams, and I don't think we can do that".   When we discussed this in the past, we certai…

I think the current understanding is that a track is the container for 
some amount of state, including the muted state, and that when you 
addTrack() to a MediaStream, the state (including mute) of the new track 
is independent of the old track after that point.

createTrack() might indeed be a better name.

---

From: "Timothy B. Terriberry" <[email-masked]>
Date: Tue, 19 Feb 2013 07:57:51 -0800

> MediaStreamTrack in two MediaStreams, and I don't think we can do that.

I don't think it can happen if you're talking to a browser. But this is 
SDP, and the other end-point might not be a browser, but may be trying 
to control a browser's behavior. It can certainly generate SDP like 
this. We should say what that behavior will be.

> the media coming on SSRC 50 as a track.

What I actually meant was this:
a=msid: 25 a x
a=msid: 25 b y
a=msid: 50 a x

I.e., both SSRC 25 and SSRC 50 are contributing to the same 
MediaStreamTrack (for any reason... FEC, layering, simulcast, etc.) in 
the first MediaStream, but only the first SSRC is declared to contribute 
to a MediaStreamTrack in the second MediaStream.

> appdata" isn't meaningful.

I think that needs to be spelled out in the draft, including what "not 
meaningful" actually means.

> same data, it seems reasonable to let it be sent once.

I was looking at this from a receiver's perspective, not a sender's.

> MediaStream to have the same CNAME, so  this is just a subset of that -

That's a good point.

> draft should do it, since it's the "how do we map onto..." document.

That sounds reasonable to me.

> I think simulcast fits best as a within-one-track concept; representing

Yeah... there's arguments for both approaches. I haven't thought about 
it enough to decide which is better.

> That seems to make sense to me. But "someone else's problem".

It certainly doesn't need to be solved in the msid draft, but it does 
need to get solved somewhere.

---

From: Magnus Westerlund <[email-masked]>
Date: Thu, 21 Feb 2013 15:29:03 +0100

On 2013-02-14 21:36, Cullen Jennings (fluffy) wrote:
>> I think it is important that we are clear on that all media sources
>> that comes from the same synchronization context must be using the
>> same CNAME.
> 
> to come from the same synchronization context and that are desirable
> to synchronize
> 
> The issues I am concurred with is a device that is producing both
> audio and video from a camera and some video from a slide share and
> may be capable of sending the two such that the receiver can
> synchronize them but that may not to syncronize them due to the
> additional latency that would impose on the audio. I want to be able
> to send the audio and video with same CNAME and the slides with a
> different CNAME. I was assuming I would do that by putting the Track
> with the slides in one RTCMediaStream and the other Tracks in a
> different RTCMediaStream.
> 

Yes, I think the requirement on RTP level so to say is that media sender
SHALL provide synchronization information for each SSRC. The receiver
choses if they use it or not.

Cheers

Magnus Westerlund

---

From: Magnus Westerlund <[email-masked]>
Date: Thu, 21 Feb 2013 15:38:59 +0100

> SSRC-multiplexed simulcast) has a similar problem.

(As individual)

We are working on updating our simulcast draft so that it supports both
simulcast between RTP session as well as within in one.

As SRCNAME is not a finished and agreed concept and depends a bit on
what SRCNAME will end up. But for simulcast I think you are correct that
what needs to be identified is
end-point:synchronization-context:media-source so that one can determine
that this is one of potential multiple encodings of a given source.

When it comes to simulcast in WebRTC perspective I am especially worried
that choices might be made that would prevent including simulcast
without much effort in the future.

Harald wrote in another reply to your email:
> I think simulcast fits best as a within-one-track concept;
> representing it as multiple tracks means that the application has to
> do explicit manipulation of it, and that doesn't sound right to me.
> It should be one of those "just ask for it, and if you get it, it
> just works" things.

I would mostly agree with this, however, an application or the receiver
of the simulcasted encodings really needs to be able to express
preferences or requirements on what type of encodings that will be
produced. The question is how this is going to be done. Through the API,
through SDP, or some other way?

Cheers

Magnus Westerlund

---

From: "Timothy B. Terriberry" <[email-masked]>
Date: Thu, 21 Feb 2013 06:50:33 -0800

> without much effort in the future.

Agreed. That's why I'm trying to spend at least a little time thinking 
about it now, even if we don't expect to ship support for it in the 
first iteration of WebRTC. I look forward to your updated draft.