Subject: [rtcweb] Fwd: New Version Notification for draft-uberti-rtcweb-plan-00.txt

From: Justin Uberti <[email-masked]>
Date: Thu, 2 May 2013 23:08:43 -0700

Scribbled down the basic concepts of what I have been referring to as "Plan
B" - a way to signal multiple MediaStreamTracks in SDP without needing a
separate m= line for each.

Hopefully this will make discussion of this topic easier.

---------- Forwarded message ----------
From: <[email-masked]>
Date: Thu, May 2, 2013 at 10:46 PM
Subject: New Version Notification for draft-uberti-rtcweb-plan-00.txt
To: Justin Uberti <[email-masked]>

A new version of I-D, draft-uberti-rtcweb-plan-00.txt
has been successfully submitted by Justin Uberti and posted to the
IETF repository.

Filename:        draft-uberti-rtcweb-plan
Revision:        00
Title:           Plan B: a proposal for signaling multiple media sources in
WebRTC.
Creation date:   2013-05-03
Group:           Individual Submission
Number of pages: 15
URL:
http://www.ietf.org/internet-drafts/draft-uberti-rtcweb-plan-00.txt
Status:          http://datatracker.ietf.org/doc/draft-uberti-rtcweb-plan
Htmlized:        http://tools.ietf.org/html/draft-uberti-rtcweb-plan-00

Abstract:
   This document explains how multiple media sources can be signaled in
   WebRTC using SDP, in a fashion that avoids many common problems and
   provides a simple control surface to the receiver.

The IETF Secretariat

---

From: Harald Alvestrand <[email-masked]>
Date: Mon, 06 May 2013 16:19:48 +0200

Note - I regard the Plan A / Plan B conflict as the "long end of the 
pole" with regards to resolving issues regarding SDP signalling in 
RTCWEB - and as such, I regard it as very important to get it resolved.

The silence on this issue is disquieting.

            Harald

On 05/03/2013 08:08 AM, Justin Uberti wrote:
> Scribbled down the basic concepts of what I have been referring to as 
> "Plan B" - a way to signal multiple MediaStreamTracks in SDP without 
> needing a separate m= line for each.
>
> From: <[email-masked] <mailto:[email-masked]>>
> To: Justin Uberti <[email-masked] <mailto:[email-masked]>>
>
>
>
> Title:           Plan B: a proposal for signaling multiple media 
> sources in WebRTC.
> URL: http://www.ietf.org/internet-drafts/draft-uberti-rtcweb-plan-00.txt
> _______________________________________________
> rtcweb mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rtcweb

---

From: Christer Holmberg <[email-masked]>
Date: Tue, 7 May 2013 10:11:55 +0000

Hi,

A few questions on the draft:

Q1_CODEC/CODEC CONFIGURATION:

The examples only show one PT per m- line, and the draft seem to assume that all PTs (ie codecs and/or codec configurations) apply to all SSRCs within an m- line. I think that is very important to point out, because at least in my opinion it is an important factor that we need to discuss.

Example: I think Cullen often talks about using a camera with built in codec X. But, Plan B does not allow to (or, at least does not describe how to) explicitly offer codec X for the track/ssrc associated with the camera. Codec X is offered for ALL tracks/ssrcs associated with the m- line.

Q2_1_LEGACY:

The text says that things must work with legacy devices, and that an endpoint can choose a single source when communicating with a legacy device. But, there is really no text about how the legacy device will know which source to use, etc.

Q2_2_LEGACY:

The example in section 5.2 (and sub sections) confuses me a little. You indicate that the m- line itself represents "main", while ssrc attributes are used to describe the left/center/right mic sources. Why is there no ssrc attribute for the main flow?

Q3_NUMBER_OF_M_LINES:

The draft does not explicitly forbid the usage of multiple m- lines for a given media type (e.g. audio). It would probably be good to point that out also.

Regards,

Christer

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]] On Behalf Of Harald Alvestrand
Sent: 6. toukokuuta 2013 17:20
To: Justin Uberti
Cc: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: Re: [rtcweb] Fwd: New Version Notification for draft-uberti-rtcweb-plan-00.txt

Note - I regard the Plan A / Plan B conflict as the "long end of the pole" with regards to resolving issues regarding SDP signalling in RTCWEB - and as such, I regard it as very important to get it resolved.

           Harald

On 05/03/2013 08:08 AM, Justin Uberti wrote:
Scribbled down the basic concepts of what I have been referring to as "Plan B" - a way to signal multiple MediaStreamTracks in SDP without needing a separate m= line for each.

---------- Forwarded message ----------
From: <[email-masked]<mailto:[email-masked]>>
Date: Thu, May 2, 2013 at 10:46 PM
Subject: New Version Notification for draft-uberti-rtcweb-plan-00.txt
To: Justin Uberti <[email-masked]<mailto:[email-masked]>>

Title:           Plan B: a proposal for signaling multiple media sources in WebRTC.

Abstract:

The IETF Secretariat

rtcweb mailing list

[email-masked]<mailto:[email-masked]>

---

From: Harald Alvestrand <[email-masked]>
Date: Tue, 07 May 2013 12:39:01 +0200

On 05/07/2013 12:11 PM, Christer Holmberg wrote:
> The examples only show one PT per m- line, and the draft seem to 
> assume that all PTs (ie codecs and/or codec configurations) apply to 
> all SSRCs within an m- line. I think that is very important to point 
> out, because at least in my opinion it is an important factor that we 
> need to discuss.
>
> Example: I think Cullen often talks about using a camera with built in 
> codec X. But, Plan B does not allow to (or, at least does not describe 
> how to) explicitly offer codec X for the track/ssrc associated with 
> the camera. Codec X is offered for ALL tracks/ssrcs associated with 
> the m- line.
>

My read: This is a very special use case. The sender always chooses the 
codec from the available codecs that the recipient can understand - and 
it's then free to choose the built in codec X if supported by the recipient.

If, for some reason, the application is dependent on segregating the 
codecs, it can use the "a=content" mechanism to segregate the streams 
onto different M-lines.

> Q2_1_LEGACY:
>
> The text says that things must work with legacy devices, and that an 
> endpoint can choose a single source when communicating with a legacy 
> device. But, there is really no text about how the legacy device will 
> know which source to use, etc.
>

The application that drives the API will have to decide that it can send 
only one source. The browser cannot (in my opinion) sensibly make that 
choice on behalf of the application.

> Q2_2_LEGACY:
>
> The example in section 5.2 (and sub sections) confuses me a little. 
> You indicate that the m- line itself represents "main", while ssrc 
> attributes are used to describe the left/center/right mic sources. Why 
> is there no ssrc attribute for the main flow?
>

The MSID identifier is, by definition, meaningless. The association 
between the MSID and the function of the stream is not carried in the 
MSID attribute. The example is a bit unfortunate in that it uses MSIDs 
that look as if they're human-readable strings that carry semantics.

> Q3_NUMBER_OF_M_LINES:
>
> The draft does not explicitly forbid the usage of multiple m- lines 
> for a given media type (e.g. audio). It would probably be good to 
> point that out also.
>

In fact the a=content mechanism (see section 3.1) depends on the ability 
to use multiple m- lines for a given media type, so it would be really 
hard to interpret the draft as forbidding this.

There's an issue with representing the semantics of an incoming call 
that uses multiple m- lines for a given media type and doesn't give 
differing a=content attributes for those media types.
I don't know if such a scenario ever occurs in practice.
>
> Regards,
>
> Christer
>
> *From:[email-masked] <mailto:[email-masked]> 
> [mailto:[email-masked]] *On Behalf Of *Harald Alvestrand
> *Sent:* 6. toukokuuta 2013 17:20
> *To:* Justin Uberti
> *Cc:* [email-masked] <mailto:[email-masked]>; [email-masked] 
> <mailto:[email-masked]>
> *Subject:* Re: [rtcweb] Fwd: New Version Notification for 
> draft-uberti-rtcweb-plan-00.txt
>
>     Scribbled down the basic concepts of what I have been referring to
>     as "Plan B" - a way to signal multiple MediaStreamTracks in SDP
>     without needing a separate m= line for each.
>
>     [email-masked]  <mailto:[email-masked]>
>

---

From: Stefan Håkansson LK <[email-masked]>
Date: Tue, 7 May 2013 12:45:23 +0200

> The silence on this issue is disquieting.

I agree. Thinking out loud: could a meeting (joint mmusic and rtcweb I 
guess), that has as objective to discuss and decided which Plan to move 
forward with, be set up?

Then there would be concrete deadlines, and most likely things would not 
stay as quiet.

I guess it is a question for the chairs of those groups.

---

From: Christer Holmberg <[email-masked]>
Date: Tue, 7 May 2013 12:08:19 +0000

Hi,
 
>> The examples only show one PT per m- line, and the draft seem to assume that all PTs (ie codecs and/or codec configurations) apply to all SSRCs 
>> within an m- line. I think that is very important to point out, because at least in my opinion it is an important factor that we need to discuss.
>>
> Example: I think Cullen often talks about using a camera with built in codec X. But, Plan B does not allow to (or, at least does not describe how to) 
> explicitly offer codec X for the track/ssrc associated with the camera. Codec X is offered for ALL tracks/ssrcs associated with the m- line.
>
> My read: This is a very special use case. The sender always chooses the codec from the available codecs that the recipient can understand - and it's then free to choose the built in codec X if supported by the recipient.
>
> If, for some reason, the application is dependent on segregating the codecs, it can use the "a=content" mechanism to segregate the streams onto different M-lines.

- Assume I have two cameras, one which can handle codec X, and one which can handle codec Y. 

- I send you an offer with a video m- line, offering X, Y and a ssrc for each camera

- You only want one of my camera streams, AND you only support codec X. Now, how do you know which ssrc to choose? You don't know which uses codec X.

Now, maybe this is a theoretical problem, but I think the draft needs to describe that codecs, and codec configurations, apply to all ssrcs associated with an m- line.

And, IF there is a need to apply codecs to a specific ssrc only, you need to use a separate m- line for that ssrc.

>> Q2_1_LEGACY:
>> 
>> The text says that things must work with legacy devices, and that an endpoint can choose a single source when communicating with a 
>> legacy device. But, there is really no text about how the legacy device will know which source to use, etc.
>
> The application that drives the API will have to decide that it can send only one source. The browser cannot (in my opinion) sensibly make that choice on behalf of the application.

Fair enough, but that needs to be clearly described.

>> Q2_2_LEGACY:
>> 
>> The example in section 5.2 (and sub sections) confuses me a little. You indicate that the m- line itself represents "main", while ssrc 
>> attributes are used to describe the left/center/right mic sources. Why is there no ssrc attribute for the main flow?
>
> The MSID identifier is, by definition, meaningless. The association between the MSID and the function of the stream is not carried in 
> the MSID attribute. The example is a bit unfortunate in that it uses MSIDs that look as if they're human-readable strings that carry semantics.

I wasn't referring to the MSID identifier, but more to the fact that you don't indicate the ssrc value for the "main" flow. I thought we were going to mandate the indication of the ssrc value for each flow, but maybe I remember wrong.

> In fact the a=content mechanism (see section 3.1) depends on the ability to use multiple m- lines for a given media type, so it would be really hard to interpret the draft as forbidding this.

Fair enough. My mistake.

Regards,

Christer

There's an issue with representing the semantics of an incoming call that uses multiple m- lines for a given media type and doesn't give differing a=content attributes for those media types.
I don't know if such a scenario ever occurs in practice.

 
From: [email-masked] [mailto:[email-masked]] On Behalf Of Harald Alvestrand
Sent: 6. toukokuuta 2013 17:20
To: Justin Uberti
Cc: [email-masked]; [email-masked]
Subject: Re: [rtcweb] Fwd: New Version Notification for draft-uberti-rtcweb-plan-00.txt

           Harald

 
---------- Forwarded message ----------
From: <[email-masked]>
Date: Thu, May 2, 2013 at 10:46 PM
Subject: New Version Notification for draft-uberti-rtcweb-plan-00.txt
To: Justin Uberti <[email-masked]>

Abstract:

The IETF Secretariat

---

From: "Roni Even" <[email-masked]>
Date: Tue, 7 May 2013 15:51:40 +0300

Hi,

I think that plan B is the right direction. I am wondering if this should be discussed in RTCWEB or MMUSIC

I like the proposed solution for multicast support but there is also an individual draft in MMUSIC see http://tools.ietf.org/html/draft-westerlund-avtcore-rtp-simulcast-02 . I think that it will be good to have a separate document on simulcast support

Some comments

1.       In section 3.1 when talking about using more than one m-line for a media type I think that there are other cases when this may be required . for example, if an attribute is not specified at SSRC level or if the SDP will be simpler when using multiple m-line  )describing different codec combinations

2.       In section 3.2 the document says that collision will not happen if the answerer sees the SDP from the Offer. I am not sure it will work when joining a multipoint conference since the offer is joining an existing session and do not know what SSRC have been used. I think that the issue is handled in RFC5576 so you can point at it.

3.       The draft, as far as I can tell, is discussing point to point calls and full meshed conferences. I think we need to look at other topologies including a centralized MCU doing video switch, for example.

4.       In section 4.1 you suggest using the MSID as an indication for supporting plan B. I think that this may be RTCWEB specific and other usages like CLUE may use a different way to associate an SSRC to a Media Capture. My proposal is to use maxssrc. I think that it will be needed since an offer may include more options than can be really sent by the offerer and the maxssrc can tell the answerer how many it can select. It can also tell the answerer how many simultaneous RTP streams the offerer can receive. Otherwise it may be better to have a specific identifier that will tell that multiple RTP streams in an m-line are supported.

5.       I have a question about the last paragraph of section 4.2. Currently the answerer can start sending media immediately when getting the offer in parallel to sending the SDP answer since the offer includes receive capabilities and receive transport address. Are you suggesting that media will flow only after the 3 way handshake?

6.       Small nit on section 5.1.4, according to section 8 of the Lennox source selection draft, the third offer must have sending:on for the streams that are being sent so probably need a=ssrc:1 msid:left-mic sending:on

Roni Even

 

From: [email-masked] [mailto:[email-masked]] On Behalf Of Justin Uberti
Sent: 03 May, 2013 9:09 AM
To: [email-masked]; [email-masked]
Subject: [rtcweb] Fwd: New Version Notification for draft-uberti-rtcweb-plan-00.txt

---------- Forwarded message ----------
From: <[email-masked]>
Date: Thu, May 2, 2013 at 10:46 PM
Subject: New Version Notification for draft-uberti-rtcweb-plan-00.txt
To: Justin Uberti <[email-masked]>

Abstract:

The IETF Secretariat

  _____

---

From: Harald Alvestrand <[email-masked]>
Date: Tue, 07 May 2013 15:08:35 +0200

> - You only want one of my camera streams, AND you only support codec X. Now, how do you know which ssrc to choose? You don't know which uses codec X.

Right. That's the part that goes all artificial on me. Two cameras, and 
I'm going to choose blindly between them if I support both codecs, but I 
can't accept both and let you deal with sending me only the one I can 
handle?

You can, if you want to, send me two M-lines: "a=content:codec-x" and 
"a=content:codec-y". This is an example of "the application is dependent 
on segregating the codecs".

> And, IF there is a need to apply codecs to a specific ssrc only, you need to use a separate m- line for that ssrc.

Agreed.

> I wasn't referring to the MSID identifier, but more to the fact that you don't indicate the ssrc value for the "main" flow. I thought we were going to mandate the indication of the ssrc value for e…

I was trying to locate the flows in the example in section 5.2, and I 
couldn't find them anywhere but in the MSID attribute. So I couldn't 
parse your comment.

The way I read the document, the first m=audio line is the main audio; 
it has 3 audio flows.
The first m=video line is the main video; it has 3 video flows.
The third m=video line is the slides video; it has 2 SSRCs, one for the 
main flow and one for the repair flow.

So - the issue seems to be that the word "main" as in "main audio" is 
easy to mistake for "a single main flow". It isn't intended to be that 
way. I guess the text could be clearer.

---

From: Harald Alvestrand <[email-masked]>
Date: Tue, 07 May 2013 15:25:35 +0200

> should be discussed in RTCWEB or MMUSIC

Good question.

> I like the proposed solution for multicast support but there is also 
> an individual draft in MMUSIC see 
> http://tools.ietf.org/html/draft-westerlund-avtcore-rtp-simulcast-02 . 
> I think that it will be good to have a separate document on simulcast 
> support
>
> Some comments
>
> 1.In section 3.1 when talking about using more than one m-line for a 
> media type I think that there are other cases when this may be 
> required . for example, if an attribute is not specified at SSRC level 
> or if the SDP will be simpler when using multiple m-line  )describing 
> different codec combinations
>
> 2.In section 3.2 the document says that collision will not happen if 
> the answerer sees the SDP from the Offer. I am not sure it will work 
> when joining a multipoint conference since the offer is joining an 
> existing session and do not know what SSRC have been used. I think 
> that the issue is handled in RFC5576 so you can point at it.
>
> 3.The draft, as far as I can tell, is discussing point to point calls 
> and full meshed conferences. I think we need to look at other 
> topologies including a centralized MCU doing video switch, for example.
>

I'm not sure the multipoint conference scenario (using a mixer, 
transport relay or multicast) is in scope here. Those scenarios can't be 
done with offer/answer anyway.

With scenarios that do centralized call termination (and not using 
CSRC), doing per-leg SSRC allocation is reasonable.

> 4.In section 4.1 you suggest using the MSID as an indication for 
> supporting plan B. I think that this may be RTCWEB specific and other 
> usages like CLUE may use a different way to associate an SSRC to a 
> Media Capture. My proposal is to use maxssrc. I think that it will be 
> needed since an offer may include more options than can be really sent 
> by the offerer and the maxssrc can tell the answerer how many it can 
> select. It can also tell the answerer how many simultaneous RTP 
> streams the offerer can receive. Otherwise it may be better to have a 
> specific identifier that will tell that multiple RTP streams in an 
> m-line are supported.
>
> 5.I have a question about the last paragraph of section 4.2. Currently 
> the answerer can start sending media immediately when getting the 
> offer in parallel to sending the SDP answer since the offer includes 
> receive capabilities and receive transport address. Are you suggesting 
> that media will flow only after the 3 way handshake?
>
> 6.Small nit on section 5.1.4, according to section 8 of the Lennox 
> source selection draft, the third offer must have sending:on for the 
> streams that are being sent so probably need a=ssrc:1 msid:left-mic 
> sending:on
>
> Roni Even
>
> *From:[email-masked] [mailto:[email-masked]] *On 
> Behalf Of *Justin Uberti
> *Sent:* 03 May, 2013 9:09 AM
> *To:* [email-masked]; [email-masked]
> *Subject:* [rtcweb] Fwd: New Version Notification for 
> ------------------------------------------------------------------------
>
>
> mmusic mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/mmusic

---

From: Christer Holmberg <[email-masked]>
Date: Tue, 7 May 2013 17:03:21 +0000

Hi,
 
> way. I guess the text could be clearer.

That is how I understood it - that "main" is a flow by itself, separate from the left/center/right mic flows :)

Maybe "media description for main audio flows" would be a move clear description, rather than simply "main audio".

Regards,

Christer

---

From: Paul Kyzivat <[email-masked]>
Date: Tue, 07 May 2013 16:58:58 -0400

Jason,

(I removed the cross posting)

I started a thread in mmusic about "bundle splitting". It is predicated 
on the assumption that the offerer may propose to bundle some m-lines 
that the answerer can't handle if bundled together. But it could handle 
some smaller bundles of the same m-lines. With "Plan A" that should be 
enough to solve the problem.

With the Plan B described here, there is a variant of that. The offerer 
may offer an m-line with several SSRCs. The answerer wants those 
"flows", but may not be able to handle them in the same m-line. (It 
needs to receive them at different addresses, corresponding to separate 
hardware.)

In my earlier posting, I proposed that a subset of the bundled m-lines 
that can be received at a single address be accepted, and the others 
refused in the answer. And that then there be a new offer in the reverse 
direction, accepting those same m-lines, but bundling them separately. 
The assumption here is that they will still be identified as being 
associated with the same flows.

Would something analogous work with Plan B as you see it? The initial 
answerer would need to refuse SSRCs. Then it would need to send a new 
offer, with a separate m-line, and include something indicating the 
flows it wanted there. It could use the same MSIDs, but it can't 
necessarily use a=ssrc with the original SSRCs, because the new m-line 
is a separate SSRC space.

This may work, but the details elude me.

I also have a concern about use of ssrc-group for multicast. That only 
works if the all the members of the multicast group share the same 
m-line (or at least the same bundle.) That may not always be the case. 
When it does't, we need a more generalized grouping, that can reference 
specific SSRCs in different m-lines.

	Thanks,
	Paul

> From: Justin Uberti <[email-masked]>
> Date: Fri, May 3, 2013 at 1:08 AM
> To: "[email-masked]" <[email-masked]>, "[email-masked]" <[email-masked]>
>
>
> clue mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/clue
>

---

From: Bernard Aboba <[email-masked]>
Date: Tue, 7 May 2013 14:56:10 -0700

Roni said:  
3.       The draft, as far as I can tell, is discussing point to point calls and full meshed conferences. I think we need to look at other topologies including a centralized MCU doing video switch, for example.
[BA] There should be an evaluation against the RTP topology bis draft which would include assessment of compatibility.  In addition to "video switch" I would be interested in RTP translator compatibility as well. 

4.       In section 4.1 you suggest using the MSID as an indication for supporting plan B. I think that this may be RTCWEB specific and other usages like CLUE may use a different way to associate an SSRC to a Media Capture. My proposal is to use maxssrc. 

[BA] Among other things, putting maxssrc in the Offer partially addresses the issue of being able to receive streams before the Answer is received.  However, since the Offerer will not know the SSRCs that it is expected to receive there has got to be a way to map the incoming streams to WebRTC mediastreams.   

5.       I have a question about the last paragraph of section 4.2. Currently the answerer can start sending media immediately when getting the offer in parallel to sending the SDP answer since the offer includes receive capabilities and receive transport address. Are you suggesting that media will flow only after the 3 way handshake?
[BA] Yes, I think that is the implication, though it doesn't seem either necessary or desirable to me.  Actually, I don't think that "Peer B" will be able to send until it gets an Answer to its Offer (the second Offer/Answer exchange), because it won't know what streams "Peer A" is prepared to accept (assuming there is no maxssrc in the initial offer).

---

From: Bernard Aboba <[email-masked]>
Date: Tue, 7 May 2013 15:02:09 -0700

Roni said: 

I
            think that plan B is the right direction. I am wondering if
            this should be discussed in RTCWEB or MMUSIC

    To the extent that the proposals are attempts to "profile" existing RFCs or MMUSIC work in progress for use in WebRTC, it seems to me like the discussion belongs in RTCWEB. One advantage of that approach is that the solution need not necessarily apply to "all uses of SDP for any scenario for all time", just to the WebRTC use cases. 

To the extent that NEW SDP functionality is needed, MMUSIC discussion would be appropriate.  But until the general direction is decided, it won't be clear what the required new functionality is (if any).

---

From: "Roni Even" <[email-masked]>
Date: Wed, 8 May 2013 01:47:02 +0300

Hi Bernard,

The reason I asked was before plan A and plan B were discussed in MMUSIC in
Orlando. Yet I understand that if the purpose of this draft and Adam's one
are to describe how to do things in RTCweb it should be discussed in RTCWEB.
And we will need a similar one in CLUE probably in
draft-ietf-clue-rtp-mapping

Roni

From: Bernard Aboba [mailto:[email-masked]] 
Sent: 08 May, 2013 1:02 AM
To: Harald Alvestrand; Roni Even
Cc: [email-masked]; [email-masked]
Subject: RE: [rtcweb] [MMUSIC] Fwd: New Version Notification for
draft-uberti-rtcweb-plan-00.txt

Roni said: 

I think that plan B is the right direction. I am wondering if this should be
discussed in RTCWEB or MMUSIC

To the extent that the proposals are attempts to "profile" existing RFCs or
MMUSIC work in progress for use in WebRTC, it seems to me like the
discussion belongs in RTCWEB. One advantage of that approach is that the
solution need not necessarily apply to "all uses of SDP for any scenario for
all time", just to the WebRTC use cases. 

To the extent that NEW SDP functionality is needed, MMUSIC discussion would
be appropriate.  But until the general direction is decided, it won't be
clear what the required new functionality is (if any).

---

From: Bernard Aboba <[email-masked]>
Date: Tue, 7 May 2013 16:07:06 -0700

Roni said: 
"If the purpose of this draft and Adam’s one are to describe how to do things in RTCweb it should be discussed in RTCWEB. And we will need a similar one in CLUE probably in draft-ietf-clue-rtp-mapping."

[BA] There are RTP issues to discuss (e.g. draft-ietf-clue-rtp-mapping relationship to draft-ietf-rtcweb-rtp-usage), and there are SDP issues.  To date, I believe we have somewhat neglected the RTP discussion, which is unfortunate, because if all the time is spent talking about the SDP Hammer, then the inclination is to use it to pound every nail, and if there is no SDP, then we have no tools at all. 

The place for the RTP discussion is probably not in RTCWEB (more like AVTCORE).

---

From: Stefan Håkansson LK <[email-masked]>
Date: Wed, 8 May 2013 10:52:56 +0200

I think this could work. There are some parts that I like a lot (and, 
yes, I tend to have a narrow rtcweb centric perspective):

* The (3-way) handshake is well aligned with the API (where the sending 
app initiates the sending of media)

* That simulcast and layered encoding is supported

* The text about end-points ensuring that its SSRC's are unique - I 
think this enables mixers/translators to form arbitrary RTP sessions

* It is outlined how sending, or stop sending, is signaled per 
MediaStreamTrack

One thing I do not understand is why the imageattr is signaled in the 
sending direction in the examples. Is not this info anyway part of the 
bitstream and will become apparent during the decoding process? It would 
be more natural to signal imageattr from the receiver (e.g. the receiver 
telling the sender the size of the video display). (An alternative would 
be to not signal it at all but leave it to app proprietary signaling and 
the APIs available to the sending application).

I think that an alternative for simulcast could be that, instead of 
sending several resolutions for the same MediaStreamTrack, it was 
designed so that a separate MediaStreamTrack would have to be used (and 
the tracks would of course have to be synced) for each resolution. The 
primary reason is that the application can only really handle data down 
to the MediaStreamTrack level - there are no handles below that level in 
the API's currently.

I thing that strikes me is that perhaps we could lift out the msid 
signaling into its own block, that established the MediaStream and 
MediaStreamTrack structure of the outgoing media and how is relates to 
SSRC's.

The MediaStream-id/MediaStreamTrack-id info is only relevant to WebRTC 
endpoints and could be ripped out if the endpoint is legacy, and the 
remaining SDP would have fewer strange (to legacy) elements.

On 2013-05-03 08:08, Justin Uberti wrote:
> From: ** <[email-masked] <mailto:[email-masked]>>

---

From: Harald Alvestrand <[email-masked]>
Date: Wed, 08 May 2013 12:05:35 +0200

On 05/08/2013 01:07 AM, Bernard Aboba wrote:
> Roni said:
> "If the purpose of this draft and Adam’s one are to describe how to do 
> things in RTCweb it should be discussed in RTCWEB. And we will need a 
> similar one in CLUE probably in draft-ietf-clue-rtp-mapping."
>
> [BA] There are RTP issues to discuss (e.g. draft-ietf-clue-rtp-mapping 
> relationship to draft-ietf-rtcweb-rtp-usage), and there are SDP 
> issues.  To date, I believe we have somewhat neglected the RTP 
> discussion, which is unfortunate, because if all the time is spent 
> talking about the SDP Hammer, then the inclination is to use it to 
> pound every nail, and if there is no SDP, then we have no tools at all.

I felt that the discussion of draft-ietf-rtcweb-rtp-usage was largely 
uncontroversial in the Stockholm interim, and that not much further 
discussion of that level of the stack was needed - so I have not been 
worrying much about it after that.

It's more like we've got the RTP nail in place, and are hammering on it 
with the SDP hammer... if we were to use an axe instead (hopefully the 
back end of it), not much in RTP would change.

> The place for the RTP discussion is probably not in RTCWEB (more like 
I think we've already done most of the discussion needed in AVTCORE - we 
seem to have the dictum of "don't send audio and video on the same 
5-tuple" changed to "if you send audio and video on the same 5-tuple, 
here's what you need to consider", which was really the only critical 
*change* I saw in RTP-land.

---

From: Paul Kyzivat <[email-masked]>
Date: Wed, 08 May 2013 15:43:46 -0400

On 5/7/13 5:56 PM, Bernard Aboba wrote:
> and full meshed conferences. I think we need to look at other topologies
> including a centralized MCU doing video switch, for example.
>
>
> [BA] There should be an evaluation against the RTP topology bis draft
> which would include assessment of compatibility.  In addition to "video
> switch" I would be interested in RTP translator compatibility as well.
>
>
> usages like CLUE may use a different way to associate an SSRC to a Media
> Capture. My proposal is to use maxssrc.

*I* think it is important that clue and rtcweb to be able to coexist in 
a single endpoint, and for a clue+rtcweb client to interoperate with 
either another clue+rtcweb client or a pure clue client. Using different 
indicators in the two environments doesn't help with that.

I don't know what MSID would mean in a pure clue environment. If it can 
serve as an identifier for a clue capture-encoding, then maybe all is 
well. But I don't know that yet.

AFAIK maxssrc has its own meaning, independent of rtcweb or clue or Plan 
B. Conflating it to mean one of those things seems like a bad idea. I'll 
change my mind if Plan B ends up meaning nothing more that allowing 
multiple RTP streams within the m-line.

	Thanks,
	Paul

---

From: Ted Hardie <[email-masked]>
Date: Wed, 8 May 2013 15:01:47 -0700

On Wed, May 8, 2013 at 12:43 PM, Paul Kyzivat <

> *I* think it is important that clue and rtcweb to be able to coexist in a
> single endpoint, and for a clue+rtcweb client to interoperate with either
> another clue+rtcweb client or a pure clue client. Using different
So, can you unpack "coexist"?  A single endpoint could certainly have
either a clue client or an rtcweb client active at a single moment in
time--do you want to say that you want them both to be active and share the
same cameras, microphones, etc?

regards,

Ted

> serve as an identifier for a clue capture-encoding, then maybe all is well.
> But I don't know that yet.
>
> ______________________________**_________________
> rtcweb mailing list
> [email-masked]
> https://www.ietf.org/mailman/**listinfo/rtcweb<https://www.ietf.org/mailman/listinfo/rtcweb>
>

---

From: Paul Kyzivat <[email-masked]>
Date: Thu, 09 May 2013 11:32:12 -0400

On 5/8/13 6:01 PM, Ted Hardie wrote:
>     *I* think it is important that clue and rtcweb to be able to coexist
>     in a single endpoint, and for a clue+rtcweb client to interoperate
>     with either another clue+rtcweb client or a pure clue client. Using
>     different indicators in the two environments doesn't help with that.
>
>
> time--do you want to say that you want them both to be active and share
> the same cameras, microphones, etc?

What I mean is that I want the capability to *implement* a clue client 
in a browser, with the media going directly to the peer clue client (or 
MCU), whether the peer is also a browser or is a native sip clue 
implementation.

Since clue isn't done yet, we can impose some constraints on it to help 
with that. Within reason.

Clue will have its own control protocol, which we intend to define to 
run over DTLS/SRTP. It should be possible for a browser client to use an 
RTCWEB data channel for that.

And of course clue has needs for many media "flows" (captures in 
clue-speak), so those need to map directly onto MediaStreamTracks, or 
something like that in the webrtc world.

Whatever signaling is used to indicate bundling usage in SDP in RTCWEB 
should be consistent with what is used in clue. It would be unpleasant 
if we had to have a translator in the web server that must translate one 
SDP notation to another.

	Thanks,
	Paul

---

From: "Cullen Jennings (fluffy)" <[email-masked]>
Date: Thu, 9 May 2013 15:49:45 +0000

> Hopefully this will make discussion of this topic easier.

I think my largest questions was does an endpoint have to know when it carets an offer if that is going to be sent to a legacy end point or not. I'm particularly interested in multiscreen video conferences systems.

---

From: "Hutton, Andrew" <[email-masked]>
Date: Thu, 9 May 2013 18:43:09 +0000

ON: 09 May 2013 16:50 Cullen Jennings (fluffy) Wrote:

> 
> I think my largest questions was does an endpoint have to know when it
> carets an offer if that is going to be sent to a legacy end point or
> not. I'm particularly interested in multiscreen video conferences
> systems.
> 

The same question also seems relevant to the respun plan A given that the temptation will be to use a=bundleonly which greatly simplifies the offer (i.e. less confusion over candidates etc.) but at the cost of interoperability with legacy multiscreen systems.  Don't you agree?

Andy

---

From: Cullen Jennings <[email-masked]>
Date: Fri, 10 May 2013 11:10:53 -0600

On May 8, 2013, at 4:01 PM, Ted Hardie <[email-masked]> wrote:

> *I* think it is important that clue and rtcweb to be able to coexist in a single endpoint, and for a clue+rtcweb client to interoperate with either another clue+rtcweb client or a pure clue client. Using different indicators in the two environments doesn't help with that.
> 
> 
> So, can you unpack "coexist"?  A single endpoint could certainly have either a clue client or an rtcweb client active at a single moment in time--do you want to say that you want them both to be active and share the same cameras, microphones, etc?  
> 
> regards,
> 
> Ted

I want to be able to build a multi-screen telepresence systems that when it constructs an Offer, that offer could be going to a webrtc compliant browser, or it could be going to another CLUE complains endpoint. When it constructs the offer, it does not know where it going and based on the answer, the right thing will happen.

---

From: Cullen Jennings <[email-masked]>
Date: Fri, 10 May 2013 11:06:53 -0600

On May 8, 2013, at 2:52 AM, Stefan Håkansson LK <[email-masked]> wrote:

> * The (3-way) handshake is well aligned with the API (where the sending app initiates the sending of media)

Interesting - I saw it sort of the opposite so perhaps you can you say some more. I saw this as a complete change to everything we had discussed up to this point in how the API would work and how it would interact with Offer/Answer. I figured dealing with this change would set  back the W3C API work over 8 months so this is good news if you see a way that it is the well alighted. Can you explain how it lines up with the current drafts, call flows, existing example applications, and APIs that are all based on a 2 way hand shake?

---

From: Ted Hardie <[email-masked]>
Date: Fri, 10 May 2013 14:03:03 -0700

On Fri, May 10, 2013 at 10:10 AM, Cullen Jennings <[email-masked]> wrote:

>
>
> I want to be able to build a multi-screen telepresence systems that when
> it constructs an Offer, that offer could be going to a webrtc compliant
> browser, or it could be going to another CLUE complains endpoint. When it
> constructs the offer, it does not know where it going and based on the
> answer, the right thing will happen.

So, I think you have elided some of the steps in this process because you
presume we share an understanding.  I'm really serious that I want this
unpacked, because I think there's been a lot of talking past each other (no
doubt some of it from me)

What you've written says the offer is going to a "webrtc compliant
browser", but the webrtc system is a compound of a browser and a specific
downloaded javascript application.  Can you unpack whether this offer is
going to a downloaded javascript application *built to work with the
telepresence system*?  For example, a company manufacturing hardware
telepresence systems could market a general web-based conference system
that interworked with it-*if you used their specific downloaded javascript
app*.

If that's not meant to be sharing resources at the same time as other apps,
it's not very different than a browser than can load a Hangout or start a
Webex app now (You'll remember we actually even mixed those two apps at the
Stockholm interim--by not mixing the resources).   But if it works that
way, the question of how to detect whether the downloaded javascript
application supports CLUE or not seems to be answered--you know because you
know what app is downloaded, right?

If you mean interworking a CLUE telepresence system with an arbitrary
downloaded javascript app, I think that's just flat off the table--why
would pokerstars want to support CLUE?

I have the strong feeling I'm missing something obvious here, which would
make it great if we could speak in long, slow sentences with short words.

thanks,

Ted

---

From: Paul Kyzivat <[email-masked]>
Date: Fri, 10 May 2013 17:53:58 -0400

On 5/10/13 5:03 PM, Ted Hardie wrote:
> On Fri, May 10, 2013 at 10:10 AM, Cullen Jennings <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>
>
>     I want to be able to build a multi-screen telepresence systems that
>     when it constructs an Offer, that offer could be going to a webrtc
>     compliant browser, or it could be going to another CLUE complains
>     endpoint. When it constructs the offer, it does not know where it
>     going and based on the answer, the right thing will happen.
>
>
> So, I think you have elided some of the steps in this process because
> you presume we share an understanding.  I'm really serious that I want
> this unpacked, because I think there's been a lot of talking past each
> other (no doubt some of it from me)

Fair enough

> browser", but the webrtc system is a compound of a browser and a
> specific downloaded javascript application.  Can you unpack whether this
> offer is going to a downloaded javascript application *built to work
> with the telepresence system*?  For example, a company manufacturing
> hardware telepresence systems could market a general web-based
> conference system that interworked with it-*if you used their specific
> downloaded javascript app*.

It could be either.

CLUE is being designed to interop with "legacy" sip endpoints. A clue 
client will send an offer that will downgrade to vanilla SIP if the 
other end doesn't understand the clue specific stuff.

But the interesting case is where the web server for the target intends 
to support clue - providing suitable javascript so that the browser can 
do the clue stuff.

> If that's not meant to be sharing resources at the same time as other
> apps, it's not very different than a browser than can load a Hangout or
> start a Webex app now (You'll remember we actually even mixed those two
> apps at the Stockholm interim--by not mixing the resources).   But if it
> works that way, the question of how to detect whether the downloaded
> javascript application supports CLUE or not seems to be answered--you
> know because you know what app is downloaded, right?

The issue is the SDP stuff that is wired into the rtcweb browser 
machinery. Specifically, if rtcweb has one way to indicate the use of 
plan B, via MSID that identifies a MediaStreamTrack, and clue uses 
something else to indicate use of plan B and to bind a capture-encoding 
to an SSRC, then we have a problem.

Maybe the answer is that CLUE needs to use MSIDs. But we first need to 
ensure that the semantics are right. Clue doesn't have media stream 
tracks. Maybe we will sort out the equivalences via the taxonomy, but 
that is moving along very well.

I would hate to require the web server to be getting SDP from the 
browser and having to translate it from rtcweb-dialect-sdp to 
clue-dialect-sdp.

> would pokerstars want to support CLUE?

Only in the "legacy" sense I mentioned above, and then only if the 
javascript is intended to support legacy sip.

> I have the strong feeling I'm missing something obvious here, which
> would make it great if we could speak in long, slow sentences with short
> words.

Does the above help?

---

From: Ted Hardie <[email-masked]>
Date: Fri, 10 May 2013 15:46:15 -0700

Inline.

>> downloaded javascript app*.
I'm sorry, either what?  Either a purpose-built javascript application
or....?

> client will send an offer that will downgrade to vanilla SIP if the other
> end doesn't understand the clue specific stuff.
>
>
Should I infer that one way forward here is for a clue client to downgrade
to vanilla sip while speaking to gateway that has a webrtc client on the
other side?

> But the interesting case is where the web server for the target intends to
> support clue - providing suitable javascript so that the browser can do the
> clue stuff.
>
>
>
Okay, in this case the web server knows what the javascript application can
do.  This means, if I understand it, that there is no problem (*for this
case*) in determining which endpoints support clue--because the webserver
should known.  Is that right?

> machinery. Specifically, if rtcweb has one way to indicate the use of plan
> B, via MSID that identifies a MediaStreamTrack, and clue uses something
> else to indicate use of plan B and to bind a capture-encoding to an SSRC,
> then we have a problem.
>
>
But this seems entirely separate from the point I thought kicked this off,
when you said:

"I* think it is important that clue and rtcweb to be able to coexist in a
single endpoint, and for a clue+rtcweb client to interoperate with either
another clue+rtcweb client or a pure clue client. Using different
indicators in the two environments doesn't help with that."

If I understand here is the clue-capable javascript application doesn't
have to coexist in any meaningful sense with a non-clue rtcweb javascript
application---it gets downloaded into a browser by a web server for the
target.  It's not coexisting with a different *active* client in that case,
even if the browser could or has downloaded some other javascript for a
different site.

> ensure that the semantics are right. Clue doesn't have media stream tracks.
> Maybe we will sort out the equivalences via the taxonomy, but that is
> moving along very well.
>
> I would hate to require the web server to be getting SDP from the browser
> and having to translate it from rtcweb-dialect-sdp to clue-dialect-sdp.
>

But if the downloaded client speaks CLUE, why would it?

> I think that's the gateway case I mentioned above,

>> would make it great if we could speak in long, slow sentences with short
I guess I won't know until I see your and Cullen's answers to some of the
above.

Ted

---

From: Stefan Håkansson LK <[email-masked]>
Date: Sat, 11 May 2013 12:20:11 +0000

On 5/10/13 8:44 PM, Cullen Jennings wrote:
>
> On May 8, 2013, at 2:52 AM, Stefan Håkansson LK
> <[email-masked]> wrote:
>
>> * The (3-way) handshake is well aligned with the API (where the
>> sending app initiates the sending of media)
>
> Interesting - I saw it sort of the opposite so perhaps you can you
> say some more.

Absolutely. The API (as of now) is aligned towards sending media. You'd 
construct a PC-stream containing PC-tracks which in turn corresponds to 
microphones and cameras. If you want to send that PC-stream to a peer 
you do "addStream" with it on a PeerConnection. If you want to add or 
remove a PC-track, there are operations for that - and it would be 
reflected on the far end.

The point is that the API is about sending - not about setting up a 
session with a certain amount of ougoing and incoming PC-tracks, and to 
me that harmonizes very well with the handshake in 
draft-uberti-rtcweb-plan-00 which has separate O/A exchanges for media 
A->B and B->A respectively. An additional benefit seems to be that those 
two exchanges are somewhat independent and would not cause glare. This 
is denoted as a 3-way handshake in the draft, that is a bit misleading 
perhaps.

I think this is actually proposed in the JSEP draft as well, look at the 
second part of 
http://tools.ietf.org/html/draft-ietf-rtcweb-jsep-03#section-4.2.3.1

> I saw this as a complete change to everything we had
> discussed up to this point in how the API would work and how it would
> interact with Offer/Answer. I figured dealing with this change would
> set  back the W3C API work over 8 months so this is good news if you
> see a way that it is the well alighted. Can you explain how it lines
> up with the current drafts, call flows, existing example
> applications, and APIs that are all based on a 2 way hand shake?

As I read draft-uberti-rtcweb-plan-00, no API changes are needed or 
proposed, and the call flows could stay the same (but as I recollect the 
discussion at the Boston interim they should be changed for other reasons).

---

From: Eric Rescorla <[email-masked]>
Date: Sat, 11 May 2013 07:14:07 -0700

> discussion at the Boston interim they should be changed for other reasons).

I wanted to address this point in isolation...

I believe that both Plan A and Plan B require at least one change to
the API: the ability to indicate that a stream is important and thus
requires backward-compatible treatment. In Plan A this means
not marking it bundle-only. In Plan B it means putting it on its own
m-line.

-Ekr

---

From: Paul Kyzivat <[email-masked]>
Date: Sat, 11 May 2013 12:51:12 -0400

I found what Stefan said below to be interesting:

> is denoted as a 3-way handshake in the draft, that is a bit misleading

A very similar situation holds for CLUE.

Maybe this is a more fundamental change than we have been thinking.

O/A works best when both sides have similar capabilities and desires. 
(It makes sense to set up to receive what I hope to receive, because it 
is likely that you will want to send that.) It works poorly when you and 
I might have a highly eclectic set of stuff we could send, and 
unpredictable interests for what we are interested in receiving.

In clue we are addressing that by exchanging (in a separate channel) 
advertisements of what the endpoints are willing to send, and only doing 
O/A to set up the media that is of interest.

In rtcweb it isn't so structured, but I suspect something is happening 
implicitly, though it may be more hard-wired per application.

In clue we may still end up negotiating separate one-way streams in each 
direction, similar to what Stefan mentions below. With bundling there 
isn't a big penalty for that (e.g., in ports and RTCP traffic) though 
there is an extra O/A required.

But for both RTCWEB and CLUE there are cases where we must negotiate 
bidirectional streams for compatibility with legacy devices. So there 
must be a way to do that.

I don't have a specific action to recommend here. This just seems like a 
somewhat fundamental shift that out to be recognized. It probably isn't 
just RTCWEB and CLUE, it is really related to more complex communication 
scenarios. ISTM that CLUE is addressing this by building a layer on top 
of O/A, while RTCWEB is *battling* with O/A.

	Thanks,
	Paul

---

From: Bernard Aboba <[email-masked]>
Date: Sat, 11 May 2013 14:08:28 -0700

> of O/A, while RTCWEB is *battling* with O/A.
[BA] IMHO, the distinction between CLUE and RTCWEB may not be that clear cut, and so the issues (and considerations) that arise in CLUE may also come up in RTCWEB, although they may not be recognized as such. 
For example, within the use case document the scenario in Section 4.2.11 (Hockey Game Viewer) refers to use of multiple cameras on a single host.  Clearly it is possible for a host to have multiple cameras as well as multiple screens attached, and existing WebRTC implementations are capable of making use of those capabilities, though perhaps not to the full extent envisioned in CLUE. 
So while I do agree with Ted Hardie when he said "If you mean interworking a CLUE telepresence system with an arbitrary downloaded _javascript_ app, I think that's just flat off the table--why would pokerstars want to support CLUE?", at the same time if you asked me what part of the functionality described in CLUE was out-of-scope of WebRTC based on the use case document,  I might not be able to come up with all that much (e.g. exchange and use of spatial information).  
This is why I find it disturbing for WebRTC and CLUE RTP usage documents to come to different conclusions about required RTP functionality *for similar scenarios*.  For example, I do not think that the concept of MSID and CLUE capture are all that different, so that I do not buy that supporting each of these concepts require different solutions. So if we are debating whether an RTP extension (or SDES item) is needed to indicate the CLUE capture to which an RTP flow pertains, then I would assert that the same logic would apply (or not apply)  to msid. 
It is one thing to make different design decisions based on aspects of a scenario which are fundamentally different than another scenario.  It's another thing to make different design decisions purely because the two scenarios are handled in different WGs.

---

From: Emil Ivov <[email-masked]>
Date: Sun, 12 May 2013 12:58:57 +0300

> of O/A, while RTCWEB is *battling* with O/A.

That's a nice way to put it. Interestingly the CLUE approach to take
this out of O/A seems to be more in line with the RTCWEB paradigm than
both Plan A or Plan B.

The decision not to implement an official signalling RTCWEB protocol was
taken very early in this working group and there was very strong
consensus on the fact that imposing a specific signalling protocol would
be incompatible with the web in general.

Still, it seems to me that we are now trying to compensate for the  lack
of such a signalling protocol by piggybacking on top of O/A and SDP with
things such as the possibility to turn off individual SSRCs.

Why do we need these things? Aren't they better handled by the API?

Emil

-- 
https://jitsi.org

---

From: Stefan Håkansson LK <[email-masked]>
Date: Sun, 12 May 2013 10:37:40 +0000

> of O/A, while RTCWEB is *battling* with O/A.

I don't know much about Clue, but I find the above being a good summary 
for rtcweb, however I would perhaps say "application specific" rahter 
than "hard-wired per application"; and perhaps it is not a "battle" but 
rather "sorting out" that is ongoing regarding O/A.

---

From: Stefan Håkansson LK <[email-masked]>
Date: Sun, 12 May 2013 17:05:01 +0000

On 5/11/13 4:14 PM, Eric Rescorla wrote:
>
>
> On Sat, May 11, 2013 at 5:20 AM, Stefan Håkansson LK
> <[email-masked]
> <mailto:[email-masked]>> wrote:
>

I think you're right. Stefan

---

From: Justin Uberti <[email-masked]>
Date: Sun, 12 May 2013 22:22:01 -0700

> The sole API change that is proposed is the new .content property, to
allow a MST to have its a=content property set, and as such be placed on
its own m= line. The rest works using the existing APIs; the necessary
a=remote-ssrc lines can be generated (to enable or disable a stream) based
on whether a received MST is hooked up to an output or not.

---

From: "Roni Even" <[email-masked]>
Date: Mon, 13 May 2013 12:55:06 +0300

Hi,
I do not think that CLUE is much different from RTCweb. Like Bernard said
the major usage of the CLUE channel is the support for the spatial
information.
CLUE is not replacing SDP o/a.  In CLUE there is still a need to have an SDP
that describe multiple RTP streams from the same media type multiplexed in a
single RTP session. There is also a need to map these streams to a media
capture in CLUE. The CLUE specific requirements stem from the need to
support different RTP topologies
(http://tools.ietf.org/html/draft-ietf-avtcore-rtp-topologies-update-00) . 
I am aware that there was no conclusion in the Stockholm interim meeting
about the need to support the different topologies for WebRTC

Roni Even

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Emil Ivov
Sent: 12 May, 2013 12:59 PM
To: Paul Kyzivat
Cc: [email-masked]
Subject: Re: [rtcweb] We are moving beyond the assumptions on which O/A is
based

On 11.05.13, 19:51, Paul Kyzivat wrote:
> I don't have a specific action to recommend here. This just seems like 
> a somewhat fundamental shift that out to be recognized. It probably 
> isn't just RTCWEB and CLUE, it is really related to more complex 
> communication scenarios. ISTM that CLUE is addressing this by building 
> a layer on top of O/A, while RTCWEB is *battling* with O/A.

That's a nice way to put it. Interestingly the CLUE approach to take this
out of O/A seems to be more in line with the RTCWEB paradigm than both Plan
A or Plan B.

The decision not to implement an official signalling RTCWEB protocol was
taken very early in this working group and there was very strong consensus
on the fact that imposing a specific signalling protocol would be
incompatible with the web in general.

Still, it seems to me that we are now trying to compensate for the  lack of
such a signalling protocol by piggybacking on top of O/A and SDP with things
such as the possibility to turn off individual SSRCs.

Emil

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 13 May 2013 09:42:45 -0400

> the major usage of the CLUE channel is the support for the spatial

Obviously I also think there is a lot of overlap.
But they are not the same thing. CLUE needs to work in situations where 
rtcweb doesn't apply, as well as those where it does. So I have been 
trying to ensure rtcweb ends up defining mechanisms that "work" for 
clue, without excessively bogging down rtcweb with clue-specific concerns.

> about the need to support the different topologies for WebRTC

IMO CLUE will end up being, in effect, a general mechanism for 
advertising and selecting desired real-time media content, that has been 
specialized for telepresence. Once it is done, I expect it will be 
possible to make relatively minor changes to it in order to specialize 
it to other application areas.

As to why we need to define a protocol for this, rather than just an 
API, the simple answer is that IETF doesn't define APIs, so if we want 
to define it we need to make it a protocol. Another answer is that the 
API approach being pursued by webrtc/rtcweb works primarily because the 
assumption underlying that is that all the communicating endpoints are 
being mediated by a single (web) server. In cases where that is not true 
a protocol will still be required.

	Thanks,
	Paul

---

From: Mary Barnes <[email-masked]>
Date: Mon, 13 May 2013 11:44:56 -0500

Responses inline below [MB].

Mary.

On Fri, May 10, 2013 at 4:53 PM, Paul Kyzivat <[email-masked]> wrote:
> CLUE is being designed to interop with "legacy" sip endpoints. A clue client
> will send an offer that will downgrade to vanilla SIP if the other end
> doesn't understand the clue specific stuff.
>
> The issue is the SDP stuff that is wired into the rtcweb browser machinery.
> Specifically, if rtcweb has one way to indicate the use of plan B, via MSID
> that identifies a MediaStreamTrack, and clue uses something else to indicate
> use of plan B and to bind a capture-encoding to an SSRC, then we have a
> problem.
>
> Maybe we will sort out the equivalences via the taxonomy, but that is moving
> along very well.
[MB] I think you meant to say"..but that is NOT moving along very
well."  I do not believe we
can make any statements as to how CLUE identifiers map to MSIDs
without doing that
taxonomy.  Of course, given that this discussion is happening entirely
on the RTCWEB mailing
list as opposed to MMUSIC or even AVTEXT, which is where we've agreed
that the taxonomy
work will happen, means that we don't have the right people to make
this decision nor is any
conclusion that might occur on this list relative to CLUE and RTCWEB
meaningful in the CLUE context.
I would love it if all the CLUE principles/document authors were on
the RTCWEB mailing list and were
following these discussions, but honestly given the volume of email on
the RTCWEB list, it's not
reasonable to expect such.  What is reasonable is for folks to take
these threads of discussion to
the MMUSIC and AVT related mailing lists.  [/MB]
> and having to translate it from rtcweb-dialect-sdp to clue-dialect-sdp.
[MB] As an individual, I think that it will be even worse than that,
as ISTM that some of the information that
CLUE might be sending in the CLUE channel will be sent in SDP in the
RTCWEB context. [/MB]

---

From: Bernard Aboba <[email-masked]>
Date: Mon, 13 May 2013 10:10:26 -0700

> [MB] I think you meant to say"..but that is NOT moving along very well."  I do not believe we can make any > statements as to how CLUE identifiers map to MSIDs without doing that taxonomy. 
[BA] MSID has some aspects similar to CLUE identifiers, but it is also trying to do some things that are WebRTC-specific.  This leads me to wonder whether in fact those two aspects couldn't be separated.  If an onaddstream event is thrown when a new SSRC is received, the event handler could handle WebRTC-specific aspects.  If this were done, it seems to me that we might be able to utilize the concept of CLUE capture in both CLUE and WebRTC, and might also get some signaling simplifications in the process.  
> Of course, given that this discussion is happening entirely> on the RTCWEB mailing list as opposed to MMUSIC or even AVTEXT, > which is where we've agreed that the taxonomy
> this decision nor is any conclusion that might occur on this list relative > to CLUE and RTCWEB meaningful in the CLUE context.
[BA] While I sympathize with the desire to try to segregate discussions by topic and WG, the reality is that this particular WebRTC topic is just one "inter-wingled" big ball of string, with inter-related RTP/RTCP and SDP discussions co-mingling with WebRTC and CLUE-specific considerations.  While I know that the orthodox response is that "we don't do architecture (well) in the IETF" if there was ever an argument for a wholistic discussion, it would probably be in a case such as this. 
> I would love it if all the CLUE principles/document authors were on> the RTCWEB mailing list and were following these discussions, but > honestly given the volume of email on the RTCWEB list, it's not
> these threads of discussion to the MMUSIC and AVT related mailing lists.  [/MB]
[BA] While it might be reasonable for folks to take easily characterized issues to the MMUSIC or AVTCORE mailing lists, I am not sure that discussions over the relative roles of RTP/RTCP and SDP or CLUE capture vs. MSID can be handled on individual lists.  Also, there may be a distinction between what is "reasonable" and what is likely to be "productive" (although I'm not sure we have enough empirical evidence yet to understand what approaches are likely to fall in the latter bucket).  
> as ISTM that some of the information that CLUE might be sending in > the CLUE channel will be sent in SDP in the RTCWEB context. [/MB]
[BA] Or even "worser" than that, some information that CLUE implementations might send in RTP or RTCP will be sent in SDP in the RTCWEB context.

---

From: Martin Thomson <[email-masked]>
Date: Mon, 13 May 2013 13:24:57 -0700

On 11 May 2013 14:08, Bernard Aboba <[email-masked]> wrote:
> [BA] IMHO, the distinction between CLUE and RTCWEB may not be that clear
> cut, and so the issues (and considerations) that arise in CLUE may also come
> up in RTCWEB, although they may not be recognized as such.

I think that the following statement - if true - would be key:

  WebRTC defines an API that I can use to implement CLUE.

I would like that statement to be true.

That is why I find the following just as disturbing as Bernard:

> This is why I find it disturbing for WebRTC and CLUE RTP usage documents to
> come to different conclusions about required RTP functionality *for similar
> scenarios*.  For example, I do not think that the concept of MSID and CLUE
> capture are all that different, so that I do not buy that supporting each of
> these concepts require different solutions. So if we are debating whether an
> RTP extension (or SDES item) is needed to indicate the CLUE capture to which
> an RTP flow pertains, then I would assert that the same logic would apply
> (or not apply)  to msid.

---

From: Martin Thomson <[email-masked]>
Date: Mon, 13 May 2013 13:35:52 -0700

On 12 May 2013 22:22, Justin Uberti <[email-masked]> wrote:
> The sole API change that is proposed is the new .content property, to allow
> a MST to have its a=content property set, and as such be placed on its own
> m= line. The rest works using the existing APIs; the necessary a=remote-ssrc
> lines can be generated (to enable or disable a stream) based on whether a
> received MST is hooked up to an output or not.

[...]
> not marking it bundle-only. In Plan B it means putting it on its own

The .content property in Plan B goes some way to addressing this need,
but it does nothing to provide control over which of the tracks are
selected from the set that are on the same m-line in the
backwards-compatibility mode.

Neither option makes addressing simulcast streams easy for this case.

---

From: Justin Uberti <[email-masked]>
Date: Mon, 13 May 2013 17:31:17 -0700

> I would like that statement to be true.

That is also how I have been looking at the situation.

---

From: Justin Uberti <[email-masked]>
Date: Mon, 13 May 2013 17:37:00 -0700

> Why do we need these things? Aren't they better handled by the API?

The same could be said for anything currently handled by SDP, it could be
better handled by a JS-friendly API. I don't see anything special about
turning on individual streams though.

IOW, if SDP is the surface we use to communicate information to the other
side, then it is de facto the best fit for these stream requests.

---

From: Emil Ivov <[email-masked]>
Date: Tue, 14 May 2013 08:56:53 +0300

On 14.05.13, 03:37, Justin Uberti wrote:
> 
> On Sun, May 12, 2013 at 2:58 AM, Emil Ivov <[email-masked]
> <mailto:[email-masked]>> wrote:
> 
> The same could be said for anything currently handled by SDP, it could
> be better handled by a JS-friendly API.

Right, and we chose to use SDP O/A, in spite of its shortcomings, for
interoperability reasons.

> I don't see anything special
> about turning on individual streams though.

The difference is that this is no longer a choice driven by
interoperability. All those applications that we wanted WebRTC clients
to easily talk to and that we sometimes refer to as legacy or
Pre-WebRTC, well they rarely support individual stream control. Those
that do, implement it without using SDP.

> IOW, if SDP is the surface we use to communicate information to the
> other side, then it is de facto the best fit for these stream requests.

SDP is not going to be the only surface we use to communicate
information to the other side. We'll always have another layer of
WebSocket signalling and we'll always have RTCP.

Emil

-- 
https://jitsi.org

---

From: Stefan Håkansson LK <[email-masked]>
Date: Tue, 14 May 2013 10:43:52 +0200

On 2013-05-13 19:10, Bernard Aboba wrote:
>  > [MB] I think you meant to say"..but that is NOT moving along
> very well." I do not believe we can make any
>  > statements as to how CLUE identifiers map to MSIDs without doing
> that taxonomy.
>
> [BA] MSID has some aspects similar to CLUE identifiers, but it is also
> trying to do some things that are WebRTC-specific. This leads me to
> wonder whether in fact those two aspects couldn't be separated. If an
> onaddstream event is thrown when a new SSRC is received, the event
> handler could handle WebRTC-specific aspects.  If this were done, it
> seems to me that we might be able to utilize the concept of CLUE capture
> in both CLUE and WebRTC, and might also get some signaling
> simplifications in the process.

As it looks like now, onaddstream would not be fired as a result of a 
new SSRC being received, but rather as a result of an SDP (offer or 
answer) informing the browser of the new SSRC. (Of course there is an 
escape mechanism that handles this if the offer or answer contains no 
SSRC info to be able to handle legacy).

I think signaling of some kind is needed - how would the browser 
otherwise know whether the new SSRC represents a new MediaStream, a new 
MediaStreamTrack (in an existing MediaStream) or just e.g. FEC data for 
an already set up track?

But, exactly how the signaling is made can be discussed (and I guess 
that is what we're doing now). And, I am always fond of simplifications!

A perhaps very naive thought: could we separate things up a bit? E.g. 
something like:

* Description and negotiation of codecs, PTs, SSRCs - this would be SDP
* Description of how those SSRCs map to a MediaStream+MediaStreamTrack 
structure // Clue captures
* Handling of in session changes (pause/resume, resolution changes)

Perhaps not all of the above should be done using SDP O/A, and perhaps 
we should not push all the functionality into the "core" SDP.

---

From: Emil Ivov <[email-masked]>
Date: Tue, 14 May 2013 12:23:12 +0300

> SSRC info to be able to handle legacy).

Hopefully at some point we'd stop dismissing non-reliance on SDP for
every single thing as "legacy".

> MediaStreamTrack (in an existing MediaStream) or just e.g. FEC data for 

If I understand the MSID draft and Harald's mail correctly, everything
would be bubbled up as separate streams and tracks and it will be up to
the application to handle them and sync them one way or another.
(Keeping in mind that syncing can also be auto-handled by CNAMEs)

I believe that option (which works with no default signalling) is one
good option. Having a second option that implements a default signalling
solution is also a good idea as long as we don't consider that to be the
"right way" and refer to the former as the "legacy" catch-all.

> * Description and negotiation of codecs, PTs - this would be SDP

Right!

> SSRCs - this would be SDP

Again, including SSRCs in SDP O/A is tricky in conferencing scenarios.

> we should not push all the functionality into the "core" SDP.

Thank you! :)

Emil

-- 
https://jitsi.org

---

From: Stefan Håkansson LK <[email-masked]>
Date: Tue, 14 May 2013 13:44:32 +0200

> Again, including SSRCs in SDP O/A is tricky in conferencing scenarios.

Here seems to be something that I have missed. That there are existing 
implementations that do not signal SSRC is fine, and in most cases there 
would be only one audio and one video SSRC used anyway. And rtcweb (and 
Clue?) should be able to interoperate (but perhaps your web app need to 
be designed in a specific way).

But if we design something forward looking, that can handle many 
streams, that can handle layered codecs and FEC data, signaling SSRCs 
seems like a must to me.

But I have missed that this is problematic in conferencing scenarios, 
could you tell me why (and I probably should know :) )?

---

From: Ted Hardie <[email-masked]>
Date: Tue, 14 May 2013 09:38:39 -0700

> implicitly, though it may be more hard-wired per application.

So, I repeat what I said before, I think you can get to there in RTCWEB,
but it won't be the same way pokerstars gets to where it wants to go.

The simplest thing I can imagine, to just put it out there, is to start
RTCWEB with just a data channel, pass the clue messages on top, then start
the media channels after that's done.  That seems to be reasonably in line
with what you've described above and it's well within the RTCWEB APIs.

What won't happen is for *every* application to adopt that procedure; some
will want behaviour that looks closer to vanilla offer/answer; some will
want behaviour that is more-or-less directed by the server, and some will
want blueberry jam (that is, something we don't know about yet).

regards,

Ted

---

From: Paul Kyzivat <[email-masked]>
Date: Tue, 14 May 2013 14:34:20 -0400

On 5/14/13 12:38 PM, Ted Hardie wrote:
> On Sat, May 11, 2013 at 9:51 AM, Paul Kyzivat <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>     advertisements of what the endpoints are willing to send, and only
>     doing O/A to set up the media that is of interest.
>
>     In rtcweb it isn't so structured, but I suspect something is
>     happening implicitly, though it may be more hard-wired per application.
>
>
> RTCWEB with just a data channel, pass the clue messages on top, then
> start the media channels after that's done.  That seems to be reasonably
> in line with what you've described above and it's well within the RTCWEB
> APIs.

I expect the same thing to happen with rtcweb as without it:

An initial offer proposes a data channel and some initial media.

If the other end also supports clue, then it will accept the data 
channel, and some or all of the media, as it sees fit, given that it 
doesn't know what the purpose of that media is yet. (E.g. the media 
might consist of one audio and one video. But it also might be 
optimistic and assume the other end is configured just like it is.)

Then the two sides will exchange clue advertisements over the data channel.

Based on the advertisements, the two sides will then decide what they 
want to receive, and send clue configure messages describing that.

Before or after the configuration messages, if necessary, a new O/A (or 
two) may be exchanged to arrange the media to be consistent with the 
advertisements and configurations.

> What won't happen is for *every* application to adopt that procedure;
> some will want behaviour that looks closer to vanilla offer/answer; some
> will want behaviour that is more-or-less directed by the server, and
> some will want blueberry jam (that is, something we don't know about yet).

Certainly. If the web server supporting the browser does't support clue 
then this isn't going to result in a clue call. But that would only come 
up for *received* calls, and receiving such a call via rtcweb depends on 
having an app that is at least addressable by a clue client. That means 
it must have a sip address. If so, then at least it will probably be 
able to connect as a "legacy" sip device.

My concern it that it be *possible* to do this with rtcweb, preferably 
obscure gateway code.

	Thanks,
	Paul

---

From: Emil Ivov <[email-masked]>
Date: Tue, 14 May 2013 22:46:07 +0300

Hey Stefan,

> could you tell me why (and I probably should know :) )?

The issue that I am referring to (I describe this in an earlier mail to
Harald here http://goo.gl/M8NbQ ) is that of a conference based on an
RTP translator. It's basically what we do in Jitsi Videobridge.

Let's say that I am a WebRTC app that can act as a conference focus and
that has access to an RTP translator somewhere. I'd like to organise a
conference with 10 people (you and I among them).

I ask the RTP translator to allocate 10 ports for me and then I start
calling participants.

You are the first one that I am going to join into this new call.

At this point, the only SSRC(s) that I could possibly give you are those
that I am going to use. I won't be able to tell you anything for the
eight other participants. For all I know, some or all of these
participants could turn out to be RTP translators themselves and there
could be a number of SSRCs behind each of them too.

The very logical and easy thing that your webapp could do here is not to
care and simply render each and every SSRC that you get. Most apps are
going to be perfectly happy with that.

If, on the other hand you needed me to re-offer you every time a new
participant joins in order to work properly, then things would easily
get quite messy even if everyone is using WebRTC.

Then, note that some of the participants can be simple SIP endpoints
(that's why we bothered with SDP in the first place right?) so the only
way I can get their SSRCs is with some extra signalling between me and
the RTP translator. This is not a problem per se, but it can only happen
after the SIP participants have joined the call and started streaming
data, that has reached the translator. And even then they will have no
MSIDs.

All in all, we would be enforcing some very laborious signalling when
it is absolutely unnecessary.

Does this make sense?
Emil

-- 
https://jitsi.org

---

From: Ted Hardie <[email-masked]>
Date: Tue, 14 May 2013 13:44:59 -0700

On Tue, May 14, 2013 at 11:34 AM, Paul Kyzivat <[email-masked]>wrote:

Certainly. If the web server supporting the browser

I assume by this you meant the javascript application; it's possible for
the web server to support the CLUE semantics but a particular client not to
(e.g. a mobile client connecting to the server).

> does't support clue then this isn't going to result in a clue call. But
> that would only come up for *received* calls, and receiving such a call via
> rtcweb depends on having an app that is at least addressable by a clue
> client. That means it must have a sip address. If so, then at least it will
> probably be able to connect as a "legacy" sip device.
>
>
So, just to confirm my understanding, in this scenario, the webrtc client
is sitting in an open screen somewhere, connected to a web server (it could
be headless, but bare with me).  When a CLUE conference starts that the
webserver believes is of interest to the webrtc client it uses the existing
connection to notify it and give it conference details.  Is that right so
far?

If that is the case, I'm not sure what you mean by "must have a SIP
address".  Can you clarify this?

> obscure gateway code.
I'm assuming that a "without" is missing here, but can you confirm?

regards,

Ted

---

From: Justin Uberti <[email-masked]>
Date: Tue, 14 May 2013 22:53:29 -0700

> it is absolutely unnecessary.
How do you plan to demultiplex the media sources without knowing whose SSRC
belongs to whom? Consider especially the case of repair flows, for which
one must know which primary flow the repair flow corresponds with.

Given the complexities here, I don't think it's entirely fair to call the
signaling "absolutely unnecessary".

---

From: "Roni Even" <[email-masked]>
Date: Wed, 15 May 2013 12:25:48 +0300

From: [email-masked] [mailto:[email-masked]] On Behalf Of Justin Uberti
Sent: 15 May, 2013 8:53 AM
To: Emil Ivov
Cc: [email-masked]
Subject: Re: [rtcweb] Why requiring pre-announcement of SSRCs is a problem for conferencing ( Was: New Version Notification for draft-uberti-rtcweb-plan-00.txt )

Hey Stefan,

How do you plan to demultiplex the media sources without knowing whose SSRC belongs to whom? Consider especially the case of repair flows, for which one must know which primary flow the repair flow corresponds with.

Given the complexities here, I don't think it's entirely fair to call the signaling "absolutely unnecessary".

RE: This is the topology where the intermediary (RTP Translator) switch the sources and pass through the SSRCs of the original source. In this topology for FEC even when using a separate UDP port for the main and repair streams the association is probably based on the SDES information having same CNAME for both streams. If we multiplex multiple RTP streams without having the SSRC explicitly signaled the combination of CNAME and SSRC are still available. What we looked at is how to distinguish between similar RTP streams ( Video was first from A and now B replaces it). In CLUE we talked about RTP header extension for signaling the replacement and we will probably need a new SDES packet to signal the switch. I do not think that you  can know the SSRC in advance. The SDP should at least say how many RTP streams can be sent and received based on the m-line using a proposed maxssrc SDP attribute.

Roni Even

---

From: Paul Kyzivat <[email-masked]>
Date: Wed, 15 May 2013 12:24:48 -0400

> we should not push all the functionality into the "core" SDP.

I agree that we need to draw a line *somewhere* about what goes into SDP 
and what does not.

I was hoping that the taxonomy document could help with this, by showing 
the relationships between all the *things*. Then we would have a better 
basis for organizing them all in layers.

	Thanks,
	Paul

---

From: Emil Ivov <[email-masked]>
Date: Wed, 15 May 2013 22:01:22 +0300

Adding to what Roni said:

On 15.05.13, 08:53, Justin Uberti wrote:
> On Tue, May 14, 2013 at 12:46 PM, Emil Ivov <[email-masked]
> How do you plan to demultiplex the media sources without knowing whose
> SSRC belongs to whom?

Demultiplexing itself shouldn't be an issue. Handling the streams in the
application (attributing them to a specific participant or discovering
inter-stream relations) does require some knowledge of the SSRCs and the
best way of solving this will always depend on the app:

First, many applications may not require such information at all, as
they may simply be rendering whatever stream they get.

Others may need this only for presentation purposes (e.g. showing a name
underneath a video). In this case they would be potentially happy with
getting it later, asynchronously and merged with other display
information that the focus may be sending their way.

Others yet may require this information in order to allow for some level
of control. One example is the case where a browser app participating in
a conference, asks the conference focus (another WebRTC browser app) to
ask the RTP translator (a server-side WebRTC compatible app) to stop
streaming a specific SSRC in their direction.

> Consider especially the case of repair flows, for
> which one must know which primary flow the repair flow corresponds with.

Applications using such repair flows would need to know about
stream/SSRC relations relatively early in the process. Roni already
indicated RTCP is widely used for this purpose and I believe this can
also be a viable option here.

Another option would be to have this information exchanged through
application-specific signalling.

Obviously, for this to work, the WebRTC APIs would need to provide means
of obtaining and setting this information from/to the browser.
Applications need to be able to start and stop specific streams, relate
them as layers or FEC repair flows.

> the signaling "absolutely unnecessary".

My point is that it could be "absolutely unnecessary" and even
prohibitive for some applications, nice to have in others, and required
by others yet.

Having SDP O/A as the (sole) mechanism that is available to all these
categories sounds like a problem.

Emil

-- 
https://jitsi.org

---

From: Bernard Aboba <[email-masked]>
Date: Wed, 15 May 2013 12:55:09 -0700

Emil said: 

>> How do you plan to demultiplex the media sources without knowing whose> > SSRC belongs to whom?>> Demultiplexing itself shouldn't be an issue.... > > First, many applications may not require such information at all, as
[BA] Scenarios where the media can be directly rendered, or the required information can arrive later are idealfor potential handling outside SDP (e.g. within RTP/RTCP).   So "known whose SSRC belongs to whom" can be handled adequately without SDP in simple cases (e.g. single media stream per participant). 
> also be a viable option here.>
[BA] I believe that RTP and RTCP can be used successfully to provide repair and FEC relationships, as well as simulcast and layered coding information.  However, this is trickier than the "simple rendering" or "name" case, because until the relationships are known to the endpoints it may be difficult for them to properly utilize those streams.  Having a reliable signaling mechanism only solves the problem once the Answer has arrived, but it may be desirable to be able to handle media before then.  Similarly, RTP/RTCP info only helps once it has arrived, and of course packets can be lost.  Overall, I believe it is possible to make things work without SDP over the wire, assuming our expectations are set appropriately.  Having RTP/RTCP alternatives available is useful within a framework such as WebRTC where there is no mandatory signaling mechanism, but also in scenarios where rapid response to changing conditions is needed (such as in congestion control).  
> them as layers or FEC repair flows.
[BA] I hope that you're not talking about application-specific RTCP packets; I would prefer to avoid that if possible.  
> prohibitive for some applications, nice to have in others, and required
[BA] There are a number of current scenarios in which frequent signaling is not desirable and in which RTP/RTCP is widely used instead.   As an example, it is one thing to use SDP to communicate the envelope of what a receiver can handle (resolutions, frame rates, etc.).  It is another to use SDP to attempt to signal changes that can happen as frequently as multiple times a second.  If we don't make it possible for that to be "absolutely unnecessary" (or worse, not make it clear that this undesirable) we will have been negligent. 
> categories sounds like a problem.
[BA] Since SDP O/A isn't a universal hammer, depending on it exclusively is a fundamental design mistake.

---

From: Emil Ivov <[email-masked]>
Date: Thu, 16 May 2013 02:41:24 +0300

On 15.05.13, 22:55, Bernard Aboba wrote:
> [BA] I hope that you're not talking about application-specific RTCP
> packets; I would prefer to avoid that if possible.

Not at all. In this particular case I was referring to the possibility
of using application-specific signalling that two browsers (or a browser
and some other app) can use to signal such things. Such signalling could
go through websockets, clue data channels or e-mail for all it matters:
it would be up to the application.

-- 
https://jitsi.org

---

From: Stefan Håkansson LK <[email-masked]>
Date: Thu, 16 May 2013 09:45:21 +0000

> it is absolutely unnecessary.

Thanks, I then understand the use-case you're envisioning.

I think it could work under certain conditions, but as soon as there are 
repair flows, enhancement layers or simulcast involved, there is a need 
anyway to somehow signal what each SSRC represents. Then it is another 
question if it happens via SDP or via RTP header extensions or some 
other means.

There was a discussion at the Stockholm rtcweb interim on what 
topologies that would be supported, but I fail to remember if this case 
was included or excluded.

It also seems to me that people have quite different views in the 
discussion. One view seems to be that SDP exchanges should be used even 
for things like pause/resume, resolution changes, another that we should 
avoid SDP exchanges even when people join/leave a conference.

---

From: Stefan Håkansson LK <[email-masked]>
Date: Thu, 16 May 2013 09:46:06 +0000

> basis for organizing them all in layers.

That is probably the right place to start.

---

From: Emil Ivov <[email-masked]>
Date: Thu, 16 May 2013 22:22:30 +0300

On 16.05.13, 12:45, Stefan Håkansson LK wrote:
> anyway to somehow signal what each SSRC represents.

I don't think anyone is arguing against this.

> Then it is another 

I couldn't find the discussion (in what WG did it happen?) but
topologies based on RTP translators of one sort or another seem to be
the default way of handling video conferencing these days so I don't see
how we could possibly rule them out.

> discussion.

Of course, where would the fun be otherwise? :)

> One view seems to be that SDP exchanges should be used even 
> for things like pause/resume, resolution changes, 

Right! That's the wrong view ;). We made two important choices in this
working group:

a) we will not impose a signalling protocol, and
b) we will use SDP O/A to make it easier to talk to legacy apps/devices.

Let's not try to compensate for the former by hijacking the latter.

> another that we should 

Even? I'd argue that this is _the_ place where avoiding SDP O/A is most
important!

Emil

-- 
https://jitsi.org

---

From: Harald Alvestrand <[email-masked]>
Date: Thu, 16 May 2013 22:28:18 +0200

> how we could possibly rule them out.

This is an interesting statement. I've also heard the claim that "RTP 
translators hardly exist outside the lab" - I think that quip was 
referring to RFC 5117 section 3.3 Topo-Trn-Translators, the story may be 
different for Topo-Media-Translators - but I thought the most common 
form was the section 3.4 Topo-Mixer?

Do we have numbers or names attached to this?

I'm afraid all I can contribute real data on is Google+ hangouts, which 
is a section 3.6 Topo-RTCP-terminating-MCU.

---

From: Emil Ivov <[email-masked]>
Date: Fri, 17 May 2013 11:30:13 +0300

> form was the section 3.4 Topo-Mixer?

Until recently, yes certainly, mixers were prevalent (which is arguably
part of the reason why very few people had access to video conferencing).

I think we'd all agree that this doesn't seem to be the case any longer
and that for various reasons (including cost, quality and flexibility),
most of the conferencing solutions today prefer shifting packets rather
than mixing content.

My statement above was made in that sense and not in the sense of
"Topo-Trn-Translators" beat all the other topos described in 5117.

> is a section 3.6 Topo-RTCP-terminating-MCU.

I can't know much about Google+ hangouts but I am quite convinced that
the problems I described earlier in this thread fully apply there. (They
certainly do for 3.6 Topo-RTCP-terminating-MCU) The reason you are not
experiencing them has much more to do with the fact that hangouts run in
a controlled environment where all clients and use cases are rather well
known. The right assumptions can therefore be made about use of repair
flows, simulcasting, presence or absence of cascaded translators (or
MCUs if you prefer) and so on.

Cheers,
Emil

-- 
https://jitsi.org

---

From: Stefan Håkansson LK <[email-masked]>
Date: Sun, 19 May 2013 11:35:46 +0000

> how we could possibly rule them out.

It was at the rtcweb interim in Stockholm last June (but I don't 
remember the details of the outcome of the discussion).

I'm no expert, but how do you handle things like congestion management 
with translators? IIUC the RMCAT group has decided to handle the simple 
point-point case only. (I'm not trying to remove translators from the 
topologies we cover - I just want to bring all the pieces to the table).

---

From: Emil Ivov <[email-masked]>
Date: Mon, 20 May 2013 12:36:19 +0300

Hey Stefan,

> topologies we cover - I just want to bring all the pieces to the table).

I believe 3551 recommends use of RTCP and monitoring packet loss.
Additional means can also be implemented over out-of-band signalling
between the translator and a controlling entity.

I already mentioned this in my response to Harald: note that my use of
the term RTP translator did not apply solely to the
"Topo-Trn-Translators" described in 5117. The problems (i.e. heavy
signalling constraints) apply to pretty much any entity that is not
mixing content and behaving like a complete single-stream B2BUA.

Emil

-- 
https://jitsi.org

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 20 May 2013 13:00:09 -0400

> most of the conferencing solutions today prefer shifting packets rather

I would like to understand: when "most" is used in this discussion, what 
are we measuring?
- number of distinct implementations (code bases) using this approach?
- number of conference sessions set up using this approach?
- number of conference user-minutes using this approach?
- something else

I'm asking because I have no idea and I'm curious.

	Thanks,
	Paul

---

From: Emil Ivov <[email-masked]>
Date: Tue, 21 May 2013 12:19:54 +0300

Hey Paul,

> - number of conference user-minutes using this approach?

I was referring to easily observable trends. I certainly don't have
exact numbers. I don't think I implied that and I don't think it's the
most important thing here. Skype, Hangouts, iChat, Asterisk, FreeSWITCH,
Jitsi Videobridge, all (seem to) have taken that approach rather than
going for content mixing.

While in RTCWEB we can't agree on the exact approach yet we all seem to
have accepted that conferencing would imply separate RTP streams. The
work you are doing in CLUE is another example since it is also about
manipulating individual RTP streams.

Similar observations could be made about the not so distant past where
content mixing seemed to be the default way of handling video conferencing.

So, the point I was making here was that:

With this shift in mind, topologies resembling RTP translators would
likely continue being increasingly popular. Constraints such as "Thou
shalt always pre-know your SSRCs" are hence likely to affect and weigh
on a big number of scenarios.

Does this make more sense?

Emil

-- 
https://jitsi.org

---

From: "Roni Even" <[email-masked]>
Date: Tue, 21 May 2013 13:53:59 +0300

> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of Emil Ivov
> Sent: 21 May, 2013 12:20 PM
> To: Paul Kyzivat
> Cc: [email-masked]
> Subject: Re: [rtcweb] Common ways of handling video conferences? (Re:
> Why requiring pre-announcement)
> 
> Hey Paul,
> 
> >>>>>> question if it happens via SDP or via RTP header extensions or
> >>>>>> some other means.
> >>>>>>
> >>>>>> topologies that would be supported, but I fail to remember if
> >>>>>> this case was included or excluded.
> >>>> topologies based on RTP translators of one sort or another seem to
> >>>> be the default way of handling video conferencing these days so I
> >>>> don't see how we could possibly rule them out.
> >>>>
> >>>
> >>> This is an interesting statement. I've also heard the claim that
> >>> "RTP translators hardly exist outside the lab" - I think that quip
> >>> was referring to RFC 5117 section 3.3 Topo-Trn-Translators, the
> >>> story may be different for Topo-Media-Translators - but I thought
> >>> the most common form was the section 3.4 Topo-Mixer?
> >>
> >> Until recently, yes certainly, mixers were prevalent (which is
> >> arguably part of the reason why very few people had access to video
> conferencing).
> >>
> >> I think we'd all agree that this doesn't seem to be the case any
> >> longer and that for various reasons (including cost, quality and
> >> flexibility), most of the conferencing solutions today prefer
> >> shifting packets rather than mixing content.
> >
> > I would like to understand: when "most" is used in this discussion,
> > what are we measuring?
> I was referring to easily observable trends. I certainly don't have exact
> numbers. I don't think I implied that and I don't think it's the most
important
> thing here. Skype, Hangouts, iChat, Asterisk, FreeSWITCH, Jitsi
Videobridge,
> all (seem to) have taken that approach rather than going for content
mixing.
> With this shift in mind, topologies resembling RTP translators would
likely
> continue being increasingly popular. Constraints such as "Thou shalt
always
> pre-know your SSRCs" are hence likely to affect and weigh on a big number
> of scenarios.
> 
[Roni Even] This make sense but I assume that this approach works better
with scalable video (SVC) or simulcast  for endpoint mixing and I did not
see that RTCweb wants to mandate such support in the MTI video codec
discussion.

---

From: "Cullen Jennings (fluffy)" <[email-masked]>
Date: Thu, 30 May 2013 19:28:04 +0000

> RTP translator. It's basically what we do in Jitsi Videobridge.

Many deployments use a mixer instead of a translator from an RTP point of view. I understand at the media level you would not decode / encode or mix the media but at the RTP layer the bridge could have it's own SSRC and put the SSRC of the original sender in the CSRC. 

Any reason you choose a translator over a mixer when building a conferencing fencing bridge?