Subject: [rtcweb] New topic: Model for multi-unicast support

From: Harald Alvestrand <[email-masked]>
Date: Thu, 01 Sep 2011 13:57:56 +0200

In the thread about draft-perkins-rtcweb-usage, the following exchange 
took place:
>> >  Speaking for some implementors of the WEBRTC API, it's also clear that there's a significant cost to implementing the multiway RTP session concept - both in code complexity and API complexity. I think this warrants more discussion, where all 3 outcomes should be on the table:
>> >  
>> >  - We recommend doing multi-unicast with a shared RTP session only
>> >  - We recommend supporting both shared and split RTP sessions for multi-unicast
>> >  - We recommend doing multi-unicast with split RTP sessions only
>> >  
>> >  We should probably do this in a new thread.
> Indeed; I'd be interested to know what complexity you're seeing. I would have expected the shared RTP session model to decrease complexity, since the RTP layer can be oblivious to the detail of the underlying transport connections, and wouldn't have to worry about tracking SSRC and payload type mapping separately for each session.
I think this is a topic that deserves its own thread.

The context is use case 4.2.7, "Multiparty video communication", the one 
without a central server; each party sends its video and audio streams 
to all participants in the meeting directly.

One can imagine supporting this with a single RTP session, where all 
video streams are equal and sent with the same SSRC to all participants, 
all participants send receiver reports (RRs) about all the SSRCs they 
see to all other participants, and all senders send their SR reports to 
all participants.

Now, this has implications for both API design, service design and 
protocol operation.

API: The model presented is no longer "this object is your connection to 
A; that object is your connection to B". There has to be a 
representation of your "cloud" as a multidestination object, with 
procedures for adding to and subtracting from the list of participants. 
Error reporting must include which participant the error relates to.

Service: The requirement for perfect equality means that you cannot 
allow any party to adapt its presence to what it can receive; for 
instance, as I understand it, nobody can drop the video and only be an 
audio participant, because that would break the model of equality, 
rendering sender reports meaningless to some participants.

Protocol: The requirement for a single agreed RTCP bandwidth means that 
this bandwidth has to be computed for the session as a whole - with the 
simplistic answer being 5% of the lowest common denominator.

In contrast, if one chooses the multiple-RTP-session approach:

- the API retains its "one object, one destination" orientation.

- the degree to which service is equal to all parties is a matter for 
the application, not the protocol

- rate adaption, choice of RTP bandwidht and so on can be negotiated on 
each link individually

I think this forum needs to pick one of the options in my initial note, 
and design its protocols and APIs accordingly.
Thoughts?

                     Harald

---

From: Stefan Håkansson LK <[email-masked]>
Date: Sat, 3 Sep 2011 07:31:36 +0200

Harald wrote:
>>> >  Speaking for some implementors of the WEBRTC API, it's also clear that there's a significant cost to implementing the multiway RTP session concept - both in code >complexity and API complexity. I think this warrants more discussion, where all 3 outcomes should be on the table:
>>> >
>> Indeed; I'd be interested to know what complexity you're seeing. I would have expected the shared RTP session model to decrease complexity, since the RTP layer can be >>oblivious to the detail of the underlying transport connections, and wouldn't have to worry about tracking SSRC and payload type mapping separately for each session.

To me an RTP session was connected to a port/IP address, so I don't understand how this could be one session (the destination port/IP would be different for each stream). (I expect to be corrected!) Is multicast assumed?

>and design its protocols and APIs accordingly.
This was to some extent discussed on the webrtc list earlier. My view is the same as then: we should go for the '"one object, one destination" and multiple RTP-session' approach. 

Stefan

                     Harald

_______________________________________________
rtcweb mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/rtcweb

---

From: Magnus Westerlund <[email-masked]>
Date: Mon, 5 Sep 2011 14:03:50 +0200

> simplistic answer being 5% of the lowest common denominator.

I agree with the issues of having it being represented like this. And so
far I don't think I have identified a use case that requires a joint RTP
session for the multi-unicast case.

The centralized mixer use case has certain benefits for a joint RTP
session, but there you can apply a RTP mixer to enable difference in
service on what being delivered and in what quality to each participant.

> - the API retains its "one object, one destination" orientation.

I mostly agree, there are a few thinks I think it is important to
discuss here. And I will have slides on this relation ship for Thursday.

> the application, not the protocol

Yes, it is not dictated by the protocol, but I don't think the
application will be in full control either. It can indicate what is
desired, but local and network resources needs to be included in the
consideration.

> - rate adaption, choice of RTP bandwidht and so on can be negotiated on 

Yes, but there is clearly a need to take the combinations into account.
An issue is clearly that one need to divide the available uplink
bit-rate between the streams one needs to send. Another issue is that if
one has three peers, and each request a different resolution one may not
have the computing resources to encode three versions, and may have to
limit it too two resolutions, where one is delivered to two peers.

Also the need for joint identifiers between peers for streams that are
the same source but may vary in representation likely needs to be kept
between the different legs. Otherwise the application logic is

> and design its protocols and APIs accordingly.

Sorry, I don't know which initial note you are referencing.

But I do think we need to agree on how we handle RTP sessions in
relation to the relevant API constructs and what binding between API
resources that are needed to allow the lower layer to correctly keep
semantics.

We will have some opportunity to discuss this on Thursday in the Interim
meeting.

Cheers

Magnus Westerlund

----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone  +46 10 7148287
Färögatan 6                | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden| mailto: [email-masked]
----------------------------------------------------------------------

---

From: Magnus Westerlund <[email-masked]>
Date: Mon, 5 Sep 2011 14:07:39 +0200

On 2011-09-03 07:31, Stefan Håkansson LK wrote:
> Harald wrote:
>> In the thread about draft-perkins-rtcweb-usage, the following
>> exchange took place:
>>>>> Speaking for some implementors of the WEBRTC API, it's also
>>>>> clear that there's a significant cost to implementing the
>>>>> multiway RTP session concept - both in code >complexity and
>>>>> API complexity. I think this warrants more discussion, where
>>>>> all 3 outcomes should be on the table:
>>>>> 
>>>>> - We recommend doing multi-unicast with a shared RTP session
>>>>> only - We recommend supporting both shared and split RTP
>>>>> sessions for multi-unicast - We recommend doing multi-unicast
>>>>> with split RTP sessions only
>>>>> 
>>> Indeed; I'd be interested to know what complexity you're seeing.
>>> I would have expected the shared RTP session model to decrease
>>> complexity, since the RTP layer can be >>oblivious to the detail
>>> of the underlying transport connections, and wouldn't have to
>>> worry about tracking SSRC and payload type mapping separately for
>>> each session.
>> The context is use case 4.2.7, "Multiparty video communication",
>> the one without a central server; each party sends its video and
>> audio streams to all participants in the meeting directly.
>> 
>> One can imagine supporting this with a single RTP session, where
>> all video streams are equal and sent with the same SSRC to all
>> participants, all participants send receiver reports (RRs) about
>> all the SSRCs they see to all other participants, and all senders
>> send their SR reports to all participants.
> 
> To me an RTP session was connected to a port/IP address, so I don't
> understand how this could be one session (the destination port/IP
> would be different for each stream). (I expect to be corrected!) Is
> multicast assumed?
> 

No, no multicast assumed. This uses the fundamental definition of an RTP
session as a joint SSRC space as the basis. What Harald talks about here
is what I call multi-unicast, where one basically configure a
replication module under the regular RTP/RTCP stack processing to send
out each RTP and RTCP packet to multiple destinations using unicast. All
incomming traffic would preferably be received on a single port (pair)
from the other peers.

Cheers

Magnus Westerlund

---

From: Harald Alvestrand <[email-masked]>
Date: Mon, 05 Sep 2011 15:46:46 +0200

---

From: Harald Alvestrand <[email-masked]>
Date: Mon, 05 Sep 2011 17:14:51 +0200

> discuss here. And I will have slides on this relation ship for Thursday.
Looking forward to it.
> desired, but local and network resources needs to be included in the
Yes, the application needs to adapt. That's an area where I expect 
applications to differ, and the successful ones will be the ones that 
adapt better.
> limit it too two resolutions, where one is delivered to two peers.
Yes, the application needs to consider all the factors when negotiating.
The typical challenging scenario I see is a 4-way call with 3 
participants on the same high speed network and 1 participant calling in 
from a 3G phone. If the meeting is not called primarily for the guy on 
the phone, the 3 other participants might want to see each other in full 
HD and listen to each other using wideband codecs; the 4th one has to 
make a reasonable apportioning of his bandwidth.

---

From: Colin Perkins <[email-masked]>
Date: Tue, 6 Sep 2011 12:13:02 +0100

On 1 Sep 2011, at 12:57, Harald Alvestrand wrote:
> In the thread about draft-perkins-rtcweb-usage, the following exchange took place:
> The context is use case 4.2.7, "Multiparty video communication", the one without a central server; each party sends its video and audio streams to all participants in the meeting directly.
> 
> One can imagine supporting this with a single RTP session, where all video streams are equal and sent with the same SSRC to all participants, all participants send receiver reports (RRs) about all the SSRCs they see to all other participants, and all senders send their SR reports to all participants.
> 
> Now, this has implications for both API design, service design and protocol operation.
> 
> API: The model presented is no longer "this object is your connection to A; that object is your connection to B". There has to be a representation of your "cloud" as a multidestination object, with procedures for adding to and subtracting from the list of participants. Error reporting must include which participant the error relates to.

Agree. This is the model RTP suggests for a session: a group-communication cloud, with the details of whether that's implemented as a single point-to-point flow, a centralised mixer, multi-unicast, or a multicast group being hidden below the RTP layer. 

You can model transport layer flow as its own RTP session, but then you get into coordination issues between the RTP sessions. Also, it doesn't solve the issue that "your connection to B" is not necessarily meaningful,  it B is a reflector that expands to multiple participants.

> Service: The requirement for perfect equality means that you cannot allow any party to adapt its presence to what it can receive; for instance, as I understand it, nobody can drop the video and only be an audio participant, because that would break the model of equality, rendering sender reports meaningless to some participants.

This is why audio and video are usually sent in separate RTP sessions. 

> Protocol: The requirement for a single agreed RTCP bandwidth means that this bandwidth has to be computed for the session as a whole - with the simplistic answer being 5% of the lowest common denominator.

This is a problem if you have flows with significantly different rates in the same RTP session, yes.

> - the degree to which service is equal to all parties is a matter for the application, not the protocol
> 
> - rate adaption, choice of RTP bandwidht and so on can be negotiated on each link individually
> 
> I think this forum needs to pick one of the options in my initial note, and design its protocols and APIs accordingly.
> Thoughts?

I'm not sure it's a simple as picking one model or the other. There are some scenarios where the model of a single RTP session across multiple transport-layer flows makes a lot of sense (e.g., centralised conferences using a mixer, for the reasons described in RFC 5117), but in other cases there are stronger arguments for treating each transport layer flow separately (e.g., multi-unicast, if you're multiplexing audio and video in a single session, and sending only a subset of media flows to some of the participants). A better model might be to provide an appropriate RTP session API, plus a higher layer API that manages the grouping of RTP sessions into a single multimedia conference. 

This would also make legacy interoperability easier, since there are legacy systems that use both the single RTP session and multiple RTP sessions models.

Colin

-- 
Colin Perkins
http://csperkins.org/