Subject: [rtcweb] JSEP: RTP demux algorithms

From: Bernard Aboba <[email-masked]>
Date: Wed, 14 Sep 2016 10:49:09 -0700

There is now a JSEP PR relating to RTP demux algorithms:
https://github.com/rtcweb-wg/jsep/pull/320

Rather than commenting in detail on that PR in this post, I would like to
prove some thoughts on how we evaluate proposed RTP demux algorithms.

The goal of an RTP demux algorithm is to determine whether an incoming RTP
packet received on a DtlsTransport can be forwarded to one of the
RtpReceivers attached to that DtlsTransport.  If none of the RtpReceivers
qualify, then the packet is either dropped (WebRTC 1.0) or generates an RTP
unhandled event (ORTC).

In principle, RTP demux algorithms do not seem complicated, since they
often can be expressed in less than half a dozen steps, typically with less
than a page of text.

However, in practice the difficulty has turned out to be not in stating an
algorithm, but in evaluating whether it is correct. This challenges
include:

1. Agreement on a set of use cases and desired outcomes that a proposed
algorithm must satisfy.  Think of this as a set of regression tests that
insure that a proposed algorithm behaves as we expect, as well as ensuring
that changes to that algorithm do not result in unintended consequences.

2. Agreement on all the inputs that the algorithm takes into account.  This
includes fields within the RTP packet (e.g. SSRC, PT, RID, MID, etc.) as
well as methods within the API (e.g. setLocalDescription,
setRemoteDescription, setParameters, etc.).

With respect to Issue #1, some of the trickiest use cases involve both PT
and SSRC signaling, SSRC "latching" and handling FEC and RTX.  Examples
include:
    a. Change of SSRCs (e.g. due to conflict).
    b. Codec switching (e.g. changing PT but same SSRC, or changing both PT
and SSRC).
    c. Use of both FEC and RTX (and perhaps RTX of FEC).

IMHO it would be useful to come up a few (less than half a dozen) use cases
with agreed-upon outcomes.

With respect to Issue #2, the proposed algorithms so far look at the SSRC &
PT fields in the RTP header as well as the MID RTP extension.  However, I
have not seen algorithms which take into account the role of the RID RTP
extension.  So agreement on whether the RID needs to be considered or not
would be helpful.

Also, for WebRTC 1.0 it would appear to me that the algorithm will depend
on the inputs to the setLocalDescription/setRemoteDescription methods, so
that there is a potential dependency on the signaling state of the
PeerConnection.

Overall, I believe that the discussion of RTP demux algorithms is likely to
go more smoothly (and conclude more quickly) if we get agreement on these
meta-issues upfront.

---

From: Cullen Jennings <[email-masked]>
Date: Thu, 15 Sep 2016 15:18:13 -0700

To propose a few use case….

“Whiskey & Soda Example”
The PT are all the same. Early packets, and any packets after a SSRC change, contain a MID. If the packet has  MID, it goes to the place that matches that MID *and* it latches that SSRC so any future SSRC that don’t have a MID go the to the same place. 

“Coke Classic Example” 
The PT are all unique. The SSRC change. There is no RID, MID, FEC, or RTX. The packets get delivered to the thing with the matching PT. 

“Bloddy Mary Example” 
The PT are the same. The SSRC do not changed and are signalled in SDP. There is no MID, RID, etc in the RTP.  The packets go to the thing the SSRC signalling would have indicated. 

“Diet Coke Example”
The PT are all unique. The packet has a MID. The MID corresponds to a an m-line that does not have PT associated with it that match the PT. (the PT is from a different m-line). The packets gets dropped (or cause event in ORTC). [ Note I think of this that the MID sends it to the right m-line aka Receiver, but then that does not have a codec for that PT so it gets dropped. ]

“Diet Pepsi Example”
The PT are all unique. The packet has a SSRC. The SSRC does not correspond to any SSRC that were signalled for the m-line that the PT is on. The packet goes to the thing associated with that PT ignoring any SSRC. 

“Red Bull Example”
The packet has a MID that does not match any MID in the signalling. The packets gets dropped (or event in ORTC).

> On Sep 14, 2016, at 10:49 AM, Bernard Aboba <[email-masked]> wrote:
> 
> https://github.com/rtcweb-wg/jsep/pull/320 <https://github.com/rtcweb-wg/jsep/pull/320>
> 
> Rather than commenting in detail on that PR in this post, I would like to prove some thoughts on how we evaluate proposed RTP demux algorithms. 
> 
> The goal of an RTP demux algorithm is to determine whether an incoming RTP packet received on a DtlsTransport can be forwarded to one of the RtpReceivers attached to that DtlsTransport.  If none of the RtpReceivers qualify, then the packet is either dropped (WebRTC 1.0) or generates an RTP unhandled event (ORTC). 
> 
> In principle, RTP demux algorithms do not seem complicated, since they often can be expressed in less than half a dozen steps, typically with less than a page of text. 
> 
> However, in practice the difficulty has turned out to be not in stating an algorithm, but in evaluating whether it is correct. This challenges include: 
> 
> 1. Agreement on a set of use cases and desired outcomes that a proposed algorithm must satisfy.  Think of this as a set of regression tests that insure that a proposed algorithm behaves as we expect, as well as ensuring that changes to that algorithm do not result in unintended consequences. 
> 
> 2. Agreement on all the inputs that the algorithm takes into account.  This includes fields within the RTP packet (e.g. SSRC, PT, RID, MID, etc.) as well as methods within the API (e.g. setLocalDescription, setRemoteDescription, setParameters, etc.). 
> 
> With respect to Issue #1, some of the trickiest use cases involve both PT and SSRC signaling, SSRC "latching" and handling FEC and RTX.  Examples include: 
>     b. Codec switching (e.g. changing PT but same SSRC, or changing both PT and SSRC).
> IMHO it would be useful to come up a few (less than half a dozen) use cases with agreed-upon outcomes.
> 
> With respect to Issue #2, the proposed algorithms so far look at the SSRC & PT fields in the RTP header as well as the MID RTP extension.  However, I have not seen algorithms which take into account the role of the RID RTP extension.  So agreement on whether the RID needs to be considered or not would be helpful. 
> 
> Also, for WebRTC 1.0 it would appear to me that the algorithm will depend on the inputs to the setLocalDescription/setRemoteDescription methods, so that there is a potential dependency on the signaling state of the PeerConnection.
> 
> Overall, I believe that the discussion of RTP demux algorithms is likely to go more smoothly (and conclude more quickly) if we get agreement on these meta-issues upfront. 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> _______________________________________________
> rtcweb mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rtcweb

---

From: Bernard Aboba <[email-masked]>
Date: Thu, 15 Sep 2016 17:52:03 -0700

Thanks, Cullen.  Those all look like good test cases.

Here are a few more:

Codec change, case 1: The PTs are all unique, and no SSRCs were signaled in
SDP  There is no RID, MID, FEC, or RTX. Sender switches codecs, and the PT
changes (but SSRC stays the same).  The packet goes to the receiver that
matches the new PT.

Codec change, case 2:  The PTs are all unique and SSRCs are signaled in
SDP.  There is no RID, MID, FEC or RTX. Sender switches codecs and the PT
changes (but SSRC stays the same).  SSRC does not match any signaled ones.
The packet goes to the receiver that matches the new PT.

These use cases relate to the following Issue (filed by Inaki):
https://github.com/w3c/ortc/issues/547

There are also some trickier use cases that derive from
draft-ietf-mmusic-bundle-negotiation Section 10.2 (which discusses PT
overlap):

PT overlap:  The PTs are not unique.  Some SSRCs are signaled in SDP.
There is no RID, MID, FEC or RTX. Packet arrives with SSRC that does not
match any signaled in the SDP, but matches PTs specified in multiple
m-lines.  Is the packet delivered or dropped, and if delivered, to which
receiver?  Some potential answers:

a. Behavior is not defined.  If reliable behavior is desired, signal the
SSRC or use MID (support for which is mandated by BUNDLE).

b. Deliver the packet to an m-line that signals the PT but that:
   1. Doesn't provide SSRCs for each encoding (e.g. encodings[i].ssrc is
not set for all i) OR
   2. Doesn't provide SSRC for each RTX entry (e.g. encodings[i].rtx.ssrc
is not set for all i) OR
   3. Doesn't provide SSRCs for each FEC entry (e.g. encodings[i].fec.ssrc
is not set for all i)

See the following Issue: https://github.com/w3c/ortc/issues/546

draft-ietf-mmusic-bundle-negotiation Section 10.2:

10.2 <https://tools.ietf.org/html/draft-ietf-mmusic-sdp-bundle-negotiation-32#section-10.2>.
Associating RTP/RTCP Packets With Correct SDP Media Description

   There are multiple mechanisms that can be used by an endpoint in
   order to associate received RTP/RTCP packets with a bundled "m="
   line.  Such mechanisms include using the payload type value carried
   inside the RTP packets, the SSRC values carried inside the RTP
   packets, and other "m=" line specific information carried inside the
   RTP packets.

   As all RTP/RTCP packets associated with a BUNDLE group are received
   (and sent) using single address:port combinations, the local
   address:port combination cannot be used to associate received RTP
   packets with the correct "m=" line.

   As described in [Section 10.1.1
<https://tools.ietf.org/html/draft-ietf-mmusic-sdp-bundle-negotiation-32#section-10.1.1>],
the same payload type value might
   be used inside RTP packets described by multiple "m=" lines.  In such
   cases, the payload type value cannot be used to associate received
   RTP packets with the correct "m=" line.

   An offerer and answerer can inform each other which SSRC values they
   will use for RTP and RTCP by using the SDP 'ssrc' attribute
   [RFC5576 <https://tools.ietf.org/html/rfc5576>].  To allow for
proper association with this mechanism, the
   'ssrc' attribute needs to be associated with each "m=" line that
   shares a payload type with any other "m=" line in the same bundle.
   As the SSRC values will be carried inside the RTP/RTCP packets, the
   offerer and answerer can then use that information to associate
   received RTP packets with the correct "m=" line.  However, an offerer
   will not know which SSRC values the answerer will use until it has
   received the answer providing that information.  Due to this, before
   the offerer has received the answer, the offerer will not be able to
   associate received RTP/RTCP packets with the correct "m=" line using
   the SSRC values.

   In order for an offerer and answerer to always be able to associate
   received RTP and RTCP packets with the correct "m=" line, an offerer
   and answerer using the BUNDLE extension MUST support the mechanism
   defined in Section 14
<https://tools.ietf.org/html/draft-ietf-mmusic-sdp-bundle-negotiation-32#section-14>,
where the remote endpoint inserts the
   identification-tag associated with an "m=" line in RTP and RTCP
   packets associated with that "m=" line.

On Thu, Sep 15, 2016 at 3:18 PM, Cullen Jennings <[email-masked]> wrote:

> The PT are all the same. Early packets, and any packets after a SSRC
> change, contain a MID. If the packet has  MID, it goes to the place that
> matches that MID *and* it latches that SSRC so any future SSRC that don’t
> have a MID go the to the same place.
>
> “Coke Classic Example”
> The PT are all unique. The SSRC change. There is no RID, MID, FEC, or RTX.
> The packets get delivered to the thing with the matching PT.
>
> “Bloddy Mary Example”
> The PT are the same. The SSRC do not changed and are signalled in SDP.
> There is no MID, RID, etc in the RTP.  The packets go to the thing the SSRC
> signalling would have indicated.
>
> “Diet Coke Example”
> The PT are all unique. The packet has a MID. The MID corresponds to a an
> m-line that does not have PT associated with it that match the PT. (the PT
> is from a different m-line). The packets gets dropped (or cause event in
> ORTC). [ Note I think of this that the MID sends it to the right m-line aka
> Receiver, but then that does not have a codec for that PT so it gets
> dropped. ]
>
> “Diet Pepsi Example”
> The PT are all unique. The packet has a SSRC. The SSRC does not correspond
> to any SSRC that were signalled for the m-line that the PT is on. The
> packet goes to the thing associated with that PT ignoring any SSRC.
>
> “Red Bull Example”
> 2. Agreement on all the inputs that the algorithm takes into account.
> This includes fields within the RTP packet (e.g. SSRC, PT, RID, MID, etc.)
> as well as methods within the API (e.g. setLocalDescription,
> With respect to Issue #2, the proposed algorithms so far look at the SSRC
> & PT fields in the RTP header as well as the MID RTP extension.  However, I
> Overall, I believe that the discussion of RTP demux algorithms is likely
> to go more smoothly (and conclude more quickly) if we get agreement on
> these meta-issues upfront.
>
>
>
>
>
>
>
>
>
>
>
>

---

From: Cullen Jennings <[email-masked]>
Date: Fri, 16 Sep 2016 08:53:36 -0700

And to add some RID based one … 

“Double Expresso”
The PT are all the same. There are MIDs. There are RIDs. When a packet with a MID and RID drives, the MID is used to find the correct m-line, then if and only if there is a RID in that m-line that matches, the packet is delivered to the place associated with that a=rid line. Any future packet with the same SSRC but no RID or MID would go the same place. 

“Capuchino”
All the PT are unique. There is no MID or RID in the RTP packet but the SDP has unique PT in every RID line in every “a=rid” line. The packet is sent to the stuff associated with that a=rid line. 

“Decaf Carmel Non-Fat Latte”
Packets is discard (or even in ORTC) for any of the following cases:
(Decaf) There is a MID but the PT does match a PT for that m-line. (Carmel) There is a MID and RID but the RID does not exist in that m-line. (Non-fat) There is RID and PT but the PT does not match the set of PT allowed for that RID. 

> On Sep 15, 2016, at 3:18 PM, Cullen Jennings <[email-masked]> wrote:
> 
> 
>> On Sep 14, 2016, at 10:49 AM, Bernard Aboba <[email-masked] <mailto:[email-masked]>> wrote:
>> 
>> [email-masked] <mailto:[email-masked]>

---

From: Cullen Jennings <[email-masked]>
Date: Fri, 16 Sep 2016 09:49:34 -0700

> On Sep 15, 2016, at 5:52 PM, Bernard Aboba <[email-masked]> wrote:
> 
> Codec change, case 1: The PTs are all unique, and no SSRCs were signaled in SDP  There is no RID, MID, FEC, or RTX. Sender switches codecs, and the PT changes (but SSRC stays the same).  The packet goes to the receiver that matches the new PT. 
agree

> 
> Codec change, case 2:  The PTs are all unique and SSRCs are signaled in SDP.  There is no RID, MID, FEC or RTX. Sender switches codecs and the PT changes (but SSRC stays the same).  SSRC does not match any signaled ones.  The packet goes to the receiver that matches the new PT. 
agree

> 
> These use cases relate to the following Issue (filed by Inaki): https://github.com/w3c/ortc/issues/547 <https://github.com/w3c/ortc/issues/547>
> 
> There are also some trickier use cases that derive from draft-ietf-mmusic-bundle-negotiation Section 10.2 (which discusses PT overlap):  
> 
> PT overlap:  The PTs are not unique.  Some SSRCs are signaled in SDP.  There is no RID, MID, FEC or RTX. Packet arrives with SSRC that does not match any signaled in the SDP, but matches PTs specified in multiple m-lines.  Is the packet delivered or dropped, and if delivered, to which receiver?  Some potential answers: 
> 
> a. Behavior is not defined.  If reliable behavior is desired, signal the SSRC or use MID (support for which is mandated by BUNDLE). 

I could live with this but prefer (a) but I think this is basically an error and would prefer that the general strategy be it is is ambiguous where to send a packet, then the packet is dropped (or event in ORTC). 

>    1. Doesn't provide SSRCs for each encoding (e.g. encodings[i].ssrc is not set for all i) OR
>    2. Doesn't provide SSRC for each RTX entry (e.g. encodings[i].rtx.ssrc is not set for all i) OR
>    3. Doesn't provide SSRCs for each FEC entry (e.g. encodings[i].fec.ssrc is not set for all i)
>

---

From: Peter Thatcher <[email-masked]>
Date: Fri, 16 Sep 2016 11:45:33 -0700

I agree 100% that having a set of unit tests for the demux alogirthm.  I
found it very difficult to write the algorithm in a way that covered all
the known cases by just keeping all the cases in my head.  Any tweak (like
swapping step 2 and step 3) can break something that you forget about.
Having a set of unit tests would help a lot.

I think we would need to formulate a unit test like this.  Each test is a
series of 3 operations:

1.  Add an RtpReceiver with MID A, SSRCs 1, 2, 3, and PTs 4, 5, 6.
2.  Route a packet with MID X (or unset), SSRC 1, PT 2.
3.  Remove an RtpReceiver

Then we have tests like:

1.  Add X
2.  Add Y
3.  Route Z

Or:

1.  Add X
2.  Route Y
3.  Add Y
4.  Remove X
4.  Route Z

Then I could write a little python to check these things.

---

From: Peter Thatcher <[email-masked]>
Date: Fri, 16 Sep 2016 11:47:33 -0700

By the way, as I mentioned elsewhere, I think we should not use RIDs for
demux between RtpReceivers, and only use them within an RtpReceiver.  Demux
between RtpReceivers should be done with MID.

On Fri, Sep 16, 2016 at 11:45 AM, Peter Thatcher <[email-masked]>
wrote:

>> The goal of an RTP demux algorithm is to determine whether an incoming
>> RTP packet received on a DtlsTransport can be forwarded to one of the
>> However, in practice the difficulty has turned out to be not in stating
>> an algorithm, but in evaluating whether it is correct. This challenges
>> include:
>>
>> on the inputs to the setLocalDescription/setRemoteDescription methods,
>> so that there is a potential dependency on the signaling state of the
>> PeerConnection.
>>

---

From: Jonathan Lennox <[email-masked]>
Date: Fri, 16 Sep 2016 18:59:34 +0000

I’m not going to come up with beverage names for my examples, but feel free to make up your own.

1. The PT are all the same.  An RTCP source description for an SSRC contains a MID.  This latches the MID.  Later, RTP packets with this SSRC arrive without a MID. They go to the place matching the MID that was in RTCP.

2. The PT are all the same.  A packet contains MID1.  It goes to the place that matches MID1, latching the SSRC to MID1.  Later, a packet with the same SSRC contains MID2. This moves the latching, so the SSRC is now latched to MID2.

On Sep 15, 2016, at 6:18 PM, Cullen Jennings <[email-masked]<mailto:[email-masked]>> wrote:

“Whiskey & Soda Example”

“Coke Classic Example”

“Bloddy Mary Example”

“Diet Coke Example”

“Diet Pepsi Example”

“Red Bull Example”

On Sep 14, 2016, at 10:49 AM, Bernard Aboba <[email-masked]<mailto:[email-masked]>> wrote:

_______________________________________________
rtcweb mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/rtcweb

_______________________________________________
rtcweb mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/rtcweb

---

From: Cullen Jennings <[email-masked]>
Date: Fri, 16 Sep 2016 12:32:02 -0700

Agree with both of these.  I think they are both the same as “Whiskey & Soda Example” - or am I missing something about theses? 

> On Sep 16, 2016, at 11:59 AM, Jonathan Lennox <[email-masked]> wrote:
> 
>> On Sep 15, 2016, at 6:18 PM, Cullen Jennings <[email-masked] <mailto:[email-masked]>> wrote:
>> 
>>

---

From: Jonathan Lennox <[email-masked]>
Date: Fri, 16 Sep 2016 19:33:28 +0000

Just that RTCP counts for latching, and that SSRCs’ MIDs can change.

On Sep 16, 2016, at 11:59 AM, Jonathan Lennox <[email-masked]<mailto:[email-masked]>> wrote:

“Whiskey & Soda Example”

---

From: Magnus Westerlund <[email-masked]>
Date: Fri, 23 Sep 2016 10:56:38 +0200

Hi,

I think the use cases looks fairly covering.

I want to add an alternative way of thinking on how the SDES item (MID, 
RID) related demultiplexing part is handled. So the RID and MID in RTP 
header extensions or in RTSP SDES packets results in these values being 
bound to the SSRC until further notice. So packets arriving without 
actual RTP header extensions can still be handled according to the SSRCs 
associated SDES items. From my perspective all the cases with MID and 
RID can be handled on a per packet basis rules without latching the SSRC 
to a specific receiver. Instead what is "latched" is the SDES values to 
the SSRC. Still at the point of change there needs to take the 
combination of SSRC and Seq into account to avoid miss routing packets.

So my general handling algorithm would be this sequence:

1. Check if RTP header extensions update any SDES item for the SSRC.
    a. If they do, mark the Sequence number and keep the old SDES item 
as valid prior to this extended sequence number.

2. Determine the set of relevant SDES items that applies for this 
packet, i.e. use SSRC + RTP seq to look up the SDES items, e.g. MID and 
RID.

3. Perform the demuxing algorithm

On another part, I find Jonathan's use case for setting the MID or RID 
using RTCP an interesting one. This put some light on the question that 
in a session where MID and RIDs are configured, if one receive packets 
with an SSRC that lacks these SDES items, what do you do, buffer or 
discard?

Cheers

Magnus

Den 2016-09-16 kl. 21:33, skrev Jonathan Lennox:
>> Agree with both of these.  I think they are both the same as “Whiskey
>> & Soda Example” - or am I missing something about theses?
>>
>>> 1. The PT are all the same.  An RTCP source description for an SSRC
>>> contains a MID.  This latches the MID.  Later, RTP packets with this
>>> SSRC arrive without a MID. They go to the place matching the MID that
>>> was in RTCP.
>>>
>>> 2. The PT are all the same.  A packet contains MID1.  It goes to the
>>> place that matches MID1, latching the SSRC to MID1.  Later, a packet
>>> with the same SSRC contains MID2. This moves the latching, so the
>>> SSRC is now latched to MID2.
>>>
>>>
>>>> change, contain a MID. If the packet has  MID, it goes to the place
>>>> that matches that MID *and* it latches that SSRC so any future SSRC
>>>> that don’t have a MID go the to the same place.
>>
>
>
>

-- 

Magnus Westerlund

----------------------------------------------------------------------
Services, Media and Network features, Ericsson Research EAB/TXM
----------------------------------------------------------------------
Ericsson AB                 | Phone  +46 10 7148287
Färögatan 6                 | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden | mailto: [email-masked]
----------------------------------------------------------------------

---

From: Christer Holmberg <[email-masked]>
Date: Fri, 23 Sep 2016 09:03:01 +0000

Hi,

If we define the RTP mux procedures in BUNDLE, do we need to define the
RTP demux algorithms in JSEP to begin with?

Is there something JSEP specific?

Regards,

Christer

On 23/09/16 11:56, "rtcweb on behalf of Magnus Westerlund"
<[email-masked] on behalf of [email-masked]>
wrote:

>> Just that RTCP counts for latching, and that SSRCs¹ MIDs can change.
>>
>>> Agree with both of these.  I think they are both the same as ³Whiskey
>>> & Soda Example² - or am I missing something about theses?
>>>
>>>> I¹m not going to come up with beverage names for my examples, but
>>>> feel free to make up your own.
>>>>
>>>>> To propose a few use caseŠ.
>>>>>
>>>>> ³Whiskey & Soda Example²
>>>>> that don¹t have a MID go the to the same place.
>>>
>>
>>
>>

---

From: Magnus Westerlund <[email-masked]>
Date: Fri, 23 Sep 2016 11:10:54 +0200

> RTP demux algorithms in JSEP to begin with?

I think BUNDLE can take care of the BUNDLE level stuff. Then we come to 
the next level which is Simulcast and RID usage. Here I think it is 
obvious that the Simulcast needs more discussion of the RTP issues. In 
Berlin we had a discussion on what behaviours a RTP receiver can expect 
to see from a middlebox selecting from simulcast streams. Those details 
needs to be written up, and that include if RID should be maintained in 
the RTP stream between the last RTP middlebox and the receiving client.

> Is there something JSEP specific?

I am not certain but I don't want to answer this question yet. Not until 
we actually written up the BUNDLE part and the Simulcast one. Then still 
remains the question if there are legacy or combination considerations 
that is not well enough handled. But, I think we should try to push as 
much of this as far down into the source specs as possible.

/Magnus

-- 

Magnus Westerlund

---

From: Bernard Aboba <[email-masked]>
Date: Fri, 23 Sep 2016 10:12:44 +0100

Magnus said:

" in a session where MID and RIDs are configured, if one receive packets
with an SSRC that lacks these SDES items, what do you do, buffer or discard?
"

[BA] Assuming that the SSRC does not match any signaled SSRC (nor any PT),
then in WebRTC 1.0, the packet would be discarded.  In ORTC, there is a
moderate buffer along with an unhandledrtp event.

On Fri, Sep 23, 2016 at 9:56 AM, Magnus Westerlund <
[email-masked]> wrote:

> bound to the SSRC until further notice. So packets arriving without actual
> RTP header extensions can still be handled according to the SSRCs
> associated SDES items. From my perspective all the cases with MID and RID
> can be handled on a per packet basis rules without latching the SSRC to a
> specific receiver. Instead what is "latched" is the SDES values to the
> SSRC. Still at the point of change there needs to take the combination of
> SSRC and Seq into account to avoid miss routing packets.
>
>    a. If they do, mark the Sequence number and keep the old SDES item as
> valid prior to this extended sequence number.
>
> 2. Determine the set of relevant SDES items that applies for this packet,
> i.e. use SSRC + RTP seq to look up the SDES items, e.g. MID and RID.
>
> using RTCP an interesting one. This put some light on the question that in
> a session where MID and RIDs are configured, if one receive packets with an
> SSRC that lacks these SDES items, what do you do, buffer or discard?
>
> Cheers
>
> Magnus
>

---

From: Bernard Aboba <[email-masked]>
Date: Fri, 23 Sep 2016 10:28:00 +0100

Christer said:

"If we define the RTP mux procedures in BUNDLE, do we need to define the
RTP demux algorithms in JSEP to begin with?"

[BA] I'd suggest that BUNDLE needs to lay out principles for RTP mux
procedures, but it
does not need to get into the details (which can be left to JSEP).

In particular, some of the most difficult use cases involve situations in
which the PTs are not unique.  IMHO, BUNDLE Section 10.2 does not provide
clear guidelines for how implementations should behave (and as a result
BUNDLE implementations currently are inconsistent in their behavior).

Some comments on Section 10.2 below:

As described in [Section 10.1.1], the same payload type value might
   be used inside RTP packets described by multiple "m=" lines.  In such
   cases, the payload type value cannot be used to associate received
   RTP packets with the correct "m=" line.

   [RFC5576].  To allow for proper association with this mechanism, the

[BA] One area of confusion relates to the meaning of the SSRCs
provided in the Answer when PTs are not unique. Is the implication
that the SSRCs in the Answer (and ONLY those SSRCs) will match the
m-line?

   defined in Section 14, where the remote endpoint inserts the

[BA] Is this only MUST support, or MUST use?  If MIDs might not be
used in a situation where PTs are not unique, then this section needs
to describe what happens if there is no SSRC match. Do RTP packets get
routed to the first m-line with a matching PT? Can a PT-match only
occur when there are no SSRCs specified (either for codecs or for
RTX/FEC)? So far, I have seen at least 3 BUNDLE implementations with
different interpretations of this section.

---

From: Christer Holmberg <[email-masked]>
Date: Fri, 23 Sep 2016 09:42:50 +0000

Hi,

In my opinion, all BUNDLE usages should perform the basic demuxing in the same way, and JSEP should reference that.

Then, only JSEP specific procedures shall be documented in JSEP.

Regards,

Christer

From: Bernard Aboba <[email-masked]<mailto:[email-masked]>>
Date: Friday 23 September 2016 at 12:28
To: Christer Holmberg <[email-masked]<mailto:[email-masked]>>
Cc: Magnus Westerlund <[email-masked]<mailto:[email-masked]>>, Jonathan Lennox <[email-masked]<mailto:[email-masked]>>, Cullen Jennings <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [rtcweb] JSEP: RTP demux algorithms

Christer said:

[BA] I'd suggest that BUNDLE needs to lay out principles for RTP mux procedures, but it
does not need to get into the details (which can be left to JSEP).

In particular, some of the most difficult use cases involve situations in which the PTs are not unique.  IMHO, BUNDLE Section 10.2 does not provide clear guidelines for how implementations should behave (and as a result BUNDLE implementations currently are inconsistent in their behavior).

[BA] One area of confusion relates to the meaning of the SSRCs provided in the Answer when PTs are not unique. Is the implication that the SSRCs in the Answer (and ONLY those SSRCs) will match the m-line?

[BA] Is this only MUST support, or MUST use?  If MIDs might not be used in a situation where PTs are not unique, then this section needs to describe what happens if there is no SSRC match. Do RTP packets get routed to the first m-line with a matching PT? Can a PT-match only occur when there are no SSRCs specified (either for codecs or for RTX/FEC)? So far, I have seen at least 3 BUNDLE implementations with different interpretations of this section.

On Fri, Sep 23, 2016 at 10:03 AM, Christer Holmberg <[email-masked]<mailto:[email-masked]>> wrote:

Hi,

Regards,

Christer

On 23/09/16 11:56, "rtcweb on behalf of Magnus Westerlund"
<[email-masked]<mailto:[email-masked]> on behalf of [email-masked]<mailto:[email-masked]>>
wrote:

>>>> On Sep 16, 2016, at 11:59 AM, Jonathan Lennox <[email-masked]<mailto:[email-masked]>
>>>> <mailto:[email-masked]<mailto:[email-masked]>>> wrote:
>>>>
>>>>> On Sep 15, 2016, at 6:18 PM, Cullen Jennings <[email-masked]<mailto:[email-masked]>
>>>>> <mailto:[email-masked]<mailto:[email-masked]>>> wrote:
>>>>>
>>>>>
>Ericsson AB                 | Phone  +46 10 7148287<tel:%2B46%2010%207148287>
>Färögatan 6                 | Mobile +46 73 0949079<tel:%2B46%2073%200949079>
>SE-164 80 Stockholm, Sweden | mailto: [email-masked]<mailto:[email-masked]>

---

From: Harald Alvestrand <[email-masked]>
Date: Mon, 26 Sep 2016 10:48:35 +0200

> 3. Perform the demuxing algorithm

I don't understand this as written.
When an RTP packet arrives before the RTCP packet containing SDES, we
can't latch to the content of the RTCP packet; we haven't seen it yet.

You can, in theory, latch RID and MID for the SSRC to the values in the
SDES packet you haven't seen yet, but that seems somewhat backwards.

The problematic sequence is:

1) RTP packet(s) with header extensions
2) RTP packet(s) without header extensions
3) SDES RTCP packet

(I'm assuming that we require consistency between SDES and RTP header
extensions.
What do we do if they're inconsistent?)

---

From: Magnus Westerlund <[email-masked]>
Date: Mon, 26 Sep 2016 11:14:55 +0200

> can't latch to the content of the RTCP packet; we haven't seen it yet.

So if neither RTP header extension nor RTCP SDES item with RID/MID for 
that SSRC has been received, then you don't have that information and 
you will have to either queue or drop that packet if the relation isn't 
otherwise clear through PT or SSRC signalling in SDP.

> 2) RTP packet(s) without header extensions

I don't see why the above is problematic. Step one gives you the SDES 
item for that SSRC. Thus in step 2 one knows that the SSRC has the 
MID/RID value that you need.

I don't understand why you are not binding the SDES info from the RTP 
header extension to the SSRC.

> What do we do if they're inconsistent?)

So, the RTP header extension is the SDES item also. So if one gets 
another SDES item value, then one should update, however one needs to 
prevent flapping so one needs to figure out which of the RTCP and RTP 
header extension was sent last. See Section 4.2.6 in RFC 7941.

Cheers

Magnus Westerlund

---

From: Harald Alvestrand <[email-masked]>
Date: Mon, 26 Sep 2016 11:35:00 +0200

> header extension to the SSRC.

OK, I had missed the change of the RID header extension from being its
own header extension to being a component of the SDES header extension only.

Then it all makes sense! Thanks!

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Mon, 26 Sep 2016 17:00:17 +0200

Hi, sorry for the late response. All my concerns will be focused on
the SFU scenario in which a peer/client receives multiple
streams/tracks from many participants over a single DTLS transport.

> The packets get delivered to the thing with the matching PT.

If multiple RtpReceivers are receiving audio tracks with same PT over
the same transport (SFU scenario) this algorithm will fail since, most
probably, all those RtpReceivers expect the same PT value for the same
codec. SSRC or MID is required here.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Bernard Aboba <[email-masked]>
Date: Mon, 26 Sep 2016 12:23:38 -0400

Inaki said:

"If multiple RtpReceivers are receiving audio tracks with same PT over
the same transport (SFU scenario) this algorithm will fail since, most
probably, all those RtpReceivers expect the same PT value for the same
codec. SSRC or MID is required here."

[BA] The above case corresponds to a separate test case.  The difficult
part of developing the routing rules is figuring out all the test cases as
well as being able to easily run them against a proposed algorithm (to
avoid regressions when making changes).

Is there any code in your SFU project that might help us with that?

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Tue, 27 Sep 2016 17:02:38 +0200

> Is there any code in your SFU project that might help us with that?

Currently it just routes based on SSRC and PT values (as signaled in
the SDP or ORTC receive() method). In fact I'm waiting for all this
stuff to be clarified and normalized in order to implement it :)

-- 
Iñaki Baz Castillo
<[email-masked]>