Subject: [rtcweb] JSEP: Issues with a=ssrc and RTP payload type switching

From: Magnus Westerlund <[email-masked]>
Date: Wed, 3 Jun 2015 14:37:13 +0200

Hi,

I want raise for discussion the fact that I think JSEP does not handle 
RTP payload type switching as currently specified.

So lets walk through a case of RTP payload type switching and what it 
means according to the specifications we do have.

So we have an established PeerConnection with one audio track for which 
we have negotiated the possibility to send both PCM and OPUS. If the 
sender decides to switch RTP payload type, lets say from OPUS to PCM. 
The reason is actually not important, but lets say due to a "replace 
track" API call as currently being discussed on W3C WebRTC group.

When that RTP payload type switch from OPUS to PCM we also have an RTP 
timestamp rate change, from 48 kHz to 8 kHz. As specified in RTP usage 
(Section 4.3):

    If performing changes between two RTP payload types that use
    different RTP clock rates, an RTP sender MUST follow the
    recommendations in Section 4.1 of [RFC7160].

So lets look at what RFC 7160 says:

4.1.  RTP Sender (with RTCP)

    An RTP Sender with RTCP turned on MUST use a different SSRC for each
    different clock rate.  An RTCP BYE MUST be sent and a new SSRC MUST
    be used if the clock rate switches back to a value already seen in
    the RTP stream.

Thus, this Payload type change forces the use of a new SSRC.

This in itself should not be a significant issue. The binding to the 
right media block in the SDP is done using MID. Which, will be sent in 
an RTP header extension to ensure that stream upon reception is 
correctly associated. The RTP synchronization information will arrive in 
RTCP or even in the RTP header extension if the peers both supports RFC 
6051. Thus minimal media impacts is to be expected.

But, discussing what happens in this case with Stefan Håkansson we did 
look into JSEP. And here we clearly have some short commings in the 
specification.

A. First of all there is an assumption that the value of a=ssrc will not 
change. For example Section 5.2.2 (Subsequent Offers):

   o  For MediaStreamTracks that are still present, the "a=msid",
       "a=ssrc", and "a=ssrc-group" lines MUST stay the same.

If an RTP payload type switching has occured that forces a SSRC change 
then I expect that the actually used SSRCs are included in the offer. 
Thus, both a=ssrc and a=ssrc-group can change in subsequent offers.

B. The next question is if a RTP payload type change actually requires 
the node that performs the change to actually initiate an Offer/answer 
exchange to update their a=ssrc values?

Clearly JSEP needs to be updated to address these issues.

Cheers

Magnus Westerlund

----------------------------------------------------------------------
Services, Media and Network features, Ericsson Research EAB/TXM
----------------------------------------------------------------------
Ericsson AB                 | Phone  +46 10 7148287
Färögatan 6                 | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden | mailto: [email-masked]
----------------------------------------------------------------------

---

From: Bernard Aboba <[email-masked]>
Date: Wed, 3 Jun 2015 11:17:18 -0700

Magnus said:

"This in itself should not be a significant issue. The binding to the right
media block in the SDP is done using MID. Which, will be sent in an RTP
header extension to ensure that stream upon reception is correctly
associated. The RTP synchronization information will arrive in RTCP or even
in the RTP header extension if the peers both supports RFC 6051. Thus
minimal media impacts is to be expected."

[BA] I do agree that this is a basic scenario that should work. However,
Section 5.2.1 of draft-ietf-rtcweb-rtp-usage indicates that RFC 6051 is
RECOMMENDED, but not mandatory.

"B. The next question is if a RTP payload type change actually requires the
node that performs the change to actually initiate an Offer/answer exchange
to update their a=ssrc values?"

[BA]Today in audio scenarios the use of a=ssrc is not very prevalent
outside of WebRTC, and yet the switching scenario you describe still
typically works without an additional O/A exchange.  So a developer
experienced in legacy systems might be surprised if another O/A exchange
was needed.  So I wonder if the a=ssrc line should always be required (and
if it is not there, whether the audio scenario should be expected to "just
work").

One argument in favor of requiring a=ssrc is that at the RTP level the
switching scenario may not be that easily distinguished from a scenario
where multiple SSRCs are being received at once (where a mediadiscarded
Event might be appropriate).  Also, in video scenarios the a=ssrc line is
much more common (e.g. where one of several simulcast streams is being
received, the SSRC can change along with characteristics of the video such
as framerate and resolution).

Overall though, IMHO the behavior in this scenario is currently
under-specified.  The Unified Plan document included some text relating to
this, but it doesn't seem to have made its way into any WG work items.

On Wed, Jun 3, 2015 at 5:37 AM, Magnus Westerlund <
[email-masked]> wrote:

> Hi,
>
> I want raise for discussion the fact that I think JSEP does not handle RTP
> payload type switching as currently specified.
>
> So we have an established PeerConnection with one audio track for which we
> have negotiated the possibility to send both PCM and OPUS. If the sender
> decides to switch RTP payload type, lets say from OPUS to PCM. The reason
> is actually not important, but lets say due to a "replace track" API call
> as currently being discussed on W3C WebRTC group.
>
> This in itself should not be a significant issue. The binding to the right
> media block in the SDP is done using MID. Which, will be sent in an RTP
> header extension to ensure that stream upon reception is correctly
> associated. The RTP synchronization information will arrive in RTCP or even
> in the RTP header extension if the peers both supports RFC 6051. Thus
> minimal media impacts is to be expected.
>
> then I expect that the actually used SSRCs are included in the offer. Thus,
> both a=ssrc and a=ssrc-group can change in subsequent offers.
>
> B. The next question is if a RTP payload type change actually requires the
> node that performs the change to actually initiate an Offer/answer exchange
> to update their a=ssrc values?
>
> _______________________________________________
> rtcweb mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rtcweb
>

---

From: Harald Alvestrand <[email-masked]>
Date: Wed, 03 Jun 2015 22:06:52 +0200

On 06/03/2015 02:37 PM, Magnus Westerlund wrote:
> So we have an established PeerConnection with one audio track for 
> which we have negotiated the possibility to send both PCM and OPUS. If 
> the sender decides to switch RTP payload type, lets say from OPUS to 
> PCM. The reason is actually not important, but lets say due to a 
> "replace track" API call as currently being discussed on W3C WebRTC 
> group.
>
> correctly associated. The RTP synchronization information will arrive 
> in RTCP or even in the RTP header extension if the peers both supports 
> RFC 6051. Thus minimal media impacts is to be expected.
>
> A. First of all there is an assumption that the value of a=ssrc will 
> not change. For example Section 5.2.2 (Subsequent Offers):
>

If we want to have a mechanism that:

a) allows PT switching
b) uses different clock rates (and therefore different SSRCs)
c) doesn't need renegotiation on SSRC change

I would think that the initial SDP would have to contain a=ssrc fields 
for *all* the possible SSRCs - that is, one "a=ssrc:<nnnn> 
cname:<cname>" line per payload type.

Section 5.2.1 already specifies that there will be multiple a=ssrc lines 
(and a=ssrc-group lines) in the case of RTX data. So it's not exactly a 
novel solution.

---

From: Roman Shpount <[email-masked]>
Date: Thu, 4 Jun 2015 17:02:44 -0400

On Wed, Jun 3, 2015 at 4:06 PM, Harald Alvestrand <[email-masked]>
wrote:

> I would think that the initial SDP would have to contain a=ssrc fields for
> *all* the possible SSRCs - that is, one "a=ssrc:<nnnn> cname:<cname>" line
> per payload type.
>
I agree with one small correction -- it is not a separate SSRC per payload
type, but separate SSRC per clock rate. For instance, if you have G.711
audio, RFC 4733 tones and CN at 8KHz, there is going to be a payload type
for each of them, but all this media should use the same SSRC. Opus, RFC
4733 tones and CN at 48Khz (and you will need to specify to different
payloads for tones and CN for different clock rates), will use second SSRC.

The reason for separate SSRCs is to have RTCP reports to work correctly,
which requires to use the same time units for all the media under
particular SSRC. Multiple payloads running with the same clock rate can
re-use the same SSRC without any issues. It actually is beneficial to use
the same SSRC since they can be synchronized based on the RTP time stamps
which would be more precise then ntp time.
_____________
Roman Shpount

---

From: Magnus Westerlund <[email-masked]>
Date: Mon, 8 Jun 2015 10:48:45 +0200

Roman Shpount skrev den 2015-06-04 23:02:
> On Wed, Jun 3, 2015 at 4:06 PM, Harald Alvestrand <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>
>     I would think that the initial SDP would have to contain a=ssrc
>     fields for *all* the possible SSRCs - that is, one "a=ssrc:<nnnn>
>     Section 5.2.1 already specifies that there will be multiple a=ssrc
>     lines (and a=ssrc-group lines) in the case of RTX data. So it's not
>     exactly a novel solution.
>
>
> I agree with one small correction -- it is not a separate SSRC per
> payload type, but separate SSRC per clock rate. For instance, if you
> have G.711 audio, RFC 4733 tones and CN at 8KHz, there is going to be a
> payload type for each of them, but all this media should use the same
> SSRC. Opus, RFC 4733 tones and CN at 48Khz (and you will need to specify
> to different payloads for tones and CN for different clock rates), will
> use second SSRC.
>
> re-use the same SSRC without any issues. It actually is beneficial to
> use the same SSRC since they can be synchronized based on the RTP time
> stamps which would be more precise then ntp time.

However, this can't work as RFC 7160 says in Section 4.1:

Note the second sentence. One would still have to do a new round of 
signalling to prime a new SSRC for the previous rate after having 
switched to be able to switch back to that Payload Type.

I would to some degree argue that renegotiation is not needed here. My 
preference would be for a solution that rather use header extensions to 
indicate what "purpose" the new RTP stream has so that one understands 
that this is the replacement for an earlier RTP stream.

For the cases where a source is encoded into a single encoded stream and 
put in one RTP stream I think all the necessary indications are there. 
MID for which m line it belongs. The PT will indicate if it a source RTP 
stream or an Redundancy RTP stream (FEC or RTX).

It is when one comes to scalability layers sent in multiple RTP streams 
it becomes problematic to determine on RTP level which is which, without 
looking into payload header info to determine which layer the payload 
contains.

Having said that I do wonder if this will be a problem, especially in 
gateways to legacy. Where this behaviour will make the legacy peer fall 
over. Thus requiring a gateway that remap back to a single stable SSRC, 
possibly with loss of synch information. Please remember that there are 
reasons why the SSRC change was made as the required behaviour.

Cheers

Magnus Westerlund

---

From: Harald Alvestrand <[email-masked]>
Date: Mon, 08 Jun 2015 11:19:14 +0200

> switched to be able to switch back to that Payload Type.

Hm. I missed that when reviewing the clock-rate doc.
So that means one uses up an SSRC for every clock rate switch. That
seems silly; is it possible that this should be considered an erratum
for RFC 7160?

> that this is the replacement for an earlier RTP stream.

That would of course be incompatible with anything that came before.

> looking into payload header info to determine which layer the payload

But I just can't imagine switching clock rates for different layers of a
scalability-encoded stream. Should we just not solve the problem of
behaving bizarrely for this case?

> reasons why the SSRC change was made as the required behaviour.

Are there legacy gateways that will NOT fall over on a clock rate + SSRC
switch?
If so - what do these legacy gateways do when switching back to the
original PT? New SSRC, or back to a previously used SSRC?

---

From: Magnus Westerlund <[email-masked]>
Date: Mon, 8 Jun 2015 11:36:55 +0200

> seems silly; is it possible that this should be considered an erratum

No, it is highly intentional as I remember the discussion.

> That would of course be incompatible with anything that came before.

All that has used multiple streams per endpoint before has had some 
implicit rules to handle different situations. What is forcing the issue 
here is the realization about RFC 7160 forcing SSRC changes. If there is 
single encoding per endpoint, then using CNAME and PT is sufficient to 
associate when one switches. If one have multiple streams then the fun 
starts. IETF has never had any specifications for solving this, so from 
that perspective we are laying down a solution where different 
interpretations has been used based on what is specific for that 
application.

> behaving bizarrely for this case?

That was not what I meant. If one switches to or from a scalable encoder 
or between different encoders, the one might be in a situation where one 
needs to identify that one set of SSRCs are now replaced with a 
different set of SSRCs.

What I was unclear on was that multiple RTP streams for a single media 
source due to scalable encoding anyway has this issue with 
identification. I think the solution to that is likely a combination of 
MID and something like the header marking header extension being 
proposed in AVTEXT:

https://datatracker.ietf.org/doc/draft-berger-avtext-framemarking/

That way the identification information is available to the receiver 
directly before having to look into payload. Something that will be 
impossible in a PERC scenario anyway.

> Are there legacy gateways that will NOT fall over on a clock rate + SSRC

Good Question!

Cheers

Magnus Westerlund

---

From: Simon Perreault <[email-masked]>
Date: Mon, 08 Jun 2015 08:15:00 -0600

> No, it is highly intentional as I remember the discussion.

Can you please explain, for those of us with short memories? :)

>> Are there legacy gateways that will NOT fall over on a clock rate + SSRC

Depends what you mean by "legacy". My gateway will not fall over.

>> original PT? New SSRC, or back to a previously used SSRC?

As RFC 7160 says: new SSRC.

Simon

---

From: Magnus Westerlund <[email-masked]>
Date: Mon, 8 Jun 2015 16:43:05 +0200

> Can you please explain, for those of us with short memories? :)

I went and checked this. This formulation has been present since the 
first versions, and no one has questioned this RFC 2119 worded 
statement. Therefore I consider there to be consensus behind it and 
being intentional.

The motivation I see for this statement is the following. First of all I 
think people have been against considering having stand bye SSRCs that 
are really likely to never be used. I think DTMF is one of these 
exceptions. Having stand by SSRC also have an overhead cost, primarily 
in the result of the number of SSRCs present in the session and their 
consumption of their share of the RTCP bandwidth resources. We also have 
the issue of correctly associating the SSRC to a media source, now that 
there are more than one.

Based on that, (with the assumption that you do PT switching rarely) the 
lower cost is to drop the SSRC after its usage. From that follows that 
one stop using it, and therefore sends BYE, then you really should not 
revive a dead SSRC. Thus, need to create a new one. Simply codifying 
this expectation in the RFC.

> Depends what you mean by "legacy". My gateway will not fall over.

I think this is an import aspect. The gateway is not legacy, as it is an 
WebRTC gateway. It is the endpoint on the other side of the gateway that 
is "legacy".

Cheers

Magnus Westerlund

---

From: Simon Perreault <[email-masked]>
Date: Mon, 08 Jun 2015 08:57:33 -0600

> WebRTC gateway. It is the endpoint on the other side of the gateway that

My modus operandi with endpoints is: if you haven't tested it and seen
it working, then you must assume it will fail. It's a sad state of
affairs. Implementers should be whipped and forced to attend SIPit. ;)

In any case, that's why we have SBCs.

Personally, having the WebRTC clients do the right thing makes my life
easier. Focusing too much on compatibility with legacy makes my life
harder. I can make the gateway do the fixup easily. But as WebRTC
inevitably grows and SIP diminishes, my life becomes gradually happier. :D

Simon

---

From: Harald Alvestrand <[email-masked]>
Date: Mon, 08 Jun 2015 17:09:07 +0200

On 06/08/2015 04:43 PM, Magnus Westerlund wrote:
> The motivation I see for this statement is the following. First of all
> I think people have been against considering having stand bye SSRCs
> that are really likely to never be used. I think DTMF is one of these
> consumption of their share of the RTCP bandwidth resources. We also
> have the issue of correctly associating the SSRC to a media source,
> now that there are more than one.
>
> Based on that, (with the assumption that you do PT switching rarely)
> the lower cost is to drop the SSRC after its usage. From that follows
> that one stop using it, and therefore sends BYE, then you really
> should not revive a dead SSRC. Thus, need to create a new one. Simply
> codifying this expectation in the RFC.

This has another implication:

When doing PT switching for comfort noise, you MUST have a CN with the
same clock rate as your normal audio codec.

CN is exactly the type of PT-switching that, if it occurs at all, occurs
very frequently.

---

From: Simon Perreault <[email-masked]>
Date: Mon, 08 Jun 2015 09:15:04 -0600

> CN is exactly the type of PT-switching that, if it occurs at all, occurs

I'm not following. We're discussing generating a new SSRC on clock
rate-switching, not on PT-switching. For the CN example, you would
typically include in your SDP one CN payload type per possible clock
type so that you can switch payload types without switching the clock rate.

Simon

---

From: Harald Alvestrand <[email-masked]>
Date: Mon, 08 Jun 2015 17:17:38 +0200

> type so that you can switch payload types without switching the clock rate.

Yep. It's only because I've seen SDP trying to match CN/8000 with
OPUS/48000 that I'm mentioning it - it's obvious, but only after it's
bitten you.

---

From: Magnus Westerlund <[email-masked]>
Date: Tue, 9 Jun 2015 10:20:05 +0200

> OPUS/48000 that I'm mentioning it - it's obvious, but only after it's

Yes, this is likely one of these things that the ones that considered 
the issues see as obvious, but is in fact not until you actually tried 
it and therefore could have benefited from a note about the need for 
having one PT configuration per timestamp rate used by the codecs that 
may use the CN.

As the RTP usage is in IESG review, I am hesitant to add such discussion 
into the RTP usage document. As it is really a configuration issue, I 
think it can be discussed in JSEP, but the audio draft might be an even 
better fit adding to the CN text.

Cheers

Magnus Westerlund

---

From: Christer Holmberg <[email-masked]>
Date: Fri, 12 Jun 2015 08:33:09 +0000

Hi,

Keep in mind that whatever changes we do in JSEP must be "compatible" with BUNDLE :)

However, BUNDLE currently doesn't mandate the usage of the SDP 'ssrc' attribute, but it does have the following rule on SSRC usage:

	o  A given SSRC MUST NOT transmit RTP packets using payload types
      	that originate from different bundled "m=" lines.

However, that doesn't seem to be related to the ongoing discussion.

Regards,

Christer

-----Original Message-----
From: rtcweb [mailto:[email-masked]] On Behalf Of Magnus Westerlund
Sent: 9. kesäkuuta 2015 11:20
To: Harald Alvestrand; Simon Perreault; Roman Shpount
Cc: [email-masked]
Subject: Re: [rtcweb] JSEP: Issues with a=ssrc and RTP payload type switching

Harald Alvestrand skrev den 2015-06-08 17:17:
>>> When doing PT switching for comfort noise, you MUST have a CN with 
>>> the same clock rate as your normal audio codec.
>>>
>>> CN is exactly the type of PT-switching that, if it occurs at all, 
>>> occurs very frequently.
>>

Yes, this is likely one of these things that the ones that considered the issues see as obvious, but is in fact not until you actually tried it and therefore could have benefited from a note about the need for having one PT configuration per timestamp rate used by the codecs that may use the CN.

As the RTP usage is in IESG review, I am hesitant to add such discussion into the RTP usage document. As it is really a configuration issue, I think it can be discussed in JSEP, but the audio draft might be an even better fit adding to the CN text.

Cheers

Magnus Westerlund