Subject: [rtcweb] Review of draft-ietf-rtcweb-rtp-usage

From: Bernard Aboba <[email-masked]>
Date: Wed, 7 May 2014 12:25:33 -0700

Here are a few things that caught my eye in re-reading
draft-ietf-rtcweb-rtp-usage:

Circuit Breaker requirements

Section 7.1

   In the absence of a concrete congestion control algorithm, all WebRTC
   implementations MUST implement the RTP circuit breaker algorithm that
   is described in
[I-D.ietf-avtcore-rtp-circuit-breakers<http://tools.ietf.org/html/draft-ietf-rtcweb-rtp-usage-13#ref-I-D.ietf-avtcore-rtp-circuit-breakers>].

[BA] What does "concrete" mean here?  There are a number of congestion
control algorithms that one might or might not consider "concrete".  Also
at various meetings, I have heard people talk about implementation of
Circuit Breakers along with congestion control algorithms.

Personally, I would consider sender-side congestion control algorithms to
qualify as "concrete", assuming that the algorithm could function
adequately when interacting with standards-based implementations only
supporting the functionality described in the RTP usage document (e.g. RTCP
reports, etc.).  I am not sure that receiver-side congestion control
mechanisms should qualify as "concrete" if they require both sender and
receiver to implement proprietary functionality for congestion control to
function adequately.

SDP dependencies

Since WebRTC is signaling-independent, at various points in discussion of
the RTP usage document, it has been pointed out that it is not appropriate
to mandate SDP signaling.  However, in re-reading the draft, a number of
SDP mandates still remain.  I'd like to see these changed to state the
requirement more generically, with guidance provided on what to do if SDP
signaling is used.

Section 4.8

   Implementations are REQUIRED to support signalled RTP synchronisation
   source (SSRC) identifiers, using the "a=ssrc:" SDP attribute defined
   in Section 4.1<http://tools.ietf.org/html/draft-ietf-rtcweb-rtp-usage-13#section-4.1>and
Section 5
of [RFC5576] <http://tools.ietf.org/html/rfc5576#section-5>.
Implementations MUST also
   support the "previous-ssrc" source attribute defined in Section 6.2 of
[RFC5576] <http://tools.ietf.org/html/rfc5576#section-6.2>.  Other per-SSRC
attributes defined in [RFC5576 <http://tools.ietf.org/html/rfc5576>] MAY be
   supported.

Propose this be changed to:

   Implementations are REQUIRED to support signalled RTP synchronisation
   source (SSRC) identifiers.  In an SDP context, this can be done using
the "a=ssrc:" SDP attribute defined in Section
4.1<http://tools.ietf.org/html/draft-ietf-rtcweb-rtp-usage-13#section-4.1>and
Section 5
of [RFC5576] <http://tools.ietf.org/html/rfc5576#section-5> in addition
to the "previous-ssrc" source attribute defined in Section 6.2; other
per-SSRC attributes defined in [RFC5576 <http://tools.ietf.org/html/rfc5576>]
MAY be supported.

---

From: Colin Perkins <[email-masked]>
Date: Thu, 8 May 2014 11:45:49 +0100

On 7 May 2014, at 20:25, Bernard Aboba <[email-masked]> wrote:
> Here are a few things that caught my eye in re-reading draft-ietf-rtcweb-rtp-usage:
> 
>    is described in [I-D.ietf-avtcore-rtp-circuit-breakers].
> 
> [BA] What does "concrete" mean here?  There are a number of congestion control algorithms that one might or might not consider "concrete".  Also at various meetings, I have heard people talk about implementation of Circuit Breakers along with congestion control algorithms. 

The fix here is to delete “In the absence of a concrete congestion control algorithm, all”, leaving the text as “WebRTC implementations MUST implement the RTP circuit breaker…” since as you note, we want the circuit breaker even if there is congestion control.

> Personally, I would consider sender-side congestion control algorithms to qualify as "concrete", assuming that the algorithm could function adequately when interacting with standards-based implementations only supporting the functionality described in the RTP usage document (e.g. RTCP reports, etc.).  I am not sure that receiver-side congestion control mechanisms should qualify as “concrete" if they require both sender and receiver to implement proprietary functionality for congestion control to function adequately.

By “concrete”, I just meant something where you could point to an RFC and say “there is agreement that this is a reasonable choice for a congestion control algorithm”. 

> SDP dependencies
> 
> Since WebRTC is signaling-independent, at various points in discussion of the RTP usage document, it has been pointed out that it is not appropriate to mandate SDP signaling.  However, in re-reading the draft, a number of SDP mandates still remain.  I’d like to see these changed to state the requirement more generically, with guidance provided on what to do if SDP signaling is used.

Other than that in Section 4.8 you highlight below, did you find any other specific places that need changing? I had thought we’d fixed these already, but if you have any we missed, please let us know.

>    in Section 4.1 and Section 5 of [RFC5576].  Implementations MUST also
>    support the "previous-ssrc" source attribute defined in Section 6.2 of [RFC5576].  Other per-SSRC attributes defined in [RFC5576] MAY be
>    supported.
> 
>    source (SSRC) identifiers.  In an SDP context, this can be done using the “a=ssrc:" SDP attribute defined in Section 4.1 and Section 5 of [RFC5576] in addition to the "previous-ssrc" source attribute defined in Section 6.2; other per-SSRC attributes defined in [RFC5576] MAY be supported.

I rephrased this slightly as:

        <t>Implementations are REQUIRED to support signalled RTP
        synchronisation source (SSRC) identifiers. If SDP is used, this MUST
        be done using the "a=ssrc:" SDP attribute defined in Section 4.1 and
        Section 5 of <xref target="RFC5576"/> and the "previous-ssrc" source
        attribute defined in Section 6.2 of <xref target="RFC5576"/>; other
        per-SSRC attributes defined in <xref target="RFC5576"/> MAY be
        supported.</t>

Colin

-- 
Colin Perkins
http://csperkins.org/

---

From: Bernard Aboba <[email-masked]>
Date: Thu, 8 May 2014 05:43:04 -0700

> On May 8, 2014, at 3:45, Colin Perkins <[email-masked]> wrote:
> 

[BA] The question I have is whether this would be widely implemented. The Circuit Breaker algorithm pauses sessions, but has no mechanism to resume them, so it increases brittleness.

---

From: Colin Perkins <[email-masked]>
Date: Thu, 8 May 2014 13:54:08 +0100

> [BA] The question I have is whether this would be widely implemented. The Circuit Breaker algorithm pauses sessions, but has no mechanism to resume them, so it increases brittleness. 

The circuit breaker is a mechanism of last resort. If we've designed it correctly, it will only stop sessions that are otherwise unusable. The resume mechanism is the redial button on the user interface. With a reasonable congestion control algorithm, the circuit breaker should never be triggered.

Colin

---

From: Emil Ivov <[email-masked]>
Date: Thu, 08 May 2014 18:47:39 +0200

On 08.05.14, 14:54, Colin Perkins wrote:
>>> On May 8, 2014, at 3:45, Colin Perkins <[email-masked]>
>>> wrote: The fix here is to delete “In the absence of a concrete
>>> congestion control algorithm, all”, leaving the text as “WebRTC
>>> implementations MUST implement the RTP circuit breaker…” since as
>>> you note, we want the circuit breaker even if there is congestion
>>> control.
>>
>> [BA] The question I have is whether this would be widely
>> implemented. The Circuit Breaker algorithm pauses sessions, but has
>> no mechanism to resume them, so it increases brittleness.
>
> The circuit breaker is a mechanism of last resort. If we've designed
> it correctly, it will only stop sessions that are otherwise unusable.

Shouldn't we have the "if" cleared before making this a MUST? It is not 
clear to me that we've managed to avoid excessive false positives for 
example.

> The resume mechanism is the redial button on the user interface. With
> a reasonable congestion control algorithm, the circuit breaker should
> never be triggered.

This is another part that bothers me with circuit breakers. If there's a 
reasonable chance that a "redial" would succeed then why did we break 
the session? If not then why are we encouraging it?

I am not saying that this can never make sense but it feels like there's 
too much application logic in there for this to be a MUST for WebRTC stacks.

Emil

-- 
https://jitsi.org

---

From: Bernard Aboba <[email-masked]>
Date: Thu, 8 May 2014 11:17:30 -0700

> On May 8, 2014, at 9:47 AM, Emil Ivov <[email-masked]> wrote:
>> 

[BA] Unusable at a given moment could change to usable later, either as a result of congestion control support and/or changing conditions. So it is one thing to stop sending for some period and indicate to the user that difficulties are being experienced then try again after backing down the send Bw, and another to stop and require a redial. 

> 
> This is another part that bothers me with circuit breakers. If there's a reasonable chance that a "redial" would succeed then why did we break the session? If not then why are we encouraging it?

[BA] As specified, circuit breakers is vulnerable to routing transients. Other transports such as TCP are explicitly designed to survive this, but circuit breakers will not.  So we need to ask ourselves why TCP never needed (or considered) such a mechanism.

---

From: Colin Perkins <[email-masked]>
Date: Thu, 8 May 2014 19:53:28 +0100

On 8 May 2014, at 19:17, Bernard Aboba <[email-masked]> wrote:
>>> The circuit breaker is a mechanism of last resort. If we’ve designed it correctly, it will only stop sessions that are otherwise unusable.
> 

The circuit breaker draft explicitly allows for this back-off and resume behaviour. From section 4.3 of draft-ietf-avtcore-rtp-circuit-breakers-05:

   Systems that usually
   send at a high data rate, but that can reduce their data rate
   significantly (i.e., by at least a factor of ten), MAY first reduce
   their sending rate to this lower value to see if this resolves the
   congestion, but MUST then cease transmission if the problem does not
   resolve itself within a further two RTCP reporting intervals (see
   Section 4.5).  An example of this might be a video conferencing
   system that backs off to sending audio only, before completely
   dropping the call.  If such a reduction in sending rate resolves the
   congestion problem, the sender MAY gradually increase the rate at
   which it sends data after a reasonable amount of time has passed,
   provided it takes care not to cause the problem to recur
   ("reasonable" is intentionally not defined here).

If there is a congestion control algorithm implemented in addition, then this ought to back off long before the circuit breaker would trigger anyway, so the problem won’t occur.

> [BA] As specified, circuit breakers is vulnerable to routing transients. Other transports such as TCP are explicitly designed to survive this, but circuit breakers will not.  So we need to ask ours…

I suggest that the user will almost certainly have hang-up the call in frustration as a result of the drop-out before the circuit breaker fires. If people think the timeouts in the circuit breaker draft are too short, or the packet loss thresholds too tight, then please comment on the circuit breaker draft (in AVTCORE) and we’ll work to improve it. So far, we haven’t had any real push-back against the approach taken in that draft. 

I do think a circuit breaker is REQUIRED for WebRTC, however, especially as RMCAT is not making rapid progress with congestion control.

Colin

-- 
Colin Perkins
http://csperkins.org/

---

From: Colin Perkins <[email-masked]>
Date: Thu, 8 May 2014 20:02:15 +0100

On 8 May 2014, at 17:47, Emil Ivov <[email-masked]> wrote:
> Shouldn't we have the "if" cleared before making this a MUST? It is not clear to me that we’ve managed to avoid excessive false positives for example.

The experiments that I, Varun, and Zahed have done look to show the circuit breaker behaving as desired. If you know of scenarios where there are false positives, I’d like to see your data so we can improve the circuit breaker.

> This is another part that bothers me with circuit breakers. If there's a reasonable chance that a "redial" would succeed then why did we break the session? If not then why are we encouraging it?

If the circuit breaker triggers then I wouldn’t expect a redial to succeed immediately. Obviously, the longer you wait before redialing, the more likely the network conditions will have changed and the call will succeed. 

-- 
Colin Perkins
http://csperkins.org/

---

From: Emil Ivov <[email-masked]>
Date: Thu, 08 May 2014 23:48:50 +0200

On 08.05.14, 21:02, Colin Perkins wrote:
>>>>> wrote: The fix here is to delete “In the absence of a
>>>>> concrete congestion control algorithm, all”, leaving the text
>>>>> as “WebRTC implementations MUST implement the RTP circuit
>>>>> breaker…” since as you note, we want the circuit breaker even
>>>>> if there is congestion control.
>>>>
>>>> implemented. The Circuit Breaker algorithm pauses sessions, but
>>>> has no mechanism to resume them, so it increases brittleness.
>>>
>>> The circuit breaker is a mechanism of last resort. If we've
>>> designed it correctly, it will only stop sessions that are
>>> otherwise unusable.
>>
>> Shouldn't we have the "if" cleared before making this a MUST? It is
>> not clear to me that we’ve managed to avoid excessive false
>> positives for example.
>
> The experiments that I, Varun, and Zahed have done look to show the
> circuit breaker behaving as desired.

That sounds interesting and I'd actually be happy to see the data 
although I can easily believe that circuit breakers could come in handy 
in certain situations.

draft-ietf-avtcore-rtp-circuit-breakers-05 contains some very useful 
guidelines on determining how and when an application might want to 
consider employing them.

So I am fine with the concept ... *as long as* the choice is left to the 
application.

I don't believe that any of the breakers however have the reliability to 
only trigger in cases where ceasing media transmission is the right 
thing to do. I don't think such a thing is possible given how use cases 
and requirements vary from one application to the next.

> If you know of scenarios where
> there are false positives, I’d like to see your data so we can
> improve the circuit breaker.

It's funny you should ask. I am currently having connectivity issues and 
my media transmissions would every now and then be interrupted for 
periods that go beyond two or three RR packets. I am very happy that my 
sessions just resume when things go back to normal.

In cases like demo stands or some forms of video surveillance, lack of 
media for several (tens) of seconds is not really a problem, whereas a 
break that requires human intervention could be one.

Cases where SFUs decide to selectively NOT forward some RTP but forward 
all RTCP would also trigger false positives, while they are not a 
problem for applications that are aware of their behaviour.

IP layer network mobility à la NEMO, temporary uplink saturation because 
someone mistakenly started a torrent client (and quit a few seconds 
later), someone tripping over a cable and then re-plugging it, buggy NAT 
gateways ... all of those could potentially trigger circuit breakers and 
in all these cases both the application and the user may prefer that 
they don't.

Obviously browsers are a special case but we already have consent 
freshness there and that likely covers most of the concerns for 
undesirable streaming that circuit breakers address.

So again, I feel circuit breakers could be a useful informative 
reference but they can be harmful if REQUIRED.

Emil

-- 
https://jitsi.org

---

From: Magnus Westerlund <[email-masked]>
Date: Fri, 9 May 2014 11:53:35 +0200

> and requirements vary from one application to the next.

The circuit breakers triggers when the conditions are really bad. I
think the one condition which is most likely to give a false positive is
the loss of feedback, i.e. that one loses all RTCP packets under 3
regular reporting intervals.

It is very hard to define programmatic rules that always do the "Right
thing" here. Because with hindsight you can determine one action more
appropriate than another.

> sessions just resume when things go back to normal.

Well, the important aspect here is how your sending client behaves
during these outages. The circuit breaker do requires you to have cut
back the transmission in these conditions to less than a 10th of the
nominal rate during these outages if the circuit breaker triggers. That
means that you can continue to send low bit-rate traffic to detect when
the path is restored and ramp up again with your congestion control.
Thus, I don't see circuit breakers preventing a congestion control
algorithm to operate under these conditions and maintain the session. If
the congestion control isn't there that is another question.

> problem for applications that are aware of their behaviour.

A SFU must provide consistent RTCP, i.e. RTCP SRs from the SFU must
provide RTCP statistics for what is actually forwarded, not what the SFU
received from the originating RTP sender.

> in all these cases both the application and the user may prefer that

Triggering the circuit breakers just requires the sending client to have
taken sufficient action to prevent persistent congestion. If it does it
can continue to transmit.

> undesirable streaming that circuit breakers address.

The point of the circuit breakers is to prevent situations where RTP
senders continue to blast away into a persistent congestion situation or
onto paths that are broken.

> reference but they can be harmful if REQUIRED.

I think they will from a protect the network perspective needs to be
REQUIRED. As I have tried to explain above, there are actions that are
allowed within the circuit breaker specification that allows a sender to
throttle its traffic sufficiently when it detects a path breakage, to
keep monitoring it until it recovers and then resume transmission. But,
if the implementation in the sender does not care to implement such
functionality, the circuit breaker do require them to cease
transmission. But, it do become a implementers choice.

In addition I don't think IETF should publish an RFC for a RTP media
plane that doesn't have some REQUIRED behaviour when it comes to
protecting the network in persistent congestion situations.

Cheers

Magnus Westerlund

----------------------------------------------------------------------
Services, Media and Network features, Ericsson Research EAB/TXM
----------------------------------------------------------------------
Ericsson AB                 | Phone  +46 10 7148287
Färögatan 6                 | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden | mailto: [email-masked]
----------------------------------------------------------------------

---

From: Emil Ivov <[email-masked]>
Date: Fri, 09 May 2014 14:12:30 +0200

Hey Magnus,

On 09.05.14, 11:53, Magnus Westerlund wrote:
> The circuit breakers triggers when the conditions are really bad.

Well, not really. They trigger when specific events occur. Whether those 
events are caused by network conditions and whether those network 
conditions actually are "really bad" is often a matter of perspective.

> I

Yes, I agree.

> received from the originating RTP sender.

That's a lot of must-s for something that is largely an implementation 
choice and for which we only have informational documents.

Yes, circuit breakers could probably be worked around by applications 
but I am not sure how trivial this would be, especially in the case of 
WebRTC.

> onto paths that are broken.

Yes I get that. However we already have "consent freshness" and that 
will do the job in a big chunk of the cases that circuit breakers are 
meant to protect. It is not clear to me that CBs would be a clear 
improvement for the remainder of the cases.

> transmission. But, it do become a implementers choice.

OK then, if we agree that this needs to be an implementation choice then 
why don't we just inform implementers of that possibility and 
potentially advise the W3C to add API calls to enable their use?

> protecting the network in persistent congestion situations.

I would agree if we actually had a flawless mechanism that would only 
trigger in cases where it absolutely has to. As you have indicated 
yourself: It is very hard to define programmatic rules that always do 
the "Right thing" here

Cheers,
Emil

-- 
https://jitsi.org

---

From: Magnus Westerlund <[email-masked]>
Date: Fri, 16 May 2014 11:22:35 +0200

Hi,

See inline.

> conditions actually are "really bad" is often a matter of perspective.

Fair enough. But from my perspective for a real-time interactive
application loss of either forward or reverse path for 10 second or more
is an event that affects the user. And continuing to blast traffic into
such a situation is bad. So either react or let the circuit breaker trip
and shutdown the transmission is for me very reasonable.

When it comes to the TCP equation the fact that the media sender are
going more than 10 times as fast as TCP would do, do indicate that the
send rate is out their into un-fair space. But, sure you can build an
application that works okay with 15% packet loss as long as the delay
isn't going through the roof.

> but I am not sure how trivial this would be, especially in the case of

But, adaptation to path characteristic aren't trivial. What circuit
breakers offers you are. Either you care about this yourself and do
something better than circuit breakers to avoid tripping it, or you
accept that you have to shutdown when tripping it.

The fundamental statement when it comes to circuit breakers is the
following: We don't want completely unresponsive traffic generators on
the internet. We want the traffic generators to shutdown when they
contribute to cause significant damage.

> improvement for the remainder of the cases.

Yes, but circuit breakers are also likely to be adopted/required in
other cases that may not have the consent freshness equivalent. But, I
do agree that consent freshness will prevent one from continuing
peerconnections across path outages that last longer than 30 seconds.

I personally don't see it acceptable to continue to transmit without any
checks even when the packet loss is high but not a complete path loss.

> potentially advise the W3C to add API calls to enable their use?

When I talk about implementation, I expect the media frameworks and
their RTP stacks in the browsers to implement both circuit breakers and
congestion control, not the javascript applications. Thus, there is no
reason to have a control API. Yes, you need an API to inform the
application about the actual condition so that it can make choices based
on what is available. For example turning off streams in the appropriate
order.

> yourself: It is very hard to define programmatic rules that always do

TCP is not flawless, nothing in this space is going to be flawless. The
point with the circuit breaker is that it should have low enough risk of
triggering unnecessary and that it also prevent persistent congestion
state in the networks. Based on the simulations for LTE networks that
Zahed presented circuit breakers do allow a media sender to be pretty
bad against others before it acts.

If you have data that it triggers to much, please share. I am not
impressed with hand waving in this matter.

Cheers

Magnus Westerlund