Subject: [tcpm] RACK, TLP and congestion control

From: marcelo bagnulo braun <[email-masked]>
Date: Thu, 23 Feb 2017 08:30:05 +0100

Hi,

After reading draft-ietf-tcpm-rack-01, I have a question about the 
impact of the impact of the use of RACK and TLP in congestion control.

If this issue has been discussed earlier, I apologize and please point 
me to the right thread in the archive.

So, consider the following situation. A TCP sender using RACK and TLP as 
defined in the draft has at a given moment a large number on packets in 
flight. Suppose that all packets are lost. I understand the reaction of 
the sender in this case is that it will fire a Tail Loss Probe after the 
PTO expires i.e. 2RTT later.

Suppose that the probe makes it and an ack for that probe is returned. 
Since the probe is a new packet or a retransmission of the latest 
packet, RACK at the sender is likely to mark all previous packets as 
lost and retransmit them. All this is done while staying in congestion 
avoidance phase, afaiu.

This seems a significant departure from current congestion control 
mechanisms, i think. I mean, this is not the case of a spurious isolated 
drop. All packets in flight were lost, which can be a pretty large 
number of packets. Staying in congestion avoidance after a single packet 
made it through seems more optimistic than the current situation, i 
would say.

Am i missing something?

Regards, marcelo

---

From: Neal Cardwell <[email-masked]>
Date: Thu, 23 Feb 2017 15:29:07 -0500

Hi,

Thanks for the question.

In the draft -- https://tools.ietf.org/html/draft-ietf-tcpm-rack-01 -- we
try to address that in section 7.5:

7.5.  Interaction with congestion control

   RACK intentionally decouples loss detection from congestion control.
   RACK only detects losses; it does not modify the congestion control
   algorithm [RFC5681][RFC6937].

Thus, in any scenario like the one you describe, where RACK (with or
without a TLP probe) marks one or more packets as lost, the sender should
use its normal congestion control response to packet loss. So if the sender
is using Reno or CUBIC, it should leave congestion avoidance and enter fast
recovery, and cut cwnd and ssthresh as it would with any other loss
detection algorithm.

Does that help clear up this question?

If there is a particular section that was unclear on this point or seemed
to imply otherwise, please let us know so we can revise it. :-)

Thanks!
neal

On Thu, Feb 23, 2017 at 2:30 AM, marcelo bagnulo braun <[email-masked]>
wrote:

> Hi,
>
> After reading draft-ietf-tcpm-rack-01, I have a question about the impact
> of the impact of the use of RACK and TLP in congestion control.
>
> If this issue has been discussed earlier, I apologize and please point me
> to the right thread in the archive.
>
> flight. Suppose that all packets are lost. I understand the reaction of the
> sender in this case is that it will fire a Tail Loss Probe after the PTO
> expires i.e. 2RTT later.
>
> Since the probe is a new packet or a retransmission of the latest packet,
> RACK at the sender is likely to mark all previous packets as lost and
> retransmit them. All this is done while staying in congestion avoidance
> phase, afaiu.
>
> drop. All packets in flight were lost, which can be a pretty large number
> of packets. Staying in congestion avoidance after a single packet made it
> through seems more optimistic than the current situation, i would say.
>
> Am i missing something?
>
> Regards, marcelo
>
> _______________________________________________
> tcpm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tcpm
>

---

From: marcelo bagnulo braun <[email-masked]>
Date: Thu, 23 Feb 2017 21:43:15 +0100

Hi,

Let me try to rephrase/expand.

As I understand it, the response to congestion depends on the intensity 
of the congestion signal.
If a few packets are lost, the CWND is reduced to half. This is why 
afaiu, the response to 3 (or more) duplicated ACKs is to reduce CWND by 
half (or some other number in cubic)

However, if all the packets in flight are lost, there are no duplicated 
acks, and the loss is detected through the RTO. In this case, the 
congestion signal is more intense and the CWND is reduced to 1 MSS.

In the example i included below, all the packets in flight are lost, but 
TLP+RACK manages to recover without using the RTO.
So, i guess my question is: in this case, what happens to the CWND? is 
it reduced to half or is it set to 1 MSS?

Regards, marcelo

El 23/02/17 a las 21:29, Neal Cardwell escribió:
> Hi,
>
> Thanks for the question.
>
> In the draft -- https://tools.ietf.org/html/draft-ietf-tcpm-rack-01 -- 
> we try to address that in section 7.5:
>
> without a TLP probe) marks one or more packets as lost, the sender 
> should use its normal congestion control response to packet loss. So 
> if the sender is using Reno or CUBIC, it should leave congestion 
> avoidance and enter fast recovery, and cut cwnd and ssthresh as it 
> would with any other loss detection algorithm.
>
> If there is a particular section that was unclear on this point or 
> seemed to imply otherwise, please let us know so we can revise it. :-)
>
> Thanks!
> neal
>
>
> On Thu, Feb 23, 2017 at 2:30 AM, marcelo bagnulo braun 
> <[email-masked] <mailto:[email-masked]>> wrote:
>
>     Hi,
>
>     If this issue has been discussed earlier, I apologize and please
>     point me to the right thread in the archive.
>
>     So, consider the following situation. A TCP sender using RACK and
>     TLP as defined in the draft has at a given moment a large number
>     on packets in flight. Suppose that all packets are lost. I
>     understand the reaction of the sender in this case is that it will
>     fire a Tail Loss Probe after the PTO expires i.e. 2RTT later.
>
>     Suppose that the probe makes it and an ack for that probe is
>     returned. Since the probe is a new packet or a retransmission of
>     the latest packet, RACK at the sender is likely to mark all
>     previous packets as lost and retransmit them. All this is done
>     while staying in congestion avoidance phase, afaiu.
>
>     mechanisms, i think. I mean, this is not the case of a spurious
>     isolated drop. All packets in flight were lost, which can be a
>     pretty large number of packets. Staying in congestion avoidance
>     after a single packet made it through seems more optimistic than
>     the current situation, i would say.
>
>     Am i missing something?
>
>     Regards, marcelo
>
>     [email-masked] <mailto:[email-masked]>
>     <https://www.ietf.org/mailman/listinfo/tcpm>
>
>

---

From: Neal Cardwell <[email-masked]>
Date: Thu, 23 Feb 2017 16:59:35 -0500

On Thu, Feb 23, 2017 at 3:43 PM, marcelo bagnulo braun <[email-masked]>
wrote:

> As I understand it, the response to congestion depends on the intensity of
> the congestion signal.
> If a few packets are lost, the CWND is reduced to half. This is why afaiu,
> the response to 3 (or more) duplicated ACKs is to reduce CWND by half (or
> some other number in cubic)
>
> So, i guess my question is: in this case, what happens to the CWND? is it
> reduced to half or is it set to 1 MSS?
>

At a high level, that is a decision for congestion control, and RACK is a
loss detection aglorithm, and tries to stay orthogonal to congestion
control.

But on a practical level, a TCP sender ideally should be using the state of
the art recovery algorithm, PRR  -- https://tools.ietf.org/html/rfc6937 --
for fast recovery. If it does that then in the situation you sketch it will
slow-start up from a cwnd of 1, allowing 1 packet in flight, up to the
ssthresh set by the congestion control algorithm. This is basically the
response that the sender would have had to an RTO as well. So from a
practical standpoint it doesn't matter much that with RACK the sender
enters recovery instead of RTO - either way the sender should be
slow-starting up from a cwnd of 1 up to ssthresh.

neal

>> On Thu, Feb 23, 2017 at 2:30 AM, marcelo bagnulo braun <
>> [email-masked] <mailto:[email-masked]>> wrote:
>>
>>     Hi,
>>

---

From: Yuchung Cheng <[email-masked]>
Date: Thu, 23 Feb 2017 14:14:34 -0800

On Thu, Feb 23, 2017 at 1:59 PM, Neal Cardwell <[email-masked]> wrote:

>
>
> On Thu, Feb 23, 2017 at 3:43 PM, marcelo bagnulo braun <[email-masked]
> > wrote:
>
>> Hi,
>>
> A more direct answer is that CWND is reduced to half.

We can put more details in the draft on the impact of C.C.. The original
rationale behind halving cwnd post Fast Recovery is that ACK clock is still
going so the congestion is not severe enough to warrant a complete cwnd
reset (=1). So if TLP+RACK was able to recover the loss within 2-3 RTTs, we
should react similar to the 3 dupack thresh based recovery. HTH

> But on a practical level, a TCP sender ideally should be using the state
> of the art recovery algorithm, PRR  -- https://tools.ietf.org/html/rfc6937
> -- for fast recovery. If it does that then in the situation you sketch it
> will slow-start up from a cwnd of 1, allowing 1 packet in flight, up to the

---

From: marcelo bagnulo braun <[email-masked]>
Date: Mon, 27 Feb 2017 12:23:24 +0100

Thanks for the replies, both from Neal and Yuchung, but i am (more) 
confused now.

 From Neal's reply, i understand that in the situation i described the 
cwnd fall to 1 MSS and from Yuchung's reply it seems it falls to half 
its value. What am i missing?

El 23/02/17 a las 23:14, Yuchung Cheng escribió:
>
>
> On Thu, Feb 23, 2017 at 1:59 PM, Neal Cardwell <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>
>
>     On Thu, Feb 23, 2017 at 3:43 PM, marcelo bagnulo braun
>         As I understand it, the response to congestion depends on the
>         intensity of the congestion signal.
>         If a few packets are lost, the CWND is reduced to half. This
>         is why afaiu, the response to 3 (or more) duplicated ACKs is
>         to reduce CWND by half (or some other number in cubic)
>
>         However, if all the packets in flight are lost, there are no
>         duplicated acks, and the loss is detected through the RTO. In
>         this case, the congestion signal is more intense and the CWND
>         is reduced to 1 MSS.
>
>         In the example i included below, all the packets in flight are
>         lost, but TLP+RACK manages to recover without using the RTO.
>         So, i guess my question is: in this case, what happens to the
>         CWND? is it reduced to half or is it set to 1 MSS?
>
> We can put more details in the draft on the impact of C.C.. The 
> original rationale behind halving cwnd post Fast Recovery is that ACK 
> clock is still going so the congestion is not severe enough to warrant 
> a complete cwnd reset (=1). So if TLP+RACK was able to recover the 
> loss within 2-3 RTTs, we should react similar to the 3 dupack thresh 
> based recovery. HTH
>
>
>
>     At a high level, that is a decision for congestion control, and
>     RACK is a loss detection aglorithm, and tries to stay orthogonal
>     to congestion control.
>
>     But on a practical level, a TCP sender ideally should be using the
>     state of the art recovery algorithm, PRR  --
>     https://tools.ietf.org/html/rfc6937
>     <https://tools.ietf.org/html/rfc6937> -- for fast recovery. If it
>     does that then in the situation you sketch it will slow-start up
>     from a cwnd of 1, allowing 1 packet in flight, up to the ssthresh
>     set by the congestion control algorithm. This is basically the
>     response that the sender would have had to an RTO as well. So from
>     a practical standpoint it doesn't matter much that with RACK the
>     sender enters recovery instead of RTO - either way the sender
>     should be slow-starting up from a cwnd of 1 up to ssthresh.
>
>     neal
>
>
>         Regards, marcelo
>
>
>             In the draft --
>             https://tools.ietf.org/html/draft-ietf-tcpm-rack-01
>             <https://tools.ietf.org/html/draft-ietf-tcpm-rack-01> --
>             Thus, in any scenario like the one you describe, where
>             RACK (with or without a TLP probe) marks one or more
>             packets as lost, the sender should use its normal
>             congestion control response to packet loss. So if the
>             sender is using Reno or CUBIC, it should leave congestion
>             avoidance and enter fast recovery, and cut cwnd and
>             ssthresh as it would with any other loss detection algorithm.
>
>             If there is a particular section that was unclear on this
>             point or seemed to imply otherwise, please let us know so
>             we can revise it. :-)
>
>             Thanks!
>             neal
>
>
>             <[email-masked] <mailto:[email-masked]>
>             <mailto:[email-masked] <mailto:[email-masked]>>>
>             wrote:
>
>                 Hi,
>
>             [email-masked] <mailto:[email-masked]> <mailto:[email-masked]
>             <mailto:[email-masked]>>
>                 <https://www.ietf.org/mailman/listinfo/tcpm
>             <https://www.ietf.org/mailman/listinfo/tcpm>>
>
>
>
>
>

---

From: Neal Cardwell <[email-masked]>
Date: Mon, 27 Feb 2017 07:57:40 -0500

On Mon, Feb 27, 2017 at 6:23 AM, marcelo bagnulo braun <[email-masked]>
wrote:

> From Neal's reply, i understand that in the situation i described the cwnd
> fall to 1 MSS and from Yuchung's reply it seems it falls to half its value.
> What am i missing?
>

In the situation you sketch, both are true, but at different time scales.
:-)

Short-term: upon the first ACK, which causes the sender to enter fast
recovery, the sender will set the cwnd to 1.

Long-term: over the course of fast recovery, the sender will gradually
increase (slow-start) the cwnd upward; at the end of recovery cwnd will
equal the ssthresh set by the congestion control algorithm, which will be
half of the cwnd before entering recovery.

(This is all assuming the sender is using PRR and IETF standard congestion
control, aka Reno.)

Hope that clarifies things a bit,
neal

---

From: Jana Iyengar <[email-masked]>
Date: Mon, 27 Feb 2017 08:52:52 -0800

An additional bit of info that might help: in PRR (
https://tools.ietf.org/html/rfc6937), sndcnt is used to gate how much a
sender can send in response to an ack (see computation of "limit" in the
algorithm.)

On Mon, Feb 27, 2017 at 4:57 AM, Neal Cardwell <[email-masked]> wrote:

> On Mon, Feb 27, 2017 at 6:23 AM, marcelo bagnulo braun <[email-masked]
> > wrote:
>
>> cwnd fall to 1 MSS and from Yuchung's reply it seems it falls to half its
>> value. What am i missing?
>>
>
> (This is all assuming the sender is using PRR and IETF standard
> congestion control, aka Reno.)
>

---

From: Yuchung Cheng <[email-masked]>
Date: Mon, 27 Feb 2017 10:05:05 -0800

> Hope that clarifies things a bit,
ack. thanks for the clarification and sorry about the confusion.

---

From: marcelo bagnulo braun <[email-masked]>
Date: Tue, 28 Feb 2017 08:45:58 +0100

thanks, makes sense to me now.

With PRR it behaves somehow similar than slow start (depending on the 
PRR mode, with CRB i think it even goes slower than slow start, while 
with SSRB it goes 1 MSS faster per RTT).
However with RFC 6675 it would send the all ssthresh packets right after 
the first ack, correct?

Should we reccomend that RACK is used with PRR?

Regards, marcelo

El 27/02/17 a las 13:57, Neal Cardwell escribió:
> On Mon, Feb 27, 2017 at 6:23 AM, marcelo bagnulo braun 
>     Thanks for the replies, both from Neal and Yuchung, but i am
>     (more) confused now.
>
>     >From Neal's reply, i understand that in the situation i described
>     the cwnd fall to 1 MSS and from Yuchung's reply it seems it falls
>     to half its value. What am i missing?
>
>
> In the situation you sketch, both are true, but at different time 
> scales. :-)
>
> increase (slow-start) the cwnd upward; at the end of recovery cwnd 
> will equal the ssthresh set by the congestion control algorithm, which 
> will be half of the cwnd before entering recovery.
>
>         On Thu, Feb 23, 2017 at 1:59 PM, Neal Cardwell
>         <[email-masked] <mailto:[email-masked]>
>         <mailto:[email-masked] <mailto:[email-masked]>>>
>         wrote:
>
>
>
>         <mailto:[email-masked] <mailto:[email-masked]>>> wrote:
>
>                 Hi,
>
>         We can put more details in the draft on the impact of C.C..
>         The original rationale behind halving cwnd post Fast Recovery
>         is that ACK clock is still going so the congestion is not
>         severe enough to warrant a complete cwnd reset (=1). So if
>         TLP+RACK was able to recover the loss within 2-3 RTTs, we
>         <https://tools.ietf.org/html/rfc6937>
>             <https://tools.ietf.org/html/rfc6937
>         <https://tools.ietf.org/html/rfc6937>> -- for fast recovery. If it
>         <https://tools.ietf.org/html/draft-ietf-tcpm-rack-01>
>                    
>         <https://tools.ietf.org/html/draft-ietf-tcpm-rack-01
>         <https://tools.ietf.org/html/draft-ietf-tcpm-rack-01>> --
>         <mailto:[email-masked] <mailto:[email-masked]>>
>                     <mailto:[email-masked]
>         <mailto:[email-masked]> <mailto:[email-masked]
>         <mailto:[email-masked]>>>>
>
>                     wrote:
>
>                         Hi,
>
>                         If this issue has been discussed earlier, I
>         apologize
>                     and please
>                         TLP as defined in the draft has at a given
>         moment a
>                     large number
>                         understand the reaction of the sender in this
>         case is
>                     that it will
>                         fire a Tail Loss Probe after the PTO expires
>         i.e. 2RTT
>                     later.
>
>                         Suppose that the probe makes it and an ack for
>         that
>                     probe is
>                         previous packets as lost and retransmit them.
>         All this
>                     is done
>                         mechanisms, i think. I mean, this is not the
>         case of a
>                     spurious
>                         isolated drop. All packets in flight were
>         lost, which
>                     can be a
>                         pretty large number of packets. Staying in
>         congestion
>                     avoidance
>         <mailto:[email-masked]>> <mailto:[email-masked]
>         <mailto:[email-masked]>
>                     <mailto:[email-masked] <mailto:[email-masked]>>>
>         <https://www.ietf.org/mailman/listinfo/tcpm>>>
>
>
>
>
>
>
>

---

From: Yoshifumi Nishida <[email-masked]>
Date: Tue, 28 Feb 2017 00:55:00 -0800

On Mon, Feb 27, 2017 at 11:45 PM, marcelo bagnulo braun <[email-masked]>
wrote:

> With PRR it behaves somehow similar than slow start (depending on the PRR
> mode, with CRB i think it even goes slower than slow start, while with SSRB
> it goes 1 MSS faster per RTT).

I think this is not very precise. It depends on the calculation of pipe
size.
Figure 3.1 in RFC6937 might be helpful to see the difference.
--
Yoshi

---

From: Yoshifumi Nishida <[email-masked]>
Date: Tue, 28 Feb 2017 01:02:59 -0800

Sorry.. I meant the figure in Section 3.1.
--
Yoshi

On Tue, Feb 28, 2017 at 12:55 AM, Yoshifumi Nishida <[email-masked]>
wrote:

> On Mon, Feb 27, 2017 at 11:45 PM, marcelo bagnulo braun <
> [email-masked]> wrote:
>

---

From: Yuchung Cheng <[email-masked]>
Date: Tue, 28 Feb 2017 09:34:37 -0800

> Figure 3.1 in RFC6937 might be helpful to see the difference.
Right. On a valid ACK, RFC6675 always send up to (cwnd - pipe) of packets.
The only exception is the first ACK that triggering the recovery. Even if
cwnd <= pipe, it still forces one retransmit. PRR (and BBR) has the same
flavor of exception to keep ACK clocking.

--