Subject: [tcpm] increased back-off across different retransmissions

From: Marc <[email-masked]>
Date: Fri, 17 Nov 2017 15:59:12 +0100

Hello,

I'm reading RFC 6298, which mentions the TCP back-off algorithm. One detail
is not very clear in this RFC, or maybe I'm missing it. Hope someone can
point me in the right direction!

In our network we have a link that's sometimes congested. When that's the
case, sometimes we have a TCP connection that just has a streak of bad luck
and loses multiple packets in a row. See attachment for an example, taken
on a switch connected to the server. Unfortunately I'm not allowed to share
the PCAP.. But I think / hope the tcpdump information contains everything
needed (time, ip, port, win, seq, len, ack, sack, sle, sre..)

Client: 1.1.1.1
Server: 2.2.2.2

So some packets from Server to Client are dropped. I've marked the packets
below in attachment with P1 until P7.

Packet 1: seq 135650, len 1348, dropped
Packet 2: seq 136998, len 1348, received <-- so this one is not dropped
Packet 3: seq 138346, len 1348, dropped
Packet 4: seq 139694, len 81, dropped
Packet 5: seq 139775, len 1348, dropped
Packet 6: seq 141123, len 1348, dropped
Packet 7: seq 142471, len 1221, dropped

Then the retransmissions, marked in attachment with R1 until R5:
- Retransmission 1: 0.5 sec after dropped 'Packet 1', packet with seq
135650, len 1348 is retransmitted.
Ack <10ms. Now the ack # is 138346, higher than seq135650 + len1348 =
136998, the client is also ack'ing 'Packet 2'.

- Retransmission 2: 1.1 seconds after Retransmission 1, packet with seq
138346, len 1348 is retransmitted.
Ack <10ms, ack # 139694 (matches seq+len of Retransmission 2). This
Retransmission covers dropped 'Packet 3'

- Retransmission 3: 2.8 seconds after Retransmission 2, packet with seq
139694, len 1348 is retransmitted.
Ack <10ms, ack # 141042 (matches seq+len of Retransmission 3). Note that
this packet contains data from dropped 'Packet 4' and 'Packet 5'

- Retransmission 4: 7.1 seconds after Retransmission 3, packet with seq
141042, len 1348 is retransmitted.
Ack <10ms, ack # 142390 (matches seq+len of Retransmission 4). Note that
this packet contains data from dropped 'Packet 5' and 'Packet 6'

- Retransmission 5: 17.8 seconds after Retransmission 4, packet with seq
142390, len 1302 is retransmitted.
Ack <10ms, ack # 143692 (matches seq+len of Retransmission 5). Note that
this packet contains data from dropped 'Packet 6' and 'Packet 7'

So the back-off is increased across _different_ retransmissions, despite
the fact those retransmissions are ack'ed very quickly. Now I know there is
a chance those acks are coming from the original tranmission so the server
can't calculate a new RTT, but well.. I'm not sure if exponentially (~ x2.5
in my case actually..) increasing the timer across different ack'ed
retransmissions is the right strategy here. 10 lost packets (which isn't
pretty, I know) in my case would mean 500 seconds of waiting.

Am I right that the RFC (or Karn's algorithm) doesn't make a statement
about this particular scenario ? In that case, should it be clarified ? Or
is this such an obvious bad strategy that it's overkill to explicitly
forbid it per RFC ?

Thanks,

Marc

---

From: Yuchung Cheng <[email-masked]>
Date: Sat, 18 Nov 2017 15:56:06 -0800

On Fri, Nov 17, 2017 at 6:59 AM, Marc <[email-masked]> wrote:

> Hello,
>
> I'm reading RFC 6298, which mentions the TCP back-off algorithm. One
> detail is not very clear in this RFC, or maybe I'm missing it. Hope someone
> can point me in the right direction!
>
The sender does not seem to implement TCP RFCs correctly. According to the
RFC5681, upon the first timeout the sender should reduce cwnd to 1 and
enter slow start. Upon receiving the ACK of rtx of P2, cwnd would be 2 so
the sender should have retransmitted P3 and P4 right away. The next round
trip would retransmit P5,P6,P7. This is called slow start after timeout and
it should take only 0.5s + 10ms * 3.

Regarding the exponential backoff, RFC6298 section 5.2 (end) states that
RTO is recomputed whenever a new RTT sample is acquired, hence clearing the
backoff. But in your case there shouldn't be backoff at all. Normally we'd
see exponentially backoffs if the first unacked packet after timeout is
lost again (and again), which indicates severe congestion.

HTH

> _______________________________________________
> tcpm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tcpm
>
>

---

From: Marc <[email-masked]>
Date: Sun, 19 Nov 2017 12:34:33 +0100

Thanks Yuchung & Richard, I think this will certainly be of help. Some more
RFC's for me to study! The misbehaving component is an SSL offloading
appliance. The vendor states they're using the BSD tcp implementation,
which could be true.. but perhaps a very dated version. Hope this info
helps them to improve.

Marc

---

From: "Richard Scheffenegger" <[email-masked]>
Date: Sun, 19 Nov 2017 14:09:37 +0100

Marc,

One thing you could be trying though - I've seen in the past misbehavior (not quite like this, with the exponential backoff - just multiple rounds of timeout loss recovery for all trailing packets) similiar to this for SACK-enabled sessions when tail packets were dropped. Loss recovery then was significantly worse than with NewReno alone...

In that particular circumstance shown in your trace, disabling SACK and forcing the sender to regular NewReno behavior should improve latency. For bulk transfers, SACK is usually better though. With incomplete SACK recovery, NewReno can have benefits for the high-percentile transaction latency.

IIRC, that observation was exactly what ultimately led to having RescueRxmt in 6675.

Something you could try with changing only on the client side (and potentially on a per session basis).

Best regards,
  Richard

  ----- Original Message ----- 
  From: Marc 
  To: Yuchung Cheng 
  Cc: [email-masked] Extensions 
  Sent: Sunday, November 19, 2017 12:34 PM
  Subject: Re: [tcpm] increased back-off across different retransmissions

  Thanks Yuchung & Richard, I think this will certainly be of help. Some more RFC's for me to study! The misbehaving component is an SSL offloading appliance. The vendor states they're using the BSD tcp implementation, which could be true.. but perhaps a very dated version. Hope this info helps them to improve. 

  Marc

       Virus-free. www.avg.com  

------------------------------------------------------------------------------