Subject: [quicwg/base-drafts] recovery: clarification on TLP when sender is outpacing the timer (#1718)

From: "Arvid E. Picciani" <[email-masked]>
Date: Sat, 01 Sep 2018 02:02:30 -0700

The TLP is reset on every sent packet.
What's happen in my implementation is that the TLP never fires if the sender just keeps sending. If an Ack never arrives, this means disconnect is never detected until the window is full.

I'm not sure what i missed in the spec that is supposed to prevent that.
timer based loss detection would catch it, but it seemed optional and not recommended.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1718

---

From: MikkelFJ <[email-masked]>
Date: Sat, 01 Sep 2018 02:44:09 -0700

As I understand it, the TLP is there to ensure the receiver sees something that it can ACK, in case a packet is lost. If you keep sending ACK'able content that problem never occurs, and it is fair that the timer is not triggered.

TLP is not meant to detect that the peer closed the connection but rather that the sender stopped sending (hence tail loss probe), possibly close to connection close, but not necessarily.

If you keep missing ACK's another trigger sets in: you loss rate gets too high and you close the connection. There is ongoing discussions about negoatiating how an ACK can be delayed.

https://github.com/quicwg/base-drafts/pull/1715#pullrequestreview-151076864

dead path dection:
https://github.com/quicwg/base-drafts/issues/1638

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1718#issuecomment-417846905

---

From: "Arvid E. Picciani" <[email-masked]>
Date: Sat, 01 Sep 2018 02:53:19 -0700

thanks, i didn't these issues are related because they talk about what happens after RTO fires, which never happens in this case.

I also could not find the spec for which loss rate is considered dead. Is this something that is simply not specified or am i blind?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1718#issuecomment-417847383

---

From: MikkelFJ <[email-masked]>
Date: Sat, 01 Sep 2018 03:03:51 -0700

Mmm - why doesn't RTO fire? If you do not get a timely ACK, you should retransmit? TLP just helps make that happen faster near the end. Retransmit can happen based on gaps in ACK or timeout, or both (from memory, spec keep changing). If you only rely on gap in ACK, then yes, it would not trigger, so that is a bad strategy to use exclusively unless you use an aggressive connection timeout.

I'm not sure the critical loss rate is documented explicitly - but perhaps a section is needed to discuss this further?  In any case, the loss rate is highly application specific because you might operate infiniband where minimal delays a considered critical and any loss unlikely, or you might be on domestic congested wifi with high loss rates, or on a satellite link with large RTT and possible large loss bursts. Loss can also happen from DoS attacks or on path attacks and may have to adjust to circumstances.

If both peers cooperate, a connection close should happen. If not, a timeout happens eventually, and and a stateless reset may happen much earlier when supported and when the path is still viable but the endpoint is not.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1718#issuecomment-417847899

---

From: MikkelFJ <[email-masked]>
Date: Sat, 01 Sep 2018 03:15:43 -0700

I just skimmed over the recover / loss spec - it seems that RTO is only triggered after TLP fires. In that case you are right. I agree that this seems wrong and not what I would have expected, but I have not read the loss doc very carefully recently.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1718#issuecomment-417848473

---

From: MikkelFJ <[email-masked]>
Date: Sat, 01 Sep 2018 03:20:35 -0700

I mixed up early retransmit timer with RTO. The early retransmit timer ought to be active in your scenario, but it requires some detailed reading to be sure of that.
https://quicwg.org/base-drafts/draft-ietf-quic-recovery.html#rfc.section.4.5.7.3

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1718#issuecomment-417848738

---

From: "Arvid E. Picciani" <[email-masked]>
Date: Sat, 01 Sep 2018 03:21:14 -0700

yeah kMaxTLPs times TLP are always sent before RTO triggers, 

https://github.com/quicwg/base-drafts/blame/master/draft-ietf-quic-recovery.md#L870

hwoever tlp_count never increases if you dont receive an ack, because ack triggers the TLP.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1718#issuecomment-417848757

---

From: "Arvid E. Picciani" <[email-masked]>
Date: Sat, 01 Sep 2018 03:24:29 -0700

good find.  early retransmit never fires for this case, that must be where the problem is. 
It doesnt fire because its only set when there's no loss.
```
 largest_acked.packet_number == largest_sent_packet
``

https://github.com/quicwg/base-drafts/blame/master/draft-ietf-quic-recovery.md#L909

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1718#issuecomment-417848905

---

From: MikkelFJ <[email-masked]>
Date: Sat, 01 Sep 2018 03:26:05 -0700

I think we need to wait for @ianswett to comment on this.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1718#issuecomment-417848964

---

From: MikkelFJ <[email-masked]>
Date: Sat, 01 Sep 2018 03:30:03 -0700

Maybe the problem is that RTO depends on TLP. TLP reset makes sense isolated but not when impacts triggering RTO. This could be a side-effect of trying to have only one timer active at time.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1718#issuecomment-417849160

---

From: Subodh Iyengar <[email-masked]>
Date: Sat, 01 Sep 2018 06:48:41 -0700

You will have to stop sending at some point because you will hit cwnd limited or flow control limit 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1718#issuecomment-417860998

---

From: MikkelFJ <[email-masked]>
Date: Sat, 01 Sep 2018 06:57:45 -0700

@aep already mentioned the window the original post, but that can take along time if you just send small status messages just below TLP timeout.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1718#issuecomment-417861517

---

From: Subodh Iyengar <[email-masked]>
Date: Sat, 01 Sep 2018 11:45:39 -0700

Why are you not getting acks? The peer will use a delayed ack timer which will be practicallly << tlp timer. If the peer is not acting at all then the peer is not going to get optimal perf either

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1718#issuecomment-417879473

---

From: "Arvid E. Picciani" <[email-masked]>
Date: Sat, 01 Sep 2018 11:49:05 -0700

@siyengar the problem discussed is dead peer detection. its not sending acks because its dead.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1718#issuecomment-417879701

---

From: Christian Huitema <[email-masked]>
Date: Sat, 01 Sep 2018 12:55:47 -0700

@aep what @mikkelfj said. TLP should not be resetting the RTO timer. That is, even if you want to manage just one timer, you should have a check that if TLP would expire after the nominal RTO, then use RTO instead.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1718#issuecomment-417883473

---

From: Subodh Iyengar <[email-masked]>
Date: Sat, 01 Sep 2018 13:12:01 -0700

Dead peers are detected by an idle timeout which can happen before the window is full. The idle timeout definition recently changed, if you were using the previous one then it’s possible that the implementation is setting the timer on every send. However idle timeout will now only be reset on the first send

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1718#issuecomment-417884266

---

From: ianswett <[email-masked]>
Date: Thu, 06 Sep 2018 08:50:42 -0700

To address your first question, PR #1652 adds some more clarity to transport on the idle timeout and makes it clear that if you're sending into a void without receiving any ACKs, you should not keep extending your idle timeout.  The recovery doc doesn't describe anything relating to the idle timer, FWIW.

The intent is for the RTO timer to only go off after TLP.  There is text about setting the TLP to the min(TLP timeout, RTO timeout) in 4.3.2(https://tools.ietf.org/html/draft-ietf-quic-recovery-14#section-4.3.2).  If you want to discuss whether that should be what this document tries to specify, please open a separate issue about whether RTO should ever fire before any TLPs are sent?

Dead peer detection(#1638) is an entirely different problem, and there's no effort to describe that in the recovery draft at the moment.  It would be nice to talk about that somewhere, but I'm not sure it's known what best practices are?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1718#issuecomment-419145274

---

From: "Arvid E. Picciani" <[email-masked]>
Date: Thu, 06 Sep 2018 09:08:32 -0700

thanks for clarification.  I'm using a separate timer on idle now that only extends on incomming ack.

It would really help to describe best practices here a little further. But i'm unsure if its needed for other implementors who just follow quic-transport after implementing recovery. 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1718#issuecomment-419151515

---

From: ianswett <[email-masked]>
Date: Thu, 06 Sep 2018 09:10:54 -0700

Q: Best practices for idle timeouts, dead path timeouts, or something else?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1718#issuecomment-419152207

---

From: "Arvid E. Picciani" <[email-masked]>
Date: Thu, 06 Sep 2018 09:35:44 -0700

all of it ;)  
Currently i'm just counting seconds since the last received ack, but that doesnt detect paths that are alive but unusable because of high packet loss for example. There's probably alot more details in this that i simply don't know about.  Should duplicated packets be filtered before the count? What about keepalive packets? Who sends them, when? etc

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1718#issuecomment-419160142

---

From: ianswett <[email-masked]>
Date: Thu, 06 Sep 2018 11:33:04 -0700

I don't think we'll provide any guidelines on how high is too high for packet loss, because I'm doubtful we could write good general purpose guidance.

I don't have any clear advice on the rest of the questions except what's already in the transport and http docs.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1718#issuecomment-419197151

---

From: "Arvid E. Picciani" <[email-masked]>
Date: Thu, 06 Sep 2018 12:39:25 -0700

fair enough. thanks for your input :)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1718#issuecomment-419216541

---

From: "Arvid E. Picciani" <[email-masked]>
Date: Thu, 06 Sep 2018 12:39:25 -0700

Closed #1718.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1718#event-1831274987