Subject: [quicwg/base-drafts] Generating Acknowledgements (#1139)

From: ianswett <[email-masked]>
Date: Fri, 23 Feb 2018 12:24:59 -0800

Fixes the remaining portion of #230 
You can view, comment on, or merge this pull request online at:

  https://github.com/quicwg/base-drafts/pull/1139

-- Commit Summary --

  * Generating Acknowledgements

-- File Changes --

    M draft-ietf-quic-recovery.md (43)

-- Patch Links --

https://github.com/quicwg/base-drafts/pull/1139.patch
https://github.com/quicwg/base-drafts/pull/1139.diff

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1139

---

From: ianswett <[email-masked]>
Date: Fri, 23 Feb 2018 20:36:55 +0000 (UTC)

@ianswett pushed 1 commit.

40ec7f9  Update draft-ietf-quic-recovery.md

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/1139/files/6d0640698573dc7bc80b1041a06c90f021d070d0..40ec7f94cc57905d85d06c97261697076514dc18

---

From: Nick Banks <[email-masked]>
Date: Sat, 24 Feb 2018 11:30:09 -0800

nibanks approved this pull request.

> +When an ACK frame is sent, one or more ranges of acknowledged packets may
+be included.  Including older packets reduces the chances previous ACK
+frames were lost, causing spurious retransmits, at the cost of larger acks.
+
+ACK frames SHOULD always acknowledge the most recently received packets,
+and the more out of order packets are, the more important it is to send
+an updated ACK frame quickly, in order to prevent the peer from declaring
+a packet as lost.
+
+Below is one recommended approach for determining what packets to include
+in an ACK frame.
+
+#### Track the most recently acknowledged ACK frame
+
+When a packet containing an ACK frame is sent, the largest acked in that
+frame can be saved.  When a packet containing and ACK frame is acknowledged,

I believe this should be `containing and ACK frame` -> `containing an ACK frame`.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1139#pullrequestreview-99121026

---

From: ianswett <[email-masked]>
Date: Sat, 24 Feb 2018 17:20:48 -0800

c4a80bc  Update draft-ietf-quic-recovery.md

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/1139/files/40ec7f94cc57905d85d06c97261697076514dc18..c4a80bc0f69d920ae07326f89048e0c645f37ebd

---

From: ianswett <[email-masked]>
Date: Sat, 24 Feb 2018 17:22:25 -0800

Thanks Nick, fixed and I tweaked some of the surrounding text.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1139#issuecomment-368274747

---

From: Mike Bishop <[email-masked]>
Date: Sun, 25 Feb 2018 05:50:54 -0800

MikeBishop commented on this pull request.

Looks fine.

> +the delay does not exceed the maximum ack delay.  QUIC recovery algorithms
+do not assume the peer generates an acknowledgement immediately when
+receiving a second full-sized segment.
+
+Out-of-order packets SHOULD be acknowledged more quickly, in order
+to accelerate loss recovery.  The receiver SHOULD send an immediate
+ACK when it receives a packet that fills in a gap in the packet numbers
+if it has already acknowledged a larger packet number.  If it has not
+sent an acknowledgement for the larger packet and there are no other
+recent gaps, then it may revert to the standard delayed ack algorithm.
+
+### Ack Ranges
+
+When an ACK frame is sent, one or more ranges of acknowledged packets may
+be included.  Including older packets reduces the chances previous ACK
+frames were lost, causing spurious retransmits, at the cost of larger acks.

No, it reduces the impact of those losses.  You can't affect the chance of loss.

> +An acknowledgement MAY be sent for every second full-sized segment,
+as TCP does {{!RFC5681}}, or may be sent less frequently, as long as
+the delay does not exceed the maximum ack delay.  QUIC recovery algorithms
+do not assume the peer generates an acknowledgement immediately when
+receiving a second full-sized segment.
+
+Out-of-order packets SHOULD be acknowledged more quickly, in order
+to accelerate loss recovery.  The receiver SHOULD send an immediate
+ACK when it receives a packet that fills in a gap in the packet numbers
+if it has already acknowledged a larger packet number.  If it has not
+sent an acknowledgement for the larger packet and there are no other
+recent gaps, then it may revert to the standard delayed ack algorithm.
+
+### Ack Ranges
+
+When an ACK frame is sent, one or more ranges of acknowledged packets may

I think at least one range is required, so "may be" is unnecessarily tentative.

+Out-of-order packets SHOULD be acknowledged more quickly, in order
+to accelerate loss recovery.  The receiver SHOULD send an immediate
+ACK when it receives a packet that fills in a gap in the packet numbers
+if it has already acknowledged a larger packet number.  If it has not
+sent an acknowledgement for the larger packet and there are no other
+recent gaps, then it may revert to the standard delayed ack algorithm.
+
+### Ack Ranges
+
+When an ACK frame is sent, one or more ranges of acknowledged packets may
+be included.  Including older packets reduces the chances previous ACK
+frames were lost, causing spurious retransmits, at the cost of larger acks.
+
+ACK frames SHOULD always acknowledge the most recently received packets,
+and the more out of order packets are, the more important it is to send

>From [this](https://www.grammarbook.com/punctuation/hyphens.asp), I would say hyphenate "out-of-order," though my wife disagrees.  Your PR, your call.  :-)

+Out-of-order packets SHOULD be acknowledged more quickly, in order
+to accelerate loss recovery.  The receiver SHOULD send an immediate
+ACK when it receives a packet that fills in a gap in the packet numbers
+if it has already acknowledged a larger packet number.  If it has not
+sent an acknowledgement for the larger packet and there are no other
+recent gaps, then it may revert to the standard delayed ack algorithm.
+
+### Ack Ranges
+
+When an ACK frame is sent, one or more ranges of acknowledged packets may
+be included.  Including older packets reduces the chances previous ACK
+frames were lost, causing spurious retransmits, at the cost of larger acks.
+
+ACK frames SHOULD always acknowledge the most recently received packets,
+and the more out of order packets are, the more important it is to send

And perhaps "the more out-of-order **the** packets are" would help improve readability.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1139#pullrequestreview-99143478

---

From: MikkelFJ <[email-masked]>
Date: Sun, 25 Feb 2018 13:55:46 +0000 (UTC)

mikkelfj commented on this pull request.

+and the more out of order packets are, the more important it is to send
+an updated ACK frame quickly, in order to prevent the peer from declaring
+a packet as lost.
+
+Below is one recommended approach for determining what packets to include
+in an ACK frame.
+
+#### Track the most recently acknowledged ACK frame
+
+When a packet containing an ACK frame is sent, the largest acked in that
+frame is saved.  When a packet containing an ACK frame is acknowledged,
+the receiver stops acknowledging packets less than or equal to the largest
+acked in the sent ACK frame.  This approach is not sensitive to packet loss
+and allows for over 1 round trip of reordering.
+

I like the general clarification. Detail: I hope it is sensitive to loss, otherwise it would be pointless. But perhaps not senstive to loss of ack frames.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1139#pullrequestreview-99143762

---

From: Martin Thomson <[email-masked]>
Date: Mon, 26 Feb 2018 00:22:25 +0000 (UTC)

martinthomson commented on this pull request.

> +## Generating Acknowledgements
+
+QUIC SHOULD delay sending acknowledgements in response to packets,
+but MUST NOT excessively delay acknowledgements of packets containing
+non-ack frames.  Specifically, implementaions MUST attempt to
+enforce a maximum ack delay to avoid causing the peer spurious
+timeouts.  The default maximum ack delay in QUIC is 25ms.
+
+An acknowledgement MAY be sent for every second full-sized segment,
+as TCP does {{!RFC5681}}, or may be sent less frequently, as long as
+the delay does not exceed the maximum ack delay.  QUIC recovery algorithms
+do not assume the peer generates an acknowledgement immediately when
+receiving a second full-sized segment.
+
+Out-of-order packets SHOULD be acknowledged more quickly, in order
+to accelerate loss recovery.  The receiver SHOULD send an immediate

There are two packets that arrive here.  Packet N+1 and packet N.  You are referring here to packet N, but you should make that clearer (packet N+1 is also out of order).

Packet N+1 benefits from rapid acknowledgment because it could accelerate retransmission.  Packet N benefits from rapid acknowledgment because it could suppress the retransmission triggered by a rapid acknowledgment of packet N+1.

+### Ack Ranges
+

"Receiver Tracking of ACK Frames"

It is customary to use title case for titles, even tiny ones like this.  You should make this `###` though instead of h4.

This sounds imperative and I don't think that it has to be.  The point here is that sending redundant acknowledgments is not especially harmful and a receiver can reduce complexity by doing this.

You should point out that if your largest acknowledged is N and you later receive N-1 (see the above reordering case for that), then dumping the acknowledgment for N-1 will mean also dropping the acknowledgment for N-1.  

Is your expectation is that the receiver will acknowledge N-1 immediately?  Or is it that you expect loss detection to have triggered for N-1 by the time that the receiver discards the acknowledgments?  I think that you need to clearer about expectations here, because the claim here suggests that this is not a problem.

It is possible that only one ACK frame contained the acknowledgment for N-1 and that if that is lost, then the sender will never learn that N-1 actually arrived.  The sender will compensate as though N-1 were lost, when in fact it was a loss of the acknowledgment on the return path.

BTW, why isn't there a loss detection delay to allow for reordering of ACKs (or the reordering of the packets that trigger acknowledgments)?  Too complicated?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1139#pullrequestreview-99162994

---

From: ianswett <[email-masked]>
Date: Tue, 27 Feb 2018 13:53:35 -0800

19d84f8  Update draft-ietf-quic-recovery.md

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/1139/files/c4a80bc0f69d920ae07326f89048e0c645f37ebd..19d84f836635b8821ebf6cdb3a897accf8c82700

---

From: ianswett <[email-masked]>
Date: Tue, 27 Feb 2018 21:53:48 +0000 (UTC)

ianswett commented on this pull request.

Text updated, PTAL.

Thanks, I clarified that it's not sensitive to ack loss.

+

I tried to clarify this some, see what you think.

Changed to not be imperative and I clarified cases when the peer could miss an acknowledgment.

In terms of loss detection delay, it would be more complex than it's worth, and at this point out of order acks are specified to not be processed in QUIC.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1139#pullrequestreview-99853502

---

From: ianswett <[email-masked]>
Date: Tue, 27 Feb 2018 22:08:36 +0000 (UTC)

204743e  Update draft-ietf-quic-recovery.md

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/1139/files/19d84f836635b8821ebf6cdb3a897accf8c82700..204743e56628b745b840c0c59cf6bbb4f4fc81a7

---

From: Martin Thomson <[email-masked]>
Date: Tue, 27 Feb 2018 16:22:28 -0800

> @@ -437,33 +437,44 @@ do not assume the peer generates an acknowledgement immediately when
 receiving a second full-sized segment.
 
 Out-of-order packets SHOULD be acknowledged more quickly, in order
-to accelerate loss recovery.  The receiver SHOULD send an immediate
-ACK when it receives a packet that fills in a gap in the packet numbers
-if it has already acknowledged a larger packet number.  If it has not
-sent an acknowledgement for the larger packet and there are no other
-recent gaps, then it may revert to the standard delayed ack algorithm.
+to accelerate loss recovery.  Specifically, the receiver SHOULD send an
+immediate ACK when it receieves a packet with a packet number more than
+1 larger than the previous largest received packet number. The receiver
+SHOULD also send an immediate ACK when it receives a packet that fills
+in a gap in the packet numbers.  If no packet larger than the packet that
+fills a gap has been acknowledged, then the peer will not invoke fast
+recovery and it may use the standard delayed ack algorithm.  This
+circumstance may arise from delaying ack transmission when
+processing multiple packets.

I struggled a little with the bit from "If no packet" onwards.  How about:

If a receiver receives acknowledgment that reveals a missed packet, it will trigger fast recovery.  Sending acknowledgments immediately on discovering a gap ensures that fast recovery is responsive.  Immediate acknowledgment for packets that fill gaps gives the sender a greater chance of avoiding unnecessary retransmissions.

 ### Ack Ranges
 
-When an ACK frame is sent, one or more ranges of acknowledged packets may
-be included.  Including older packets reduces the chances previous ACK
-frames were lost, causing spurious retransmits, at the cost of larger acks.
+When an ACK frame is sent, one or more ranges of acknowledged packets are
+included.  Including older packets reduces the chance of spurious
+retransmits caused by losing previously sent ACK frames, at the cost of
+larger ACK frames.

sometimes larger

 ACK frames SHOULD always acknowledge the most recently received packets,
-and the more out of order packets are, the more important it is to send
-an updated ACK frame quickly, in order to prevent the peer from declaring
-a packet as lost.
+and the more out-of-order the packets are, the more important it is to send
+an updated ACK frame quickly, to prevent the peer from declaring a packet
+as lost and spuriusly retransmitting the frames it contains.

I think that any amount of out-of-order is a risk, because gaps trigger fast recovery, which is more often than not spurious transmissions.

Also, we don't retransmit frames either.

 When a packet containing an ACK frame is sent, the largest acked in that
-frame is saved.  When a packet containing an ACK frame is acknowledged,
-the receiver stops acknowledging packets less than or equal to the largest
-acked in the sent ACK frame.  This approach is not sensitive to packet loss
-and allows for over 1 round trip of reordering.
+frame may be saved.  When a packet containing an ACK frame is acknowledged,
+the receiver can stop acknowledging packets less than or equal to the
+largest acked in the sent ACK frame.
+
+This approach does not guarantee every acknowledgement is seen by the

"does not guarantee *that* every acknowledgment" reads a little better

 When a packet containing an ACK frame is sent, the largest acked in that
-frame is saved.  When a packet containing an ACK frame is acknowledged,
-the receiver stops acknowledging packets less than or equal to the largest
-acked in the sent ACK frame.  This approach is not sensitive to packet loss
-and allows for over 1 round trip of reordering.
+frame may be saved.  When a packet containing an ACK frame is acknowledged,
+the receiver can stop acknowledging packets less than or equal to the
+largest acked in the sent ACK frame.
+
+This approach does not guarantee every acknowledgement is seen by the
+sender before it is no longer included in the ack frame, because packets

"ACK frame"

 When a packet containing an ACK frame is sent, the largest acked in that
-frame is saved.  When a packet containing an ACK frame is acknowledged,
-the receiver stops acknowledging packets less than or equal to the largest
-acked in the sent ACK frame.  This approach is not sensitive to packet loss
-and allows for over 1 round trip of reordering.
+frame may be saved.  When a packet containing an ACK frame is acknowledged,
+the receiver can stop acknowledging packets less than or equal to the
+largest acked in the sent ACK frame.
+
+This approach does not guarantee every acknowledgement is seen by the
+sender before it is no longer included in the ack frame, because packets
+could be received out of order and all subsequent ACK frames containing them
+could be lost.  It does guarantee the sender is making forward progress and

Do you want to insert: "In that case, the loss recovery algorithm will mark the packets as lost and cause data to be sent again anyway."?

In other words, make it clear that trying to fix this isn't going to do much good anyway.  It does mean that if the sender is unable to schedule the retransmissions before the ACKs are repaired, it could avoid sending again, but that's somewhat unlikely given the way that retransmission is generally prioritized.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1139#pullrequestreview-99905329