Subject: [quicwg/base-drafts] MUST ACK each ack-eliciting packet once (#3092)

From: ianswett <[email-masked]>
Date: Mon, 14 Oct 2019 22:40:15 -0700

The editors assumed this was true, but I think it&#39;s worth stating this as a MUST to ensure clarity.

Fixes #3091 
You can view, comment on, or merge this pull request online at:

  https://github.com/quicwg/base-drafts/pull/3092

-- Commit Summary --

  * MUST ACK each ack-eliciting packet once

-- File Changes --

    M draft-ietf-quic-transport.md (2)

-- Patch Links --

https://github.com/quicwg/base-drafts/pull/3092.patch
https://github.com/quicwg/base-drafts/pull/3092.diff

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3092

---

From: Martin Thomson <[email-masked]>
Date: Tue, 15 Oct 2019 16:25:48 -0700

martinthomson requested changes on this pull request.

> @@ -3013,6 +3013,8 @@ guidance offered below seeks to strike this balance.
 
 An ACK frame SHOULD be generated for at least every second ack-eliciting packet.
 This recommendation is in keeping with standard practice for TCP {{?RFC5681}}.
+Every ack-eliciting packet MUST be acknowledged in at least one transmitted
+ACK frame, and SHOULD typically be acknoweldged in more than one ACK frame.

I think that we want to say that every ack-eliciting packet MUST be acknowledged at least once, and ideally all packets are acknowledged until one of those acknowledgements are also acknowledged.

```suggestion
ACK frame, and SHOULD typically be acknowledged in more than one ACK frame.
```

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3092#pullrequestreview-302250928

---

From: ianswett <[email-masked]>
Date: Tue, 15 Oct 2019 20:22:02 -0700

@ianswett pushed 1 commit.

aa408593de2e13669e66403474ac99df9ff325da  Update draft-ietf-quic-transport.md

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/3092/files/24e19f14d0b03ce3c11a986a8dc041b8728301ac..aa408593de2e13669e66403474ac99df9ff325da

---

From: ianswett <[email-masked]>
Date: Wed, 16 Oct 2019 21:36:52 -0700

dd3c2214fa4d84ecc80229b44d9c817abcba71b8  Update draft-ietf-quic-transport.md

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/3092/files/aa408593de2e13669e66403474ac99df9ff325da..dd3c2214fa4d84ecc80229b44d9c817abcba71b8

---

From: Jana Iyengar <[email-masked]>
Date: Thu, 17 Oct 2019 11:25:42 -0700

janaiyengar commented on this pull request.

> @@ -3013,18 +3013,21 @@ guidance offered below seeks to strike this balance.
 
 An ACK frame SHOULD be generated for at least every second ack-eliciting packet.
 This recommendation is in keeping with standard practice for TCP {{?RFC5681}}.
+Every ack-eliciting packet MUST be acknowledged in at least one transmitted
+ACK frame, and SHOULD typically be acknowledged in more than one ACK frame.

I would rephrase this. "A receiver is expected to commonly acknowledge a received packet in multiple subsequent ACK frames, to increase the chances of the acknowledgement being received by the sender.  A receiver MUST acknowledge every received ack-eliciting packet at least once."

Also, move this to the top of the section.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3092#pullrequestreview-303457180

---

From: MikkelFJ <[email-masked]>
Date: Sun, 20 Oct 2019 03:24:50 -0700

I'm a bit worried this misses the point. Ack-eliciting packets should be acknowledged yes, but so should non-ack-eliciting packets. By stating that ack-eliciting packets should be ACK'ed at least once, it is implied that this is not the case for other packets. This may be correct but not necessarily clear. For example for long transmissions where the peer does nothing but sending ACKs, not all of those ACKs will necessarily have to be ACK'ed. But what is the guiding principle for that? And, since you in this case are sending lots of ACK eliciting packets, you can easily ACK all the peers ACK's. So when would you not want to do that? A special case that could deadlock - e.g. too many ACKs to fit in a packet?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3092#issuecomment-544239492

---

From: MikkelFJ <[email-masked]>
Date: Sun, 20 Oct 2019 03:27:27 -0700

Also, when ACKs packets don't fit in packets, this would also hold for ACK'ing ack-eliciting packets. So you could end up having to not ACK'ing some packets and induce loss. Requiring an ACK for all packets would be problematic. I'm not sure we want that. The result could be very hard to implement while the practical benefit would be small: sending multiple ACK packets with different content due to overflow.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3092#issuecomment-544239674

---

From: Martin Thomson <[email-masked]>
Date: Mon, 21 Oct 2019 18:57:38 -0700

martinthomson commented on this pull request.

To Mikkel's comment, could we say that *every* packet SHOULD be acknowledged at least once, and that ack-eliciting packets MUST be acknowledged within the maximum ack delay.  ?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3092#discussion_r337308612

---

From: ianswett <[email-masked]>
Date: Mon, 21 Oct 2019 20:02:41 -0700

ianswett commented on this pull request.

Sure, that could work.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3092#discussion_r337316512

---

From: ianswett <[email-masked]>
Date: Mon, 21 Oct 2019 20:08:29 -0700

5b3d744d7ac478df9c211712680948537cbca604  Merge branch 'master' into ianswett-must-ack-once

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/3092/files/dd3c2214fa4d84ecc80229b44d9c817abcba71b8..5b3d744d7ac478df9c211712680948537cbca604

---

From: ianswett <[email-masked]>
Date: Mon, 21 Oct 2019 20:10:42 -0700

I just synced to master and now this seems to be an editorial change, at least to me.

That being said, PTAL at the text and tell me if there are any other changes I should make.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3092#issuecomment-544788847

---

From: Martin Thomson <[email-masked]>
Date: Tue, 22 Oct 2019 03:20:38 +0000 (UTC)

> @@ -3007,9 +3007,6 @@ guidance offered below seeks to strike this balance.
 
 ### Sending ACK Frames {#sending-acknowledgements}
 
-An ACK frame SHOULD be generated for at least every second ack-eliciting packet.
-This recommendation is in keeping with standard practice for TCP {{?RFC5681}}.
-
 An endpoint MUST NOT excessively delay acknowledgements of ack-eliciting

I think that we're still missing the "An endpoint MUST send at least one acknowledgment for every packet it receives and successfully processes."  This paragraph is mostly about max ack delay, which is fine, but it only really says that you can't excessively delay.

Sure, you could say that "never send" == "excessive delay", but you could also say that you can't delay something that never existed in the first place.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3092#pullrequestreview-304948609

---

From: MikkelFJ <[email-masked]>
Date: Mon, 21 Oct 2019 23:15:11 -0700

mikkelfj commented on this pull request.

 

What if you run out if space in the ACK frame?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3092#discussion_r337342373

---

From: Martin Thomson <[email-masked]>
Date: Tue, 22 Oct 2019 01:30:18 -0700

If you send an ACK for every other received packet and drop all older acknowledgments, then you use at most 19 bytes for an ACK, probably less.  I don't think that will break any limits.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3092#discussion_r337385994

---

From: ianswett <[email-masked]>
Date: Tue, 22 Oct 2019 04:29:46 -0700

@mikkelfj Even if you only ACK the last 2 packets per ACK, QUIC should work correctly.  It's if packets never get acknowledged, especially consistently, there are potential problems.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3092#discussion_r337461035

---

From: MikkelFJ <[email-masked]>
Date: Tue, 22 Oct 2019 04:36:21 -0700

I can see this works on average - but if you opt to max out on max delay to minimize transmissions in one direction you could find you self in trouble. A MUST would then force to send more frequently than optimal whereas SHOULD would allow for occasionally dropping the ball. Dropping the ball frequently would be be bad, of course, suggesting that ACKs are not sent often enough.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3092#discussion_r337463479

---

From: ianswett <[email-masked]>
Date: Tue, 22 Oct 2019 06:06:50 -0700

c619547e6376dadc2e5e148ba1812dcc75b1a4d7  Martin's previous suggestion

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/3092/files/5b3d744d7ac478df9c211712680948537cbca604..c619547e6376dadc2e5e148ba1812dcc75b1a4d7

---

From: ianswett <[email-masked]>
Date: Tue, 22 Oct 2019 06:08:11 -0700

@martinthomson I worked your previous suggestion into the text, which I like better than what was there anyway.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3092#discussion_r337503940

---

From: ianswett <[email-masked]>
Date: Tue, 22 Oct 2019 06:15:21 -0700

@mikkelfj I don't understand the issue you're concerned about.  One can fit an ACK of any two packets into a datagram, which is the minimum requirement of this text.  Yes, you could set a very large max_ack_delay, which would cause enormously long PTO timeouts and be very bad for performance in most cases.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3092#discussion_r337507912

---

From: MikkelFJ <[email-masked]>
Date: Tue, 22 Oct 2019 06:24:09 -0700

I don't want to make this a hard argument, I'm just slightly concerned. Maybe it is a non-issue, or maybe potential issues are shadowed by other concerns such as PTO. Some devices might not want to fire up a transmitter radio more often than necessary. But generally I'm just concerned with a MUST that cannot be enforced or fully controlled - you can easily claim it works under usual conditions but it might not under abnormal conditions. A SHOULD would make it possible to drop packets under those a conditions. A MUST would either have to make assumptions that are hard to prove correct, or have complex code to handle edge case where ACK's queue op, or violate the MUST. I suspect most implementations would go for the latter and hope they don't violate by sending ACK's often enough.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3092#discussion_r337512807

---

From: Nick Banks <[email-masked]>
Date: Tue, 22 Oct 2019 07:03:54 -0700

nibanks commented on this pull request.

Generally looks good, but I don't think we can have an absolute requirement that every ack-eliciting packet is acknowledged.

> +Every packet SHOULD be acknowledged at least once, and ack-eliciting packets
+MUST be acknowledged at least once within the maximum ack delay. An endpoint

I think I agree with @mikkelfj that MUST might not be the right word here. It's obviously unenforceable for one. But also, I think there are valid reasons where a packet won't be acknowledged. For one, we have a memory limit for the data structure we use to keep track of packets that need to be acknowledged. At a high enough rate, with enough (random?) packet loss you could have enough gaps to hit that limit. In that case, we start forgetting older packet numbers, and will never end up acknowledging them.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3092#pullrequestreview-305230869

---

From: Martin Thomson <[email-masked]>
Date: Tue, 22 Oct 2019 15:23:44 -0700

Merged #3092 into master.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3092#event-2735026081

---

From: Martin Thomson <[email-masked]>
Date: Tue, 22 Oct 2019 16:43:18 -0700

We have a memory limit too, but that doesn't mean that we don't send acknowledgments.  We just send packets more often as a result.

As for untestable, that's true for a bunch of things we require.  The point is to require actions that produce an interoperable, functional protocol.  In the general case, ignoring this requirement would be bad.  At insane incoming packet rates, maybe this breaks down, and that's probably not ideal, but I find it hard to believe that you can't generate enough packets outbound to acknowledge every inbound packet at least once.  Even if you only receive every other packet, you are only spending two bytes per packet to acknowledge them.  That's almost a 600:1 advantage to the receiver.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3092#discussion_r337796493

---

From: MikkelFJ <[email-masked]>
Date: Wed, 23 Oct 2019 00:34:37 -0700

I think the biggest problem is if the peer assumes a packet was not processed if it eventually concludes it was not ACK'ed. That could lead to deadlocks or double spending in some poor transactional design.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3092#discussion_r337888881

---

From: ianswett <[email-masked]>
Date: Wed, 23 Oct 2019 07:11:33 -0700

An acknowledgement indicates a packet was received and processed.  It makes no claim that other packets are never processed.  This is true throughout the QUIC design.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3092#discussion_r338072464

---

From: MikkelFJ <[email-masked]>
Date: Wed, 23 Oct 2019 07:15:05 -0700

The MUST de facto makes that claim, although timing can make it hard to decide.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3092#discussion_r338074660

---

From: Mike Bishop <[email-masked]>
Date: Thu, 24 Oct 2019 12:34:12 -0700

MikeBishop commented on this pull request.

No, it doesn't, because you have no guarantee that you received every ACK the peer sent.  This requires the peer to put every packet into an ACK frame sent in at least one packet; there's no requirement to ensure that it's delivered reliably to the peer.  Thus, if you haven't received an ACK, that doesn't mean the peer didn't received the packet anyway.  It could be delayed and still in transit, or the ACK might have been lost.

That said, there's no actual difference between "MUST ack every packet, but not reliably" and "SHOULD ack every packet" from the other side's point of view.  This is something that could be bench-tested on an implementation, but can't be ascertained about the peer over a lossy network.

> @@ -3007,25 +3007,26 @@ guidance offered below seeks to strike this balance.

+Every packet SHOULD be acknowledged at least once, and ack-eliciting packets
+MUST be acknowledged at least once within the maximum ack delay. An endpoint
+communicates its maximum delay using the max_ack_delay transport parameter;
+see {{transport-parameter-definitions}}.  max_ack_delay declares an explicit

It's probably correct to start the sentence with a lower-case letter, since this is a literal name of a protocol element, but it looks weird.  Consider rephrasing to avoid the question.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3092#pullrequestreview-306812844

---

From: MikkelFJ <[email-masked]>
Date: Thu, 24 Oct 2019 13:13:29 -0700

True, I forgot that a specific PN can be present in one ACK frame on not in the next, even if there are both older and younger PNs in the next ACK frame. This makes it inconclusive.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3092#discussion_r338770021