Subject: [quicwg/base-drafts] Possible SETTINGS_ACK proposal (#84)

From: Mike Bishop <[email-masked]>
Date: Tue, 13 Dec 2016 10:23:44 -0800

This is what the ACK-on-all-streams option would look like, I think.  For #75, but depends on #80.
You can view, comment on, or merge this pull request online at:

  https://github.com/quicwg/base-drafts/pull/84

-- Commit Summary --

  * SETTING_ACK across all streams -- for #75

-- File Changes --

    M draft-ietf-quic-http.md (82)

-- Patch Links --

https://github.com/quicwg/base-drafts/pull/84.patch
https://github.com/quicwg/base-drafts/pull/84.diff

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/84

---

From: Martin Thomson <[email-masked]>
Date: Tue, 13 Dec 2016 15:54:13 -0800

martinthomson commented on this pull request.

> -here that need to be solved.
-
+recipient MUST immediately emit the following:
+
+ - On the connection control stream, a SETTINGS_ACK frame listing the 
+   identifiers whose values were understood and applied. (If none of the values 
+   were understood, the SETTINGS_ACK frame will be empty, but MUST still be
+   sent.) 
+
+ - On each request control stream which is not in the "half-closed (local)" or
+   "closed" state, an empty SETTINGS_ACK frame.
+
+The SETTINGS_ACK frame on the connection control stream contains the highest
+stream number which was open at the time the SETTINGS frame was received.  All
+streams with higher numbers can safely be assumed to have the new settings in
+effect when they open.

Let's say that the opening of a stream is delayed (loss looks like a delay for this stuff).  Stream N gets through, but stream N-2 takes another RTT or so to open.  What is the receiver of the SETTINGS frame expected to do in this case?  Is it expected to play out the SETTINGS_ACK when the stream finally makes it through?

The problem here is that stream N-2 might have been skipped by the other side.  How does a receiver know what to do with that uncertainty?  It can't hold onto the SETTINGS_ACK indefinitely.

I think that we need to mandate no stream skipping in this case.

The next problem is in correlating the SETTINGS_ACK with the SETTINGS.  I think that your scheme almost works, but we probably want to include some text on how this correlation is expected to work:
 - for every SETTINGS that requests acknowledgment, there is a SETTINGS_ACK
 - for every SETTINGS_ACK, every open stream less or equal to the included stream number will contain a SETTINGS_ACK

Where this doesn't work is when the receiver of a SETTINGS frame opens a stream.  Say a server sends SETTINGS at and then a client opens stream 7.  The server cannot know whether stream 7 was open at the time that the client received the SETTINGS frame.  Even if the opening frame on stream 7 is in the same packet as the client's SETTINGS_ACK, there is no good way to know what order the client actually performed these two actions.  Thus, if stream 7 is still open when a second SETTINGS is sent, the SETTINGS_ACK that the server receives on stream 7 could correspond to either settings.

The only fix I'm aware of is to set start AND end markers on the control stream SETTINGS_ACK.

> -SETTINGS identifiers which the sender has understood and applied. This 
-list MAY be empty. 
+of specific values in the peer's SETTINGS frame. Depending on the stream where
+it is sent, it takes two different forms.
+
+On the connection control stream, it contains information about how and when the
+sender has processed the most recently-received SETTINGS frame, and has the
+following payload:
+
+~~~~~~~~~~~~~~~
+    0                   1                   2                   3
+    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |                      Highest Local Stream (32)                |
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |                      Highest Remote Stream (32)               |

See above.

> +of specific values in the peer's SETTINGS frame. Depending on the stream where
+it is sent, it takes two different forms.
+
+On the connection control stream, it contains information about how and when the
+sender has processed the most recently-received SETTINGS frame, and has the
+following payload:
+
+~~~~~~~~~~~~~~~
+    0                   1                   2                   3
+    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |                      Highest Local Stream (32)                |
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |                      Highest Remote Stream (32)               |
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |                    Recognized Identifiers (*)               ...

Would it be more efficient to include unrecognized identifiers?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/84#pullrequestreview-12806466

---

From: Mike Bishop <[email-masked]>
Date: Wed, 14 Dec 2016 11:45:22 -0800

MikeBishop commented on this pull request.

-

The first case is the corner-case from line 572 -- because QUIC mandates that streams be opened "sequentially," stream-skipping is already prohibited.  (Should that be made clearer?)  The fact that I've seen stream N opened means N-2 is in-flight, even if I haven't seen it yet.  That means, having received a SETTINGS frame, I can go ahead and send a SETTINGS_ACK on N-2 *without having yet received a frame* because it's not out-of-order at the remote side.

Your suggested correlation text looks a lot like what I have at line 576 about what's needed to consider the ACK "finished."  I might need to add a note that all previous SETTINGS frames have to be "finished" before a subsequent one can be -- ACKs on-stream are fully ordered.

I don't see the issue when the receiver of SETTINGS opens the stream.  There are two possible orderings:

- **SETTINGS processed before stream created.**  Then the SETTINGS_ACK on stream 3 will not cover the new stream, and the stream abides by the new settings from the start.
- **Stream created before SETTINGS processed.**  Then the SETTINGS_ACK on stream 3 *will* cover the new stream, and the sender will expect a STREAM_ACK (which may be the first frame on the stream).

What am I missing?

---

From: Mike Bishop <[email-masked]>
Date: Wed, 14 Dec 2016 11:53:06 -0800

More relevant on #80.  I'm indifferent on that point -- just that it's useful to have a yes/no for each entry after processing the ACK.

If we want to be as compact as possible, we could even do a bitmap of the identifiers and mark them understood/not-understood.  That requires a little more state at the sender of the SETTINGS frame to recall the order, but not much.

---

From: Martin Thomson <[email-masked]>
Date: Wed, 14 Dec 2016 15:09:20 -0800

-

I think that you have it.  Thanks for walking through it with me.  The sequential thing definitely needs work.  "Sequential" just means in order, which doesn't correspond to a mandate to use every number.

---

From: Martin Thomson <[email-masked]>
Date: Wed, 14 Dec 2016 15:10:43 -0800

The efficiency I'm looking for is in processing and understanding.  An endpoint that receives a negative list can more easily mark those settings as "do not try again".  As you say, a bit vector is the most compact, but given the relative frequency of settings changes, I'm not interested in optimizing that much.

---

From: Mike Bishop <[email-masked]>
Date: Thu, 15 Dec 2016 09:01:51 -0800

My initial thought here was that creating a mechanism to trigger the echo of arbitrary identifiers was a potential security threat, but as the attacker has to send at least 32 bits per desired 16-bit echo, there's no amplification issue here.  A negative list should work -- I'll make the change.

---

From: Mike Bishop <[email-masked]>
Date: Thu, 15 Dec 2016 09:18:13 -0800

@MikeBishop pushed 1 commit.

00c56ae  Martin's feedback and some alignment

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/84/files/6800c5a950fa3ea80aa373b13c51cbdcb7d3bbaa..00c56ae130917e15701b35194e8518039db35a7a

---

From: Mike Bishop <[email-masked]>
Date: Thu, 15 Dec 2016 09:19:14 -0800

-

Added some text to the Stream Usage section explicitly requiring no stream-skipping, just to be thorough.

---

From: Mike Bishop <[email-masked]>
Date: Thu, 15 Dec 2016 14:18:07 -0800

Merged #84.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/84#event-895952336