Subject: [quicwg/base-drafts] HTTP/3 unidirectional streams arms race (#2559)

From: Lucas Pardue <[email-masked]>
Date: Tue, 26 Mar 2019 20:17:04 -0700

In Tokyo 2019, we discussed #2100 (Avoid creating QPACK streams when unnecessary) and came upon the conclusion:

> Discussed in Tokyo; do not require creating the streams, but creating them is not forbidden even if the setting prevents their use.

Until such a change is applied to the HTTP and QPACK specs, we rely on the guidance in section 3.2:

> Both clients and servers SHOULD send a value of three or greater for
   the QUIC transport parameter "initial_max_uni_streams".

There is an implicit assumption in this present state that assumes these three streams will be used for the "critical streams" - Control stream, QPACK Encoder Stream and QPACK Decoder stream. The outcome of the Tokyo discussion didn't seem to me to identify the impact on initial_max_uni_streams. 

HTTP/3 reserves code points for GREASING but doesn't provide much guidance on their usage. It says "MUST ignore" and CAN send reserved stream types. In contrast Transport is a little stronger and says that:

> A server SHOULD include a reserved version (see Section 15) while generating a Version Negotiation packet.

Not sending GREASE negates its powers. 

GREASE streams are not the only consumer of unidirectional credit. It seems feasible that an endpoint might optimise for some use cases and decide push on 2-out-of-three streams it was given the credit for. This depends on receiving MAX_PUSH_ID on the control stream, of course.

The control stream unfortunately has dubious language:

> Each side MUST initiate a single control stream at the beginning of
   the connection.

When is the beginning of beginning and when is the end of beginning? Mike somewhat answered this question in #2038 (Default settings in HTTP):

> You're still required to send/consume SETTINGS -- you're just allowed to use a basic version of the protocol for the (hopefully very brief) period where you haven't seen it yet. I'm dubious anyone would decide it's better to send an empty SETTINGS frame and continue using the defaults forever, certainly not to the point that extensibility withers.

Is the end the beginning once SETTINGS have been sent?

PR #2529 was an attempt at scurry along the spec change #2100 wanted. Created in response to hitting an interop issue causes by the old requirement to "MUST create one QPACK Encoder and Decoder stream" conflicting with the "SHOULD allow initial max 3". However, the issue I see with the way the specs would look like with this specification is that there is not much determinism on how an endpoint should use MAX_STREAMS and how it can the unidirectional streams it makes available will be used. For example,

* a client advertises 3 initial max uni streams, likewise a server advertises 3
* a server creates a 2 grease streams and 1 encoder stream
  * Recall the Tokyo outcome was "creating them is not forbidden even if the setting prevents their use" and by default value of SETTINGS_QPACK_MAX_TABLE_CAPACITY is 0 if you never receive a SETTING that tells you otherwise
* the client expects 3 "critical streams", so banks the difference (2) and gives unidirectional stream credit by sending a MAX_STREAMS frame of value 5
* the client opens a grease stream, a control stream, and 1 encoder stream
* the client sends SETTINGS_QPACK_MAX_TABLE_CAPACITY = Y
* the client sends MAX_PUSH_ID = 3
* the client sends a request with static compression
* the server sends 3 PUSH_PROMISES and attempts to push 3 resources.
* the server wishes to open N unidirectional extension streams
* the server sends STREAMS_BLOCKED with the value of 5
* the client sends a request with dynamic compression
* the server wishes to open a decoder and sends STREAMS_BLOCKED with the value of 5

a) What value of MAX_STREAMS is suitable for a client to send?
b) if the value is < N+2 what is a suitable rate to send STREAMS_BLOCKED?
c) will this just devolve to a cycle of one endpoint sending STREAMS_BLOCKED and the other echoing back MAX_STREAMS (value +1) until the other side shuts up?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2559

---

From: Kazuho Oku <[email-masked]>
Date: Tue, 26 Mar 2019 20:26:17 -0700

> c) will this just devolve to a cycle of one endpoint sending STREAMS_BLOCKED and the other echoing back MAX_STREAMS (value +1) until the other side shuts up?

It seems to me that this is the worst case, and that if we think it's fine then every other questions can be considered fine.

My position is that having this worst case is fine.

Many types of greasing (e.g., version negotiation, retry, HelloRetryRequest) introduces one extra roundtrip. So having one extra round-trip when opening one greasing stream sounds not unreasonable.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2559#issuecomment-476955236

---

From: Lucas Pardue <[email-masked]>
Date: Tue, 26 Mar 2019 20:53:19 -0700

> So having one extra round-trip when opening one greasing stream sounds not unreasonable.

I'm ok with that but it also risks affecting all unidirectional stream types. Server push somewhat skirts the issue by providing a signal (counting PUSH_PROMISEs) that enhances the clients knowledge. QPACK is part way there because non-zero SETTINGS_QPACK_MAX_TABLE_CAPACITY implies that some credit from both ends is required. 

Perhaps we could summarise some guidance for people about the RTT incurred by these streams and ways to avoid them.

A totally different alternative that might turn out in practice easier is to simply set max_initial_uni_streams to a very large number, and set the initial_max_stream_data_uni to a low number (8 bytes would be enough to peak at the stream type). This way the client has far more insight into what the other side is attempting to do and can exercise stream control as suited to its needs. All while avoiding this weird cycle.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2559#issuecomment-476961392

---

From: Kazuho Oku <[email-masked]>
Date: Tue, 26 Mar 2019 21:11:37 -0700

> Perhaps we could summarise some guidance for people about the RTT incurred by these streams and ways to avoid them.

Agreed. Recommending "at least three, but more is better because there could be other types of streams" makes sense to me.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2559#issuecomment-476965131

---

From: Martin Thomson <[email-masked]>
Date: Tue, 26 Mar 2019 23:28:23 -0700

I think that this has resolved into an editorial-level conclusion.  (That I agree with.)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2559#issuecomment-476997190

---

From: Lucas Pardue <[email-masked]>
Date: Wed, 27 Mar 2019 05:43:04 -0700

> I think that this has resolved into an editorial-level conclusion. (That I agree with.)

I'll create a PR

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2559#issuecomment-477135062