Subject: [quicwg/base-drafts] Asymmetry in the handling of SETTINGS frame (#1846)

From: Kazuho Oku <[email-masked]>
Date: Tue, 09 Oct 2018 02:24:12 -0700

[Section 2.3 of the editor's draft](https://quicwg.org/base-drafts/draft-ietf-quic-http.html#rfc.section.2.3) forbids a server from sending data on any other stream until it receives a SETTINGS frame on the control stream, while such requirement does not exist for the client even for the 1-RTT case.

I hereby propose to remove the requirement for the server as well. The requirement is a source of unnecessary complexity; it requires a latchet that blocks the server to send a HTTP response until it receives the first octet of the control stream.

IMO, not knowing the QPACK table size cannot justify having the blocking rule for the server-only; the same rule does not exist for the client and therefore a client is allowed to start sending requests without knowing the QPACK table size. Having symmetry has the potential of simplifying the specification and implementations. For example, a HQ stack can send HTTP messages without using the dynamic table until it receives the SETTINGS frame on *both client-side and server-side*.

Note that even if we relax the specification, the endpoints will have the freedom to block until it receives the first octet of the control stream.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1846

---

From: afrind <[email-masked]>
Date: Tue, 09 Oct 2018 09:04:10 -0700

I'm generally ok with this concept.

SETTINGS_MAX_HEADER_LIST_SIZE is also in the settings, with a default value of unlimited (side note, this meaning of this setting is not described anywhere in the HQ spec).  If we allow sending header blocks before receiving SETTINGS, is the peer required to accept arbitrarily long header lists?  Or we should reintroduce a lower default?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1846#issuecomment-428251470

---

From: Luca Niccolini <[email-masked]>
Date: Tue, 09 Oct 2018 09:12:48 -0700

just pointing out that the latest draft does specify the client behavior in the 1-RTT case

> When a 1-RTT QUIC connection is being used, the client MUST NOT send requests prior to receiving and processing the server’s SETTINGS frame.

but I agree that if we can find a way to start sending without having received the peer settings, that would simplify the implementation

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1846#issuecomment-428254560

---

From: Mike Bishop <[email-masked]>
Date: Tue, 09 Oct 2018 18:50:05 +0000 (UTC)

This isn't asymmetrical, actually.  Both client and server MUST know the settings before they can send requests/responses.  It's just that there are two cases, which you're conflating:

- For 1-RTT, the client is required to see the server's (0.5-RTT flight) SETTINGS frame before sending requests, and the server is required to see the client's SETTINGS frame before sending responses.
- For 0-RTT, the client is assuming (and the server is consenting to, by accepting 0-RTT) minimum values for settings remembered from last time, which is what permits it to send without seeing the SETTINGS.  The server still needs the client's SETTINGS, because there is no implicit contract on the client's parameters in 0-RTT.

In HTTP/2, both parties have to tolerate peers having lower values than the defaults: the defaults apply until the real SETTINGS frame is seen and then get revised downward, so you have to tolerate reductions after you've already violated the setting unknowingly.  In the HQ version, there is never a downward revision, only an upward revision, so you eliminate the unknowing violation.  That was the motivation to introduce the restriction.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1846#issuecomment-428307396

---

From: Mike Bishop <[email-masked]>
Date: Tue, 09 Oct 2018 18:54:53 +0000 (UTC)

Also, because settings are allowed to negotiate non-conformant behavior (and we no longer have a SETTINGS_ACK), in the general case, you can't guarantee that anything received on any other stream actually means what you think it means until you've seen SETTINGS.  In the more common case where you don't support any such behavior, of course, this isn't a problem.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1846#issuecomment-428308995

---

From: Lucas Pardue <[email-masked]>
Date: Tue, 09 Oct 2018 19:01:19 +0000 (UTC)

>  In the HQ version, there is never a downward revision,

Hmm, so I assumed the default for max header size list was unlimited (inferred from the language used to define SETTINGS_MAX_HEADER_LIST_SIZE). Which would mean that is it possible to reduce the size. Have I misunderstood?

Perhaps it would be better to define QPACK related settings in QPACK?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1846#issuecomment-428311258

---

From: Dmitri Tikhonov <[email-masked]>
Date: Tue, 09 Oct 2018 12:12:35 -0700

In HTTP2 world, this setting is defined in RFC 7540, not in the HPACK RFC (7541):

https://tools.ietf.org/html/rfc7540#section-6.5.2

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1846#issuecomment-428315051

---

From: Lucas Pardue <[email-masked]>
Date: Tue, 09 Oct 2018 12:15:46 -0700

Perhaps H2 got it wrong? :P

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1846#issuecomment-428316136

---

From: Mike Bishop <[email-masked]>
Date: Tue, 09 Oct 2018 12:17:16 -0700

@LPardue, the defaults apply if the SETTINGS frame arrives and the value is not specified.  There simply isn't a value until the SETTINGS frame arrives.

We did move the QPACK-specific settings over in a previous issue/PR; that setting is not *quite* QPACK-specific.  The precise meaning is poorly defined in HTTP/2, and perhaps we should open an issue to clarify:  Is it the maximum length of a HEADERS frame, or the maximum length of the decoded headers?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1846#issuecomment-428316705

---

From: Dmitri Tikhonov <[email-masked]>
Date: Tue, 09 Oct 2018 12:19:43 -0700

> Is it the maximum length of a HEADERS frame, or the maximum length of the decoded headers?

RFC 7540 makes it clear:

> This advisory setting informs a peer of the maximum size of header list that the sender is prepared to accept, in octets.  The value is based on the uncompressed size of header fields, including the length of the name and value in octets plus an overhead of 32 octets for each header field.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1846#issuecomment-428317517

---

From: Mike Bishop <[email-masked]>
Date: Tue, 09 Oct 2018 12:20:42 -0700

Yes, I had already updated above to reflect that.  Implementations I'm familiar with don't use that precise definition of header length, though, and it's purely advisory to begin with.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1846#issuecomment-428317811

---

From: Lucas Pardue <[email-masked]>
Date: Tue, 09 Oct 2018 12:31:09 -0700

> @LPardue, the defaults apply if the SETTINGS frame arrives and the value is not specified. There simply isn't a value until the SETTINGS frame arrives.

We did move the QPACK-specific settings over in a previous issue/PR; that setting is not quite QPACK-specific. The precise meaning is poorly defined in HTTP/2, and perhaps we should open an issue to clarify: Is it the maximum length of a HEADERS frame, or the maximum length of the decoded headers?

ACK. I've gotten myself confused with this setting and QPACK-specific ones. Sorry!

I think it would be helpful to redefine what this settings means explicitly, or link to the HTTP/2 section if we are carrying that over. 

Back on the true topic of this ticket, I agree with Mike that we need to block on SETTINGS to know exactly what HTTP-level connection characteristics are being 'negotiated'. For example, my candidate [unbound server push extension](https://datatracker.ietf.org/doc/draft-pardue-quic-http-unbound-server-push) would allow a server to start sending PUSH_PROMISE frames on server-initiated unidirectional streams, something that would normally not be allowed in HTTP/QUIC. How would a non-blocking client handle such a case?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1846#issuecomment-428321027

---

From: Dmitri Tikhonov <[email-masked]>
Date: Tue, 09 Oct 2018 12:31:34 -0700

We perform precisely that calculation for GQUIC in [lsquic](https://github.com/litespeedtech/lsquic-client/blob/3b55e6ae0a91500b2612f9d20a57d3d2fb777886/src/liblsquic/lsquic_frame_writer.c#L377).  So know you are familiar with one. :-)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1846#issuecomment-428321170

---

From: Kazuho Oku <[email-masked]>
Date: Tue, 09 Oct 2018 14:26:52 -0700

@lnicco @MikeBishop Thank you for clarifying that we have a restriction for 1-RTT case, client-side.

@MikeBishop 
> Also, because settings are allowed to negotiate non-conformant behavior (and we no longer have a SETTINGS_ACK), in the general case, you can't guarantee that anything received on any other stream a…

If that is the case, shouldn't we better block the client from sending anything (including SETTINGS) until it receives SETTINGS from server sent in 0.5 RTT?

In my view, without such requirement, current design does *not* allow the endpoints to *negotiate* their settings. Rather, it is unilateral declaration of SETTINGS by two parties. Assuming that we are fine with the lock-step approach on exchanging SETTINGS, I think adding one latch that requires the client for server's SETTINGS before sending it's SETTINGS would be fine.

If we have that latch, exchange of SETTINGS becomes a *negotiation*. That would allow us to evolve in various ways, e.g. have a different static table (#1343), or even have a different compression algorithm.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1846#issuecomment-428358560

---

From: Lucas Pardue <[email-masked]>
Date: Tue, 09 Oct 2018 15:01:52 -0700

This is a good point. I've been operating under an assumption that a QUIC transport acknowledgement is equivalent to a SETTINGS_ACK but I'm now not sure that is true; the application layer may still not have processed the received SETTINGS.

I think the unilateral declaration is ok. For the asymmetric case like headers, the settings are independent and endpoints use the received value. For a new feature that affects the connection equally simple AND logic seems ok (feature is only 'on' if both declare it so). That's how Server Push used to work before the setting was replaced by MAX_PUSH_ID.

I don't know if @kazuho's latch proposal allows more than one type basic negotiation - server advertises multiple integer values (A, B, or C), client selects a value. On another ticket I suggested that a design that uses integer only SETTINGS parameters with an AND-logic approach can be used to enable a more complicated negotiation mechanism i.e. use SETTINGS to inform the peer that you are intending to send a new negotiation FRAME type followed later by the feature FRAME type.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1846#issuecomment-428368292

---

From: Mike Bishop <[email-masked]>
Date: Tue, 09 Oct 2018 15:50:40 -0700

I think the mechanism for "negotiation" looks like this:  each side unilaterally declares something (the set of options it supports for a particular extension, say), then each side performs the same calculation and derives the same result (mutually supported values, select most preferred by some measure if a single choice is required).

The implied ACK currently is that neither side is supposed to send until it knows what the other side supports, so if you're seeing bytes on other streams, they know what your settings are.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1846#issuecomment-428379797

---

From: Mike Bishop <[email-masked]>
Date: Tue, 09 Oct 2018 16:07:25 -0700

@kazuho, am I right that the general statement for your model is that there are three sets of settings exchanged:
1. Server's initial (SETTINGS frame in 1-RTT, remembered in 0-RTT)
2. Client's initial (SETTINGS frame always)
3. Server's update (not present in 1-RTT, SETTINGS frame in 0-RTT; cannot reduce values, but can increase them)

You'd then say that you must have received the previous item before you can generate the subsequent one?  That would certainly enable consistent offer-select semantics for settings; the limitation is that any option the server adds during (3) isn't actually available until the next connection, when it's part of the implicit (1).  However, that's true of the existing model as well unless there's an explicit change mechanism built in (see QPACK table size).

(BTW, change of compressor / static table doesn't actually require an offer-select negotiation because the two directions are independent.  Each side sends the set that it supports in preference order, then the result is that the other side will receive its most preferred (first in its list) which occurs in the overlap.  That could result in different selections in each direction, but that's not a bad thing in many cases -- separate static tables for clients and servers, for example.)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1846#issuecomment-428383163

---

From: Kazuho Oku <[email-masked]>
Date: Tue, 09 Oct 2018 19:33:15 -0700

@MikeBishop Yes. That is precisely the model I am interested in having. I also agree with your observation stated in the last paragraph.

I think I might rephrase my question to: why don't we block the client from sending *anything* until it receives SETTINGS, rather than just blocking the requests?

The benefits are:
* We can have offer-select semantics.
* We have the chance of reducing the number of unidirectional streams:
  * We might want to state that an endpoint opens a QPACK encoder stream only when it sees the peer advertising a non-zero table size, and that an endpoint opens a QPACK decoder stream only when it sees the peer opening the encoder stream. This would be a natural if we block the client from sending anything until it sees SETTINGS.
  * Then, a memory-constrained device would need to have only one slot for unidirectional streams in each direction, assuming that it advertises a QPACK table size of zero. This is better than the current requirement which requires us of having slots for three incoming unidirectional streams even though they might become idle.
* Expanding on the previous point, we even have the freedom to replace QPACK with something else; for example, we could have XPACK that uses a unidirectional stream identifier of 'X'.

Compared to these benefits, I do not see any downside in particular.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1846#issuecomment-428417336

---

From: Dmitri Tikhonov <[email-masked]>
Date: Wed, 10 Oct 2018 06:08:16 -0700

@LPardue writes:
> I've been operating under an assumption that a QUIC transport acknowledgement is equivalent to a SETTINGS_ACK but I'm now not sure that is true; the application layer may still not have processed the received SETTINGS.

Theoretically, because the SETTINGS frame is a _stream_ frame, not a QUIC frame, the receipt of a packet carrying a SETTINGS frame does not mean that the stream is readable.  This is because the stream type `C` may be in a different packet from the SETTINGS frame.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1846#issuecomment-428564382

---

From: Lucas Pardue <[email-masked]>
Date: Wed, 10 Oct 2018 12:12:58 -0700

thanks @dtikhonov. With that in mind, my thought experiments of attempting to determine if the remote endpoint received and processed a SETTINGS frame seems kludgy and fragile.

@MikeBishop / @kazuho Does your line of thinking risk introducing delay of _sending_ things? I realise the overall timing  might net out the same. 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1846#issuecomment-428696277

---

From: Ryan Hamilton <[email-masked]>
Date: Wed, 10 Oct 2018 12:29:49 -0700

@LPardue I dont' think it's kludgy or fragile at all :) You just need to ensure that the SETTINGS data *AND* all previous data on that stream have been acked. Depending on the data structure that you use to store data for retransmission that may well be trivial.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1846#issuecomment-428701307

---

From: Lucas Pardue <[email-masked]>
Date: Wed, 10 Oct 2018 13:22:28 -0700

In the limited experience I have, we read data off the transport as fast as possible but the application is then free to take as long as it wants to actually process stream data. Which is why I'm not so sure that transport acks are a definitive signal.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1846#issuecomment-428717268

---

From: Mike Bishop <[email-masked]>
Date: Wed, 10 Oct 2018 20:25:42 +0000 (UTC)

There would be almost no new delay; in the 1-RTT case, if the server's 1-RTT packet containing the SETTINGS frame is
- not coalesced with the Handshake packet containing the ServerFinished, and
- lost
then in the current scheme the client could *only* send SETTINGS, while in Kazuho's proposal the client couldn't send SETTINGS either.  It doesn't induce delay in sending requests.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1846#issuecomment-428718363

---

From: Dmitri Tikhonov <[email-masked]>
Date: Wed, 10 Oct 2018 13:46:56 -0700

@RyanAtGoogle, but there is no guarantee that the application layer has processed the _SETTINGS_ frame even if all the packets carrying the corresponding _STREAM_ frames have been acked by the transport layer.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1846#issuecomment-428725690

---

From: Ryan Hamilton <[email-masked]>
Date: Wed, 10 Oct 2018 14:37:22 -0700

@dtikhonov Oh, agreed. I was merely addressing the potential issue of the 'C type' indication going in a different packet.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1846#issuecomment-428741452

---

From: Kazuho Oku <[email-masked]>
Date: Wed, 10 Oct 2018 18:02:52 -0700

> in the current scheme the client could _only_ send SETTINGS, while in Kazuho's proposal the client couldn't send SETTINGS either. It doesn't induce additional delay in sending requests.

To amend what @MikeBishop states here, I would like to point out that we currently have two ways of doing negotiation.

For bidirectional streams (omitting the 0-RTT case), we currently use unidirectional declaration of SETTINGS. An endpoint is not allowed to send data using a bidirectional stream until it receives SETTINGS.

For unidirectional streams, we do not negotiate prior to using them. Instead, we have a type octet, and a endpoint that receives an unknown type is expected to close the stream by sending STOPPING.

The design works. But I wonder if there is a reason to have two different ways of "negotiation." My suggestion is to use SETTINGS-based negotiation for both bidirectional *and* unidirectional streams. The benefits in doing so are listed in 
https://github.com/quicwg/base-drafts/issues/1846#issuecomment-428417336.

Doing everything in lock-step will not introduce additional delay to processing HTTP requests in practice. If we are to care about such delay, I would prefer unblocking the use of bidirectional streams as well. That is what I originally proposed in this issue.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1846#issuecomment-428782407

---

From: Mike Bishop <[email-masked]>
Date: Thu, 11 Oct 2018 12:11:28 -0700

While I'm coming to like this proposal, I will also point out that the unilateral use of frame types is a model carried forward from HTTP/2, and our charter says that we will "accommodate the extension mechanisms defined in the HTTP/2 specification."  Unilateral use of unidirectional stream types is in keeping with that.

The reason currently stated for having these two different mechanisms is that you can unilaterally use optional-to-understand extensions (ALTSVC, ORIGIN, etc.), but MUST negotiate breaking changes before using them.  (Also, note that the server might begin initiating unidirectional stream types before having received the client SETTINGS.)

However, a modified SETTINGS exchange that permits true offer-select negotiation would greatly simplify negotiating breaking changes; currently the spec just handwaves that a coordination mechanism for when the change comes into effect might be required.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1846#issuecomment-429082881

---

From: Lucas Pardue <[email-masked]>
Date: Thu, 11 Oct 2018 12:34:38 -0700

@kazuho the way I think about unidirectional streams is that they are the _only_ way to extend HTTP/QUIC. This isn't a hard rule and isn't really true (as Mike points out, there's hand waving coordination that would let anyone do anything). 

The unbound server push extension does actually require negotiation because it modifies the rule about allowing a PUSH_PROMISE frame on server-initiated streams (a behavioural change). Without a stream type to help identify what is an endpoint is explicitly trying to tell you it is doing, we end up with implicit rules like assuming there is a control stream or QPACK encoder streams using a particular. I actually think your proposal works well with the unidirectional stream types because it reduces implicit assumption. And if that comes without additional delays then it really encourages me for the flexibility of HTTP over QUIC. 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1846#issuecomment-429089754

---

From: Subodh Iyengar <[email-masked]>
Date: Tue, 16 Oct 2018 10:39:06 -0700

One alternative might be to use transport settings to negotiate these. Transport settings have the nice property of being used for both 0-rtt and as well as 1-rtt data. It is also an explicit synchronization point for this kind of stuff vs having to deal with cases of what would happen if the SETTINGS frame was lost.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1846#issuecomment-430329202

---

From: Kazuho Oku <[email-masked]>
Date: Tue, 16 Oct 2018 19:03:42 -0700

@siyengar FWIW, we had some discussion about that in #904 starting from 
https://github.com/quicwg/base-drafts/issues/904#issuecomment-384449861.

And in addition to the risk of transmitting the setting in clear, I think @MikeBishop made a good point stating that having HQ-level settings negotiation _after_ finishing ALPN negotiation is a cleaner design than doing both in parallel (see 
https://github.com/quicwg/base-drafts/issues/904#issuecomment-384789205).

Considering that we can use SETTINGS to negotiate HQ-level configurations without any time penalty, I think there is no reason to switch to using transport parameters.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1846#issuecomment-430463049