Subject: [quicwg/base-drafts] Consider making h3 frame types varint (#2253)

From: Martin Thomson <[email-masked]>
Date: Sun, 23 Dec 2018 22:12:58 -0800

Reusing the language from the transport frame might make this easier, though we might decide not to worry about enforcing the minimum-sized encoding for HTTP.

I'm not advocating for this in particular, but we seem to have a bunch of issues on our plate around the basic frame formats.  See also #2233, which proposes this for settings types.  And #2235, which proposes reordering of the frame.  Considering these issues together might be a good idea.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2253

---

From: Kazuho Oku <[email-masked]>
Date: Mon, 24 Dec 2018 00:18:40 -0800

As stated in 
https://github.com/quicwg/base-drafts/pull/2235#issuecomment-449610469, I do not think that we should make this change.

Our experience from HTTP/2 tells us that it's unlikely that we would run out of the 8-bit number space, and therefore requiring every stack to decode type using varint seems like an unnecessary complexity.

Note that the endpoints would be required to decode the varint even if we decide to enforce minimum-sized encoding. This is because H3 allows use of frame IDs without negotiation.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2253#issuecomment-449701397

---

From: David Schinazi <[email-masked]>
Date: Mon, 24 Dec 2018 04:03:03 -0800

I'm in favor of varint with the min-size requirement. The added ability to perform experiments without coordination is worth the addition of a 5-line inline function.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2253#issuecomment-449725780

---

From: Dmitri Tikhonov <[email-masked]>
Date: Mon, 24 Dec 2018 06:22:11 -0800

@DavidSchinazi, what do you mean by _without coordination_?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2253#issuecomment-449739793

---

From: ekr <[email-masked]>
Date: Mon, 24 Dec 2018 08:01:13 -0800

> @DavidSchinazi, what do you mean by _without coordination_?

The idea is that there is a lot of space in the 8-byte version, and so if you just pick a random code point in that space, the chance of collision is vanishingly small. This means people can do experiments without having some registry or other mechanism to prevent people choosing the same code point.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2253#issuecomment-449749716

---

From: Dmitri Tikhonov <[email-masked]>
Date: Mon, 24 Dec 2018 08:06:07 -0800

> if you just pick a random code point in that space, the chance of collision is vanishingly small.

That's what I figured; but it's not zero. (Note that you only get 62, not 64, bits.)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2253#issuecomment-449750218

---

From: ekr <[email-masked]>
Date: Mon, 24 Dec 2018 08:12:08 -0800

No random selection mechanism ever has a 0 chance of collision, but we routinely build systems that rely on not having collisions in these circumstances.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2253#issuecomment-449750816

---

From: Lucas Pardue <[email-masked]>
Date: Mon, 24 Dec 2018 08:39:04 -0800

How do we decide how the varint 62-bits are used? I recall this being a discussion topic that led to the invariants saying the space is used however the endpoints decide. Would H3 frames need to coordinate with SETTINGS or just assume varint encodes 1-byte updwards?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2253#issuecomment-449753311

---

From: Dmitri Tikhonov <[email-masked]>
Date: Mon, 24 Dec 2018 08:41:09 -0800

That's a good point. It would be bad to have to support small values encoded using the full eight bytes.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2253#issuecomment-449753499

---

From: ianswett <[email-masked]>
Date: Mon, 24 Dec 2018 09:28:28 -0800

I'm generally in favor of this, because it adds consistency and I believe it will make experimentation easier without explicit approval of the HTTP WG.

@martinthomson Thanks for filing a separate issue.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2253#issuecomment-449757446

---

From: Kazuho Oku <[email-masked]>
Date: Mon, 24 Dec 2018 14:20:49 -0800

@DavidSchinazi 
> The added ability to perform experiments without coordination is worth the addition of a 5-line inline function.

IMO, the endpoints willing to do experiments should spend their 5-line worth of coding effort to negotiate the experiment, rather than asking every implementation to accept the complexity.

As suggested in the last paragraph of [HTTP/3 section 7](
https://quicwg.org/base-drafts/draft-ietf-quic-http.html#rfc.section.7), endpoints that are willing to do an experiment can exchange their own SETTINGS value to confirm that the peer supports the experiment, then use any of the unused 8-bit frame types for the experiment.

The benefits of the approach are:
* does not introduce the need to deal with varint types
* less network overhead in experiments (due to being able to use 8-bit types)
* less chance of the experiment becoming a de-facto standard

The last argument is based on the fact that the experiments use negotiation. The fear of doing experiments is that sometimes the code point used for the experiment gets ossified; you cannot decide when to stop sending the extensions. Negotiating the use is one way to mitigate the problem, because with negotiation you can observe if the extension is actually used by the peer.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2253#issuecomment-449774738

---

From: Martin Thomson <[email-masked]>
Date: Sun, 30 Dec 2018 17:08:33 -0800

One thing that SETTINGS doesn't support is the use of new frame types prior to receiving SETTINGS.  For instance, if I wanted to push out an origin policy in 0.5RTT on a regular connection (using the context-free push that @ekr and @LPardue talked about), I could maybe do that with a new frame type.  Both h2 and h3 allow unannounced use of new frame types, so this would work fine.

If the space is 8-bit, then I can't experiment with that spontaneously and without coordination.

I'm leaning toward the varint on that basis.  The larger encodings will push things off the fast path, but I think that's OK.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2253#issuecomment-450598736

---

From: Kazuho Oku <[email-masked]>
Date: Sun, 30 Dec 2018 18:00:04 -0800

@martinthomson While I agree to the fact that it is impossible with the 8-bit design to send new frame types in 0-RTT, I am not sure if I am convinced with the argument that it represents the necessity to use varint frame types.

Consider the 0-RTT context-free push case. We not only need to have new frame types but change the semantics of push. The request headers of a context-push cannot be sent in a way that a client that do not understand the extension can ignore.

Therefore, to me it seems that anything to be sent from the server in 0-RTT needs to have a different unidirectional stream type. Therefore I think #2275 is the way to go, but that having varint frames types is not.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2253#issuecomment-450600968

---

From: Martin Thomson <[email-masked]>
Date: Sun, 30 Dec 2018 18:54:30 -0800

> The request headers of a context-push cannot be sent in a way that a client (that does not understand the extension) can ignore.

I don't think that is true.  If you assume that this is a QPACK-encoded header block, that would be true (because it would create an expectation of acknowledgment), but you don't need to use QPACK.

That example, is - as you say - better on a unidirectional stream.  And - with a small cost in head-of-line - can be done based on SETTINGS.  But I can't say confidently that there is no use of 0.5-RTT that would benefit from having relative ordering with other activity on the control stream.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2253#issuecomment-450603284

---

From: Kazuho Oku <[email-masked]>
Date: Sun, 30 Dec 2018 20:55:53 -0800

> I don't think that is true. If you assume that this is a QPACK-encoded header block, that would be true (because it would create an expectation of acknowledgment), but you don't need to use QPACK.

Maybe I should have pointed out that push streams start with a Push ID, however if a server has a usable Push ID is not determined until it receives a MAX_PUSH_ID frame from the client. Doesn't sending a push stream in 0.5 RTT violate that requirement?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2253#issuecomment-450608362

---

From: Martin Thomson <[email-masked]>
Date: Sun, 30 Dec 2018 21:21:50 -0800

That's not really relevant.  If the intent is to enable pushing of stuff in 0.5RTT, then the design wouldn't necessarily use push IDs.  It doesn't need to associate the push with a request, so it doesn't need them, it could just use a new stream type.  But I think we're well into the weeds here.  Point is that there *might* be valid uses for sending extension frames before receiving SETTINGS.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2253#issuecomment-450609463

---

From: Kazuho Oku <[email-masked]>
Date: Sun, 30 Dec 2018 21:33:15 -0800

> That's not really relevant. If the intent is to enable pushing of stuff in 0.5RTT, then the design wouldn't necessarily use push IDs. It doesn't need to associate the push with a request, so it doesn't need them, it could just use a new stream type.

The point is that how such a context-free push in 0.5 RTT would look like to a client that only implements H3. H3 requires every push stream to carry a Push ID (immediately following the stream type). Every non-negotiated extension including context-free push need to obey the requirements.

> Point is that there *might* be valid uses for sending extension frames before receiving SETTINGS.

Yeah. My point is that there would be not a single case for that, because my understanding is that a server cannot start sending frames on an unidirectional stream due to the restriction I point out.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2253#issuecomment-450609959

---

From: Martin Thomson <[email-masked]>
Date: Sun, 30 Dec 2018 22:11:28 -0800

Sorry, I'm being dense, obviously.  Which restriction?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2253#issuecomment-450611786

---

From: Kazuho Oku <[email-masked]>
Date: Mon, 31 Dec 2018 00:58:08 -0800

No, I was imprecise in my last sentence.

What I was trying to clarify is that it is a requirement of H3 that a server cannot start sending frames on a *push stream* before it receives a MAX_PUSH_ID frame from the client, which arrives after SETTINGS. Therefore, any use of an extended frame on the *push stream* can be negotiated.

*Unidirectional streams* other than the push stream are not restricted to the frame definition of H3, so we have the room to do whatever we want, regardless of if we have varint frame types in H3.

The only case a server might want to send extended frames before seeing SETTINGS is when the client's request arrives before the SETTINGS frame due to packet loss or out-of-order delivery (I haven't considered this case before; please forgive me if you assumed this case).

But if we do care about such case (i.e. using 0.5 RTT or SETTINGS being delivered later), I think we should revisit the discussion of if we should use Transport Parameters for sending H3 settings. We would have the following benefit by exchanging H3 settings using Transport Parameters, :
* QPACK dynamic table can always be used; there is no need to operate in restricted mode until receiving SETTINGS from the peer
* Use of 0.5-RTT push (or any other extension) can be negotiated. That avoids wasting the bandwidth by sending extensions that the client does not understand.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2253#issuecomment-450622014