Subject: [quicwg/base-drafts] VarInt all the HTTP things (#2437)

From: Mike Bishop <[email-masked]>
Date: Thu, 07 Feb 2019 23:57:06 +0000 (UTC)

As discussed in Tokyo, varints all the things.  This led to a few ancillary changes along the way:
- I stole transport's frame type IANA rules and used it for our three new varint registries.  (This was alluded to in discussion in Tokyo.)
- Unidirectional stream types are now a fixed sequence starting from 0x00, rather than scattered code points that "just happen" to have ASCII values.  (This was explicitly requested in discussion in Tokyo.)
- It's impractical to enumerate and have IANA reserve the grease values for a much larger space; therefore, I shifted the requirement to saying that IANA MUST NOT allocate those code points to anyone.  (This also avoids IANA having a publicly-visible list of the grease codepoints, which is a nice perk.)
- The `0x?a?a?a?a` is less effective greasing, as it falls entirely in the private use range.  I changed everything to use the `0x1F * N` formulation, which has the side benefit of being easy to produce and less obvious to identify on first glance.  (However, only two of these values fall in the standards-assigned region.)

If any of these ancillary changes are controversial, I'm happy to split them out into separate issues for discussion, but they make these changes fit more naturally.

Fixes #2233.
Fixes #2253.
Fixes #2275.
You can view, comment on, or merge this pull request online at:

  https://github.com/quicwg/base-drafts/pull/2437

-- Commit Summary --

  * Varint the frame types
  * Concomitant change to HTTP_MALFORMED_FRAME_TYPE
  * Varint the stream types
  * Format values
  * Varint the Settings Identifiers
  * Mention the difference in HTTP/2 considerations
  * Make frame type greasing consistent

-- File Changes --

    M draft-ietf-quic-http.md (161)
    M draft-ietf-quic-qpack.md (14)

-- Patch Links --

https://github.com/quicwg/base-drafts/pull/2437.patch
https://github.com/quicwg/base-drafts/pull/2437.diff

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2437

---

From: Mike Bishop <[email-masked]>
Date: Thu, 07 Feb 2019 15:59:47 -0800

@MikeBishop pushed 1 commit.

6e5f470ee4e1b6f8bff1f3a24b20b29bcfed38f7  Explicit value in fig-push-stream-header

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/2437/files/33cbc34885bb59c464218d48ed204bce3a0f585b..6e5f470ee4e1b6f8bff1f3a24b20b29bcfed38f7

---

From: Mike Bishop <[email-masked]>
Date: Thu, 07 Feb 2019 16:03:34 -0800

f06211da71550e9a7ed03cc42d96ec5dc84dd3f0  Non-zero values of N only

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/2437/files/6e5f470ee4e1b6f8bff1f3a24b20b29bcfed38f7..f06211da71550e9a7ed03cc42d96ec5dc84dd3f0

---

From: Christian Huitema <[email-masked]>
Date: Thu, 07 Feb 2019 16:56:28 -0800

huitema commented on this pull request.

The use of reserved values is a bit loose; what about `0x1f * N` when `N=0`? Apart from that, looks fine.

> @@ -881,12 +885,12 @@ the DUPLICATE_PUSH.
 
 ### Reserved Frame Types {#frame-grease}
 
-Frame types of the format `0xb + (0x1f * N)` are reserved to exercise the
-requirement that unknown types be ignored ({{extensions}}). These frames have no
-semantic value, and can be sent when application-layer padding is desired. They
-MAY also be sent on connections where no request data is currently being
-transferred. Endpoints MUST NOT consider these frames to have any meaning upon
-receipt.
+Frame types of the format `0x1f * N` for non-zero values of N are reserved to
+exercise the requirement that unknown types be ignored ({{extensions}}). These
+frames have no semantic value, and can be sent when application-layer padding is
+desired. They MAY also be sent on connections where no request data is currently
+being transferred. Endpoints MUST NOT consider these frames to have any meaning
+upon receipt.

When you write  `0x1f * N` , is 0 a valid value of N?

> @@ -384,11 +385,12 @@ client MUST treat this as a connection error of type HTTP_DUPLICATE_PUSH.
 
 ### Reserved Stream Types {#stream-grease}
 
-Stream types of the format `0x1f * N` are reserved to exercise the requirement
-that unknown types be ignored. These streams have no semantic meaning, and can
-be sent when application-layer padding is desired.  They MAY also be sent on
-connections where no request data is currently being transferred. Endpoints MUST
-NOT consider these streams to have any meaning upon receipt.
+Stream types of the format `0x1f * N` for non-zero values of N are reserved to
+exercise the requirement that unknown types be ignored. These streams have no
+semantic meaning, and can be sent when application-layer padding is desired.
+They MAY also be sent on connections where no request data is currently being
+transferred. Endpoints MUST NOT consider these streams to have any meaning upon
+receipt.

What if `N=0` in `0x1f * N`

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2437#pullrequestreview-201393225

---

From: Christian Huitema <[email-masked]>
Date: Thu, 07 Feb 2019 16:59:31 -0800

My previous comments are addressed by the  "non zero values of N" fix.

 

Oh, this is addressed by the "non zero values of N" fix.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2437#pullrequestreview-201395017

---

From: Dmitri Tikhonov <[email-masked]>
Date: Thu, 07 Feb 2019 20:16:19 -0800

dtikhonov requested changes on this pull request.

> @@ -373,7 +374,7 @@ MUST be treated as a stream error of type HTTP_WRONG_STREAM_DIRECTION.
  0                   1                   2                   3
  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
-|Stream Type (8)|                  Push ID (i)                ...
+|    0x01 (8)   |                  Push ID (i)                ...

The `(8)` suggests that this is an octet, whereas the stream type is a varint.  I'd make this figure use the same format as _fig-stream-header_.

 

"Semantic meaning" sounds like a tautology.

(This formulation is not introduced in this change, but I figured I'd highlight it while I am at it.)

> @@ -671,14 +673,16 @@ treat the presence of the same parameter more than once as a connection error of
 type HTTP_MALFORMED_FRAME.
 
 The payload of a SETTINGS frame consists of zero or more parameters, each
-consisting of an unsigned 16-bit setting identifier and a value which uses the
+consisting of a setting identifier and a value, each of which uses the

I'd put it as "...identifier and a value, each encoded as a QUIC variable-length integer," simply because I don't see how the identifier and the value _use_ the encoding.

(Again, this is not a new thing in this change, but simply something I've noticed.)

> @@ -1824,7 +1815,9 @@ CONTINUATION (0x9):
 
 Frame types defined by extensions to HTTP/2 need to be separately registered for
 HTTP/3 if still applicable.  The IDs of frames defined in {{!RFC7540}} have been
-reserved for simplicity.  See {{iana-frames}}.
+reserved for simplicity.  Note that the frame type space in HTTP/3 is
+substantially larger (62 bits versus 8 bits), so many HTTP/3 frame types have no
+equivalent HTTP/2 code point.   See {{iana-frames}}.

Number agreement: `s/point/points/`

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2437#pullrequestreview-201425291

---

From: Lucas Pardue <[email-masked]>
Date: Fri, 08 Feb 2019 03:37:38 -0800

LPardue requested changes on this pull request.

I requested some changes, but if they are controversial we can move them to separate issues for further discussion.

> @@ -303,15 +303,16 @@ specify a value of zero for the QUIC transport parameter
 ## Unidirectional Streams
 
 Unidirectional streams, in either direction, are used for a range of purposes.
-The purpose is indicated by a stream type, which is sent as a single byte header
-at the start of the stream. The format and structure of data that follows this
-header is determined by the stream type.
+The purpose is indicated by a stream type, which is sent as a variable-length
+integer at the start of the stream. The format and structure of data that

you've replaced 'byte header' with 'variable-length integer', but the next sentence refers to 'this header'. This might be confusing to the reader? 

  0                   1                   2                   3

The alternative option is to remove it from the figure altogether. It was not great before the change - see #2338 

> @@ -433,12 +437,10 @@ All frames have the following format:
 A frame includes the following fields:
 
   Type:
-  : An 8-bit type for the frame.
-
+  : A variable-length type for the frame.

nit: make this consistent with the frame payload description. e.g.

A variable-length integer that identifies the frame type.

"both encoded as" would work too

 

Not a new problem but I wonder if request data can be simplified to data, which covers all cases such as QPACK instructions, response data, control data etc.

> @@ -1589,15 +1593,9 @@ The entries in the following table are registered by this document.
 | NUM_PLACEHOLDERS             |  0x8   | {{settings-parameters}}   |
 | ---------------------------- | ------ | ------------------------- |

Not a new issue but: RFC 7540 has a column for `Initial Value`. I think it makes sense that HTTP/3 does similar.

> @@ -1868,7 +1861,9 @@ use the full 32-bit space.  Settings ported from HTTP/2 might choose to redefine
 the format of their settings to avoid using the 62-bit encoding.
 
 Settings need to be defined separately for HTTP/2 and HTTP/3. The IDs of
-settings defined in {{!RFC7540}} have been reserved for simplicity. See
+settings defined in {{!RFC7540}} have been reserved for simplicity.  Note that
+the settings identifier space in HTTP/3 is substantially larger (62 bits versus
+16 bits), so many HTTP/3 settings have no equivalent HTTP/2 code point. See

Unfortunately, `0x1` is reserved by QPACK so it makes a little more difficult to see that this is true. And a quick check of the [HTTP/2 IANA registry](https://www.iana.org/assignments/http2-parameters/http2-parameters.xhtml#settings) shows that 0x0 is also reserved, but I can't seem to find where that is defined in RFC 7540.

Checking that, it seems we have a clash on 0x8 due to RFC 8441 - WebSocket over HTTP/2. I think its reasonable that someone might write a WebSocket over HTTP/3 extension and reusing the code point would be nice. Can we move NUM_PLACEHOLDERS to 0x9 and leave 0x8 open?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2437#pullrequestreview-201516984

---

From: ianswett <[email-masked]>
Date: Fri, 08 Feb 2019 08:39:21 -0800

ianswett commented on this pull request.

+follows this header is determined by the stream type.

```suggestion
follows this integer is determined by the stream type.
```

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2437#pullrequestreview-201673543

---

From: ianswett <[email-masked]>
Date: Fri, 08 Feb 2019 08:41:53 -0800

Type:

+1

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2437#discussion_r255148468

---

From: Mike Bishop <[email-masked]>
Date: Fri, 08 Feb 2019 13:18:59 -0800

MikeBishop commented on this pull request.

  0                   1                   2                   3

A varint whose value is specified, and therefore fits in one octet.  But point taken -- you're not required to use the minimal encoding, so this is overly restrictive.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2437#discussion_r255238599

---

From: Mike Bishop <[email-masked]>
Date: Fri, 08 Feb 2019 13:22:43 -0800

Opened #2442.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2437#discussion_r255239595

---

From: Mike Bishop <[email-masked]>
Date: Fri, 08 Feb 2019 21:25:22 +0000 (UTC)

8d9bf8577027bc2639f3cc9d20b276de2863272d  Update draft-ietf-quic-http.md

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/2437/files/f06211da71550e9a7ed03cc42d96ec5dc84dd3f0..8d9bf8577027bc2639f3cc9d20b276de2863272d

---

From: Mike Bishop <[email-masked]>
Date: Fri, 08 Feb 2019 14:41:59 -0800

@MikeBishop pushed 3 commits.

5379931cb0dc3931c0dde09839420adb4f68dfb9  Revert "Explicit value in fig-push-stream-header"
3e2dbea894c2e3fe4fe4d63e9592ddf4c1e0f1bc  Varint explicit value
0c33013556840e9f16f3e701cbb372f68b0ab124  Review feedback

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/2437/files/8d9bf8577027bc2639f3cc9d20b276de2863272d..0c33013556840e9f16f3e701cbb372f68b0ab124

---

From: Mike Bishop <[email-masked]>
Date: Fri, 08 Feb 2019 14:44:11 -0800

Updated from feedback.

Where do you see "this header"?

 

Changed to "semantics" both places this occurs.

  0                   1                   2                   3

I reverted this particular commit and re-did it to use the explicit value but still show it as a varint, which seems the most accurate.  (I know @LPardue would prefer I remove it entirely.)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2437#pullrequestreview-201814181

---

From: Mike Bishop <[email-masked]>
Date: Fri, 08 Feb 2019 22:46:04 +0000 (UTC)

05b784f6985de0597a30faa611b6317d2bd56051  Plurals

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/2437/files/0c33013556840e9f16f3e701cbb372f68b0ab124..05b784f6985de0597a30faa611b6317d2bd56051

---

From: Mike Bishop <[email-masked]>
Date: Fri, 08 Feb 2019 15:15:16 -0800

I'm inclined to think that either is correct, since there can be many types (each of which has no equivalent code point) or a range of types (which has no corresponding range of code points).  However, I don't care that much.  :-)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2437#discussion_r255264589

---

From: Mike Bishop <[email-masked]>
Date: Fri, 08 Feb 2019 15:15:20 -0800

c30019f681d092f3a04dee62bfed3bf868f610d2  More plurals

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/2437/files/05b784f6985de0597a30faa611b6317d2bd56051..c30019f681d092f3a04dee62bfed3bf868f610d2

---

From: Martin Thomson <[email-masked]>
Date: Sun, 10 Feb 2019 16:07:30 -0800

martinthomson approved this pull request.

The word "header" appears in the figure title.

> @@ -384,11 +387,11 @@ client MUST treat this as a connection error of type HTTP_DUPLICATE_PUSH.

Rather than grapple with the 0 value, why not 0x1f*N+0x21 or something like that?

#2443 (that feature that creates an issue is nice, but it should leave a tombstone so that people know not to reply).

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2437#pullrequestreview-201930606

---

From: Lucas Pardue <[email-masked]>
Date: Sun, 10 Feb 2019 16:12:51 -0800

LPardue commented on this pull request.

"this header" is now "this integer" on line 308

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2437#discussion_r255364310

---

From: ianswett <[email-masked]>
Date: Mon, 11 Feb 2019 17:16:30 +0000 (UTC)

ianswett approved this pull request.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2437#pullrequestreview-202230809

---

From: Mike Bishop <[email-masked]>
Date: Mon, 11 Feb 2019 16:42:48 -0800

Hm, that might be easier.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2437#discussion_r255757309

---

From: Mike Bishop <[email-masked]>
Date: Mon, 11 Feb 2019 16:47:59 -0800

630bc71c3e54f9149ae472e00618e67da3ead32f  Martin's suggested alternative to non-zero

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/2437/files/c30019f681d092f3a04dee62bfed3bf868f610d2..630bc71c3e54f9149ae472e00618e67da3ead32f

---

From: Mike Bishop <[email-masked]>
Date: Mon, 11 Feb 2019 16:50:06 -0800

Seconded.  GitHub feedback submitted.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2437#discussion_r255758734

---

From: Mike Bishop <[email-masked]>
Date: Mon, 11 Feb 2019 16:52:03 -0800

Merged #2437 into master.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2437#event-2132254926