Subject: [quicwg/base-drafts] Interleaved Stream IDs are awkward (#1430)

From: Mike Bishop <[email-masked]>
Date: Fri, 08 Jun 2018 04:25:27 -0700

In case we hadn't noticed before, at the Kista interim it became very clear how we all trip over the N/N+4 progression of stream IDs of the same type.  It's quite awkward.  As someone said, we'll all eventually get used to it, and so will the world... unless we don't have to.

It's possible to envision the lowest two bits of the Stream ID as flags which indicate the type, leaving the "real" ID as the upper bits.  However, by encoding them in the low bits of an integer, that becomes annoying to represent and work with easily.

If they were the high-order bits, that would be more convenient, but would of course blow out the varint encoding.  (Of course, two high-order bits of stream type and a 62-bit Stream ID fit nicely into a 64-bit variable assuming that implementations want to store them together once the varint is expanded.)  If varint could start mid-byte, that would also be an option, but that makes the varint parser more annoying.

The obvious solution, if we want to bite the bullet and go there, is to move the stream type being referenced into the frame type.  This was awkward when client-initiated bidirectional stream 0 was special, but now that crypto is moving to a special stream outside the Stream ID space, I think this transition makes more sense.  (Crypto is essentially a fifth type of stream with special properties: there can be only one, and most stream management frames don't apply to it.)

Besides making Stream IDs easier to keep track of, this resolves the inconsistency between `initial_max_(uni|bidi)_streams` and MAX_STREAM_ID -- the Stream ID and the count are the same thing, if the type is specified separately.  By reclaiming two bits per stream reference, this also saves a small number of bytes on the wire -- every STREAM, STOP_SENDING, RST_STREAM, etc. becomes a byte shorter for streams 16-63, 4096-16384, etc. of each stream type.  The same benefits will accrue to HTTP PRIORITY frames, where the stream type is already known from the fact that it's a stream carrying an HTTP request.

The biggest cost comes in combination with our decision to make frame types varint:  By spending two bits of frame type to indicate length and moving stream types into the type byte as well, there are only two frame types values left before we spill into a second byte, and we'll those for CRYPTO and CRYPTO_CLOSE.  (If we add something else and something has to spill over, I'd start with things that are sent only once per connection, like the *_CLOSE frames.)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1430

---

From: MikkelFJ <[email-masked]>
Date: Fri, 08 Jun 2018 04:43:32 -0700

Finally !!!

but 

> The biggest cost comes in combination with our decision to make frame types varint: By spending two bits of frame type to indicate length and moving stream types into the type byte as well, there are only two frame types values left before we spill into a second byte, 

why do you want to do this? If you want an extensible frame type, use 0xff for extensions. Waisting two bits just to use the same varint encoding as for 62 bit integers makes little sense.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1430#issuecomment-395736075

---

From: MikkelFJ <[email-masked]>
Date: Fri, 08 Jun 2018 04:51:16 -0700

On frame type extensions:
more specifically, if you later want to add partial reliable frames or single frame messages, these would benefit the most from short frame types, but they will be forced into a two byte frame type. This can be avoided by using 0xff (or similar to encode extensions). On second byte 0xff could again code for extensions, or you could use varint encoding in the second byte.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1430#issuecomment-395737681

---

From: Mike Bishop <[email-masked]>
Date: Tue, 12 Jun 2018 18:20:32 +0000 (UTC)

I'm wondering whether we could alternatively fix this in text only.  If we say that the stream type is indicated by the two low bits of the field, and the Stream ID within the type is indicated by the remaining upper bits of the number, then you get to the same place (indicating streams as 1us, 3bc, etc.) without modifying the bits on the wire.  That's basically just codifying a change in presentation / reference terminology.

It doesn't get the same small byte savings as #1431, because you're not effectively expanding the varint by two bits, but those savings are modest anyway.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1430#issuecomment-396686466

---

From: MikkelFJ <[email-masked]>
Date: Tue, 12 Jun 2018 18:30:22 +0000 (UTC)

It's beneficial to have the stream type detectable in the frame type because it is then possible to dispatch to separate handler logic before reading the frame and that logic then doesn't have to do special bit magic and branch on what it discovers. OTOH separate handlers can easily call shared support functions for streams.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1430#issuecomment-396689436

---

From: Kazuho Oku <[email-masked]>
Date: Tue, 12 Jun 2018 19:29:03 -0700

> I'm wondering whether we could alternatively fix this in text only. If we say that the stream type is indicated by the two low bits of the field, and the Stream ID within the type is indicated by t…

@MikeBishop I am not sure if I like the approach. Within the implementation, I would like to use an integral type to determine a stream. I'd assume that would be the same for others.

Using the low-bits for determining the type will mean that we will be calling, for example, a stream represented by a number of `4` as `1bc`. To me, it is more confusing than the current scheme.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1430#issuecomment-396793738