Subject: [quicwg/base-drafts] Make HTTP frames a TLV format. This has several advantages: (#2235)

From: ekr <[email-masked]>
Date: Thu, 20 Dec 2018 15:21:21 -0800

I know this is a late change, but I think it's an improvement....

- You can switch on the frame type before you even read a
  variable-length value
- You can have special-case (negotiated) frame types with no lengths

If this has already been discussed and rejected somewhere, maybe
I missed it.
You can view, comment on, or merge this pull request online at:

  https://github.com/quicwg/base-drafts/pull/2235

-- Commit Summary --

  * Make HTTP frames a TLV format. This has several advantages:

-- File Changes --

    M draft-ietf-quic-http.md (11)

-- Patch Links --

https://github.com/quicwg/base-drafts/pull/2235.patch
https://github.com/quicwg/base-drafts/pull/2235.diff

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2235

---

From: ekr <[email-masked]>
Date: Thu, 20 Dec 2018 15:40:13 -0800

@DavidSchinazi 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2235#issuecomment-449177170

---

From: MikkelFJ <[email-masked]>
Date: Thu, 20 Dec 2018 15:46:31 -0800

Me thinks it makes sense. As to 0-length special DATA frames (until end), I also think it could be a length-less separate type instead of 0-length. The downside is that you can't iterate frames without understanding all types, but that is no different from transport.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2235#issuecomment-449179082

---

From: ekr <[email-masked]>
Date: Thu, 20 Dec 2018 15:47:28 -0800

I think the way you would make iterating all frames possible is by requiring explicit opt-in (or standards or something) for any types that didn't have the TLV format.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2235#issuecomment-449179361

---

From: David Schinazi <[email-masked]>
Date: Thu, 20 Dec 2018 15:57:28 -0800

I like this. You could imagine reserving a part of the type space (e.g. 0-32) for non-TLV frames, and if you ever receive something in that range that you don't understand you fail the connection. But if you receive a TLV type you don't understand, you can safely ignore it since you know how long it is.

Minor: I'd make the type a variable-length integer

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2235#issuecomment-449181514

---

From: ekr <[email-masked]>
Date: Thu, 20 Dec 2018 15:58:44 -0800

I would be fine with type being a VLI.

On Thu, Dec 20, 2018 at 3:57 PM David Schinazi <[email-masked]>
wrote:

> I like this. You could imagine reserving a part of the type space (e.g.
> 0-32) for non-TLV frames, and if you ever receive something in that range
> that you don't understand you fail the connection. But if you receive a TLV
> type you don't understand, you can safely ignore it since you know how long
> it is.
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/quicwg/base-drafts/pull/2235#issuecomment-449181514>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ABD1oVtlQi7p7siJy9Y9WKwIWjwPPuL6ks5u7CPmgaJpZM4ZdEz2>
> .
>

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2235#issuecomment-449181703

---

From: Dmitri Tikhonov <[email-masked]>
Date: Thu, 20 Dec 2018 16:00:16 -0800

We've had this discussion haven't we.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2235#issuecomment-449181973

---

From: MikkelFJ <[email-masked]>
Date: Thu, 20 Dec 2018 16:28:19 -0800

Is it even possible to have a QUIC discussion that hasn't been discussed at lease once before?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2235#issuecomment-449186805

---

From: janaiyengar <[email-masked]>
Date: Thu, 20 Dec 2018 16:28:31 -0800

@MikeBishop would remember -- I recall a discussion about this, perhaps about L = 0 being allowable or not. 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2235#issuecomment-449186834

---

From: MikkelFJ <[email-masked]>
Date: Thu, 20 Dec 2018 16:34:58 -0800

L=0 definitely was discussed in the context of whether only DATA frames or all frames could be 0 length. But that is a different issue.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2235#issuecomment-449187885

---

From: Martin Thomson <[email-masked]>
Date: Thu, 20 Dec 2018 20:41:32 -0800

This has been discussed a few times.  We discussed this in h2 and explicitly made the opposite choice.  Though that was a while back, and I don't know if it was well documented.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2235#issuecomment-449246669

---

From: ianswett <[email-masked]>
Date: Sat, 22 Dec 2018 18:32:53 -0800

This seems like a good change to me, I was rather confused when I realized the length came before the type.  If we're really going to drink the varint kool-aid, maybe the type should be a varint too?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2235#issuecomment-449610128

---

From: Kazuho Oku <[email-masked]>
Date: Sat, 22 Dec 2018 18:44:21 -0800

I'm fine with moving frame type in front of the length.

OTOH, I am not sure if I prefer using varints for frame types.

IIRC, using varints for frame types of the transport, with the requirement that the minimal-sized encoding MUST be used, was based on the fact that it reserves more code points to future extensions, at the same time not making existing implementations complex. The reason the change did not make existing implementations complex was because new frame types cannot be used without negotiation.

The situation is different in H3.

H3 allows frames to be used without negotiation. In fact, we allow endpoints to grease the codepoints.

That means that every implementation would be required to support frame types using varints, even though in practice only the single-octet versions would be used.

To me that sounds like an unnecessary complexity.

I am also skeptical if we would ever run out of H3 frame types even if they are kept 8 bits, considering how few we have used in H2.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2235#issuecomment-449610469

---

From: Dmitri Tikhonov <[email-masked]>
Date: Sat, 22 Dec 2018 19:31:12 -0800

I don't see an advantage to having type come before length.  I would be against special-casing some frames to have no length, which is the only scenario where this change would be necessary for.

HTTP/2 frames have length before type; why change?  Seems gratuitous.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2235#issuecomment-449611960

---

From: MikkelFJ <[email-masked]>
Date: Sun, 23 Dec 2018 02:57:33 -0800

Nearly all binary self-describing formats introduced these days use some form of TLV because it offers the a good size/flexibility tradeoff, because the L and/or V part can omitted or specialised. For example there can be compact integers up to 4 bits in length using only a type field and a separate var length type. That is not in itself an argument for using it in HTTP/3.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2235#issuecomment-449628660

---

From: Lucas Pardue <[email-masked]>
Date: Sun, 23 Dec 2018 04:04:30 -0800

I'm not sure about omitting length. I can see there could be benefit in stating that some frame types, that are part of H3 core, which contain only varints could be optimised by not including the length. 

However, from an implementation perspective it seems simpler to have code that can read frames out of the stream using the frame length. 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2235#issuecomment-449632012

---

From: Dmitri Tikhonov <[email-masked]>
Date: Mon, 24 Dec 2018 07:07:30 -0800

The implementation is easy either way, which is why I ask: "why change?" The current frame format derives from HTTP/2, which makes me inclined to leave it as-is.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2235#issuecomment-449744350

---

From: ekr <[email-masked]>
Date: Mon, 24 Dec 2018 07:59:18 -0800

On Mon, Dec 24, 2018 at 7:07 AM Dmitri Tikhonov <[email-masked]>
wrote:

> The implementation is easy either way, which is why I ask: "why change?"
>
I listed the arguments for this in the original comments on the issue.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2235#issuecomment-449749494

---

From: Dmitri Tikhonov <[email-masked]>
Date: Mon, 24 Dec 2018 08:03:39 -0800

@ekr, I addressed the original comments above. The proposed change is only advantageous if we want to be able to special-case some frame types to exclude length. We should get a consensus on that first.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2235#issuecomment-449749970

---

From: ekr <[email-masked]>
Date: Mon, 24 Dec 2018 08:14:49 -0800

No, I don't think we need consensus on that first, because (for instance) someone might want a private extension that permitted excluding the length, which the current design makes very difficult. TLV is just inherently more flexible. And so far the only argument I've heard against it is "why change".

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2235#issuecomment-449751081

---

From: Dmitri Tikhonov <[email-masked]>
Date: Mon, 24 Dec 2018 08:36:30 -0800

Excluding the length has diminished benefit if it takes 8 bytes to encode the frame type (I am referring to #2253, with which this proposal is linked).

I am not against future-proofing the format, but both this and the varint type proposal seem very theoretical: _there might be someone who may want..._ etc.  For example, there have only been [two extensions](https://www.iana.org/assignments/http2-parameters/http2-parameters.xhtml#frame-type) to the HTTP/2 frames.  Has extending or experimenting with HTTP/2 frames ever been a real problem?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2235#issuecomment-449753035

---

From: ianswett <[email-masked]>
Date: Mon, 24 Dec 2018 10:42:23 -0800

After thinking about this more, I think this is a very important change and making it will cause HTTP/3 align better with the QUIC transport.

In retrospect, PR#2098(Allow DATA to extend to the end of the stream) violated what appears to be a core design choice of HTTP/2, that all frames explicitly specify a length, so they're easily skippable.  Admittedly, skipping DATA doesn't make much sense, but it is still inconsistent.

To me, that's one of many signals that TLV is a better approach.  I was intending to create an extension that was DATA without length, but I realized that this new frame type would have a length byte, but it would be ignored or specified as a particular incorrect value(ie: 0), neither of which are ideal.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2235#issuecomment-449762657

---

From: Dmitri Tikhonov <[email-masked]>
Date: Mon, 24 Dec 2018 10:49:26 -0800

>  this is a very important change and making it will cause HTTP/3 align better with the QUIC transport.

1. What is meant by "better alignment?"
1. Why is it important?  QUIC and HTTP/3 are different protocols, we touched on this a bit in #2232.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2235#issuecomment-449763101

---

From: ianswett <[email-masked]>
Date: Mon, 24 Dec 2018 11:08:24 -0800

@dtikhonov By better alignment, I meant more consistent design choices between QUIC transport and HTTP/3.

Previously HTTP over QUIC adopted varints because it was straightforward and QUIC transport had recently adopted them.  Moving HTTP/3 to TLV seems like a similar change.

More importantly, I'd like to argue that TLV is fundamentally better than LTV.  If someone can provide an example in HTTP/2 where LTV has shown clear value over TLV, I'd be interested in hearing about it.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2235#issuecomment-449764387

---

From: Dmitri Tikhonov <[email-masked]>
Date: Mon, 24 Dec 2018 11:11:47 -0800

> Previously HTTP over QUIC adopted varints because it was straightforward and QUIC transport had recently adopted them. Moving HTTP/3 to TLV seems like a similar change.

One does not follow from the other; TLV vs LTV and fixed-size vs varint are two different, orthogonal questions.

> If someone can provide an example in HTTP/2 where LTV has shown clear value over TLV, I'd be interested in hearing about it.

I'd be curious to know why HTTP/2 chose one over the other in the first place.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2235#issuecomment-449764592

---

From: ekr <[email-masked]>
Date: Mon, 24 Dec 2018 11:16:18 -0800

On Mon, Dec 24, 2018 at 11:11 AM Dmitri Tikhonov <[email-masked]>
wrote:

> Previously HTTP over QUIC adopted varints because it was straightforward
> and QUIC transport had recently adopted them. Moving HTTP/3 to TLV seems
> like a similar change.
>

This is true at some level, but once we've changed anything, we might as
well ask what the best design is.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2235#issuecomment-449764905

---

From: ianswett <[email-masked]>
Date: Mon, 24 Dec 2018 11:19:44 -0800

The varint question is completely orthogonal, sorry to bring it up here. @martinthomson filed a separate issue for that: #2253 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2235#issuecomment-449765107

---

From: Kazuho Oku <[email-masked]>
Date: Mon, 24 Dec 2018 14:29:28 -0800

While I think switching from LTV to TLV is fine (it does not change the semantics), I think it's worth noting that QUIC transport is not even a TLV; it's TV. Therefore, I do not think that switching to TLV aligns H3 better to the QUIC transport.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/2235#issuecomment-449775101