Subject: [quicwg/base-drafts] Allow extra data after self-terminating h3 frames (#2291)

From: David Schinazi <[email-masked]>
Date: Thu, 03 Jan 2019 17:29:21 -0800

The current spec is overly restrictive when it comes to self-terminating h3 frames. If implementations ignore extra data after the expected bytes, this will allow us to add fields down the road without the need to specify new frame types.

Created this issue to host discussion regarding PR #2289.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2291

---

From: Lucas Pardue <[email-masked]>
Date: Thu, 03 Jan 2019 17:48:43 -0800

Thanks!

Some further observations:

0) It reminds me a bit of ARINC 661 wire format :)
1) this seems like it could work with most frames that carry a fixed-size fields and/Or varint payload because the delta between H3 frame length and payload field sizes is the size of postfix fields. However, how is it expected to work with DATA frames? Does HEADERS have a similar problem?

2) why is experimentation with frame types difficult for this use case? 

3) I think I might have hit a similar problem when trying to design the HELIUM Http/2 frames: https://tools.ietf.org/html/draft-pardue-httpbis-http-network-tunnelling-01#section-6.3

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2291#issuecomment-451333551

---

From: David Schinazi <[email-masked]>
Date: Thu, 03 Jan 2019 18:09:54 -0800

This only applies to self-terminating frames (ones that have a known length regardless of the h3 frame LTV length) so for example it applies to PRIORITY but not to DATA or HEADERS. The idea is to apply the Postel principle when inside the encryption boundary - we may want to add fields later so why prevent ourselves from doing it?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2291#issuecomment-451336194

---

From: Lucas Pardue <[email-masked]>
Date: Thu, 03 Jan 2019 18:27:00 -0800

Ah ok, understood. I guess the reaction  is that there are so few applicable core H3 frames that are likely to be sent through the course of a typical connection* that the merits of this flexibility might be hard to realise. 

Also, by invoking Postel, I fear the wrath of https://tools.ietf.org/html/draft-iab-protocol-maintenance-01 :)

* of the 7 frames, 4 of them relate to Server push. SETTINGS can't be sent multiple times. PRIORITY, the H3 style might change this up compared to H3, it's also one-way. GOAWAY is one-way too.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2291#issuecomment-451338208

---

From: MikkelFJ <[email-masked]>
Date: Fri, 04 Jan 2019 01:38:57 -0800

Are self-terminating frames not limited to DATA frames to avoid the general complexity and because DATA is the only frame needing this?

If more complex engineering is needed, I think designing separate streams for DATA is better, and probably a v2 feature.

A TLV format can have frames with implicit lengths, iif all types with implicit lengths are known.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2291#issuecomment-451396743

---

From: Kazuho Oku <[email-masked]>
Date: Fri, 04 Jan 2019 03:37:13 -0800

Generally speaking, I think having the ability to express self-terminating frames is a good thing.

However, our charter requires us to create a mapping that is "a description of HTTP/2 semantics using QUIC," and my preference goes to adhering the charter rather than chasing the possible optimizations. I am not against changing things in H3 if it clearly helps us, but I am not sure if there is a provable benefit here.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2291#issuecomment-451421429

---

From: ianswett <[email-masked]>
Date: Mon, 07 Jan 2019 08:02:04 -0800

After thinking about this more, I'd prefer to change to TV for cases when forcing implementations to supply a length is adding complexity.  That fixes the same core issue: "What to do if the specified length doesn't match the actual length" in a different way.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2291#issuecomment-451983730

---

From: Lucas Pardue <[email-masked]>
Date: Mon, 07 Jan 2019 08:13:08 -0800

I don't understand how that works though, how does the receiver parse the frame if the length is unknown? 

For the DATA frame, in #2098 the exception was made for len=0 because we could use the EOS signal. 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2291#issuecomment-451987473

---

From: ianswett <[email-masked]>
Date: Mon, 07 Jan 2019 12:01:16 -0800

Some frames need a length(ie: DATA), but some, such as GOAWAY, clearly do not.  In the case of GOAWAY, computing the length is trivial, so length is just a wasted byte and it doesn't really matter for an infrequently sent frame.  In other cases, it's more of a challenge to add a length after the fact.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2291#issuecomment-452062793

---

From: Lucas Pardue <[email-masked]>
Date: Mon, 07 Jan 2019 20:16:09 +0000 (UTC)

Ok but that seems tangential to this ticket's request AFAIK (which may be wrong). I thought the ask was to enable arbitrary fields after the core H3 frame fields. Done by using the delta between how large the frame says it is and the sum of all frame fields.

In the case of GOAWAY it's simple. But imagine I start sending extension frames on the control stream. If you know the type is not supported in your receiver, you can just ignore length bytes and carry on. 

I appreciate the want to reduce some complexity but it seems like we'll end up with an inconsistent set of frame types (TLV vs TV) without much spec text /guidance to support it. 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2291#issuecomment-452067141

---

From: David Schinazi <[email-masked]>
Date: Mon, 07 Jan 2019 20:34:52 +0000 (UTC)

My thought when I opened this issue was that if we have self-terminating frames that include a redundant length, then we should use that length for extensibility. I'm also fine with @ianswett's proposal of removing the length from self-terminating frames, [even if some consider it over-engineering](https://github.com/quicwg/base-drafts/pull/2235#issuecomment-451296608).

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2291#issuecomment-452072557

---

From: MikkelFJ <[email-masked]>
Date: Mon, 07 Jan 2019 12:43:03 -0800

I'm not against TLV vs TV when omitting the length is given by a known type (I have argued that before). But reserving half the numeric space for TV values when you need to know the type anyway to decode the TV field, that is what I consider overengineering.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2291#issuecomment-452074997

---

From: Mike Bishop <[email-masked]>
Date: Tue, 08 Jan 2019 11:23:50 -0800

(Editor hat off.)

If you know that the peer understands the extended fields, then you've created an extension that redefines a frame type by mutual agreement, which is already legal.  Just redefine the frame in your extension, possibly with optional fields.

If you don't negotiate, the risk is that there are conflicting interpretations of this after-the-fields data.  In order to avoid that, you should move your particular interpretation / the extra fields to a new frame type.

I don't see a path in which stuffing excess data into a frame without negotiation is a wise choice.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2291#issuecomment-452420720

---

From: Mike Bishop <[email-masked]>
Date: Wed, 30 Jan 2019 21:21:38 -0800

Discussed in Tokyo; no.  Security concerns, and better ways to do extensions.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2291#issuecomment-459219621

---

From: Mike Bishop <[email-masked]>
Date: Wed, 30 Jan 2019 21:21:40 -0800

Closed #2291.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2291#event-2108459172