Subject: Re: [quicwg/base-drafts] Forwarding upstream errors, and the implications (#3300)

From: Kazuho Oku <[email-masked]>
Date: Wed, 01 Jan 2020 17:13:39 -0800

@MikeBishop I agree to your comments, with the only exception that I do not think we have "overreached."

In my view, HTTP/2 is underspecified in sense that it allows any stream error to be promoted to a connection error, even though as we agree, a stream error that might have originated from an origin cannot be promoted to a connection error.

I think it is an improvement in HTTP/3, to be more explicit about the distinction between the two types of errors. Therefore, even though I am not sure if I like the complexity of defining the type of error for every possible scenario, I'd prefer trying to do that until we fail.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3300#issuecomment-570099327

---

From: Martin Thomson <[email-masked]>
Date: Wed, 01 Jan 2020 19:13:23 -0800

> HTTP/2 is underspecified in sense that it allows any stream error to be promoted to a connection error

I think that we might disagree on this point.  Even if the spec didn't expressly include this permission, you can always reduce this to a question of local policy.  That is, my policy could be to cease communicating with a peer that fails to follow the spec.

The effect of such a policy has been beneficial in that it provides some incentive for new implementations to follow the spec.  The same policy has also been bad in those cases where this was overzealously implemented: if we can't used new frame types without explicit negotiation, that's not a great state to be in.

> even though as we agree, a stream error that might have originated from an origin cannot be promoted to a connection error

I don't think "cannot" is agreed.  We might agree on the outcome of propagating the error, but I personally don't think that we should externalize these effects even in this way.  We might highlight the possibility of problems, but if an intermediary is forwarding streams without looking at their contents, it is still ultimately responsible for the effect of their contents.  If clients decide to hold the intermediary responsible for a failing at an origin server that is invisible to them, that's still the intermediary's problem.

I'm going to suggest that we acknowledge that this is tricky and then move on.  I don't want to create another application of the Robustness Principle before we've even shipped the protocol.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3300#issuecomment-570109275

---

From: Kazuho Oku <[email-masked]>
Date: Wed, 01 Jan 2020 20:45:13 -0800

@martinthomson 
> I'm going to suggest that we acknowledge that this is tricky and then move on. I don't want to create another application of the Robustness Principle before we've even shipped the protocol.

I might argue that we *have had* that principle. [RFC 7540](https://tools.ietf.org/html/rfc8336) by default allows a client to coalesce requests going to multiple origins (assuming that the DNS lookup and the provided certificate allows that), then let the server opt-out (by sending 421). [RFC 8336](https://tools.ietf.org/html/rfc8336) and [draft-ietf-httpbis-http2-secondary-certs](https://datatracker.ietf.org/doc/draft-ietf-httpbis-http2-secondary-certs/) talk about coalescing requests going to multiple origins. It is my understanding that we have collectively thought that coalescing connections when possible is beneficial for privacy and performance reasons (prevents leak of SNI, better network utilization by not having multiple congestion controllers between the same endpoints).

Saying that a response from one origin might disrupt responses from other origins that are coalesced on one HTTP connection will be a wind against these previous attempts.

I am not necessarily against creating a new connection per each origin (or set of origins owned by single entity). The privacy benefit will be covered by [ESNI](https://datatracker.ietf.org/doc/draft-ietf-tls-esni/), and the performance reason can be covered by something like [draft-kazuho-quic-address-bound-token](https://datatracker.ietf.org/doc/draft-kazuho-quic-address-bound-token/).

However, as I have stated, I think we need to take a principled approach on when coalescing should (or should not) happen. This includes reconsidering if it is a good idea to allow the client to coalesce by default.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3300#issuecomment-570117390

---

From: ianswett <[email-masked]>
Date: Thu, 02 Jan 2020 13:25:06 -0800

I think coalescing is still a good thing.  I don't expect this issue to have much impact on that in practice.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3300#issuecomment-570355904

---

From: Mike Bishop <[email-masked]>
Date: Mon, 13 Jan 2020 13:09:30 -0800

Taking a look at the text, I still think the right principle here is that errors which might have been tunneled from an origin shouldn't be required to be stream errors.  Looking at the text, here are the things that might fall into this category:

- "Malformed" requests or responses are already stream errors
- Other frame types on CONNECT streams are connection errors, but with a CONNECT stream, the tunneled content is the DATA payload.  This could go either way.
- Frames that don't have the right size/content are connection errors, including final frames on streams that close cleanly.
- Control frames on non-control streams are connection errors.

Does this principle hold for the opposite direction?  Should servers consider garbage on streams as stream errors, because the "client" might actually be a proxy for several clients?  It's probably the simplest to keep things symmetrical.

I've put #3336 as an alternative solution to this.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3300#issuecomment-573871628

---

From: Kazuho Oku <[email-masked]>
Date: Mon, 13 Jan 2020 21:04:18 -0800

I think that this issue discusses two orthogonal problems:
* This issue was originally about how an intermediary can close a HTTP message that it has been forwarding, when that intermediary recognizes that the message being forwarded is malformed (e.g., when upstream closes the request stream mid-frame, when the amount of body provided does not match the value of content-length response header), at the same time forwarding all the content that it has received from upstream. The only approach that works is delaying the transmission of stream resets until receiving ACKs for all the data it has sent on that stream. H2 does not have the first problem because a stream reset can never overtake the delivery of other frames that have already been sent.
* The scope of an error that happens on the request stream (or the push stream). At the moment, some errors (i.e. invalid frames) are handled at connection-level, while others (i.e. malformed response) are stream-level. This is a departure from the design that we have in H2, that localizes stream-level issues to stream-level errors. Note that H2 and H3 both allow endpoints to escalate stream-level errors to connection-level.

It might be a good idea to split this issue in two; both #3303 and #3336 are attempts to address the second problem, but not the first problem.

Regarding the second problem, there are three paths:
* a) status quo
* b) go back to what H2 does - #3336 
* c) require those errors to be stream-level, forbidding promotion to connection-level - #3303 

As stated in [my previous comment](https://github.com/quicwg/base-drafts/issues/3300#issuecomment-570117390), my preference goes to #3303, as it maximizes the chance of requests being coalesced. Unless we forbid the promotion of stream-level errors to connection level (or without stating that such promotion might have negative effects), CDNs cannot allow coalescing of HTTP requests that go to different origins, as one misbehaving origin might obstruct the exchanges between the client and other origins that share the same connection. While this is not an issue for all the server deployments, it would be a backlash against our previous efforts (e.g., ORIGIN frame, secondary certificates) to maximize the chance of requests going to different origins getting coalesced, as there would be less interest in implementing these extensions from CDNs.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3300#issuecomment-574003033

---

From: Mike Bishop <[email-masked]>
Date: Tue, 14 Jan 2020 07:33:34 -0800

I agree that these are separate problems; for the first problem, I'm strongly inclined toward the status quo.  If you want to ensure delivery of partial content, there is a workaround; for the default case, "play stupid games, win stupid prizes" seems like a reasonable point in the design space.

I'm willing to add cautionary text around the promotion permission, but I think an outright prohibition is both unwise and unenforceable.  Implementers will do what they will do, and when there's garbage coming from the peer, it's a totally sensible design to close aggressively.

Frankly, I'd say CDNs shouldn't be shuffling raw frames anyway; they should be shuttling content, at the very least, and mostly have to anyway for HPACK/QPACK to work (unless everything is literal).  At that point, we're really only talking about malformed headers.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3300#issuecomment-574231598

---

From: Ryan Hamilton <[email-masked]>
Date: Tue, 14 Jan 2020 08:14:02 -0800

I agree with @MikeBishop completely.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3300#issuecomment-574251111

---

From: Kazuho Oku <[email-masked]>
Date: Tue, 14 Jan 2020 10:36:34 -0800

@MikeBishop 
> Frankly, I'd say CDNs shouldn't be shuffling raw frames anyway; they should be shuttling content, at the very least, and mostly have to anyway for HPACK/QPACK to work (unless everything is literal)…

It is true that implementations would do whatever they do. To that end, I think it is correct to argue that there is no difference between a MUST NOT and a cautionary text with a clarification on what might happen. We'd have the same design as that in HTTP/2, and having clarification on the side-effects would be an improvement.

I also agree that CDNs shouldn't be shuffling raw frames anyway. That's not something on table. The original issue, for which I am seeing complexity, is about transmitting an error at the same time forwarding all the bytes up to where the error was detected. As previously stated, doing that has become much more complicated in HTTP/3 compared to HTTP/2. That said, it's not impossible, and if others implementing proxies are happy with living with such an workaround (or do not care about the problem), I can concede with that workaround.

Re malformed headers, I am starting to wonder the intent. Which of the following two are we suggesting?
* headers considered malformed in the HTTP/3 specification
* headers consider malformed by the specifications that define those headers

If it's the latter, it would be impossible for an HTTP/3 intermediary to detect all the malformed headers (consider the case of a new HTTP extension defining a authentication header - a type of header field that is forbidden from being sent as part of a trailer). Maybe it's a good idea to clarify that it is the former, assuming that that is the intent.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3300#issuecomment-574312708

---

From: Mike Bishop <[email-masked]>
Date: Tue, 14 Jan 2020 15:50:57 -0800

"Malformed" is defined by [Section 4.1.3](https://quicwg.org/base-drafts/draft-ietf-quic-http.html#name-malformed-requests-and-resp).  If a specific header is not used properly, that's an error at the semantic layer; the server might send you a 400, but that's a successful request/response as far as HTTP/3 is concerned.

#3334 adds the advice without any change to what is a stream/connection error, meaning that malformed messages are stream errors and the rest remain connection errors.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3300#issuecomment-574428860

---

From: Kazuho Oku <[email-masked]>
Date: Tue, 14 Jan 2020 16:02:33 -0800

@MikeBishop Yeah my point is that "prohibited header fields" is ambiguous. I've opened #3345.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3300#issuecomment-574432098

---

From: Mike Bishop <[email-masked]>
Date: Thu, 16 Jan 2020 12:14:07 -0800

Closed #3300 via #3344.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3300#event-2955916816

---

From: Mike Bishop <[email-masked]>
Date: Thu, 16 Jan 2020 12:16:44 -0800

Note:  editorial fix; issue hadn't been triaged as Design.  However, @LPardue / @mnot  / @larseggert, if you'd like to run the process on *not* handling the case of reliable delivery prior to a RESET_STREAM, please re-open.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3300#issuecomment-575327795