Subject: Re: [quicwg/base-drafts] Why are there two ways of associating push with requests? (#3275)

From: Ryan Hamilton <[email-masked]>
Date: Mon, 02 Dec 2019 21:22:26 -0800

This is likely the wrong place for this observation, but I'll mention it here. Ian and I were chatting about this issue and realized a couple of things about HTTP/3 push which were quite surprising.

Recall that in HTTP/2, the motivating use case for PUSH_PROMISE was the ability to promise more resources than the open stream limit would allow to be pushed. (Consider an image search in which the server wants to push a grid of 100 thumbnails when the open stream limit is 10). 

The server would be able to push all 100 of these thumbnails in a single RTT (assuming sufficient BDP) because in HTTP/2 the sever can open a new stream as soon as the previous stream is closed.

However, in HTTP/3 this is not the case. The server will be unable to open a new push stream until it receives a new MAX_STREAMS from the client increasing the limit. But that burns an RTT. In which case, there's no point in pushing at all. So to avoid this, a client could increase the MAX_STREAMS to some sort of enormous value. But that does not behave like the HTTP/2 open stream limit. 

As such I'm not sure the original motivation for PUSH_PROMISE makes sense in HTTP/3. On the other hand, PUSH_PROMISE does something totally different in HTTP/3. It provides an ordering guarantee between the promised headers and the the reference to the resource in the main response body.

What's the conclusion of all this? I'm not sure. But it raises a few points:
* HTTP/3 server push works incredibly differently than HTTP/2
* At a minimum we should probably provide advice about how the client should manage MAX_STREAMS and MAX_PUSH_ID
* I could almost imagine replacing PUSH_PROMISE completely by a BLOCK_UNTIL_PUSH_RECEIVED frame which specifies a QUIC stream ID and prevents the app from processing the rest of the body until the specified push stream's initial headers are received.

Thoughts? Does this resonate with anyone?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3275#issuecomment-561003936

---

From: Kazuho Oku <[email-masked]>
Date: Mon, 02 Dec 2019 22:15:17 -0800

@RyanAtGoogle I kind of agree, and kind of disagree.

> * HTTP/3 server push works incredibly differently than HTTP/2

I am not sure how different they are. As you correctly point out, the benefit you achieve by pushing is limited to one BDP in HTTP/2. As there is the potential of pushing a response that has already been cached by the client, pushing too much has been a bad idea in HTTP/2. IMO, the fact that we are additionally capped by the push stream concurrency in HTTP/3 does not change the big picture.

> * At a minimum we should probably provide advice about how the client should manage MAX_STREAMS and MAX_PUSH_ID

SGTM.

> * I could almost imagine replacing PUSH_PROMISE completely by a BLOCK_UNTIL_PUSH_RECEIVED frame which specifies a QUIC stream ID and prevents the app from processing the rest of the body until the…

I presume that's the same as what I originally proposed in this issue? Personally, I think I am fine with either ways. What I'm unhappy with status-quo is that there is two ways of doing one thing, which is causing complexity.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3275#issuecomment-561018182

---

From: Ryan Hamilton <[email-masked]>
Date: Tue, 03 Dec 2019 07:21:16 -0800

> I am not sure how different they are. As you correctly point out, the benefit you achieve by pushing is limited to one BDP in HTTP/2. As there is the potential of pushing a response that has alread…

In HTTP/2, pushing too much is only a problem when you push something that the client already has cached (or otherwise does not need) or if you invert the priorities and push something instead of sending something more important. But in HTTP/2 there is no performance problem with pushing an unlimited number of resources, so long as they are all "required" and "important".

This is not the case with push. Promising more resources than the concurrent stream limit is a performance regression. I think this is non-obvious.

> I presume that's the same as what I originally proposed in this issue? Personally, I think I am fine with either ways. What I'm unhappy with status-quo is that there is two ways of doing one thing,…

I think this could be what you proposed initially, though I have one point of clarification. I understood the DUPLICATE_PUSH semantics to be that the client would continue to process the body of the stream on which the DUPLICATE_PUSH was sent, but that any requests would be blocked until the associated PUSH_PROMISE was received. I don't like this idea because it blocks all requests, not just those which the server intended to promise. Instead, I'd propose that the frame (whatever we call it) blocks the processing of the stream on which the frame was sent, which prevents the client from discovering the link to the promised resource until the push headers have arrived. (Of course the server should send the push headers on the push stream before it sends the frame on the original stream thus typically the push headers will arrive before the frame).

Is this latter behavior what you were thinking?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3275#issuecomment-561215368

---

From: Lucas Pardue <[email-masked]>
Date: Tue, 03 Dec 2019 07:39:13 -0800

> Recall that in HTTP/2, the motivating use case for PUSH_PROMISE was the ability to promise more resources than the open stream limit would allow to be pushed. (Consider an image search in which the…

Was that a feature or a bug? By my understanding, we've been attempting to fix that accounting loophole in HTTP/3.

Some of what is being discussed here is a retread of #2559 (in particular https://github.com/quicwg/base-drafts/issues/2559#issuecomment-476961392)

> However, in HTTP/3 this is not the case. The server will be unable to open a new push stream until it receives a new MAX_STREAMS from the client increasing the limit. But that burns an RTT. In whic…

I don't follow. The HTTP/2 behaviour seems to equate to constrained concurrent requests, and uncontrained pushes. Therefore, setting a sensible value for `MAX_STREAMS bidi` and an enormous value for `MAX_STREAMS uni` seems like what you want. Although I not convinced that is a feature.

> At a minimum we should probably provide advice about how the client should manage MAX_STREAMS and MAX_PUSH_ID

If you're trying to mimic the "push 100 tumbnails without an RTT" behaviour, you'll have to keep in mind the value of initial_max_stream_data_uni too.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3275#issuecomment-561223695

---

From: Ryan Hamilton <[email-masked]>
Date: Tue, 03 Dec 2019 08:07:48 -0800

> Was that a feature or a bug? By my understanding, we've been attempting to fix that accounting loophole in HTTP/3.

It was definitely a feature! (Though the lack of *any* limit on the number of promised resources might have been a bug). In particular, the feature came into existence because, iirc, because Roberto tried to do a server push demo of exactly this use case (page of thumbnails) but it didn't work because the stream limits got in the way. After that, he proposed the push/promise distinction with the explicit intent of allowing more promises than pushes.
 
> Some of what is being discussed here is a retread of #2559 (in particular [#2559 (comment)](https://github.com/quicwg/base-drafts/issues/2559#issuecomment-476961392))

Agreed. To be clear, I rather like the explicit MAX_STREAMS signal. For bidirectional streams and web browsing, this seems to work out extremely nicely.

For server push, on the other hand, it's a non-obvious sharp edge, particular with respect to HTTP/2. (So at a minimum, I think we should provide guidance on how to push with HTTP/3 effectively)

> I don't follow. The HTTP/2 behaviour seems to equate to constrained concurrent requests, and uncontrained pushes. Therefore, setting a sensible value for `MAX_STREAMS bidi` and an enormous value for `MAX_STREAMS uni` seems like what you want. 

Not quite. HTTP/2 provides unlimited *promises*, but limited concurrent *pushes* (via the open steams limit). HTTP/3 can limit promises via MAX_PUSH_ID, and limit concurrent pushed via MAX_STREAMS uni. So to replicate HTTP/2, and HTTP/3 client could try a huge MAX_PUSH_ID and a small MAX_STREAMS uni. That would have the same concurrent behavior as HTTP/2, but would run the risky of adding a 1-RTT delay for pushes above the MAX_STREAMS limit.

> Although I not convinced that is a feature.

That's a fair point. If so we should probably have guidance how managing MAX_PUSH_ID and MAX_STREAMS uni together to make this obvious. (Though at this point I'm coming around the to position of other folks in Chrome that push in general is rarely a performance win and typically a footgun and we should stop using it)
 
> If you're trying to mimic the "push 100 tumbnails without an RTT" behaviour, you'll have to keep in mind the value of initial_max_stream_data_uni too.

Yes, flow control also comes into play for sure. (Thankfully that's not much different from HTTP/2). If a client's flow control limits are less than a BDP, the client is "gonna have a bad time" :) 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3275#issuecomment-561236987

---

From: Kazuho Oku <[email-masked]>
Date: Tue, 03 Dec 2019 18:36:02 -0800

I tend to agree with what @RyanAtGoogle says.

IIUC, one of the reasons we have push ID in addition to stream IDs is to limit the exposure of transport features to the application protocol. However, as @ianswett points out in #3273, there is already a leak in the abstraction.

Therefore, I think we can (and should) directly use stream IDs. Doing so would resolve the concern that the push might get blocked by stream concurrency.

>> I presume that's the same as what I originally proposed in this issue? Personally, I think I am fine with either ways. What I'm unhappy with status-quo is that there is two ways of doing one thing, which is causing complexity.>
> 

To me, those two approaches sounds like something that a client can choose from, though I agree that the latter is better as it introduces less blocking. If the HTTP/3 draft is not already clear about the intention, it might be a good idea to open an editorial issue, because IMO that is a problem with the DUPLICATE_PUSH frame that already exists.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3275#issuecomment-561448819

---

From: Martin Thomson <[email-masked]>
Date: Tue, 03 Dec 2019 19:17:53 -0800

Unlimited pushes in HTTP/2 was a bug, not a feature.  The concurrency limit might have applied in theory, but the ability to run the entire stream lifecycle without feedback meant that there was no limit in practice other than those imposed by TCP - on either promises or responses.

The important feature, which we retained, was the ability to decouple the stream limits and the number of promises.  That's why we have push IDs.  Assuming the client is configured for that, the separate limit on push promises ensures that promises can greatly exceed the number of active streams.  I think that is the ideal situation if the client is somehow resource-limited, so it can accept N promises and then receive M (<N) responses for those promises.

In other words, I see no reason to change the current design.  If this is the result of general antipathy toward server push, that's fine, but we're not in a stage of development where we can afford to make changes at such a fundamental level without serious impact on our schedule.  Besides, I'm not seeing a proposal here that would be anything less than a regression in some way.

If someone wants to contribute text that walks through some of the trade-offs inherent in setting different limits (whether that be streams, flow control, or push IDs) for people who might want to implement or deploy HTTP, that seems fine.  As this is highly context-dependent, it might be hard to avoid certain biases.  I certainly wouldn't want to block publication if such guidance were absent.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3275#issuecomment-561458016

---

From: ianswett <[email-masked]>
Date: Wed, 04 Dec 2019 05:54:22 -0800

I'd suggest removing DUPLICATE_PUSH and allowing duplicate PUSH_PROMISE frames on different response streams.  I'd be hesitant to make larger changes at this point and I think Push IDs are an improvement over HTTP/2.

DUPLICATE_PUSH saves a few bytes over an equivalent PUSH_PROMISE, but I believe has no value.  Think of the two cases:
1) No loss or reordering, the client has already received the PUSH_PROMISE.  DUPLICATE_PUSH should not change the client response.
2) DUPLICATE_PUSH arrives before PUSH_PROMISE.  But the client doesn't know what was pushed.  It has the choice of indefinitely delaying all requests on the page, which is almost certainly a performance degradation in most cases or ignoring it.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3275#issuecomment-561655341

---

From: Lucas Pardue <[email-masked]>
Date: Wed, 04 Dec 2019 06:05:46 -0800

> I'd suggest removing DUPLICATE_PUSH and allowing duplicate PUSH_PROMISE frames on different response streams.
> DUPLICATE_PUSH saves a few bytes over an equivalent PUSH_PROMISE, but I believe has no value.

What we have now addresses a specific concern raised on #1864 about how to handle multiple PUSH_PROMISES with the same ID. Mike suggested some points that could be address on that ticket https://github.com/quicwg/base-drafts/issues/1864#issuecomment-430697483 if we continued to allow multiple PUSH_PROMISE:

>     * Maintains a list of active promises for which it is still willing to accept an additional reference to the promise, including the uncompressed headers for the promise (likely until the promise is fulfilled, possibly plus a timeout)
>     * Verifies that the headers match exactly when receiving an additional reference to one of these promises; MUST error if they don't.
>     * If a promise is received for an ID which has already been flushed from the list, the promise is ignored

A PR that proposes removing DUPLICATE_PUSH should address the above.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3275#issuecomment-561659879

---

From: Mike Bishop <[email-masked]>
Date: Mon, 16 Dec 2019 08:34:04 -0800

> Nevertheless, the current design must have some rationale behind it, which I expect @MikeBishop to share with us on this thread.

I think I've been adequately channeled in the meantime.  😉

As others have pointed out, this was a compromise that no one was thrilled about, but was considered better than the status quo and no ingenious solution to all issues has yet presented itself.  The solution space that I see is:

- **One request, one promise, one push.**  H2-style, every push is promised to one and only one request.
- **Multiple PUSH_PROMISE frames on different streams.**  Old H3-style; wastes some bytes repeating yourself, and now we have to talk about consistency between them:
  - **Allowed to differ**; each promise separately contains the headers of the request, and if different requests would have garnered identical responses, this is fine.  (For example, pushing "empty list" for lots of subresources the client might have wanted to retrieve as part of an API.)
  - **Not allowed to differ**; client has to check that the headers are the same.  What if they're not?  How long does the client have to keep the headers?  Need it be the same bytes, or simply the same resulting header set?
- **Headers sent once; DUPLICATE_PUSH frame.**  This is the compromise we landed on.  It's true that this means the headers might not be available at the point DUPLICATE_PUSH is received.
- **Headers sent once on push stream; DUPLICATE_PUSH becomes PUSH_PROMISE.**  Conceptually easier, but means headers are almost certainly not available at the point the promise is received.  Means promised request can't be delivered without enough stream credit to do the push itself.

The idea of including a URI in DUPLICATE_PUSH is an interesting one, and might reduce the amount of blocking clients need to do until the push request is received.  It also makes that last solution more palatable.  Presumably, if the server is pushing a request for a URI, the odds that the request will match what the client would have sent better be high.

I don't personally believe this change is warranted in late stage, but I'm willing to take an improvement the working group reaches consensus on.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3275#issuecomment-566138148

---

From: Kazuho Oku <[email-masked]>
Date: Mon, 16 Dec 2019 20:53:13 -0800

@MikeBishop Thank you for laying down the options.

My preference goes to:

> * **Multiple PUSH_PROMISE frames on different streams.** Old H3-style; wastes some bytes repeating yourself, and now we have to talk about consistency between them:
>   * **Not allowed to differ**; client has to check that the headers are the same. What if they're not? How long does the client have to keep the headers? Need it be the same bytes, or simply the same resulting header set?

Regarding the points listed as questions, I think we should state that the header set that the server sends MUST contain the exact same bytes (after decompression), and that a client MUST either validate that they are the same, or ignore the PUSH_PROMISE.

By using MAX_PUSH_ID frame, the client controls the number of pushes that the server can initiate. It can track the unused push IDs much like the QUIC transport tracks unused stream IDs. By tracking that, the client can tell if a PUSH_PROMISE frame contains a new push or if is a duplicate. If it is a duplicate that it has already forgotten, then it can simply ignore it.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3275#issuecomment-566379821

---

From: Ryan Hamilton <[email-masked]>
Date: Mon, 16 Dec 2019 21:20:45 -0800

+1 to everything @kazuho said. Let's do it!

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3275#issuecomment-566385558

---

From: Lucas Pardue <[email-masked]>
Date: Mon, 16 Dec 2019 23:10:14 -0800

What do you mean by "exact same bytes"? After decompression I'm converting to my own header list object that isn't the same layout as any interop standard, is that ok?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3275#issuecomment-566412333

---

From: Kazuho Oku <[email-masked]>
Date: Mon, 16 Dec 2019 23:42:26 -0800

@LPardue I mean that the set of headers should appear in the same order, and that each of the header fields should contain the same bytes (after decompression).

Or to paraphrase, they should be the same byte-by-byte once decompressed, rather than semantically being the same. For example, the number of white spaces between the cache-control directives cannot change.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3275#issuecomment-566422452

---

From: Ryan Hamilton <[email-masked]>
Date: Fri, 20 Dec 2019 08:07:41 -0800

Not sure if I'm doing Gituhub right, but I just cooked up a PR to remove DUPLICATE_PUSH and permit duplicate PUSH_PROMISE frames which must contain the same headers.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3275#issuecomment-567977291