Subject: [quicwg/base-drafts] Consider a new stream type to act like MAX_PUSH_ID (#2418)

From: Lucas Pardue <[email-masked]>
Date: Tue, 05 Feb 2019 02:19:52 -0800

This straw man idea stems from the discussion on #2412.

Consider defining a new unidirectional stream type for Max Push ID updates. This can be expressed as unframed data: a series of increasing var ints. The use of MAX_PUSH_ID frame is entirely, so using a uni stream is a slightly better fit to the concept. Another reason to consider this approach is that it lends itself to extracting server push into an extension. It provides a pattern for how future extensions could manage themselves without needing to pollute the core control stream.

The max Push ID stands out as a connection-level state that has no associated H3 Setting. The reintroduction of an ENABLE_PUSH setting was recently dismissed in Tokyo, for valid reasons.

In this proposal, one client initiated unidirectional stream of type "Max Push ID update" may be created by a client as a clear signal of its interest in enabling server push. Push IDs sent on the stream can only increase in value, any decrease is a *Stream* error of type TBD (possibly just generic HTTP protocol error). Judicious endpoints can upgrade stream error to connection error if they so wish. For reference, closing the H3 control streams is verboten. This solves the problem with the draft 18 text that suggests sending a connection error of MALFORMED_FRAME if a MAX_PUSH_ID frame is received with a value lower than previously seen.

Applying validation at the stream level may be easier to implement, needing only compare next push ID to previous push ID.

One problem with this proposal is that it adds another stream that a server needs to coordinate with. However, server push is already coordinating over the control stream, request streams and push streams.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2418

---

From: Martin Thomson <[email-masked]>
Date: Tue, 05 Feb 2019 03:24:43 -0800

If the sequence is monotonic increasing, a series of deltas will make it more efficient.  Slightly.  I mean, the stream overhead here will dominate anyway.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2418#issuecomment-460603067

---

From: Lucas Pardue <[email-masked]>
Date: Tue, 05 Feb 2019 03:31:42 -0800

> If the sequence is monotonic increasing, a series of deltas will make it more efficient. Slightly. I mean, the stream overhead here will dominate anyway.

Neat idea. unsigned deltas may actually be a more natural mapping for what the client is trying to express in terms of managing concurrent pushes. However, this might add some work for the server to reconstruct the push ID. Not sending explicit values seems like an easy way to misunderstand the clients intent.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2418#issuecomment-460604870

---

From: Jana Iyengar <[email-masked]>
Date: Tue, 05 Feb 2019 03:47:24 -0800

This is a nice idea.

Encoding deltas avoids the server having to check if the new MAX_PUSH_ID is
smaller, but meh.

The only issue I see here is HoL blocking. It would be nicer to not
introduce dependencies and send these updates in DATAGRAM frames, but of
course, we don't have those.

On Tue, Feb 5, 2019 at 5:01 PM Lucas Pardue <[email-masked]>
wrote:

> If the sequence is monotonic increasing, a series of deltas will make it
> more efficient. Slightly. I mean, the stream overhead here will dominate
> anyway.
>
> Neat idea. unsigned deltas may actually be a more natural mapping for what
> the client is trying to express in terms of managing concurrent pushes.
> However, this might add some work for the server to reconstruct the push
> ID. Not sending explicit values seems like an easy way to misunderstand the
> clients intent.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/quicwg/base-drafts/issues/2418#issuecomment-460604870>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AKjg1I6DUGrP6xli1nutis540wJZ4rTzks5vKWudgaJpZM4ai5xW>
> .
>

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2418#issuecomment-460609051

---

From: ianswett <[email-masked]>
Date: Tue, 05 Feb 2019 07:27:13 -0800

To clarify, I believe the problems this is solving/benfits are:
 1) Impossible to have an invalid value in MAX_STREAM_ID
 2) Makes push more like an extension
 3) Saves 2+bytes per update to MAX_STREAM_ID
 4) Slight reduction in odds of head of line blocking from moving off the control stream.

I'm not sure if this is worth the extra complexity of another dedicated stream or not.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2418#issuecomment-460679724

---

From: Lucas Pardue <[email-masked]>
Date: Tue, 05 Feb 2019 07:50:42 -0800

It might also solve the problem that the server cannot tell the client that there will be no server push used in a H3 session. You could do that with this design by resetting the "push control stream" or whatever we want to call it. 

This is a different way to achieve the lack of SETTINGS_(DIS|EN)ABLE_PUSH that we no longer have.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2418#issuecomment-460689489

---

From: Kazuho Oku <[email-masked]>
Date: Tue, 05 Feb 2019 19:00:26 +0000 (UTC)

> Consider defining a new unidirectional stream type for Max Push ID updates. This can be expressed as unframed data: a series of increasing var ints.

Am I correct in assuming that the design have the same issue to what you tried to address in #2412? If so, I wonder what the benefit of changing the design is.

> The use of MAX_PUSH_ID frame is entirely unidirectional, so using a uni stream is a slightly better fit to the concept. Another reason to consider this approach is that it lends itself to extracting server push into an extension. It provides a pattern for how future extensions could manage themselves without needing to pollute the core control stream.

I am not sure if that is the preferred approach. Although I acknowledge some of the upsides, the downsides of using unidirectional stream are that it requires more state (due to the need for it's own send / receive buffers) and that it is sometimes complex to synchronize the states that are communicated over multiple streams. There are multiple cases where we could use a dedicated stream instead of sending a frame on the control stream; PRIORITY, CANCEL_PUSH, GOAWAY, DUPLICATE_PUSH, I believe that the payload of all these frames can be sent using their own streams, the same way the proposal tries to carry the payload of MAX_PUSH_ID frame.

To put it another way, it seems to me that our current design pattern is to use the control stream for transmitting various data instead of allocating a different unidirectional stream for each type of information. Allocating a different stream for just MAX_PUSH_ID would be a inconsistency in the design. Do we want to have dedicated unidirectional streams for each type of control information? To me that sounds like a design that's unnecessarily heavyweight.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2418#issuecomment-460761115

---

From: Lucas Pardue <[email-masked]>
Date: Tue, 05 Feb 2019 12:29:58 -0800

No other frame has the same combination of characteristics as MAX_PUSH_FRAME:

* Valid in one direction only
* Valid on one stream type only
* Hard requirements on order of values carried in frame, with failure requiring a connection error

GOAWAY comes close but is more lenient 

>. A server that is attempting to gracefully shut down a connection SHOULD send an initial GOAWAY frame with the last Stream ID set to the current value of QUIC’s MAX_STREAM_ID and SHOULD NOT increase the MAX_STREAM_ID thereafter.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2418#issuecomment-460791582

---

From: Lucas Pardue <[email-masked]>
Date: Tue, 05 Feb 2019 12:35:28 -0800

I might also dare to incite the qpackers and say that we could live without encoder and decoder streams. Insteaf creating some new frame types that hold the instructions and exchange them on control stream

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2418#issuecomment-460793296

---

From: Mike Bishop <[email-masked]>
Date: Tue, 05 Feb 2019 13:37:47 -0800

@LPardue, QPACK encoder data was originally in HEADERS frames on the control stream; the reason for moving to unidirectional streams was the insight that the blocks of updates weren't themselves significant; it was really the state of individual updates that needed to be tracked.  Because they were an ordered sequence with no semantics to the division into frames, we decided that moving to a stream would be simpler to process.

In this case, the semantics are a little different -- you really only ever care about the most recent value.  In the transport and in H2, we carry such things in frames because that's what we have.  An application on top of QUIC doesn't really have an equivalent mechanism until DATAGRAM exists.  Putting the current frame in a DATAGRAM would make the most sense.  In the meantime, @martinthomson is correct that only deltas are needed, and that's actually true even with the frame -- they're on the control stream in order, so a delta would be sufficient.  The current design does rather straddle the two options, delivering full updates in order.  The main argument for keeping the current design seems like avoiding the need to change the structure when we are able to adopt DATAGRAM.

I don't think moving it to a unidirectional stream makes it any more extension-like than defining a frame type -- extensions can do either.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2418#issuecomment-460813704

---

From: Lucas Pardue <[email-masked]>
Date: Tue, 05 Feb 2019 21:48:04 +0000 (UTC)

This is a valiant explanation, thanks Mike.

I don't see how unreliable DATAGRAM fixes the problem with handling MAX_PUSH_ID. I think it risks making the problem worse with the current rules. 

Consider a client that sends a ID 5, then ID 3 erroneously. If these are in separate DATAGRAMS, the error may go unnoticed if the first frame was lost. Furthermore, loss could effectively starve the server of Push ID credits, with neither endpoint being the wiser as to what happened.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2418#issuecomment-460816890

---

From: Mike Bishop <[email-masked]>
Date: Tue, 05 Feb 2019 16:28:28 -0800

We have a mandatory error because they arrive in a fixed order and therefore an incorrect receipt order means an incorrect send order.  Once DATAGRAMs are used, there's no ordering guarantee, so no check can be performed on the receiver.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2418#issuecomment-460858943

---

From: Lucas Pardue <[email-masked]>
Date: Tue, 05 Feb 2019 16:48:52 -0800

I think depends if we ascribe special meaning to the fact that a client took N steps to a max value, or if we just care that we arrive at the max value.

To me, the endstate is the same. And since you can't "steal back" push ID credits, neither side should care if that manifests. 

But there may be gotchas in ignoring lower limits so I can fall in line of popular opinion.

I'm not seeing an outright no to this strawman. I think there are some unique benefits and drawbacks. What is a sensible path forward, a more solid PR?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2418#issuecomment-460862806

---

From: Martin Thomson <[email-masked]>
Date: Tue, 05 Feb 2019 17:08:14 -0800

I'd say that we need better justification than "not an outright no".  For something like this, control stream seems fine.  The frames are small (and could be made smaller by making them deltas, something I'd be happy to see).

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2418#issuecomment-460866532

---

From: Lucas Pardue <[email-masked]>
Date: Tue, 05 Feb 2019 17:20:51 -0800

Ok.

The problem I see with deltas is that the client can easily blow the limit of Push ID e.g. send 2 maximum varints in a pathological case. This kind of error is not possible with the current design. 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2418#issuecomment-460868818