Subject: Re: [quicwg/base-drafts] STOP_SENDING in Ready state (#1797)

From: Ryan Hamilton <[email-masked]>
Date: Wed, 14 Nov 2018 20:21:29 -0800

On the list, I raised the following question which seems quite related to this issue:

The section on STOP_SENDING says:

   Receipt of a STOP_SENDING frame is only valid for a send stream that
   exists and is not in the "Ready" state (see Section 3.1).  Receiving
   a STOP_SENDING frame for a send stream that is "Ready" or non-
   existent MUST be treated as a connection error of type
   PROTOCOL_VIOLATION.

However, I'm not sure I'm clear on what should be done in the face of reordering, but maybe this is obvious. Say an HTTP client sends a request. This sends a STREAM frame to the server. This move the stream to the "Send" state. Then, the user cancels the request. So the client sends a RST_STREAM and a STOP_SENDING in order to completely nuke the stream. However, both the RST_STREAM and STREAM frames are lost/reordered such that the STOP_SENDING is the first frame to arrive at the server. As such, the stream is non-existent I think, which is prohibited. But maybe I'm not thinking about this the right way?

>From @MikeBishop  "Ah, I see what you’re saying – the STREAM frame in the C2S direction is what moves the S2C direction from Ready to Send.  If STOP_SENDING can be reordered relative to that frame, this could cause a valid-when-sent STOP_SENDING frame to arrive while the S2C direction is still in the Ready state.

I think you’re correct; the prohibition on receiving STOP_SENDING in the Ready state is erroneous.  Please file an issue, and thank you!"

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1797#issuecomment-438912358

---

From: MikkelFJ <[email-masked]>
Date: Thu, 15 Nov 2018 04:01:36 -0800

Also note the related problem:

If a stream is opened, lower valued streams are also opened. It is not possible to send STOP_SENDING to those lower valued streams because they are in ready state and have not received any data yet. It would cause a protocol violation.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1797#issuecomment-439017071

---

From: MikkelFJ <[email-masked]>
Date: Thu, 15 Nov 2018 04:04:17 -0800

@martinthomson I think this issue needs to be reopened based on the above comments.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1797#issuecomment-439017747

---

From: Ryan Hamilton <[email-masked]>
Date: Thu, 15 Nov 2018 07:00:50 -0800

(]@mikkelfj I thought we dropped the language about streams being opened implicitly by virtue of receive information for a higher numbered stream? My recollection is that this was no longer needed which we switched from limiting the number of open streams to using max stream id.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1797#issuecomment-439069654

---

From: MikkelFJ <[email-masked]>
Date: Thu, 15 Nov 2018 07:07:02 -0800

@RyanAtGoogle that is true, but a lot of people wanted it back, so it reentered. My primary concern was the ability to have inifinite number of streams open - either very old are very future - but there is now a limited set of ID's you need to track. For this to work, it is very important that messages on old or uncreated streams are hard errors. This is the motiviation for the STOP_SENDING in Ready being a hard error. But there are some edge cases as you discovered.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1797#issuecomment-439071934

---

From: Mike Bishop <[email-masked]>
Date: Thu, 15 Nov 2018 10:17:38 -0800

Reopened #1797.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1797#event-1969396200

---

From: Mike Bishop <[email-masked]>
Date: Thu, 15 Nov 2018 10:19:06 -0800

STOP_SENDING on a stream that's already closed is a no-op, so they can be ignored.  @mikkelfj, would it satisfy your concerns if STOP_SENDING on a stream above the MAX_STREAM_ID were an error?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1797#issuecomment-439139494

---

From: MikkelFJ <[email-masked]>
Date: Thu, 15 Nov 2018 18:27:00 +0000 (UTC)

@MikeBishop No I don't think it is quite right, although also not catastrophic since impact is bounded.

I think the general idea is that the non-initiator cannot respond before seeing physical evidence of a stream. To honor that principle, STOP_SENDING should only be sent for streams with an ID at below a stream that has left Ready state. I can't really wrap my head fully around this atm. because there are different kinds of streams.

If you allowed up to MAX_STREAM_ID you could open a stream that the initiator is not prepared for yet. Which is why sending STOP_SENDING is a violation in the first place.

Perhaps the solution is to have a stream state earlier than Ready which is not pemitted to receive anything unprovoked, or perhaps Ready is that state, but then its semantics should be slightly redefined?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1797#issuecomment-439142065

---

From: Mike Bishop <[email-masked]>
Date: Thu, 15 Nov 2018 18:31:58 +0000 (UTC)

The specific race condition that @RyanAtGoogle raised is a STOP_SENDING on a bidirectional stream *by the initiator* of that stream.  I think a constraint on non-initiators is probably reasonable -- there's some risk that the initiator will have "leaked" knowledge of that stream in a different context, but I would expect that will typically reflect that the initiator has moved beyond the Ready state.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1797#issuecomment-439143605

---

From: Mike Bishop <[email-masked]>
Date: Thu, 15 Nov 2018 10:37:45 -0800

The difficulty is that, in this specific race condition, STOP_SENDING is the first frame received from the initiator.  Perhaps the right solution is that there's a different start state for peer-initiated streams than for locally-initiated streams?  Locally-initiated streams begin as "Not Allocated" and become "Ready" when the peer locally decides to allocate them; receiving any frame is an error.  Peer-initiated streams become "Ready" upon receipt of any frame in either direction.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1797#issuecomment-439145320

---

From: MikkelFJ <[email-masked]>
Date: Thu, 15 Nov 2018 18:43:00 +0000 (UTC)

I think it should be permissable to receive STOP_SENDING on a Ready stream because a stream is only ready if a higher stream has been created and started sending (I think). When the receiver sees that transmission it would know that earlier streams are potentially able to transmit (and addition to being ready to send). If the initiator does not want to waste state on tracking STOP_SENDING, it should not transmit on a higher ID first.

Alternatively, we should just accept this limitation - that you can't stop a transmission before seeing evidence of it being active. All the implicit open logic is about being able to transmit while STOP_SENDING affects the peers send state - which is unfortunate. It's really bordering cause/effect in particle physics (not entirely coincidental).

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1797#issuecomment-439146964

---

From: MikkelFJ <[email-masked]>
Date: Thu, 15 Nov 2018 18:45:57 +0000 (UTC)

btw: not arguing against your proposal on "Not Allocated" I think it is already there somewhat implicit. That might be the way to go - the bigger question is - what is the desired behavior in the first place.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1797#issuecomment-439147949

---

From: Mike Bishop <[email-masked]>
Date: Thu, 15 Nov 2018 10:49:13 -0800

I would say the overall desired behavior is that the initiator of a stream must be the first one to send frames on it, but that any frame it chooses to send should be considered valid, regardless of reordering.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1797#issuecomment-439148988

---

From: Mike Bishop <[email-masked]>
Date: Thu, 15 Nov 2018 18:50:26 +0000 (UTC)

The other way to frame it is that a peer-initiated stream moves out of Ready on receipt of any frame, and the prohibition on STOP_SENDING gets scoped to locally-initiated streams.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1797#issuecomment-439149368

---

From: Ryan Hamilton <[email-masked]>
Date: Thu, 15 Nov 2018 11:21:51 -0800

"I would say the overall desired behavior is that the initiator of a stream must be the first one to send frames on it, but that any frame it chooses to send should be considered valid, regardless of reordering."

"The other way to frame it is that a peer-initiated stream moves out of Ready on receipt of any frame, and the prohibition on STOP_SENDING gets scoped to locally-initiated streams."

@MikeBishop well said. I completely agree with both framings of this issue.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1797#issuecomment-439159062

---

From: MikkelFJ <[email-masked]>
Date: Thu, 15 Nov 2018 11:41:36 -0800

I think the answer to @RyanAtGoogle 's problem is that it can't really send a STOP_SENDING together with a RESET_STREAM unless a higher stream was already open. Not because of the current formulation, but because it violates the principle of sender creating the stream state for its own half-stream, regardless of who is the initiater.

To work around that, either STOP_SENDING should be permitted up to MAX_STREAM_ID, which I dislike, or a new STOP_RESET_STREAM message is needed, or STOP_SENDING is only permitted when coalesced in the same packet, which is messy.

So it really is the same problem as the uninitated peer problem - STOP_SENDING is invasive on streams that are not yet ready.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1797#issuecomment-439165290

---

From: Ryan Hamilton <[email-masked]>
Date: Thu, 15 Nov 2018 19:45:34 +0000 (UTC)

"but because it violates the principle of sender creating the stream state for its own half-stream, regardless of who is the initiater."

Is that correct? I think the stream state machine disagrees:
```
          o
          | Create Stream (Sending)
          | Create Bidirectional Stream (Receiving)
          v
      +-------+
```
This says that the send side of a bidi stream is opened when a bidi stream is opened on the receive side. Or perhaps I'm misreading that?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1797#issuecomment-439166479

---

From: MikkelFJ <[email-masked]>
Date: Thu, 15 Nov 2018 11:52:46 -0800

I'm note sure, I have a hard time reading these diagrams. You need to also read the associated text at the very least.

Even with a STOP_RESET_STREAM that closed both half-streams, you still have the problem that you then potentially opened streams with lesser ID's. You cannot send STOP_SENDING on these due to the same race condition.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1797#issuecomment-439168740

---

From: MikkelFJ <[email-masked]>
Date: Thu, 15 Nov 2018 11:54:20 -0800

> This says that the send side of a bidi stream is opened when a bidi stream is opened on the receive side. Or perhaps I'm misreading that?

I don't think that is wrong, but you cannot know the state at both ends at the same time. So the question is how you know that the other end opened the stream. It can happen by receiving data, or via implicit open in some cases.

I start to dislike implicit open even more now.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1797#issuecomment-439169198

---

From: Mike Bishop <[email-masked]>
Date: Thu, 15 Nov 2018 12:12:38 -0800

Looking at the existing transitions....

> The sending part of a bidirectional stream initiated by a peer (type 0 for a server, type 1 for a client) enters the “Ready” state then immediately transitions to the “Send” state if the receiving part enters the “Recv” state (Section 3.2).

And in "Send," STOP_SENDING is valid.  So what causes the receiving part to enter "Recv"?

> The receiving part of a stream initiated by a peer (types 1 and 3 for a client, or 0 and 2 for a server) is created when the first STREAM, STREAM_DATA_BLOCKED, RESET_STREAM, or MAX_STREAM_DATA (bidirectional only, see below) is received for that stream. The initial state for a receive stream is “Recv”.

It's odd to say that STOP_SENDING causes a state change on the receive stream, but that's one way to formulate this.  It's evidence that the peer is starting to use this bidirectional stream, and is no more weird than a MAX_STREAM_DATA (which is similarly talking about the other stream direction).

Maybe STOP_SENDING needs to join the bidirectional-only category of frames that can open a stream in both directions.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1797#issuecomment-439174620

---

From: MikkelFJ <[email-masked]>
Date: Thu, 15 Nov 2018 12:16:57 -0800

> Maybe STOP_SENDING needs to join the bidirectional-only category of frames that can open a stream in both directions.

And maybe there or more hidden race conditions in other frame types?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1797#issuecomment-439175861

---

From: ianswett <[email-masked]>
Date: Thu, 15 Nov 2018 12:53:39 -0800

I do think STOP_SENDING needs to join the set of frames that open a stream in both directions.

In particular, if I sent a GET in HTTP, and then I send a STOP_SENDING and RESET_STREAM in the same packet a few ms later, but the packet containing the GET is lost, I think that needs to be valid, since I expect webapps to do that fairly frequently.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1797#issuecomment-439186196

---

From: Mike Bishop <[email-masked]>
Date: Thu, 15 Nov 2018 14:28:46 -0800

>From a cursory glance over the frame types, STOP_SENDING and MAX_STREAM_DATA appear to be the two frames sent by receivers on a stream.  Everything else is either connection-level or sent by senders.  The sender frames seem to be well-covered in terms of opening a stream.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1797#issuecomment-439213670

---

From: Martin Thomson <[email-masked]>
Date: Mon, 19 Nov 2018 13:38:59 -0800

Closed #1797 via #2013.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1797#event-1975816774