Subject: [quicwg/base-drafts] Editorial rework of streams sections (#1950)

From: janaiyengar <[email-masked]>
Date: Fri, 02 Nov 2018 22:59:25 -0700

I've tried to eliminate redundancies and rearrange a few things in a better order. This might be hard to read the diffs on, so maybe just look at the Stream sections in the compiled draft.
You can view, comment on, or merge this pull request online at:

  https://github.com/quicwg/base-drafts/pull/1950

-- Commit Summary --

  * Rework of stream sections
  * rework of stream states

-- File Changes --

    M draft-ietf-quic-transport.md (642)

-- Patch Links --

https://github.com/quicwg/base-drafts/pull/1950.patch
https://github.com/quicwg/base-drafts/pull/1950.diff

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1950

---

From: janaiyengar <[email-masked]>
Date: Fri, 02 Nov 2018 23:01:03 -0700

@janaiyengar pushed 1 commit.

fdeafb2  Fix reference

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/1950/files/2d5ce9101ef9350daadaedce7fa10addb935e84b..fdeafb2ae55e15ee7f88639cdd826564d1413152

---

From: janaiyengar <[email-masked]>
Date: Fri, 02 Nov 2018 23:08:19 -0700

1894181  rebase

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/1950/files/3389bda3ee81fa5c893a63ff022f5d15c15156f3..1894181aa1b7913b6219936433fb483bb0da2da0

---

From: janaiyengar <[email-masked]>
Date: Fri, 02 Nov 2018 23:22:40 -0700

7f0dd50  s/SHOULD/MUST

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/1950/files/1894181aa1b7913b6219936433fb483bb0da2da0..7f0dd505ab37f4b44271c15edb52c32abd0d73a7

---

From: Martin Thomson <[email-masked]>
Date: Sat, 03 Nov 2018 00:52:45 -0700

martinthomson commented on this pull request.

> @@ -240,178 +240,119 @@ x (*) ...
 
 # Streams {#streams}
 
-Streams in QUIC provide a lightweight, ordered byte-stream abstraction.
+<!-- TODO: Replace bytes with octets -->

we collectively decided on bytes

```suggestion
```

+<!-- Stream ID or stream ID -->

the latter.  We're using capitalization for field names, not concepts.

-Either type of stream can be created by either endpoint, can concurrently send
-data interleaved with other streams, and can be cancelled.
+QUIC does not provide any means of ensuring ordering between octets on different
+streams.

Too early for this concept.

 Streams can be created by sending data. Other processes associated with stream
 management - ending, cancelling, and managing flow control - are all designed to
 impose minimal overheads. For instance, a single STREAM frame ({{frame-stream}})
 can open, carry data for, and close a stream. Streams can also be long-lived and
 can last the entire duration of a connection.
 
-Stream offsets allow for the bytes on a stream to be placed in order.  An
-endpoint MUST be capable of delivering data received on a stream in order.
-Implementations MAY choose to offer the ability to deliver data out of order.
-There is no means of ensuring ordering between bytes on different streams.
-
-Streams are individually flow controlled, allowing an endpoint to limit memory
-commitment and to apply back pressure.  The creation of streams is also flow
-controlled, with each peer declaring the maximum stream ID it is willing to
-accept at a given time.
+QUIC allows for an arbitrary number of streams to operate concurrently and for
+an arbitrary amount of data to be sent on any stream, subject to flow control
+constraints (see {{flow-control}}).

Maybe the inter-stream ordering belongs here.

-## Stream Identifiers {#stream-id}
+Streams can be unidirectional or bidirectional.  Unidirectional streams carry
+data in one direction: from the initiator of the stream to its peer.
+Bidirectional streams allow for data to be sent in both directions.  Streams can
+be created by either endpoint, can concurrently send data interleaved with other
+streams, and can be cancelled.  Any stream can be initiated by either enndpoint.

This might be better before the header.  Concentrating this bit on identifiers might be easier.

> -the protocol functions efficiently.  That is, prioritizing frames other than
-STREAM frames ensures that loss recovery, congestion control, and flow control
-operate effectively.
-
-CRYPTO frames SHOULD be prioritized over other streams prior to the completion
-of the cryptographic handshake.  This includes the retransmission of the second
-flight of client handshake messages, that is, the TLS Finished and any client
-authentication messages.
-
-STREAM data in frames determined to be lost SHOULD be retransmitted before
-sending new data, unless application priorities indicate otherwise.
-Retransmitting lost stream data can fill in gaps, which allows the peer to
-consume already received data and free up the flow control window.
+resources to, the implementation SHOULD use the information provided by the
+application.  Failure to account for priority of streams can result in
+suboptimal application performance.

 # Stream States: Life of a Stream {#stream-states}

```suggestion
# Stream States {#stream-states}
```

-Streams are identified by an unsigned 62-bit integer, referred to as the Stream
-ID.  Stream IDs are encoded as a variable-length integer (see
-{{integer-encoding}}).  The least significant two bits of the Stream ID are used
-to identify the type of stream (unidirectional or bidirectional) and the
-initiator of the stream.
+Streams are identified within a connection by a numeric value, referred to as
+the Stream ID.  Stream IDs are unique to a stream: a QUIC endpoint MUST NOT

```suggestion
the Stream ID.  Stream IDs are unique to a stream. A QUIC endpoint MUST NOT
```

> -
-The stream limit is specific to each endpoint and applies only to the peer that
-receives the setting.  That is, the client limits the number of streams of each
-type the server can initiate, and the server limits the number of streams the
-client can initiate.  Each endpoint can respond on streams initiated by the
-other peer, regardless of whether it is permitted to initiate new streams.
-
-Endpoints MUST NOT exceed the limit set by their peer.  An endpoint that
-receives a STREAM frame with an ID greater than the limit it has sent MUST treat
-this as a stream error of type STREAM_LIMIT_ERROR ({{error-handling}}).
-
-A receiver cannot renege on an advertisement; that is, once a receiver
-advertises a stream limit using the MAX_STREAMS frame, advertising a smaller
-limit has no effect.  A receiver MUST ignore any MAX_STREAMS frame that does
-not increase the stream limit.
+Within each type, streams are expected to be created with numerically increasing

```suggestion
Within each type, streams are created with numerically increasing
```

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1950#pullrequestreview-171318802

---

From: Martin Thomson <[email-masked]>
Date: Sat, 03 Nov 2018 01:10:00 -0700

## Sending and Receiving Data
 
-Endpoints uses streams to send and receive data. Endpoints send STREAM frames,
-which encapsulate data for a stream. STREAM frames carry a flag that can be used
-to signal the end of a stream.
+Endpoints use STREAM frames to encapsulate data sent by an application on a
+stream ({{frame-stream}}). An endpoint uses the stream ID and offset fields in a

STREAM frames ({{frame-stream}}) encapsulate data sent by an application.  An endpoint uses the Stream ID and Offset fields in STREAM frames to place data in order.

-QUIC makes no specific allowances for partial reliability or delivery of stream
-data out of order.  Endpoints MUST be able to deliver stream data to an
-application as an ordered byte-stream.  Delivering an ordered byte-stream
-requires that an endpoint buffer any data that is received out of order, up to
-the advertised flow control limit.
+An endpoint could receive data for a stream at the same stream offset multiple
+times; data that has already been received can be discarded.  The data at a

```suggestion
times. Data that has already been received can be discarded.  The data at a
```

Drop the last sentence.

> @@ -420,8 +361,9 @@ conditions for opening a stream are slightly more complex for a bidirectional
 stream because the opening of either send or receive sides causes the stream
 to open in both directions.
 
-An endpoint can open streams up to its maximum stream limit in any order,
-however endpoints SHOULD open the send side of streams for each type in order.
+An endpoint can open streams up to its maximum stream limit in any order of
+stream ID. However, within each type, endpoints SHOULD open the send side of
+streams in numerically increasing order of stream ID.

I think that this text was rotten to begin with.  There might be a simpler statement.

"An endpoint MUST open streams of the same type in increasing order of stream ID."

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1950#pullrequestreview-171319992

---

From: Martin Thomson <[email-masked]>
Date: Sat, 03 Nov 2018 01:19:06 -0700

-Before creating a stream, all lower-numbered streams of the same type MUST be
-created.  That means that receipt of a frame that would open a stream causes all
-lower-numbered streams of the same type to be opened in numeric order.  This
-ensures that the creation order for streams is consistent on both endpoints.
+Before creating a stream, all streams of the same type with lower-numbered
+stream IDs MUST be created.  This means that receipt of a frame that would open
+a stream causes all streams of the same type with lower-numbered stream IDs to
+be opened in numeric order.  This ensures that the creation order for streams is

Drop the middle sentence of this paragraph.

> @@ -640,19 +584,23 @@ STREAM_DATA_BLOCKED ({{frame-stream-data-blocked}}), and RST_STREAM
 
 A sender MUST NOT send any of these frames from a terminal state ("Data Recvd"
 or "Reset Recvd").  A sender MUST NOT send STREAM or STREAM_DATA_BLOCKED after
-sending a RST_STREAM; that is, in the "Reset Sent" state in addition to the
-terminal states.  A receiver could receive any of these frames in any state, but
-only due to the possibility of delayed delivery of packets carrying them.
+sending a RST_STREAM; that is, in the terminal states and in the "Reset Sent"
+state.
+
+A receiver could receive any of these three frames in any state, due to the

"these three frames" refers to something in another paragraph, so I think that your split should be reverted.

 The receiver of a stream sends MAX_STREAM_DATA ({{frame-max-stream-data}}) and
 STOP_SENDING frames ({{frame-stop-sending}}).
 
 The receiver only sends MAX_STREAM_DATA in the "Recv" state.  A receiver can
 send STOP_SENDING in any state where it has not received a RST_STREAM frame;
 that is states other than "Reset Recvd" or "Reset Read".  However there is
-little value in sending a STOP_SENDING frame after all stream data has been
-received in the "Data Recvd" state.  A sender could receive these frames in any
-state as a result of delayed delivery of packets.
+little value in sending a STOP_SENDING frame in the "Data Recvd" state, since
+all stream data has been received.
+
+A sender could receive either of these two frames in any state as a result of
+delayed delivery of packets.

Same as previous.

 # Flow Control {#flow-control}
 
-It is necessary to limit the amount of data that a sender may have outstanding
-at any time, so as to prevent a fast sender from overwhelming a slow receiver,
-or to prevent a malicious sender from consuming significant resources at a
-receiver.  To this end, QUIC employs a credit-based flow-control scheme similar
-to that in HTTP/2 {{?HTTP2}}.  A receiver advertises the number of bytes it is
-prepared to receive on a given stream and for the entire connection.  This leads
-to two levels of flow control in QUIC:
+It is necessary to limit the amount of data that a receiver may have to buffer,

```suggestion
It is necessary to limit the amount of data that a receiver could buffer,
```

 # Flow Control {#flow-control}
 
-It is necessary to limit the amount of data that a sender may have outstanding
-at any time, so as to prevent a fast sender from overwhelming a slow receiver,
-or to prevent a malicious sender from consuming significant resources at a
-receiver.  To this end, QUIC employs a credit-based flow-control scheme similar
-to that in HTTP/2 {{?HTTP2}}.  A receiver advertises the number of bytes it is
-prepared to receive on a given stream and for the entire connection.  This leads
-to two levels of flow control in QUIC:
+It is necessary to limit the amount of data that a receiver may have to buffer,
+so as to prevent a fast sender from overwhelming a slow receiver, or to prevent

```suggestion
to prevent a fast sender from overwhelming a slow receiver, or to prevent
```

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1950#pullrequestreview-171320303

---

From: Martin Thomson <[email-masked]>
Date: Sat, 03 Nov 2018 01:51:07 -0700

martinthomson approved this pull request.

-A receiver MAY advertise a larger offset at any point by sending MAX_STREAM_DATA
-or MAX_DATA frames.  A receiver cannot renege on an advertisement; that is, once
-a receiver advertises an offset, advertising a smaller offset has no effect.  A
-sender MUST therefore ignore any MAX_STREAM_DATA or MAX_DATA frames that do not
-increase flow control limits.
+with the Stream ID set appropriately.  A MAX_STREAM_DATA frame indicates the
+maximum absolute byte offset of a stream.  A receiver could use the current
+offset of data consumed to determine the flow control offset to be advertised.
+A receiver MAY send MAX_STREAM_DATA frames in multiple packets in order to make
+sure that the sender receives an update before running out of flow control
+credit, even if one of the packets is lost.
+
+A receiver advertises credit for a connection by sending a MAX_DATA frame, which
+indicates the maximum of the sum of the absolute byte offsets of all streams.  A
+receiver maintains a cumulative sum of bytes received on all contributing

```suggestion
receiver maintains a cumulative sum of bytes received on all
```

> @@ -831,29 +768,46 @@ larger resource commitments at the receiver.  Thus there is a trade-off between
 resource commitment and overhead when determining how large a limit is
 advertised.
 
-A receiver MAY use an autotuning mechanism to tune the frequency and amount that
-it increases data limits based on a round-trip time estimate and the rate at
-which the receiving application consumes data, similar to common TCP
-implementations.
+A receiver MAY use an autotuning mechanism to tune the frequency and amount of
+advertised additional credit.

The cut here might be too much.  I'd say either cut the whole paragraph, or keep it.  I think keeping it is best.

-If a sender runs out of flow control credit, it will be unable to send new
-data. That is, the sender is blocked. A blocked sender SHOULD send a
-STREAM_DATA_BLOCKED or DATA_BLOCKED frame.  A receiver uses these frames for
-debugging purposes.  A receiver MUST NOT wait for a STREAM_DATA_BLOCKED or
-DATA_BLOCKED frame before sending MAX_STREAM_DATA or MAX_DATA, since doing so
-will mean that a sender will be blocked for an entire round trip and the peer
-might never send a STREAM_DATA_BLOCKED or DATA_BLOCKED frame.
+If a sender runs out of flow control credit, it will be unable to send new data
+and is considered blocked.  It is generally considered best to not let the
+sender go into quiescence.  To avoid blocking a sender, and to reasonably

```suggestion
sender to become blocked.  To avoid blocking a sender, and to reasonably
```

-A sender sends a single STREAM_DATA_BLOCKED or DATA_BLOCKED frame only once when
-it reaches a data limit.  A sender SHOULD NOT send multiple STREAM_DATA_BLOCKED
-or DATA_BLOCKED frames for the same data limit, unless the original frame is
-determined to be lost.  Another STREAM_DATA_BLOCKED or DATA_BLOCKED frame can be
-sent after the data limit is increased.
+
+## Handling Stream Cancellation {#stream-cancellation}
+
+Given enough time, both endpoints must agree on flow control state, to avoid

Endpoints need to eventually agree on the amount of flow control credit that has been consumed, to avoid either exceeding limits or deadlocking.

> @@ -2762,6 +2719,28 @@ streams as necessary in outgoing packets without losing transmission efficiency
 to underfilled packets.

+## Frame Priority {#frame-priority}
+
+Often, there will be limits on what can be transmitted as a result of connection
+flow control or the current congestion controller state.
+
+Giving preference to the transmission of its own management functions ensures
+that a protocol functions efficiently.  That is, prioritizing frames other than
+STREAM frames ensures that loss recovery, congestion control, and flow control
+operate effectively.
+
+CRYPTO frames SHOULD be prioritized over STREAM frames prior to the completion
+of the cryptographic handshake.  This includes the retransmission of the second

You already effectively said this in the last paragraph, so I think that you might want to drop this paragraph.

> @@ -4838,9 +4816,11 @@ Stream Data:
 When a Stream Data field has a length of 0, the offset in the STREAM frame is
 the offset of the next byte that would be sent.
 
-The first byte in the stream has an offset of 0.  The largest offset delivered
-on a stream - the sum of the re-constructed offset and data length - MUST be
-less than 2^62.
+When new data is to be sent on a stream, a sender MUST set the encapsulating
+STREAM frame's offset field to the stream offset of the first byte of this new
+data.  The first byte in the stream has an offset of 0.  The largest offset
+delivered on a stream - the sum of the re-constructed offset and data length -

```suggestion
delivered on a stream - the sum of the offset and data length -
```

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1950#pullrequestreview-171320545

---

From: MikkelFJ <[email-masked]>
Date: Sat, 03 Nov 2018 01:59:11 -0700

mikkelfj commented on this pull request.

-There are two basic types of stream in QUIC.  Unidirectional streams carry data
-in one direction: from the initiator of the stream to its peer;
-bidirectional streams allow for data to be sent in both directions.  Different
-stream identifiers are used to distinguish between unidirectional and
-bidirectional streams, as well as to create a separation between streams that
-are initiated by the client and server (see {{stream-id}}).
+Streams in QUIC provide a lightweight, ordered byte-stream abstraction to an
+application. An alternative view of QUIC streams is as an elastic "message"
+abstraction, similar to the way ephemeral streams are used in SST
+{{?SST=DOI.10.1145/1282427.1282421}}, which may be a more appealing description

SST is a very dense description that may not be very helpful as an introduction for most users.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1950#pullrequestreview-171321256

---

From: MikkelFJ <[email-masked]>
Date: Sat, 03 Nov 2018 02:01:28 -0700

```suggestion
QUIC does not provide any means of ensuring ordering between bytes on different
```

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1950#pullrequestreview-171321289

---

From: MikkelFJ <[email-masked]>
Date: Sat, 03 Nov 2018 02:03:51 -0700

```suggestion
constraints (see {{flow-control}}) and a 62-bit stream offset limit.
```

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1950#pullrequestreview-171321341

---

From: janaiyengar <[email-masked]>
Date: Sat, 03 Nov 2018 02:07:45 -0700

janaiyengar commented on this pull request.

 to underfilled packets.

I think this paragraph is good to have since it is more specific. We can drop the previous paragraph, which is simply general advice.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1950#discussion_r230549347

---

From: janaiyengar <[email-masked]>
Date: Sat, 03 Nov 2018 02:08:43 -0700

e58c992  mt comments

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/1950/files/7f0dd505ab37f4b44271c15edb52c32abd0d73a7..e58c99220cdfe8fb1931c2c62b29661275ea00c2

---

From: MikkelFJ <[email-masked]>
Date: Sat, 03 Nov 2018 02:23:56 -0700

-Streams are identified by an unsigned 62-bit integer, referred to as the Stream
-ID.  Stream IDs are encoded as a variable-length integer (see
-{{integer-encoding}}).  The least significant two bits of the Stream ID are used
-to identify the type of stream (unidirectional or bidirectional) and the
-initiator of the stream.
+Streams are identified within a connection by a numeric value, referred to as
+the Stream ID.  Stream IDs are unique to a stream: a QUIC endpoint MUST NOT
+reuse a Stream ID within a connection.  Stream IDs are encoded as a
+variable-length integer (see {{integer-encoding}}).

plural integers

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1950#pullrequestreview-171321832

---

From: janaiyengar <[email-masked]>
Date: Sat, 03 Nov 2018 02:26:16 -0700

97294c8  more trimming

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/1950/files/e58c99220cdfe8fb1931c2c62b29661275ea00c2..97294c8a6b441dbd832b6406fd149b4d14f897c4

---

From: MikkelFJ <[email-masked]>
Date: Sat, 03 Nov 2018 02:28:48 -0700

+Stream IDs.  A Stream ID that is used out of order results in all streams of

```suggestion
Stream IDs.  A stream opened an out of order Stream ID results in all streams of
```
(using a stream does not necessarily imply opening it, and therefore also not lower numbered streams)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1950#pullrequestreview-171321951

---

From: MikkelFJ <[email-masked]>
Date: Sat, 03 Nov 2018 02:36:42 -0700

```suggestion
suboptimal application performance, and potentially starvation or deadlocks.
```

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1950#pullrequestreview-171322154

---

From: MikkelFJ <[email-masked]>
Date: Sat, 03 Nov 2018 02:41:13 -0700

I'm not sure how to formulate this, but there is a difference between the endpoints and view, including the state machine, and the overall distributed view: A stream can only be opened by the initiator, but the first materialized view of a frame can be receiver side sending a packet to a bidi stream. This does not mean that the receiver is opening the stream, but to the receiver it looks like the stream is being opened (although it already knows that is is open from receiving a higher valued Stream ID).

In short, I think it is wrong to state that receive side can open a stream, but the correct formulation is not clear to me.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1950#pullrequestreview-171322248

---

From: MikkelFJ <[email-masked]>
Date: Sat, 03 Nov 2018 02:44:27 -0700

> @@ -489,27 +431,30 @@ which can allow for better stream prioritization.

Perhaps a note here on loose coupling between application stream handles and QUIC internal stream ID's?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1950#pullrequestreview-171322333

---

From: janaiyengar <[email-masked]>
Date: Sat, 03 Nov 2018 03:09:32 -0700

Merged #1950 into master.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1950#event-1944320798

---

From: janaiyengar <[email-masked]>
Date: Sat, 03 Nov 2018 03:38:00 -0700

@mikkelfj : Sorry, missed your comments when I merged my PR. I'll take them in separately.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1950#issuecomment-435577473

---

From: janaiyengar <[email-masked]>
Date: Sat, 03 Nov 2018 03:45:36 -0700

Fair point, though as you note, it's not clear exactly what to replace this with. Feel free to suggest a PR for it if you come up with something here.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1950#discussion_r230551381