Subject: Re: [quicwg/base-drafts] Add a section on flow control performance (#3793)

From: Martin Thomson <[email-masked]>
Date: Tue, 07 Jul 2020 15:50:06 -0700

@martinthomson pushed 1 commit.

1804018ab832a872b35e479b795863cb9241b095  Remove problematic paragraph

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/3793/files/c040125303cbaa88054c7359aaa7d6ea3075416d..1804018ab832a872b35e479b795863cb9241b095

---

From: krasic <[email-masked]>
Date: Tue, 07 Jul 2020 16:24:35 -0700

On Fri, Jun 26, 2020, 12:31 AM Martin Thomson <[email-masked]>
wrote:

> QUIC doesn't really address the question of tuning flow control very
> well. This just admits that fact.
>
> It also indicates why: which essentially boils down to "RAM is cheap".
>
Hi Martin. I stumbled across this.  I don't know if you are joking, but if
not I'd like to disagree.  There are a significant class of devices,
commodity streaming devices, which are very large volume but for which the
trend is to shave as much RAM budget as possible, HD video devices in
markets like India, Brazil, etc.  In aggregate, these devices consume a
significant amount of overall internet traffic.    RAM is cheap is most
definitely not true there, and they certainly don't ship with TCP receive
windows anywhere near as large as other device segments.  This is not a
legacy issue either, for the foreseeable future it looks like the amount of
RAM in newer devices will be lucky to stay stable as opposed to even
decrease.

> Closes #3722 <https://github.com/quicwg/base-drafts/issues/3722>.
> ------------------------------
> You can view, comment on, or merge this pull request online at:
>
>   https://github.com/quicwg/base-drafts/pull/3793
> Commit Summary
>
>    - Add a section on flow control performance
>
> File Changes
>
>    - *M* draft-ietf-quic-transport.md
>    <https://github.com/quicwg/base-drafts/pull/3793/files#diff-db016291106766877c4921a79f8596e0>
>    (18)
>
> Patch Links:
>
>    - https://github.com/quicwg/base-drafts/pull/3793.patch
>    - https://github.com/quicwg/base-drafts/pull/3793.diff
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/quicwg/base-drafts/pull/3793>, or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ABRABBQI2QR6RBOI65V4JCLRYRFDHANCNFSM4OJCIA6A>
> .
>

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3793#issuecomment-655190036

---

From: Mike Bishop <[email-masked]>
Date: Wed, 08 Jul 2020 14:04:39 -0700

@MikeBishop approved this pull request.

> @@ -968,6 +968,17 @@ signal before advertising additional credit, since doing so will mean that the
 peer will be blocked for at least an entire round trip, and potentially for
 longer if the peer chooses to not send STREAMS_BLOCKED frames.
 
+## Flow Control Performance
+
+An endpoint that is unable to ensure that a peer has flow control credit in the

```suggestion
An endpoint that is unable to ensure that a peer has flow control credit on the
```

> @@ -968,6 +968,17 @@ signal before advertising additional credit, since doing so will mean that the
 peer will be blocked for at least an entire round trip, and potentially for
 longer if the peer chooses to not send STREAMS_BLOCKED frames.
 
+## Flow Control Performance
+
+An endpoint that is unable to ensure that a peer has flow control credit in the
+order of the current BDP will have receive throughput limited by flow control
+and not other limiting factors like congestion control.  Timely sending of
+updates to flow control limits can improve performance, however an excessive

```suggestion
updates to flow control limits can improve performance. However, an excessive
```

> @@ -968,6 +968,24 @@ signal before advertising additional credit, since doing so will mean that the
 peer will be blocked for at least an entire round trip, and potentially for
 longer if the peer chooses to not send STREAMS_BLOCKED frames.
 
+## Flow Control Performance
+
+Unlike TCP, QUIC decouples flow control from congestion control. This can
+release pressure on implementing a good flow control algorithm that governs both
+how limits are increased and when new limits are advertised. An endpoint can
+use flow control to constrain resource commitments without an optimized
+algorithm by allocating buffers for receiving data that are significantly larger
+than the bandwidth-delay product (BDP).  That is, by a factor of 2 or more.
+
+An endpoint that is unable to ensure that a peer has flow control credit in the
+order of the current BDP will have receive throughput limited by flow control
+and not other limiting factors like congestion control.  Timely sending of
+updates to flow control limits can improve performance, however an excessive
+rate of updates can also adversely affect performance.

I suspect "excessive" means "more frequently than ACKs and/or in their own packets."  Should we say that?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3793#pullrequestreview-445125932

---

From: Martin Thomson <[email-masked]>
Date: Wed, 08 Jul 2020 18:21:20 -0700

36b91f1afd0e4be4774e4f307556068ac2138d14  Tweaks

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/3793/files/1804018ab832a872b35e479b795863cb9241b095..36b91f1afd0e4be4774e4f307556068ac2138d14

---

From: ianswett <[email-masked]>
Date: Thu, 09 Jul 2020 07:01:18 -0700

@ianswett commented on this pull request.

Congestion controllers sometimes do silly things, so I'd simplify this, but add an additional consideration.

```suggestion
order of the current BDP will have receive throughput limited by flow control.
Lost packets can cause gaps in the receive buffer, delaying the application
from consuming data and freeing up flow control window. Timely sending of
```

> +updates to flow control limits can improve performance. However, an excessive

```suggestion
updates to flow control limits can improve performance. An excessive
rate of updates sent in their own packet increases network load and
can adversely affect performance.  Bundling updates with ACK
frames provides frequent updates without adding significant load.
```

+## Flow Control Performance
+
+An endpoint that is unable to ensure that a peer has flow control credit on the
+order of the current BDP will have receive throughput limited by flow control
+and not other limiting factors like congestion control.  Timely sending of
+updates to flow control limits can improve performance. However, an excessive
+rate of updates can also adversely affect performance.
+
+This document does not specify techniques for tuning flow control performance

I don't think this paragraph is necessary, but that's me.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3793#pullrequestreview-445637439

---

From: Martin Thomson <[email-masked]>
Date: Thu, 09 Jul 2020 17:21:05 -0700

@martinthomson commented on this pull request.

Should we instead say "Sending of packets just to provide flow control updates can increase network load and adversely affect performance.  Sending flow control updates along with other frames, such as ACK frames, can reduce the cost of those updates."

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3793#discussion_r452558379

---

From: ianswett <[email-masked]>
Date: Fri, 10 Jul 2020 06:43:07 -0700

SGTM, there's no reason it only needs to be ACK frames.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3793#discussion_r452851573

---

From: Martin Thomson <[email-masked]>
Date: Mon, 13 Jul 2020 18:43:47 -0700

b4faf1b2099262ec466b830548555c970ece47e9  Add note about gaps in receive buffers

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/3793/files/36b91f1afd0e4be4774e4f307556068ac2138d14..b4faf1b2099262ec466b830548555c970ece47e9

---

From: Martin Thomson <[email-masked]>
Date: Mon, 13 Jul 2020 18:48:24 -0700

df7fe2664621d48bbd7e01802d9897af6a3e9d83  Negotiated changes, plus a split of a large paragraph

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/3793/files/b4faf1b2099262ec466b830548555c970ece47e9..df7fe2664621d48bbd7e01802d9897af6a3e9d83

---

From: ianswett <[email-masked]>
Date: Tue, 14 Jul 2020 15:50:07 -0700

@ianswett approved this pull request.

> @@ -969,6 +969,19 @@ peer will be blocked for at least an entire round trip, and potentially for

+## Flow Control Performance
+
+An endpoint that is unable to ensure that a peer has flow control credit on the
+order of the current BDP will have receive throughput limited by flow control.
+Lost packets can cause gaps in the receive buffer, delaying the application
+from consuming data and freeing up flow control window.
+
+Timely sending of updates to flow control limits can improve performance.

```suggestion
Sending timely updates of flow control limits can improve performance.
```

> @@ -969,6 +969,19 @@ peer will be blocked for at least an entire round trip, and potentially for

+## Flow Control Performance
+
+An endpoint that is unable to ensure that a peer has flow control credit on the
+order of the current BDP will have receive throughput limited by flow control.
+Lost packets can cause gaps in the receive buffer, delaying the application
+from consuming data and freeing up flow control window.
+
+Timely sending of updates to flow control limits can improve performance.
+Sending of packets just to provide flow control updates can increase network

```suggestion
Sending packets only to provide flow control updates can increase network
```

> @@ -969,6 +969,19 @@ peer will be blocked for at least an entire round trip, and potentially for

+## Flow Control Performance
+
+An endpoint that is unable to ensure that a peer has flow control credit on the
+order of the current BDP will have receive throughput limited by flow control.
+Lost packets can cause gaps in the receive buffer, delaying the application
+from consuming data and freeing up flow control window.
+
+Timely sending of updates to flow control limits can improve performance.
+Sending of packets just to provide flow control updates can increase network
+load and adversely affect performance. Sending flow control updates along with
+other frames, such as ACK frames, can reduce the cost of those updates.

```suggestion
other frames, such as ACK frames, reduces the cost of those updates.
```

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3793#pullrequestreview-448517737

---

From: Martin Thomson <[email-masked]>
Date: Tue, 14 Jul 2020 15:51:26 -0700

e504e117d8b93ac9ea075fb3546eba4c3f7348b7  Wording tweaks

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/3793/files/df7fe2664621d48bbd7e01802d9897af6a3e9d83..e504e117d8b93ac9ea075fb3546eba4c3f7348b7

---

From: Martin Thomson <[email-masked]>
Date: Tue, 14 Jul 2020 15:51:54 -0700

Merged #3793 into master.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3793#event-3546208989