Subject: Re: [quicwg/base-drafts] introduce a max_connection_ids transport parameter (#1998)

From: Marten Seemann <[email-masked]>
Date: Wed, 06 Feb 2019 19:04:49 -0800

@marten-seemann pushed 1 commit.

95a13b37bfe63289a7c67985857e27eaeed99944  remove the CONNECTION_ID_LIMIT_ERROR error code

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/1998/files/b43ca5b505697373a46bb789f1080be3f4e21d40..95a13b37bfe63289a7c67985857e27eaeed99944

---

From: Marten Seemann <[email-masked]>
Date: Wed, 06 Feb 2019 19:04:58 -0800

I removed the error code. PHAL.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1998#issuecomment-461272578

---

From: Martin Thomson <[email-masked]>
Date: Wed, 06 Feb 2019 19:12:06 -0800

martinthomson commented on this pull request.

> -unused connection IDs.  While each endpoint independently chooses how many
-connection IDs to issue, endpoints SHOULD provide and maintain at least eight
-connection IDs.  The endpoint SHOULD do this by always supplying a new
-connection ID when a connection ID is retired by its peer or when the endpoint
-receives a packet with a previously unused connection ID.  Endpoints that
-initiate migration and require non-zero-length connection IDs SHOULD provide
-their peers with new connection IDs before migration, or risk the peer closing
-the connection.
+unused connection IDs. Endpoints store received connection IDs for future use.
+An endpoint uses a transport parameter to advertise the number of connection IDs
+it can store for future use. An endpoint SHOULD NOT provide more connection IDs
+than this limit. If an endpoint has provided its peer with the maximum number of
+connection IDs, it SHOULD only provide a new connection ID when the peer retires
+one. Endpoints that initiate migration and require non-zero-length connection
+IDs SHOULD provide their peers with new connection IDs before migration, or risk
+the peer closing the connection.

There are a lot of SHOULDs here, but it's a bit light on the associated justification.  I think that we'll need some more text about why an endpoint might choose not to follow this advice.  Having everything in the one paragraph will make that difficult.

> @@ -4106,6 +4102,12 @@ A client MUST NOT include an original connection ID, a stateless reset token, or
 a preferred address.  A server MUST treat receipt of any of these transport
 parameters as a connection error of type TRANSPORT_PARAMETER_ERROR.
 
+max_connection_ids (0x000e):
+
+: The maximum number of connection IDs that the peer is willing to store.
+  This value includes only connection IDs sent in NEW_CONNECTION_ID frames.
+  If this parameter is absent, a default of 0 is assumed.

I'd really like to be able to not send this.  Is 0 really the right answer?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1998#pullrequestreview-200906639

---

From: erickinnear <[email-masked]>
Date: Wed, 06 Feb 2019 19:16:27 -0800

erickinnear commented on this pull request.

We had a recommended value previously, let's pull that value over and have the default value conversation separately? 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1998#discussion_r254535453

---

From: Marten Seemann <[email-masked]>
Date: Wed, 06 Feb 2019 19:26:49 -0800

marten-seemann commented on this pull request.

We discussed this before in https://github.com/quicwg/base-drafts/pull/1998#discussion_r233121475. We're using 0 as the default value for all flow control and stream limit values, so at least this seems consistent.

In Tokyo we had disagreement about what a reasonable default should be. @ianswett advocated for 2 or 3, if I remember correctly, while other people want 8. One reason for moving forward with this PR was exactly the fact that people have different use cases in mind, and there's no one-size-fits-all value.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1998#discussion_r254536703

---

From: Christian Huitema <[email-masked]>
Date: Wed, 06 Feb 2019 19:47:23 -0800

huitema commented on this pull request.

> @@ -3981,6 +3979,12 @@ A client MUST NOT include an original connection ID, a stateless reset token, or

If a server has sent a CID as part of SPA, what sequence number does it use for the next NCID?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1998#discussion_r254538947

---

From: Christian Huitema <[email-masked]>
Date: Wed, 06 Feb 2019 19:48:59 -0800

What if an endpoint set both "migration disabled = 0" (default) and "max CID=0" (default)? Should that trigger a protocol error?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1998#discussion_r254539120

---

From: Christian Huitema <[email-masked]>
Date: Wed, 06 Feb 2019 19:52:28 -0800

By the way, we already use no zero defaults for some parameters: ack_delay_exponent defaults to 3, and max_ack_delay defaults to 25. There is no rule that says that forbids us to set a non zero default for the proposed MAX CID parameter. I would think that 3 is a good number.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1998#discussion_r254539518

---

From: Marten Seemann <[email-masked]>
Date: Wed, 06 Feb 2019 19:52:42 -0800

@huitema I think this is unrelated to this PR. We have text about that in https://quicwg.org/base-drafts/draft-ietf-quic-transport.html#issue-cid.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1998#discussion_r254539549

---

From: Christian Huitema <[email-masked]>
Date: Wed, 06 Feb 2019 19:57:03 -0800

It is *not* unrelated. The sequence number is a direct statement on the number of CID out there. I would very much like to have just one number to test. Say, state "sequence number should be lower than the number of retired CID plus the max number of CID accepted by the peer".

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1998#discussion_r254540015

---

From: ianswett <[email-masked]>
Date: Thu, 07 Feb 2019 09:05:48 -0800

ianswett commented on this pull request.

I think the restriction you're suggesting is a nice simplification Christian.

I'd also like to close the connection if the limit is exceeded, but I know there was some opposition to that in Tokyo that I didn't fully understand.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1998#discussion_r254780743

---

From: Mike Bishop <[email-masked]>
Date: Thu, 07 Feb 2019 10:53:19 -0800

MikeBishop commented on this pull request.

In HTTP, we use zero defaults for everything because those values apply until the SETTINGS frame arrives.  In QUIC, however, you can't send anything until transport parameters are in hand.  Any value can be a reasonable default.  (Note that the requirement not to reduce after accepting 0-RTT will apply here, though.)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1998#discussion_r254820903

---

From: Mike Bishop <[email-masked]>
Date: Thu, 07 Feb 2019 18:59:11 +0000 (UTC)

Iff you count the handshake and SPA CIDs, an overflow just from transport parameters is possible if set to 0/1, as @marten-seemann pointed out earlier.  Overflows are also possible based on the advice to replace upon seeing a new CID enter use, though I think that's best addressed by just not recommending/allowing that.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1998#discussion_r254823526

---

From: Christian Huitema <[email-masked]>
Date: Thu, 07 Feb 2019 20:49:12 +0000 (UTC)

@MikeBishop yes indeed, the server always sends at least 1 CID, so a limit of 0 appears to break. But this is easily worked around by only applying the test to NCID frames. This is easier than special-casing the handling of CID[0] and CID[1]/SPA. In particular, the logic for handling RETIRE_CID and creation of new NCID becomes much simpler.

As I mentioned before, a default of 0 does not work very well, because the default for migration disabled is also 0 -- yet enabling migration without any NCID would be a protocol violation. The default should be at least 2 (initial and SPA), and I would prefer 3.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1998#discussion_r254862789

---

From: Martin Thomson <[email-masked]>
Date: Thu, 07 Feb 2019 18:58:48 -0800

This is one where reducing after 0-RTT is perfectly OK.  It's not a hard limit, it's more like a guideline.  We should allow an exception here.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1998#discussion_r254943260

---

From: MikkelFJ <[email-masked]>
Date: Fri, 08 Feb 2019 00:12:47 -0800

Just throwing in a comment I made on the rate limiting PR: maintaining more CID's than necessary can have performance impacts: https://github.com/quicwg/base-drafts/pull/2428#issuecomment-460951783 therefore I'm in favour of lowering from 8. I'm less concerned with the default being 0 or 2.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1998#issuecomment-461723518

---

From: Mike Bishop <[email-masked]>
Date: Fri, 08 Feb 2019 21:07:55 +0000 (UTC)

True -- we have an exemption for the one HTTP setting which is purely advisory.  Since this is a suggestion, reducing it wouldn't necessarily hurt anything.  (But there's no prohibition on the sender to not blow up on exceeding the limit, either.)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1998#discussion_r255235836

---

From: Martin Thomson <[email-masked]>
Date: Tue, 12 Feb 2019 15:28:57 -0800

To summarize state: we should pick a better default (3, 4, 8, I don't care).  And make an explicit exception to the "must not decrease" rule for 0-RTT.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1998#issuecomment-462984327

---

From: Kazuho Oku <[email-masked]>
Date: Tue, 12 Feb 2019 17:01:45 -0800

kazuho commented on this pull request.

I think what @janaiyengar is pointing out is that "the _peer_ is willing to store" needs to be changed to "the _endpoint_ is willing to store".

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1998#discussion_r256211254

---

From: Kazuho Oku <[email-masked]>
Date: Tue, 12 Feb 2019 18:47:16 -0800

@martinthomson 
> To summarize state: we should pick a better default (3, 4, 8, I don't care). And make an explicit exception to the "must not decrease" rule for 0-RTT.

Maybe we could recommend ignoring the value recovered from the ticket. Endpoints can wait for the transport parameter exchanged during the handshake, then send NEW_CONNECTION_ID frames.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1998#issuecomment-463034065

---

From: Martin Thomson <[email-masked]>
Date: Tue, 12 Feb 2019 18:57:08 -0800

@kazuho 

> Maybe we could recommend ignoring the value recovered from the ticket.

SGTM.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1998#issuecomment-463036124

---

From: martinduke <[email-masked]>
Date: Wed, 13 Feb 2019 08:10:29 -0800

martinduke commented on this pull request.

Isn't the number of stored connection IDs symmetric? In practice, each address change requires both sides to change their CIDs (except for zero-length cases).

I'm not sure I see any point in ever storing more than min(my_max_connection_ids, your_max_connection_ids)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1998#discussion_r256467839

---

From: martinduke <[email-masked]>
Date: Wed, 13 Feb 2019 08:12:32 -0800

I am a little hazy on the proper behavior for NAT rebinding vs. migration, but how is communicating a zero here different from disable_migration? Do we still need that TP?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1998#discussion_r256468778

---

From: Mike Bishop <[email-masked]>
Date: Wed, 13 Feb 2019 10:53:21 -0800

Yes, because you might want to rotate CIDs on a connection that isn't actively migrating.  Whether you'll survive NAT rebinding depends on the server infrastructure -- a 2-tuple load balancer will survive a port change, but a 4-tuple load balancer can't guarantee it.  An IP+port change isn't guaranteed to survive on anything but a CID-aware load balancer.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1998#discussion_r256538312

---

From: Martin Thomson <[email-masked]>
Date: Wed, 13 Feb 2019 17:08:10 -0800

The need for reciprocity in most cases that consume a connection ID does tend to lead to the numbers being approximately the same.  But it's not symmetric in the sense that they need to be equal.  The design is therefore asymmetric.  

If you consider a client that is moving between networks of questionable quality, they can burn connection IDs on futile probes, leading to greater consumption of connection IDs.  That suggests that a client might store more spares than a server.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1998#discussion_r256655813

---

From: Jana Iyengar <[email-masked]>
Date: Thu, 21 Feb 2019 19:16:35 -0800

@marten-seemann : can you update the PR based on the discussion?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1998#issuecomment-466258704

---

From: Kazuho Oku <[email-masked]>
Date: Thu, 21 Feb 2019 19:24:23 -0800

FWIW it would be easier to clarify that endpoints should not retain the max_connection_ids TP when resuming a connection once #2467 gets merged.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1998#issuecomment-466259901

---

From: Marten Seemann <[email-masked]>
Date: Fri, 22 Feb 2019 09:16:38 +0000 (UTC)

Thanks @kazuho, I've been waiting for this PR to land.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1998#issuecomment-466329182