Subject: Re: [quicwg/base-drafts] Client connection IDs are broken (#2844)

From: Benjamin Saunders <[email-masked]>
Date: Mon, 08 Jul 2019 14:53:08 -0700

Would it make sense to solve this by placing restrictions on QUIC configuration at the application protocol level? E.g. forbid HTTP/3 servers from using zero-length CIDs. That avoids forcing increased header size on other application protocols which may have no need for the type of proxy described here.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2844#issuecomment-509406216

---

From: David Schinazi <[email-masked]>
Date: Mon, 08 Jul 2019 21:55:27 +0000 (UTC)

Connection IDs are a property of the transport layer, and the proxy use-case described above uses HTTP/3. I don't think we should vary any of this by application.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2844#issuecomment-509406818

---

From: David Schinazi <[email-masked]>
Date: Tue, 09 Jul 2019 13:47:22 -0700

I've updated the associated PR #2851 in case anyone following here would like to go check it.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2844#issuecomment-509801053

---

From: Igor Lubashev <[email-masked]>
Date: Mon, 22 Jul 2019 19:37:48 -0700

What do people think about allowing the server to use a zero-length connection ID only if the client chose to use a zero-length connection ID?

By selecting a zero-length connection ID, the client is indicating it will not reuse the port. So the server can rely on just the source ip and port to identify the connection.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2844#issuecomment-514029964

---

From: Benjamin Saunders <[email-masked]>
Date: Mon, 22 Jul 2019 19:45:56 -0700

That seems better than forbidding it outright, at least.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2844#issuecomment-514031468

---

From: David Schinazi <[email-masked]>
Date: Tue, 23 Jul 2019 05:57:59 -0700

@igorlord That doesn't work. As I client we use zero-length connection IDs and migration. The server cannot use our address and port as that would break connection migration and NAT rebinding.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2844#issuecomment-514198308

---

From: Benjamin Saunders <[email-masked]>
Date: Tue, 23 Jul 2019 07:08:53 -0700

If a server wishes to support migration/rebinding, it could still choose to use non-zero-length CIDs, as today.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2844#issuecomment-514226155

---

From: Lars Eggert <[email-masked]>
Date: Tue, 23 Jul 2019 08:22:09 -0700

Discussed in Montreal, @DavidSchinazi to revise the PR as discussed and then we'll attempt to see if there is consensus

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2844#issuecomment-514258095

---

From: Martin Thomson <[email-masked]>
Date: Tue, 23 Jul 2019 08:27:31 -0700

Discussed in Montreal.  Some concern about enabling a TCP-like usage for QUIC, but concerns about the effect that a choice of an endpoint could have on its peer.

Options are:

1. MUST use only local information (local host, port, cid) to route packets.

2. SHOULD, with qualification, but leaving this open to people to shoot each other in the feet.

3. MUST unless you know something about what your peers (as you identify them) are doing.  This precludes peers that attempt to use connection IDs if you intend to use source-address-based routing.

I think that we might want to end up in the third bucket.  Which acknowledges and endorses the use case, but properly defines the conditions under which this is acceptable.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2844#issuecomment-514260400

---

From: David Schinazi <[email-masked]>
Date: Tue, 23 Jul 2019 08:56:13 -0700

I've updated the associated PR #2851 to make this a SHOULD, and explain that it MUST NOT be violated unless an endpoint knows that these features are not in use.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2844#issuecomment-514272371

---

From: Benjamin Saunders <[email-masked]>
Date: Tue, 23 Jul 2019 09:13:26 -0700

The new text addresses my concerns 👍 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2844#issuecomment-514279217

---

From: martinduke <[email-masked]>
Date: Tue, 23 Jul 2019 17:17:03 -0700

At the risk of tying this in with other issues, there are two separate problems here.
1) Migration: if I change address, I have no means of expressing that it is the same connection. On the other hand "DISABLE_MIGRATION" promises that things will fall apart if the address changes. In that event I am not sure why we would force the server to do anything.
2) Multiplexing. I agree that this case does not work if the server has no CID. But are servers required to support the same-port case? As someone that will be stuck with a 4-tuple based infrastructure for at least another year, trying to handle multiple connections on one 4-tuple is quite laborious and I'm not likely to support it in the near term. I'd much rather spend the time fixing my internal routing rather than band-aiding this use case.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2844#issuecomment-514429604

---

From: MikkelFJ <[email-masked]>
Date: Tue, 23 Jul 2019 23:46:55 -0700

2. multiplexing:

I'm not sure if same port should always be supported, but aside from your temporary difficulties, it is highly valuable to servers with the opposite view: band-aiding ephemeral hosts when it is much simpler to just multiplex on the raw packet based on a connection ID hash. A lot of infrastructure is artificially limited by the number of ephemeral ports available. Getting rid of this problem requires getting rid of ephemeral ports.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2844#issuecomment-514503241

---

From: David Schinazi <[email-masked]>
Date: Fri, 26 Jul 2019 11:26:50 -0700

@martinduke I agree that there are two separate issues.

My opinion is that we should require servers to support clients multiplexing multiple connections on the same port. It's OK if your routing infrastructure only looks at the four-tuple, but the end-server than receives the packet needs to demultiplex based on the server connection ID.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2844#issuecomment-515554237

---

From: Igor Lubashev <[email-masked]>
Date: Fri, 26 Jul 2019 12:03:38 -0700

@DavidSchinazi Note that supporting multiplexing clients does not mean always using a server CID. A server that does not support client migration may decide to use a CID only if the client is using non-zero-length CID. (And there is little reason for a client CID other than for multiplexing.)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2844#issuecomment-515565411

---

From: David Schinazi <[email-masked]>
Date: Fri, 26 Jul 2019 12:08:07 -0700

@igorlord As a server you should not use zero-length connection IDs unless you can demultiplex locally based on your own IP and port. Violating that breaks several protocol features - see #2851 .

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2844#issuecomment-515566614

---

From: Igor Lubashev <[email-masked]>
Date: Fri, 26 Jul 2019 12:33:45 -0700

@DavidSchinazi I obviously disagree with this (and we had a discussion about this in Montreal). Support for connection migration is optional. Hence, if the server does not support connection migration or knows that a particular client will not migrate, there is no need for a server to supply a non-zero-length CID, when such a client supplies a zero-length CID.

Which protocol feature is missing when the client will not migrate, if the server is using a 4-tuple to demux and there is a zero-length client and server CID?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2844#issuecomment-515573666

---

From: Marten Seemann <[email-masked]>
Date: Fri, 26 Jul 2019 12:49:55 -0700

@igorlord A client that is using a single UDP connection would not be able two establish two connections to such a server at the same time. Even worse, it would have no way of knowing if it’s safe to dial two connections concurrently.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2844#issuecomment-515578003

---

From: Igor Lubashev <[email-masked]>
Date: Fri, 26 Jul 2019 12:57:37 -0700

@marten-seemann such client would not use a zero-length client CID, do the argument is moot.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2844#issuecomment-515580181

---

From: Marten Seemann <[email-masked]>
Date: Fri, 26 Jul 2019 13:15:33 -0700

@igorlord The client would not use zero-length CIDs for *receiving* packets. The problem occurs when the server uses zero-length CIDs, since it won't be able to distinguish between the two connections.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2844#issuecomment-515585239

---

From: David Schinazi <[email-masked]>
Date: Fri, 26 Jul 2019 13:22:03 -0700

+1 to what @marten-seemann said

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2844#issuecomment-515587290

---

From: Igor Lubashev <[email-masked]>
Date: Fri, 26 Jul 2019 15:26:42 -0700

@marten-seemann So we are saying the same thing. I do not see where you are disagreeing.  I said, verbatim:

> if the server does not support connection migration or knows that a particular client will not migrate, there is no need for a server to supply a non-zero-length CID, *when such a client supplies a zero-length CID*.

Both of us are agreeing that a client who is interested in multiplexing will *not* chose to use a zero-length client CID. Hence the server will also not use a zero-length server CID.

But when clients chose a zero-length client CID (making they will not multiplex on that 4-tuple), the server should be free to choose a zero-length server CID, if it does not support connection migration.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2844#issuecomment-515617560

---

From: David Schinazi <[email-masked]>
Date: Fri, 26 Jul 2019 15:43:00 -0700

@igorlord How does that work in practice? You can't use both zero-length and non-zero-length connection IDs on the same server without trial decryption. What happens when your first client uses zero-length-client-CIDs and you second one doesn't?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2844#issuecomment-515620497

---

From: Igor Lubashev <[email-masked]>
Date: Fri, 26 Jul 2019 15:57:17 -0700

@DavidSchinazi You can use both on the same server, and no trial decryption is required.

```
Map < 4-tuple , connection_ctx > no_cid_connections_by_4_tuple;
Map < CID , connection_ctx > connections_by_cid;
```

The the incoming 4-tuple is found in `no_cid_connections_by_4_tuple`, use that connection_ctx.  Otherwise, assume there is a CID and look up in `connections_by_cid`.  That's all.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2844#issuecomment-515622963

---

From: David Schinazi <[email-masked]>
Date: Fri, 26 Jul 2019 16:01:48 -0700

@igorlord thanks for clarifying. I have to admit this approach does seem convoluted. What's the benefit of this over non-zero-length server connection IDs? (they have the advantage of surviving NAT rebinding)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2844#issuecomment-515623740

---

From: Igor Lubashev <[email-masked]>
Date: Fri, 26 Jul 2019 16:16:33 -0700

@DavidSchinazi **Short answer**: saving bytes on the wire.

**Long answer**:

A server may need to support both regular clients (so using a server CID) as well as peer servers in the datacenter (or some SDN). It may be trivial to identify such peers (anyone using 10.0.0.0/8), so why waste CID in each packet when talking to 10.0.0.0/8 machines?  Moreover, QUIC now supports huge Server CIDs (up to 20 bytes), and if that's what's needed for regular clients, that's wasting 20 bytes per packet on internal communications.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2844#issuecomment-515626185

---

From: Kazuho Oku <[email-masked]>
Date: Fri, 26 Jul 2019 17:46:30 -0700

@igorlord I think no one would stop you from doing something like what you describe in [your comment](https://github.com/quicwg/base-drafts/issues/2844#issuecomment-515622963).

However, most of us do not want to be forced do something as complex as that. Therefore, we are trying to clarify when the 4-tuple can be used by the server for identifying the connections.

If it is your intent is to ensure that your mixed approach is allowed _as one of the approaches_, then I think we can clarify that in #2851 (I think it's already implied).

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2844#issuecomment-515637483

---

From: Igor Lubashev <[email-masked]>
Date: Fri, 26 Jul 2019 19:09:58 -0700

@kazuho This discussion is about not arbitrarily forbidding something that is useful. Of course different implementations will chose to do what they need to do. Once the implementation is deployed at scale, people tend to look for ways to find even 1% improvements.

I am puzzled why you think it is "complex", though.

Is it that much more complex to do

```connection = by_4tuple[pkt.4tuple()] ? : by_cid[pkt.cid()];```

than

```connection b by_cid[pkt.cid()];```

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2844#issuecomment-515643427

---

From: Kazuho Oku <[email-masked]>
Date: Fri, 26 Jul 2019 23:31:58 -0700

@igorlord 
> This discussion is about not arbitrarily forbidding something that is useful. 

Thank you for the clarification. Assuming that we can give clients the freedom coalesce multiple connections onto a single 5-tuple, I do not have a strong argument against giving servers some wiggle room.

> `connection = by_4tuple[pkt.4tuple()] ? : by_cid[pkt.cid()];`
> 
> than
> 
> `connection = by_cid[pkt.cid()];`

I think it is more complex than that.

The approach you are describing can be modeled as a daisy-chain of two QUIC servers.

What would the server do if there is no corresponding state for a short header packet that you receive, neither in `by_4tuple` nor `by_cid`?

The server needs to send a stateless reset. But because the server cannot tell if the packet were to be identified by the 4-tuple or the CID (because you no longer have state!), the server needs to send two stateless reset packets, one generated using the 4-tuple as the key, and the other generated using the CID. However, simply doing that could lead to an amplification attack (as we know, the amount of data (or packets) the server can send as a reset needs to be smaller than the amount of data that the server has received).

This _might_ be a problem htat we want to solve. But is it worth spending the time of the WG to discuss such thing? I'm not sure.

And I'm not sure if this is the only problem that exists in the proposed design.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2844#issuecomment-515657133

---

From: MikkelFJ <[email-masked]>
Date: Sat, 27 Jul 2019 01:04:05 -0700

> However, simply doing that could lead to an amplification attack (as we
know, the amount of data (or packets) the server can send as a reset needs
to be smaller than the amount of data that the server has received).

The main reason for this is not amplification but because it could lead to
infinite reset loops. This could be by mistake, or by deliberate injection
of packets to trigger this behaviour.

Mikkel

On 27 July 2019 at 08.31.52, Kazuho Oku ([email-masked]) wrote:

However, simply doing that could lead to an amplification attack (as we
know, the amount of data (or packets) the server can send as a reset needs
to be smaller than the amount of data that the server has received).

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2844#issuecomment-515664068

---

From: Igor Lubashev <[email-masked]>
Date: Mon, 29 Jul 2019 10:56:14 -0700

@kazuho 

> But because the server cannot tell if the packet were to be identified by the 4-tuple or the CID (because you no longer have state!), the server needs to send two stateless reset packets, one generated using the 4-tuple as the key, and the other generated using the CID.

This is a good point that Stateless Reset must also be considered.  In case that the server can identify a certain population of clients (10.0.0.0/8) as not needing migration, it can also identify which clients to send what kind of Stateless Reset.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2844#issuecomment-516096764