Subject: [quicwg/base-drafts] Long Header Packets and Routing Connection IDs (#2834)

From: Nick Banks <[email-masked]>
Date: Sun, 23 Jun 2019 08:42:31 -0700

I discussed this on Slack and opened a related issue on the [QUIC-LB doc](https://github.com/martinduke/draft-duke-quic-load-balancers/issues/43), but I think the real issue needs to be discussed here. The issue here is how a server (it's LB specifically) is supposed to statelessly route Long Header packets. Right now, the client immediately changes the destination CID it puts in its packets to the server chosen CID. This results in all Initial and Handshake packets it sends after the first (successful) flight of packets are received and acknowleded/responded to by the server.

So, practically, that means Long Header packets could have one of two types of CIDs: client chosen or server chosen. To prevent attack vectors from being exposed, the LB practically needs to use different routing logic for each type of CID. We don't want an attacker to explicitly control which backend server it creates a new connection to. The problem arrises because we have no way to differentiate which type of CID is included in the Long Header packets processed by the LB.

**Proposal**: All Long Header packets use the client chosen CID, to allow for consistent routing of these packets.

Now, if we agree that the CID doesn't change until Short Header packets are used, then that means we don't have to use the current implicit mechanism for changing the CID in response to the first server packet received. We can instead exchange it, encrypted with the Handshake keys; possibly a new transport parameter; or just the first NEW_CONNECTION_ID.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834

---

From: MikkelFJ <[email-masked]>
Date: Sun, 23 Jun 2019 09:17:37 -0700

>  All Long Header packets use the client chosen CID, to allow for consistent routing of these packets.

That would prevent a load balancer from balancing load to a specific server that handles the handshake. It would also allow a client to decide where state is managed during a handshake, to a degree, which opens to an attack.

But I do agree that it should be possible to distinguish the original CID from other CID's. Isn't it already via the header type?

Or is the problem that you don't want dual logic? You don't need to maintain state for OCID, so that is a fast test and no crypto.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-504766188

---

From: Nick Banks <[email-masked]>
Date: Sun, 23 Jun 2019 09:23:01 -0700

@mikkelfj currently, when processing a LH Initial packet, from just the packet header, the LB cannot determine how to route the packet, if it has two different sets of logic: one for client chosen CIDs and one for server chosen CIDs.

Instead of waiting until short header packets to change the CID, you could wait until Handshake packets, but I don't see how this makes things better. I don't understand your statement:

> It would also allow a client to decide where state is managed during a handshake, to a degree, which opens to an attack.

The whole point here is that you want to route all long header packets to the same place. Why would you route the initial packets one place, and then handshake packets somewhere else? If you didn't like where they went in the first place, don't route them there.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-504766651

---

From: MikkelFJ <[email-masked]>
Date: Sun, 23 Jun 2019 10:52:56 -0700

> The whole point here is that you want to route all long header packets to the same place. Why would you route the initial packets one place, and then handshake packets somewhere else? If you didn't…

The initially discovering LB can choose a random or least loaded endpoint to send the initial packet to. This endpoint decides a new CID it sends to the client. When client responds, the original LB sees a new CID in a LH, it knows that it is not a random OCID because it is no longer an initial packet. Therefore it routes the CID to the correct server assuming the LB understands the CID that the server generated.

If the client retransmits initial, it might land on another server, but that should work fine.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-504773171

---

From: Nick Banks <[email-masked]>
Date: Sun, 23 Jun 2019 11:02:42 -0700

> If the client retransmits initial, it might land on another server, but that should work fine.

The client can send several things in initial packets:

1. ACKs of the server initial packets.
2. TLS renegotiation packets. For instance, HRR response.

Bottom line, the client continues to use Initial packets beyond the first one it sends. But with the current CID design, those packets use a new CID.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-504773912

---

From: MikkelFJ <[email-masked]>
Date: Sun, 23 Jun 2019 11:08:52 -0700

Well, if that is possible, I agree that should be fixed. But I think that it is those packets that should not use an initial packet then.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-504774346

---

From: Marten Seemann <[email-masked]>
Date: Sun, 23 Jun 2019 17:59:30 -0700

> So, practically, that means Long Header packets could have one of two types of CIDs: client chosen or server chosen. 

The LB could append an auth tag to CIDs that it generates, which would allow it to distinguish between its own and client generated CIDs. 

> To prevent attack vectors from being exposed, the LB practically needs to use different routing logic for each type of CID.

I'd like to make sure that I understand the attack vector that you're concerned about. An attacker could reuse CIDs which it obtained on an old connection to reconnect to exactly the same backend server, and flood it with (computationally expensive) handshakes. Is that the scenario you had in mind, @nibanks?
Mitigating this attack would could be done by sending Retries, or by effectively blocking a certain CID from reestablishing a new connection for a certain time.
Requiring all Long Header packets to use the same (client chosen) CID, as you propose, would indeed make it easier to mitigate this attack: A LB would then require Long Header packets to use a client chosen CID (i.e. without an auth tag), and discard all Long Header packets that have a valid auth tag.

Thinking a bit more about routing during the handshake, it seems to me that the current spec would make sense if it wasn't for 0-RTT: In that case, you could randomly assign an Initial packet with a client-chosen CID to any backend server, and the immediate switch to server-chosen CIDs would make sure that all packets sent in the second flight are routed back to that server. However, 0-RTT breaks that assumption, since the first flight contains (a lot) more than just one packet, and 0-RTT packets need to be routed to that server as well. A LB therefore needs to implement two different routing schemes anyway. I might be missing something here, but I can't see a big disadvantage of using the routing scheme based on client-chosen CIDs until the completion of the handshake, as @nibanks proposes, and only switching to server-chosen CIDs for Short Header packets.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-504805050

---

From: Nick Banks <[email-masked]>
Date: Sun, 23 Jun 2019 18:21:53 -0700

The attack vector would only be for designs without an auth tag in the CID. I'm personally not a big fan of the really large CID designs, nor am I convinced that they are truly necessary for a complete and secure system. So, I continue to look into how it would be possible to route with smaller CIDs. For that kind of design, the "server ID" or SID would be encoded somehow in the CID, but not encrypted. If you applied this design to all CIDs, including client chosen CIDs, then it would allow an attacker to directly create new connections to a particular server, by specifying the correct SID. That is definitely not what we want in a design though. So, instead of extracting the SID out of the CID for client chosen CIDs, we'd just hash the CID with some private key instead, and use the hash to pick a SID. BUT that only works if you can differentiate client chosen and server chosen.

As you indicated, an auth tag would help with that, but require additional complexity for the overall design. I'm looking for solutions that don't necessarily increase the length of the CID.

Being able to route based on the CID consistently just by looking at the long header bit and then interpreting the destination CID accordingly would definitely make for simpler load balancing. It could even be version independent. I find the idea of a version independent LB very attractive.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-504807474

---

From: ianswett <[email-masked]>
Date: Sun, 23 Jun 2019 19:05:52 -0700

I think there are two obvious designs here, though there may be more:
 1) Change the CID routing based on packet type (Nick's suggestion above)
 2) Use 5-tuple routing for all long header packets(Because 5-tuple can't change during the handshake) and CID routing after that. 

If we care about the first case, I think we need to change the current design, though I'd be happy to be proven wrong.

One idea is to change to the new CID when sending packets of the next encryption level, since you have to parse the previous encryption level to start sending the next, but not change the current encryption level.

ie: if a client processes an Initial and Handshake, then all short header packets would be sent with the new Handshake CID, or if the CID was changed in Initial, then all Handshake and short header packets would be sent with that.

That leaves Retry.  I assume we want all Initial, Handshake, and 0-RTT packets to be sent with the new CID provided by the Retry, but I'd be curious what others think?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-504813305

---

From: Kazuho Oku <[email-masked]>
Date: Sun, 23 Jun 2019 19:23:45 -0700

@ianswett 
> 1. Change the CID routing based on packet type (Nick's suggestion above)

I do not think that this is a viable design.

An attacker controls both the client tuple and the initial CID. A stateless load balancer has to have the capability to change the CID, otherwise, it cannot balance the load between the servers running behind.

> 2. Use 5-tuple routing for all long header packets(Because 5-tuple can't change during the handshake) and CID routing after that.

Actually, there's a third design, that is possible under the current specification (and IIRC that multiple stacks already implement).

As @marten-seemann points out, a server can use authenticate the CID, to tell at a very high probability if a CID was generated by the server. Initial / 0-RTT packets that contain an authenticated CID is passed to the corresponding backend server (or the corresponding pthread). Initial / 0-RTT packets that do not are considered as new connections.

When not under attack, packets with CIDs that do not authenticate are routing using the 5-tuple. Servers will generate authenticated CIDs that route to themselves. Therefore, all the Initial and 0-RTT packets reach the correct backend.

When under attack, 0-RTT is disabled / dropped. Initial packets carrying CIDs that do not authenticate are distributed to the backends using an algorithm that the load balancer selects. Initial packets that carry CIDs that authenticate are routed to the corresponding servers. While it is true that routing of a tiny portion of Initial packets is not controlled by the load balancer, it is fine assuming that the CID encryption key is unknown to the attacker.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-504827612

---

From: Nick Banks <[email-masked]>
Date: Mon, 24 Jun 2019 06:28:29 -0700

> An attacker controls both the client tuple and the initial CID. A stateless load balancer has to have the capability to change the CID, otherwise, it cannot balance the load between the servers run…

@kazuho I don't understand how your statement shows that changing the CID only with Short Header packets is not a viable design. First, the LB isn't the one to change the CID, the backend server is the one. Second, the CID would be changing, but until a little later in the exchange.

Also, while I agree that an auth tag would likely be a solution, I do not believe it is acceptable that the only way to statelessly (and securely) load balance Long Header packets is to have an auth tag as part of the CID. We should allow for less complex designs.

Finally, using 5-tuple routing cannot adequately load balance parallel new connections on the same tuple (proxy scenario or possible attacker).

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505009704

---

From: ianswett <[email-masked]>
Date: Mon, 24 Jun 2019 06:43:44 -0700

I agree with @nibanks that auth hash should be a 3rd option, but I don't believe it should be the only option.

I also think option 1 can be viable with some small changes.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505015677

---

From: MikkelFJ <[email-masked]>
Date: Mon, 24 Jun 2019 10:11:54 -0700

So what are the thoughts on limiting the initial packet to, well, initial packets?

Then you can route based on packet type. You can randomly or intelligently decide a backend based on OCID. You do not need to hash 5-tuples or attached extra information.

But you must define home, and possibly a CID, for other packets that are squatting the initial packet header.

I think I have previously argued against reusing the OCID in new connections 0-RTT, RETRY, or otherwise, but I really don't recall the details. However, using OCID for only one ongoing connection attempt is important to make the above work.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505098666

---

From: Nick Banks <[email-masked]>
Date: Mon, 24 Jun 2019 10:50:20 -0700

> So what are the thoughts on limiting the initial packet to, well, initial packets?

What exactly do you mean by this @mikkelfj? Do you mean change the CID in handshake packets instead of short header packets? If so, that's definitely an improvement over what we have, but then requires the load balancing logic to be version specific, and I'm unsure how to load balanced new/unknown version numbers then. That's why I'd prefer changing with short header packets.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505112319

---

From: MikkelFJ <[email-masked]>
Date: Mon, 24 Jun 2019 11:18:31 -0700

Couldn't we have something that isn't version specific? Initial packets are always initial and have a special OCID, the rest server selected. Temporary for handshake CID if need be? But not version specific to LB? That could possible also make 5-tuple stability go away which is great for mesh networks that don't do IP, though this is not in the charter.
(I'm working a bit all over the place atm so not into enough detail here.)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505122410

---

From: Mike Bishop <[email-masked]>
Date: Mon, 24 Jun 2019 11:57:22 -0700

Initial packets are already version-specific, so you can't define any not-version-specific property of Initial packets.  Any packet type is version-specific, because the only concept defined by invariants is long vs. short, and that version zero is VN.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505136249

---

From: MikkelFJ <[email-masked]>
Date: Mon, 24 Jun 2019 12:50:45 -0700

> Initial packets are already version-specific

You could add a signal that is universal, just like long vs short packet. But I agree it needs to be added to the invariants.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505154099

---

From: Nick Banks <[email-masked]>
Date: Mon, 24 Jun 2019 12:53:52 -0700

At this point, I'm pretty sure everyone wants to restrict any additional changes to the invariants. This likely wouldn't make that bar. The best way to have an invariant load balancer, without changing the invariants, is to have a general statement that the CIDs don't change in the long headers.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505155167

---

From: MikkelFJ <[email-masked]>
Date: Mon, 24 Jun 2019 12:59:55 -0700

I agree the bar should be high, but the only time to change the invariants is now. So it depends on how messy the alternatives are.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505157185

---

From: Nick Banks <[email-masked]>
Date: Mon, 24 Jun 2019 13:11:19 -0700

@mikkelfj  can you explain again why you don't like waiting until short header packets to change the CID?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505160990

---

From: MikkelFJ <[email-masked]>
Date: Mon, 24 Jun 2019 13:20:17 -0700

I'm a bit of a loss here because I thought the OCID was only in the first packet and not in all long headers (and perhaps in 0-RTT/HelloRetry - we can discuss these separately). But assuming you have OCID in anything but the very first packet:

The first user created CID is name OCID for original CID. This one is random, or supposed to be. It cannot be trusted, and it does not contain routing information nor reliable entropy.

If the LB is to use CID for routing, OCID cannot be used, but it is important to know that it is an OCID. This is the information that I want in the invariants.

If the OCID only appears in the first packet there is no state. If it can be used in multiple packets (and not a retransmitted first packet), then routing must be consistent. There is help in the OCID so you must either use another CID if possible, or some other mechanism.

If that other mechanism is 5-tuple, you need to store state, or you need to route consistently. And you have to do it until you know that you have a useful CID.

If instead you can trust a server assigned CID on anything but the first packet things start to improve. Since we don't have a full handshake, any CID is a bit dubious still, but then CID's are always dubious to external observers. The important part is that the receiver can reject. The server can reject a server assigned CID.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505164170

---

From: MikkelFJ <[email-masked]>
Date: Mon, 24 Jun 2019 13:21:32 -0700

What I really want in invariants is a signal to know that a CID which is supposed to be server chosen is in fact client chosen. And secondary, I want a given QUIC version to use this OCID signal very carefully, but that is not an invariant thing.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505164629

---

From: MikkelFJ <[email-masked]>
Date: Mon, 24 Jun 2019 13:28:27 -0700

(I have a related issues to prevent replay attacks on 0-RTT based on early CID).

>From memory, 0-RTT uses OCID to ensure the packet lands with the issuer of 0-RTT token. But that implies that OCID is used for routing, which is bad.

If the first server assigned CID is used for 0-RTT instead, you get proper routing to target without the mess. It is important that OCID is in the transport parameters somewhere for security reasons, but that is different.

I'm sorry if I mix things up here, this is just how I recall it.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505167491

---

From: Nick Banks <[email-masked]>
Date: Mon, 24 Jun 2019 13:40:52 -0700

> What I really want in invariants is a signal to know that a CID which is supposed to be server chosen is in fact client chosen.

I agree with this and that's why I proposed this:

All Long Headers use client chosen CID. All Short Headers use server chosen CID. Therefore, the Long Header bit (invariant) indicates the CID type.

With that design, an LB has two routing algorithms:

1) Long Header (client CID) routing - For example, hash the tuple and/or client CID and use that to select a backend server.
2) Short Header (server CID) routing - Extract server identifier directly from CID. This might entail decryption or just decoding a value.

Both algorithms can be QUIC invariant, but don't have to be.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505172292

---

From: MikkelFJ <[email-masked]>
Date: Mon, 24 Jun 2019 13:48:02 -0700

But while we agree on the signal, I disagree that it is a good idea to base routing on a client chosen CID.  I can live with that in v1, but not v2. If long headers is that signal, you a locked into this pattern forever.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505174891

---

From: Nick Banks <[email-masked]>
Date: Mon, 24 Jun 2019 13:55:48 -0700

If you don't want to be locked into anything, I don't think it can be invariant. That's the whole point of invariants.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505177645

---

From: MikkelFJ <[email-masked]>
Date: Mon, 24 Jun 2019 13:58:24 -0700

Well, call it locked into ossification then.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505178572

---

From: Kazuho Oku <[email-masked]>
Date: Mon, 24 Jun 2019 14:51:52 -0700

@nibanks 
> @kazuho I don't understand how your statement shows that changing the CID only with Short Header packets is not a viable design. First, the LB isn't the one to change the CID, the backend server is…

The problem is this "a little later in the exchange" that is introduced by the proposal.

In QUIC v1, if we ignore use of 0-RTT packets (that we do not need to care under attack), the only type of packet that has client-chosen CID is the first flight sent from the client. All the packets that the client sends in response to server's packet contains a server-chosen CID.

This is an excellent property.

When receiving an Initial packet carrying a CID that does not authenticate, a load balancer can assign the connection to a server that is the least loaded (or a server can choose a thread that is the least loaded). The assigned server (or the thread) would generate an authenticated CID that maps to itself, and responds with an Initial packet that contains that CID. Then, all the subsequent packets from the client that need to be processed would carry that authenticated CID, allowing these packets to be routed using only the CID.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505195683

---

From: Kazuho Oku <[email-masked]>
Date: Mon, 24 Jun 2019 15:02:00 -0700

> Being able to route based on the CID consistently just by looking at the long header bit and then interpreting the destination CID accordingly would definitely make for simpler load balancing. It c…

Routing packets based on client-chosen CID is vulnerable to DoS attacks, compared to routing based on server-chosen CIDs (the way I've described in my previous comment).

Besides, if you are fine with routing packets based on client-chosen CID, I'd assume that you'd also be fine with routing long header packets based on the 5-tuple.

Assuming that is the case, it is my view that *not* applying CID change to long header packet narrows the designs we can choose with no practical merit.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505198558

---

From: Nick Banks <[email-masked]>
Date: Mon, 24 Jun 2019 15:13:59 -0700

>All the packets that the client sends in response to server's packet contains a server-chosen CID.

>This is an excellent property.

@kazuho with the current way things are spec'ed, and assuming you do not use auth tags in the CIDs and that you do not try to decrypt the actual payload, when a stateless LB receives an Initial packet from the client, how exactly is it supposed to chose which backend server to send it to?

In other words, how do you differentiate the very first Initial packet from the client (using the client CID) vs a response Initial packet (using the server CID)?

> Besides, if you are fine with routing packets based on client-chosen CID, I'd assume that you'd also be fine with routing long header packets based on the 5-tuple.

As far as exposing attacks, existing (TCP/UDP) deployments already use hashing of the 5-tuple to route packets. Using similar logic (hash of CID+tuple) to route just initial QUIC packets doesn't expose any additional attack surface.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505201750

---

From: Mike Bishop <[email-masked]>
Date: Mon, 24 Jun 2019 15:47:25 -0700

I think the different viewpoints here implicitly assume different properties of the CID.

- If server-generated CIDs are easy for the server to recognize, but hard for an attacker to duplicate, then @kazuho's design makes perfect sense -- check whether the CID in an Initial is yours, and route based on it if so.
- If they're either difficult to verify or easy to forge, then you should only route based on CID if you retain enough state / embed enough state in the token to recognize when the client has switched to your CID and assume anything else is client-generated and untrustworthy.

Handshake packets can't be sent until you've seen the server's Initial, which means that Handshake *can* always carry the server's selected CID, but we could decide otherwise without breakage.  0-RTT packets have to start with the ODCID, and they could change or not.  Initial packets after the first round trip could change or not.

My inclination is that server infrastructure SHOULD NOT route based on the CID in an Initial or 0-RTT packet, even if the client has switched to the one you gave it; you also SHOULD NOT route based on CID for a version you don't understand.  In that case, it really doesn't matter whether the client switches upon receipt of your Initial.  They can start using the server's CID in Handshake and short-header packets.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505209548

---

From: Nick Banks <[email-masked]>
Date: Mon, 24 Jun 2019 16:02:53 -0700

Thanks @MikeBishop for articulating the different sides of things so well.

In summary you're suggesting that CID should not be used to route Initial and 0-RTT packets as they could/would be using the client chosen CID. Ok. Then that only leaves the UDP 4-tuple if you're routing statelessly. That has been just fine for TCP in the past, so at first glance it seems an acceptable solution for QUIC.

But QUIC connections are identified by CID not 4-tuple, and the 4-tuple can/will be shared for multiple parallel connections. So, if you route purely on 4-tuple then a client on a single local ip/port that creates multiple connections (proxy) gets all of its traffic sent to a single backend server. That's not a great behavior and I am trying not to get stuck with that.

Instead of all long header packets using the client CID are folks more inclined to accept all Initial and 0-RTT packets use the client CID? In other words, the CID doesn't change until Handshake packets?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505213070

---

From: Mike Bishop <[email-masked]>
Date: Mon, 24 Jun 2019 16:11:44 -0700

And I think that leads to why the DCID can change in a Retry -- so that the server being routed to by the 4-tuple can bounce you to a different back-end instance if it's overloaded.  However, another way to do that is to forward the packets internally and let that server respond with the new CID.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505215048

---

From: ianswett <[email-masked]>
Date: Mon, 24 Jun 2019 16:16:46 -0700

To restate my suggestion, above, I'd suggest:
 1) Retry can change Initial CID - Presumably that's the point of Retry?
 2) Initial can change Handshake CID
 3) Handshake can change initial 1RTT CID
 4) ORTT only changes as a result of Retry

 That provides consistency per packet type, with the exception of Retry.  Maybe we'd prefer Retry doesn't change the CID, of Initial and 0RTT, but if that's true, it seems like we should change the design of Retry?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505216115

---

From: Nick Banks <[email-masked]>
Date: Mon, 24 Jun 2019 16:34:23 -0700

@MikeBishop the fact that Retry can change the CID only helps if you can then use the next Initial packet's CID to route packets. But you've already said that you SHOULD NOT use the CID to route.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505219934

---

From: Kazuho Oku <[email-masked]>
Date: Mon, 24 Jun 2019 16:53:15 -0700

@nibanks 
> @kazuho with the current way things are spec'ed, and assuming you do not use auth tags in the CIDs and that you do not try to decrypt the actual payload, when a stateless LB receives an Initial pac…

My assumption has been that it's the 5-tuple.

As you state, that's how TCP load balancers have operated, and we can do the same for QUIC.

Using client-generated CID for load balancing incoming connections is not a good idea because it is more prone to attacks than relying on the 5-tuple, because spoofing the payload of a UDP packet is easier than spoofing the source address.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505223634

---

From: Kazuho Oku <[email-masked]>
Date: Mon, 24 Jun 2019 16:55:11 -0700

@MikeBishop Thank you for articulating the differences. I think I agree with your comments, with the exception being that

> My inclination is that server infrastructure SHOULD NOT route based on the CID in an Initial or 0-RTT packet, even if the client has switched to the one you gave it

needs "unless the server has a way to test if CID is server-generated."

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505223970

---

From: Nick Banks <[email-masked]>
Date: Mon, 24 Jun 2019 16:56:20 -0700

@kazuho why is using a `hash(tuple + CID)` not acceptable? I'm arguing that for QUIC, tuple I not enough. QUIC is different from TCP because the tuple can be shared.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505224182

---

From: MikkelFJ <[email-masked]>
Date: Mon, 24 Jun 2019 16:58:44 -0700

If LB is behind proxy, tuple has no entropy. If CID is, say, 0, everything lands in the same spot.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505224625

---

From: Kazuho Oku <[email-masked]>
Date: Mon, 24 Jun 2019 17:03:00 -0700

@ianswett IIUC, the design principle has been to allow the server to initiate a CID change _for the packets that the client sends in response_, when the server receives the incoming packet for the first time. Generally speaking, I think that having the capability to switch the CID just once is sufficient.

That's why we allow server's Initial to change the CID.

However, because we'd also want a way to validate reachability, we added Retry. Then, because we thought that having the capability to redistribute the load using Retry would be a good idea, we added the capability to switch CID in Retry.

We can forbid the change of CID using Initial packets when Retry is used. But IIRC the question was what we gain from having such prohibition. Assuming that it's just an addition of complexity without any gain, we just allowed CID to change at most twice, once by Retry, and then by Initial.

While I agree that the design is not clean-cut, I think it's very pragmatic and fullfills our needs.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505225402

---

From: Nick Banks <[email-masked]>
Date: Mon, 24 Jun 2019 17:03:08 -0700

@mikkelfj if the CID is the same, then it's the same connection.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505225428

---

From: Nick Banks <[email-masked]>
Date: Mon, 24 Jun 2019 17:06:00 -0700

> While I agree that the design is not clean-cut, I think it's very pragmatic and fullfills our needs.

@kazuho until we have a design that allows for load balancing unique connections, independent of tuple, without using an auth tag in the CID, I will have to disagree with you.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505225991

---

From: Kazuho Oku <[email-masked]>
Date: Mon, 24 Jun 2019 17:07:59 -0700

@nibanks 
> why is using a hash(tuple + CID) not acceptable? I'm arguing that for QUIC, tuple I not enough. QUIC is different from TCP because the tuple can be shared.

I had thought that sharing 5-tuple among multiple QUIC connections is easy, but I no longer think that is correct.

A client cannot establish multiple QUIC connections using the same 5-tuple, unless it knows beforehand that the server is using a non-zero-length server CID. That requires out-of-band negotiation beforehand, and I am afraid how much we need to care about such usage.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505226409

---

From: David Schinazi <[email-masked]>
Date: Mon, 24 Jun 2019 17:22:33 -0700

@kazuho can't a client share the same 5-tuple by using client connection IDs?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505229076

---

From: Marten Seemann <[email-masked]>
Date: Mon, 24 Jun 2019 17:24:20 -0700

@kazuho 
> needs "unless the server has a way to test if CID is server-generated."

This check only works if you assume that a client's connection ID is generated randomly, right?
This assumption doesn't hold in the case of an attacker, who, as I've pointed out earlier, could reuse a connection ID that it obtained on an earlier connection to target the specific backend server it was connected to.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505229389

---

From: Kazuho Oku <[email-masked]>
Date: Mon, 24 Jun 2019 17:27:32 -0700

@marten-seemann 
> This assumption doesn't hold in the case of an attacker, who, as I've pointed out earlier, could reuse a connection ID that it obtained on an earlier connection to target the specific backend serve…

It is true that an attacker can forge a CID it has seen previously, and reuse that to mount DoS attacks. 

However, the issue is not related to long header packets (or how a connection is assigned to a particular node behind a load balancer). An attacker can for example, forge a CID of a short header packet and create multiple copies of short header packets.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505229947

---

From: Marten Seemann <[email-masked]>
Date: Mon, 24 Jun 2019 17:29:18 -0700

> However, the issue is not related to long header packets (or how a connection is assigned to a particular node behind a load balancer). An attacker can for example, forge a CID of a short header pa…

The impact of that attack is different though. For a short header packet, all an attacker can do is force a trial decryption of a packet (which is cheap). For a long header packet, it can initiate a new TLS handshake (which is expensive).

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505230263

---

From: Kazuho Oku <[email-masked]>
Date: Mon, 24 Jun 2019 17:29:23 -0700

@DavidSchinazi 
> can't a client share the same 5-tuple by using client connection IDs?

It's the packets going to the server. A server using 0-byte CID relies on the 5-tuple to identify the connections. When there a chance that the server is as such, a client cannot reuse it's client port.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505230276

---

From: Kazuho Oku <[email-masked]>
Date: Mon, 24 Jun 2019 17:33:45 -0700

@marten-seemann 
> The impact of that attack is different though. For a short header packet, all an attacker can do is force a trial decryption of a packet (which is cheap). For a long header packet, it can initiate…

No. Because for every one CID, only one TLS handshake is performed by the server.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505231087

---

From: David Schinazi <[email-masked]>
Date: Mon, 24 Jun 2019 17:34:15 -0700

@kazuho Ah, right. Thanks. That said, a server shouldn't be using zero-length connection IDs if it expects to service multiple connections on the same IP+port. Perhaps we need text clarifying that? The current text doesn't discuss this use-case:

> A zero-length connection ID MAY be used when the connection ID is not needed for routing and the address/port tuple of packets is sufficient to identify a connection.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505231170

---

From: Kazuho Oku <[email-masked]>
Date: Mon, 24 Jun 2019 17:39:20 -0700

@DavidSchinazi 
> That said, a server shouldn't be using zero-length connection IDs if it expects to service multiple connections on the same IP+port.

Would you mind clarifying why?

I think that a server can handle multiple clients using zero-length connection IDs if it disables migration. While I am not interesting deploying such a server, I am not sure if we need to prohibit such possibility.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505232009

---

From: MikkelFJ <[email-masked]>
Date: Mon, 24 Jun 2019 17:45:16 -0700

> That said, a server shouldn't be using zero-length connection IDs if it expects to service multiple connections on the same IP+port. Perhaps we need text clarifying that?

I believe this is already covered:

>A zero-length connection ID MAY be used when the connection ID is not needed for routing and the address/port tuple of packets is sufficient to identify a connection. An endpoint whose peer has selected a zero-length connection ID MUST continue to use a zero-length connection ID for the lifetime of the connection and MUST NOT send packets from any other local address.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505233016

---

From: Kazuho Oku <[email-masked]>
Date: Mon, 24 Jun 2019 18:42:47 -0700

@nibanks PS. FWIW I think you can use `hash(5-tuple || SCID)` of Initial packets as a stable source to distribute the connections if you do expect to have connections sharing 5-tuple.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505243242

---

From: Nick Banks <[email-masked]>
Date: Mon, 24 Jun 2019 19:44:53 -0700

@kazuho that's a good suggestion. I will think on it some more, but I think that is an acceptable solution.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505257141

---

From: ianswett <[email-masked]>
Date: Tue, 25 Jun 2019 09:17:36 -0700

@kazuho So the suggestion is: If there's a non-zero length SCID, use that.  If SCID is 0 length, use 5-tuple?

Given you can't share a 5-tuple without using a SCID, that seems like a reasonable approach.  If so, it should probably be written down in the QUIC LB draft?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505516692

---

From: David Schinazi <[email-masked]>
Date: Tue, 25 Jun 2019 09:59:19 -0700

@kazuho it sounds we can have either:
(1) server can share port and use zero-length CIDs (which prohibits migration)
(2) client can share port and use non-zero-length CIDs

We can't have both (1) and (2) because the client wanting to do (2) doesn't know if the server will do (1) or not.

@huitema and I have a use-case for (2) which is a QUIC proxy that only uses one port for outbound connections and relies on client CIDs for demultiplexing. I don't see a use for (1).

I'd recommend prohibiting (1) so we can have (2).

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505532481

---

From: ianswett <[email-masked]>
Date: Tue, 25 Jun 2019 11:05:28 -0700

Anytime an endpoint would like to terminate multiple connections on the same 5-tuple, it would need an incoming connection ID.

Given that, I don't think we need to change anything in the existing spec.  The combination of 5-tuple and SCID will provide enough entropy during the handshake and both are guaranteed not to change during the handshake.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505557394

---

From: David Schinazi <[email-masked]>
Date: Tue, 25 Jun 2019 11:22:08 -0700

@ianswett that won't work because short headers don't carry the SCID and if the server is reusing a port and has zero-length CIDs then it won't be able to differentiate short headers from the client if the client also shares its port between connections.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505563641

---

From: ianswett <[email-masked]>
Date: Tue, 25 Jun 2019 11:24:04 -0700

I thought @nibanks wants to route short header packets based only on the server's chosen CID?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505564383

---

From: Nick Banks <[email-masked]>
Date: Tue, 25 Jun 2019 11:30:20 -0700

@ianswett that's correct.

Get Outlook for Android<https://aka.ms/ghei36>

________________________________
From: ianswett <[email-masked]>
Sent: Tuesday, June 25, 2019 2:24:04 PM
To: quicwg/base-drafts
Cc: Nick Banks; Mention
Subject: Re: [quicwg/base-drafts] Long Header Packets and Routing Connection IDs (#2834)

I thought @nibanks<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnibanks&data=02%7C01%7Cnibanks%40microsoft.com%7C3d4dade33012488b5be708d6f99a4dfa%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636970838458340332&sdata=frkRvMK6X4J%2BnyFO1a75GIKe1uctHd9a18iwR1tbdxw%3D&reserved=0> wants to route short header packets based only on the server's chosen CID?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fquicwg%2Fbase-drafts%2Fissues%2F2834%3Femail_source%3Dnotifications%26email_token%3DAE5U2BP2QZPAGK2MI34XHBTP4JPEJA5CNFSM4H2ZGBS2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYREZXY%23issuecomment-505564383&data=02%7C01%7Cnibanks%40microsoft.com%7C3d4dade33012488b5be708d6f99a4dfa%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636970838458340332&sdata=oHpNVBXLnxqGeyZSbGzczbx3cJWXMJ5gSo4SFWb6ccA%3D&reserved=0>, or mute the thread<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAE5U2BIU2PFAPJVEY6KIC23P4JPEJANCNFSM4H2ZGBSQ&data=02%7C01%7Cnibanks%40microsoft.com%7C3d4dade33012488b5be708d6f99a4dfa%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636970838458350286&sdata=AlNeGmWNTs3nJeoe4LUHkluk9UC5%2Bh0RzlitYgmQ3vo%3D&reserved=0>.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505566779

---

From: David Schinazi <[email-masked]>
Date: Tue, 25 Jun 2019 12:32:15 -0700

Sorry I may have derailed this issue off topic, I'm moving my topic to new issue #2844.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505589807

---

From: Kazuho Oku <[email-masked]>
Date: Tue, 25 Jun 2019 14:55:51 -0700

@ianswett 
> So the suggestion is: If there's a non-zero length SCID, use that. If SCID is 0 length, use 5-tuple?
> 

Yes, that's the suggestion, and I think we should write it down somewhere.

I might even advocate for having it clarified in the transport draft, especially if we decide to disallow servers using the client's tuple to distinguish between the connections (#2844).

We have said that long header packets can be identified to which connections they belong, by looking at the 5-tuple. However, for clients to establish multiple QUIC connections in parallel on a single 5-tuple, the server _has_ to identify the connections using `5-tuple || client CID` as the key. It's crucial, but not obvious.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505637716

---

From: ianswett <[email-masked]>
Date: Tue, 25 Jun 2019 15:02:52 -0700

Given we discuss stability of 5-tuple during the handshake in the transport draft, discussing stability of SCID makes sense as well.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/2834#issuecomment-505639783