Subject: Re: [quicwg/base-drafts] Coalescing different CIDs for same connection (#3800)

From: Lars Eggert <[email-masked]>
Date: Fri, 03 Jul 2020 05:59:28 -0700

But this complicates the coalescing logic, because you no longer only need to look at packet types but now also CID fields. At least in my stack, that is an ugly change to make at this stage.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3800#issuecomment-653536489

---

From: Martin Thomson <[email-masked]>
Date: Tue, 07 Jul 2020 16:04:15 -0700

The alternative, which I can now see is probably a smaller change is:

> Receivers SHOULD ignore any subsequent packets for a different connection than the first packet in the datagram.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3800#issuecomment-655182971

---

From: Martin Thomson <[email-masked]>
Date: Tue, 07 Jul 2020 16:05:27 -0700

What motivated this was two observations:

* we don't coalesce except during the handshake, where we are not migrating

* some stacks generate packets for coalescing while processing, and so they might generate a datagram with multiple connection IDs

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3800#issuecomment-655183410

---

From: Dmitri Tikhonov <[email-masked]>
Date: Fri, 10 Jul 2020 12:27:28 -0700

@martinthomson writes:
> I'm of the opinion that the first should change to different connection IDs rather than connections; the second might change to a MUST NOT, but it could remain a SHOULD and achieve the same end.

This is much simpler alternative than what is being proposed.  Now an implementation can only drop a packet after doing a connection lookup, whereas before it could just compare the CID to the previous value.

If this is just a matter of personal preference, I vote for this approach, so now the score is 1:1 (with @larse being on the opposite side).

---

To take a step back: why is recommendation to drop packets exist in the first place?  If a sender wants to coalesce packets from different connections, why not let it?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3800#issuecomment-656851626

---

From: Quoc-Viet Nguyen <[email-masked]>
Date: Fri, 10 Jul 2020 17:31:04 -0700

I think it should *not* allow having multiple packets for different connections in a single UDP datagram - please.
Currently I see implementations (including mine) have something like:
  1. Receive UDP packet.
  2. Parse packet header.
  2. Check CID, find the connection handler.
  3. Pass the packet to the respective connection (thread) to handle the packet. 

I think it makes implementations much more complicated (already). 
Also I wonder it may cause security issue if having bad assumption?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3800#issuecomment-656949630

---

From: Christian Huitema <[email-masked]>
Date: Sun, 12 Jul 2020 08:59:15 -0700

The classic example would be coalescing Handshake packets, whose DCID is the SCID used by the server, and Initial or Zero-RTT packets, whose DCID is the client-chosen Initial DCID. The client MAY switch to using the server chosen CID for Initial packets. Maybe we should say that it MUST do that if it is going to coalesce Initial and Handshake packets. Not so clear about 0-RTT packets. Is it actually allowed to send them with a different DCID than the Initial DCID?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3800#issuecomment-657241368

---

From: Jana Iyengar <[email-masked]>
Date: Sun, 12 Jul 2020 17:33:12 -0700

@dtikhonov : Our assumption has been that load balancers, for instance, can simply route on the CID in the first header they see in the UDP datagram. If we allow different connections to be coalesced, that complicates things. It may also start to leak some things about the way load is distributed within the server network: if some CIDs when combined cause loss and some don't, that reveals some things about how the server load is distributed.

Is there a strong argument for changing this behavior? This new language in #3870 was always the intent. I suspect it didn't change when we started allowing multiple CIDs within a connection. Unless there's an argument for changing behavior, this is really an editorial PR.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3800#issuecomment-657300915

---

From: Marten Seemann <[email-masked]>
Date: Sun, 12 Jul 2020 18:23:19 -0700

> Maybe we should say that it MUST do that if it is going to coalesce Initial and Handshake packets. Not so clear about 0-RTT packets. Is it actually allowed to send them with a different DCID than the Initial DCID?

@huitema If I understand correctly, every time you change the connection ID, you change the connection ID for all packet number spaces. Assuming that my reading of the spec is correct in this regard, it seems like there's no compelling reason for making this change?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3800#issuecomment-657311225

---

From: Christian Huitema <[email-masked]>
Date: Sun, 12 Jul 2020 18:41:50 -0700

@marten-seemann the spec actually says (7.2): _The first flight of 0-RTT packets use the same Destination Connection ID and Source Connection ID values as the client's first Initial packet. Upon first receiving an Initial or Retry packet from the server, the client uses the Source Connection ID supplied by the server as the Destination Connection ID for subsequent packets, including any 0-RTT packets._
SO you are correct, there is no ambiguity.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3800#issuecomment-657315419

---

From: Marten Seemann <[email-masked]>
Date: Sun, 12 Jul 2020 20:18:54 -0700

@huitema Thank you for looking this up.
Then I'm opposed to making any change here. Not only is it unnecessary (that alone should be enough), it would also make it more difficult to evolve the way we deal with coalesced packets in the future: I can imagine a future QUIC version introducing a way to authenticate the entire coalesced packet, while at the same time compressing the header. This will be easier if we keep the explicit property that all connection IDs of the coalesced packets match.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3800#issuecomment-657336969

---

From: Kazuho Oku <[email-masked]>
Date: Sun, 12 Jul 2020 21:39:30 -0700

@marten-seemann 
>  I can imagine a future QUIC version introducing a way to authenticate the entire coalesced packet, while at the same time compressing the header. This will be easier if we keep the explicit property that all connection IDs of the coalesced packets match.

I'm not sure if I agree.

Coalescing of QUIC packets is a feature specific to v1, entities that operate without the knowledge of the connection (e.g., load balancers, *datagram* dispatchers) are not expected to consult the CID values of QUIC packets that are appended. Therefore, what we are talking here does not affect what we can do in the future.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3800#issuecomment-657353978

---

From: Marten Seemann <[email-masked]>
Date: Sun, 12 Jul 2020 22:27:33 -0700

That argument was based on the thought  that it will be easier in a future QUIC version because the diff to v1 would be smaller.

My main point is that this change is unnecessary because when endpoints change connection IDs, they change it for all packer types at the same time.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3800#issuecomment-657366298

---

From: Kazuho Oku <[email-masked]>
Date: Mon, 13 Jul 2020 04:29:24 -0700

@marten-seemann Thank you for the clarification.

> My main point is that this change is unnecessary because when endpoints change connection IDs, they change it for all packer types at the same time.

I think I do not agree to this, too. IIUC, the consensus we reached with #3438 was that the CID of a Handshake packet and that of the 1-RTT packet can be different. While it is true that the initial DCID set by the client is expected to change when the client receives the first "active" connection ID supplied by the server, the fact does not mean that the CIDs of all QUIC packets within a datagram has to be the same in all the cases.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3800#issuecomment-657501031

---

From: Mike Bishop <[email-masked]>
Date: Mon, 13 Jul 2020 12:59:59 -0700

I think we need to make *a* change, in one direction or the other.  It's unfortunate to mix CIDs, since it clearly links them to any observer, but doesn't seem like a big deal.  For some implementations, it sounds like it's a big deal to avoid mixing them.  Therefore, while I'm content with either resolution, it seems like it makes more sense to resolve toward being more permissive; e.g. #3870.

Mixing packets from different connections seems like a generally bad idea, since it assumes an implementation structure on the far end for it to work.  The more likely outcome is that coalesced packets from other connections will simply be dropped as invalid.  (Of course, this very property could be used for padding / misleading observers.)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3800#issuecomment-657761307

---

From: hardie <[email-masked]>
Date: Mon, 13 Jul 2020 13:09:09 -0700

Hi Mike,

On Mon, Jul 13, 2020 at 12:59 PM Mike Bishop <[email-masked]>
wrote:

> I think we need to make *a* change, in one direction or the other. It's
> unfortunate to mix CIDs, since it clearly links them to any observer, but
> doesn't seem like a big deal.
>
Can you say more about why you don't see this as an issue?  Is it because
this use case is most likely during the handshake?  Or is there some other
linkability already present that makes this less of an issue?

regards,

Ted

> For some implementations, it sounds like it's a big deal to avoid mixing
> them. Therefore, while I'm content with either resolution, it seems like it
> makes more sense to resolve toward being more permissive; e.g. #3870
> <https://github.com/quicwg/base-drafts/pull/3870>.
>
> Mixing packets from different connections seems like a generally bad idea,
> since it assumes an implementation structure on the far end for it to work.
> The more likely outcome is that coalesced packets from other connections
> will simply be dropped as invalid. (Of course, this very property could be
> used for padding / misleading observers.)
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/quicwg/base-drafts/issues/3800#issuecomment-657761307>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAKVXZG2NRCVNV3NJV7ADS3R3NRTRANCNFSM4OMZFPYA>
> .
>

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3800#issuecomment-657765332

---

From: Christian Huitema <[email-masked]>
Date: Mon, 13 Jul 2020 13:31:15 -0700

Some implementations try to change the CID as soon as possible, to avoid linking 1RTT traffic with the CID visible during the handshake and to improve privacy. There will be a brief window when they still need to send both some handshake packets and some 1RTT packets. But coalescing a Handshake packet with the old CID and a 1RTT packet with the new CID links old and new CID and thus negates the effort.

And just in case someone asks, no, you don't really need to decrypt the packets to access the second CID. You just need the coalesced packet and another 1RTT packet. Not hard.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3800#issuecomment-657777896

---

From: Kazuho Oku <[email-masked]>
Date: Mon, 13 Jul 2020 16:40:03 -0700

@huitema If that is the concern, would it be better for endpoints to migrate to a new CID after learning that the peer has dropped the Handshake keys?

If an endpoint switches to the new CID before the handshake keys are dropped by the peer, there's chance that the peer might use a new CID in the Handshake packets that the peer sends. That at least leaks *when* the connection was established for that CID.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3800#issuecomment-657877461

---

From: Christian Huitema <[email-masked]>
Date: Mon, 13 Jul 2020 18:06:15 -0700

@kazuho yes, the simplest way to not run into coalescing problems is to not start migrations or CID changes before Handshake Done.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3800#issuecomment-657907007

---

From: Jana Iyengar <[email-masked]>
Date: Mon, 13 Jul 2020 18:15:16 -0700

Apologies, I'm getting lost in the weeds here. I agree with @MikeBishop that we need to make a change, and that #3870 is really meant to be an editorial fix to what the intent always was.

@huitema, @marten-seemann, @kazuho: Are we all agreeing on this point?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3800#issuecomment-657909556

---

From: Kazuho Oku <[email-masked]>
Date: Mon, 13 Jul 2020 18:18:44 -0700

@janaiyengar That is my view. I think #3870 to be an editorial amendment of #3438. Regarding linkability problem, the simple and safe way of mitigating the problem (as proposed in https://github.com/quicwg/base-drafts/issues/3800#issuecomment-657877461) does not interfere with the proposed resolution.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3800#issuecomment-657910578

---

From: Christian Huitema <[email-masked]>
Date: Mon, 13 Jul 2020 18:33:51 -0700

@janaiyengar I personally prefer the old version, force single CID in coalesced packet. Loosening the text to allow multiple CID requires a code change in my implementation.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3800#issuecomment-657914609

---

From: Nick Banks <[email-masked]>
Date: Mon, 13 Jul 2020 18:58:15 -0700

I'm with @huitema. Single CID.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3800#issuecomment-657920744

---

From: Mike Bishop <[email-masked]>
Date: Tue, 14 Jul 2020 14:02:04 -0700

> Can you say more about why you don't see this as an issue? Is it because this use case is most likely during the handshake? Or is there some other linkability already present that makes this less of an issue? regards, Ted

Both, actually.  This occurs during the handshake, when addresses are required to be stable.  Unless you're running a large number of parallel connections, all CIDs used on that 4-tuple during the handshake belong to the same connection with very high probability.  You can break that linkage by deliberately changing CID and port at the same time, but you can't do that until the handshake is confirmed.  If you want the linkage broken, you need to do that jump post-handshake anyway.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3800#issuecomment-658411474

---

From: hardie <[email-masked]>
Date: Tue, 14 Jul 2020 14:35:51 -0700

Hi Mike,

On Tue, Jul 14, 2020 at 2:01 PM Mike Bishop <[email-masked]>
wrote:

> linkability already present that makes this less of an issue? regards, Ted
>
> Both, actually. This occurs during the handshake, when addresses are
> required to be stable. Unless you're running a large number of parallel
> connections, all CIDs used on that 4-tuple during the handshake belong to
> the same connection with very high probability.
>
I think "high probability" is probably app-dependent, because there may be
cases where there are significant numbers of parallel connections.  In the
case of proxied queries, for example, the use of the same 4-tuple but a
different CID could turn out to be common, because the proxy may prefer to
keep traffic from its different clients separate even if they could be
served by the same upstream server.

A DNS example:  if you have an DNS resolver talking to an upstream proxy or
a common authoritative server, that resolver may want to set a different
Client Subnet for different query streams, so that input can be used to
determine the preferred reply.*

> You can break that linkage by deliberately changing CID and port at the
> same time, but you can't do that until the handshake is confirmed. If you
> want the linkage broken, you need to do that jump post-handshake anyway.
>
In the presence of a NAT or CGNAT, there could be interesting interaction
with the dwell timers in the state table of the middlebox.  If you change
it too late, the middlebox may recycle the most recently freed port, which
could result in you getting the same 4-tuple to an observer on that part of
the path.

Speaking strictly as an individual, I think using a single CID per
coalesced packet as Christian prefers is cleaner and leaves open app
behaviors we may not have yet identified.

regards

Ted

*I'm not a big fan of Client Subnet, and these particular parallel
connections may or may not pose a risk; it's just a proxy situation where
the parallelism might occur.

> —
> You are receiving this because you commented.
> <https://github.com/quicwg/base-drafts/issues/3800#issuecomment-658411474>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAKVXZBEI4ITAAKO3YE2ARTR3TBUFANCNFSM4OMZFPYA>
> .
>

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3800#issuecomment-658425864

---

From: Lucas Pardue <[email-masked]>
Date: Tue, 14 Jul 2020 15:42:22 -0700

The discussion is active, I'm marking this as design.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3800#issuecomment-658448654

---

From: Kazuho Oku <[email-masked]>
Date: Tue, 14 Jul 2020 16:30:14 -0700

@hardie 
> I think "high probability" is probably app-dependent, because there may be cases where there are significant numbers of parallel connections.  In the case of proxied queries, for example, the use of the same 4-tuple but a different CID could turn out to be common, because the proxy may prefer to keep traffic from its different clients separate even if they could be served by the same upstream server.

I might be missing something, but if an endpoint wants to cut linkability between the packets used during the handshake and the packets used in 1-RTT, that endpoint has to switch to a new CID *after* the peer has dropped its Initial and Handshake keys, regardless of the outcome of this issue. That is because if the endpoint switches to a new CID while the peer is still sending Initial or Handshake packets, we allow the peer to use that new CID in the long header packets sent in response.

Therefore, I do not think that requiring endpoints to use one DCID in all the coalesced QUIC packets help us in practice.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3800#issuecomment-658462267

---

From: Tatsuhiro Tsujikawa <[email-masked]>
Date: Wed, 15 Jul 2020 08:39:45 -0700

In my implementation, ngtcp2, it processes packet in the way that PR #3870 proposes; which means I just over looked existing text, but anyway, I think that processing a packet atomically is cleaner approach in implementation wise rather than passing a some kind of pointer to the very first packet header.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3800#issuecomment-658836551

---

From: Marten Seemann <[email-masked]>
Date: Wed, 15 Jul 2020 09:04:40 -0700

@tatsuhiro-t Once a non-coalesced packet is passed to a connection, in my implementation I don't have to do any checking. The server already makes sure to pass packets to the correct connection based on the DCID.
So there's additional validation logic required for coalesced packets. And the easiest way is to just check that all DCIDs match. Sure, it would be possible to somehow query which other DCIDs would also be valid for the same session, but it's more complicated. And given that nobody has come up with any good reason to coalesce packets with different DCIDs in the first place, I'm wondering how this additional complexitiy would be justified.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3800#issuecomment-658847305

---

From: ianswett <[email-masked]>
Date: Wed, 15 Jul 2020 10:58:29 -0700

I don't foresee a large problem with the proposed change, but I also don't think it's necessary.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3800#issuecomment-658914188

---

From: Lars Eggert <[email-masked]>
Date: Mon, 20 Jul 2020 23:52:02 -0700

Do we have a proposed resolution for this one?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3800#issuecomment-661670797

---

From: Martin Thomson <[email-masked]>
Date: Mon, 20 Jul 2020 23:54:48 -0700

I think we settled on requiring connection IDs to be the same, but I haven't had the time to write that up, so not just yet.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3800#issuecomment-661671875

---

From: Jana Iyengar <[email-masked]>
Date: Tue, 21 Jul 2020 12:04:50 -0700

@larseggert: FTR, discussion happened on the mailing list [here](https://mailarchive.ietf.org/arch/msg/quic/JL0UthjhdFQ7h_ysW0LDrdmowlo/).

My sense is that it's still not clear to some (including me) why the current fix in #3870 (requiring "connections" to be the same) doesn't work, but nobody is arguing strongly against what @martinthomson said above (requiring "connection IDs" to be the same). So that might be the least common denominator.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3800#issuecomment-662049962

---

From: Martin Thomson <[email-masked]>
Date: Wed, 29 Jul 2020 06:15:37 -0700

Closed #3800 via #3930.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3800#event-3598991337