Subject: [quicwg/base-drafts] 5-tuple routing and SPA (#3608)

From: Mike Bishop <[email-masked]>
Date: Wed, 29 Apr 2020 08:48:42 -0700

Offshoot from #3536 review.  Servers behind a load balancer, and the load balancer isn't CID-aware.  Each server gives out an IP:port per client in the `preferred_address`  parameter.  _If_ the client moves to the SPA, then future migration is fine -- it will continue to talk to the same server using the preferred address.  If the client doesn't move, those future migrations will break.  There's currently no way to set `disable_migration` to "only if you don't use my preferred address".

Similarly, if the primary connection is to an anycast address, if the client fails to move to the preferred address, subsequent migration will likely break.

Should we blanket recommend that clients not actively migrate if they ignore / are unsuccessful in using the SPA?  Or should there be a way for the server to indicate that?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3608

---

From: ianswett <[email-masked]>
Date: Wed, 29 Apr 2020 11:27:12 -0700

Good question.  This brings up a related question for me about disable_active_migration.  Given a server can decide not to give out any additional connection IDs, doesn't that give it a mechanism for disabling active migration?

Ironically, given recent decisions on connection CIDs and SPA, if an SPA is supplied, then the client always has a second connection ID it can use on any path, not just the SPA.  So my suggestion of not giving out extra CIDs fails for the original path, but not the SPA.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3608#issuecomment-621384055

---

From: Eric Kinnear <[email-masked]>
Date: Wed, 29 Apr 2020 15:12:03 -0700

What we’ve said for this so far is that SPA is limited to offering a minimum of a single additional CID that needs to work in both places, the original server and the preferred server.

As Ian says, the original server should not issue the client any more CIDs (if you have some that are deployment specific) until after they’ve successfully migrated to the new endpoint, at which point the preferred server can handle that issuance.

And as he notes, you still have to deal with the fact that there are two CIDs that the client could use, from any incoming address. 

It seems as though the full way to solve this is to define potentially disjoint sets of TPs that different servers could set to match their particular requirements (around more than just migration). In that world, if you had an initial server that couldn’t handle migration and a preferred server that did handle migration, you could set different TPs to indicate that. It goes well beyond that though, any other TP might have a different values for the original server and then have those be different with the preferred server. But that does feel an awful lot like something that goes with full server migration.

In the limited form that we have now, if you want to be able to push someone off to a different server, there are some limitations that you need to deal with, like being able to set up enough of a shared configuration for the two that the same TPs work with both. 

=================

In practice, I think this means we end up with the following situation: 

The client either moves to the requested address or it doesn’t. 
- If it does, then you’re in good shape, in theory you gave it TPs that matched what the server wanted and everybody is happy.
- If it doesn’t, then you’re unhappy, since in theory you had a reason you asked it to move in the first place. You might choose to close it down more aggressively than you would on the preferred server to save resources, or give it smaller limits for flow control, etc. 

When you’re in the “client didn’t move” state:
- If the client then tries to use the original 2nd CID that you gave, from the same IP address/port, then a non-CID aware load balancer should be fine.
- If the client then tries to use the original 2nd CID that you gave, from a different IP address/port, then a non-CID aware load balancer will not be happy. 

Of course, the problematic case is even narrower, since if the client is probing the new path they’ll see it as though there isn’t any connectivity, just like they will if there’s anything else on the new path that prevents their connection from working — this is always a risk when migrating, so they should be unsurprised and stick with their current path. 

If they’re forced to move for whatever reason, then it will be just as though they didn’t have connectivity over the new path (which they don’t) and whether or not you disable migration you’ll see the same outcome — the connection is closed.

Interestingly, I’m not actually seeing a difference from the client’s perspective in terms of outcome whether you let them try with that CID and fail or if you told them to not try.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3608#issuecomment-621492586

---

From: Eric Kinnear <[email-masked]>
Date: Wed, 29 Apr 2020 15:13:28 -0700

In very much shorter terms: isn’t this a situation that all clients already need to be able to handle for other reasons?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3608#issuecomment-621493103

---

From: Kazuho Oku <[email-masked]>
Date: Wed, 29 Apr 2020 16:25:38 -0700

@MikeBishop 
> Should we blanket recommend that clients not actively migrate if they ignore / are unsuccessful in using the SPA? Or should there be a way for the server to indicate that?

I do not like the first option, because IMO banning migration on the original address is a premature optimization based on what we have now, has negative effect on what we would have in the future.
My assumption is that the majority of the server deployments would be CID-aware, especially in the long term, especially in terms of traffic volume. Note that the volume of traffic is more important than the number of server deployments, as migration is about (improving) end-user experience.

IMO there are two ways forward:
* (a) State that disable_active_migration TP only applies to the original address. Per-definition, server-preferred address is an address that is capable of receiving packets from any address. There is no need to prohibit clients doing migration when connected to SPA.
* (b) Do nothing. Clients are RECOMMENDED to implement SPA, and this issue is about a small fraction of deployments that rely on non-CID-aware load balancers. We can ignore the problem.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3608#issuecomment-621521229

---

From: Mike Bishop <[email-masked]>
Date: Thu, 30 Apr 2020 06:05:57 -0700

> This brings up a related question for me about disable_active_migration. Given a server can decide not to give out any additional connection IDs, doesn't that give it a mechanism for disabling active migration?

IIRC, we had that conversation when deciding whether to add `disable_active_migration` in the first place.  The short version is that taking this approach prevents the client from rotating CIDs on the primary path; if we want the client to be able to rotate CIDs but not migrate, they need to be distinct signals.

@erickinnear, in a way you're correct that clients will have to deal with the fallout regardless of what the server says.  However, the actual use of `disable_active_migration` is likely to trigger a little more than just "don't do that" -- a client might PING more actively to prevent NAT rebindings, for example.  I think it's this hinting aspect that's actually more valuable, since as you note, clients need to be prepared for attempted migrations to fail even with servers that tacitly support them.

> Per-definition, server-preferred address is an address that is capable of receiving packets from any address.

I don't think this is necessarily true, though I agree that we're talking about smaller and smaller fractions of server space.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3608#issuecomment-621822010

---

From: Kazuho Oku <[email-masked]>
Date: Thu, 30 Apr 2020 19:31:41 -0700

@MikeBishop 
> I don't think this is necessarily true, though I agree that we're talking about smaller and smaller fractions of server space.

While I agree that it might not necessarily be true at the moment, I might argue that it is going to be true.

The basis of the resolution for #3559 and #3565 (both on consensus call) is that servers using preferred address would not be using the server's address for distinguishing between connections. We also know that a server cannot use client's address (unless there is out-of-band knowledge), because clients are not required to use the same 2-tuple when migrating to a different address (and also because there are NATs).

The outcome is that the server has to rely on CID when using SPA. And therefore I might insist that servers using SPA would always be capable of handling further migration.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/3608#issuecomment-622219056