Subject: [quicwg/base-drafts] Connection migration must be negotiated (#1271)

From: Praveen Balasubramanian <[email-masked]>
Date: Fri, 06 Apr 2018 16:27:53 +0000 (UTC)

Connection migration is an optional feature of the QUIC transport protocol. An implementation should be allowed to omit it.

Server considerations:
Scalable  server deployment requires load balancing. Connection migration will require QUIC aware support from the load balancer. Most load balancers available today do not have this support. If a server does not support connection migration or is behind a load balancer that is known to not support QUIC, then all migrated traffic looks like attack traffic and will likely trip up DoS mitigation and cause a lot of traffic to failover to TCP unnecessarily.

Client considerations:
Many client devices will only have one network card or only one network available at a time. Most desktops, laptops and many tablets will qualify. Even of mobile devices, OS policy may dictate that metered connections must not be used aggressively or automatically. Applications may have policy or requirement that their traffic must not flow over specific interfaces - for example update traffic today must not flow over metered networks. 

For these reasons, clients which do implement migration, should make a negotiate attempt for connection migration support based on application and OS policy. Servers that know that they can support migration should negotiate and accept the request. 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1271

---

From: Nick Banks <[email-masked]>
Date: Fri, 06 Apr 2018 16:49:17 +0000 (UTC)

On the client vs server option, I think it would be fine to state that if a client doesn't want to allow/support connection migration, it just doesn't do it. No negotiation needed. This is because server migration isn't currently a supported scenario for V1.

As to how to negotiate it server side, I'd prefer to use QUIC transport parameters, instead of different version numbers. A new, optional TP could be defined, `disallow_connection_migration`, that the server could set to indicate it doesn't allow/support connection migration. Instead, if needed, the client should close existing connections and start a new (0-RTT) connection.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1271#issuecomment-379311402

---

From: ianswett <[email-masked]>
Date: Fri, 06 Apr 2018 16:52:30 +0000 (UTC)

>From an operational perspective, I definitely support having a way to indicate to the client that connection migration is not supported, since there are servers that don't support connection migration in the gQUIC deployment, for various reasons.

Currently gQUIC has the equivalent of a tranport param to indicate it's not supported.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1271#issuecomment-379312234

---

From: Tommy Pauly <[email-masked]>
Date: Fri, 06 Apr 2018 10:07:59 -0700

@pravb I disagree that this would be an uncommon client use case. It's important to distinguish migration from multipath; the former does not imply simultaneous use of two interfaces, just a transition between interfaces. This is exceedingly common on all mobile devices (moving between Wi-Fi, LTE, and back again), and even quite common for laptops that get plugged into Ethernet, and may be unplugged. The lack of support for resiliency at the transport layer has plagued many connections.

I agree that if there *is* an option to be negotiated, it should be an explicit disabling of the feature, and we should assume that migration is enabled by default. Opt-out is fine, opt-in is too limiting.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1271#issuecomment-379316349

---

From: Praveen Balasubramanian <[email-masked]>
Date: Fri, 06 Apr 2018 17:13:29 +0000 (UTC)

Not all devices are mobile :-), please see "client considerations" section or various reasons the client app needs control over whether to do connection migration. I believe Apple devices also have a setting to prevent aggressive use of metered connections. So app and OS policy should dictate opt-in. Of course you may choose to design the API as default opt-in but currently I am leaning the other way. This is crucial because many apps are on top of HTTP and unaware of the QUIC/TCP fork underneath and I would prefer making it opt-in to avoid things like user bill shock. 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1271#issuecomment-379317790

---

From: Praveen Balasubramanian <[email-masked]>
Date: Fri, 06 Apr 2018 10:39:29 -0700

Also decades of protocol design has always followed the negotiate pattern for any optional functionality. Case in point TCP options. Side initiating the connection requests support and the other side agrees or vetoes. Any deviation from this common design pattern will need justification. 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1271#issuecomment-379324897

---

From: Tommy Pauly <[email-masked]>
Date: Fri, 06 Apr 2018 17:54:00 +0000 (UTC)

A few points:
- Generally, restrictions are around using metered interfaces when unmetered interfaces are available; when only a metered interface is available, things are more open. Migration in this case should be generally allowed.
- Preventing use of interfaces based on metering is separate from allowing migration. I can either let user policy or API specify that a connection should never use metered interfaces, but still migrate a connection between two Wi-Fi networks as my association changes. This also works for going between Wired Ethernet and Wi-Fi.

My main concern is that implementations, especially server implementations, MUST implement migration support, even if they allow it to be configured "off".

One of the points of migration is the path validation, which applies even in the case of NAT re-binding. For clients behind NATs, they may look to the server that they are "migrating" between ports, which is quite likely with UDP to happen quickly. By default, we should make sure connections are resilient here. The connection ID does give us the necessary support to allow the server to detect the case, and validate the path back to the client.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1271#issuecomment-379328731

---

From: Nick Banks <[email-masked]>
Date: Fri, 06 Apr 2018 17:58:56 +0000 (UTC)

One problem that this is trying to address is a load balancer that does 4-tuple load balancing. In that case, even a NAT rebind might end up sending the client's new traffic to a new server. So in that case, if the client really need to keep the connection bound properly, keep alives must be used.

I agree that this option shouldn't be a copout for servers to just not support migration. Though it will be in their best interest to support it (if their infrastructure does) to provide the best experience to their clients.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1271#issuecomment-379329991

---

From: Praveen Balasubramanian <[email-masked]>
Date: Fri, 06 Apr 2018 18:17:23 +0000 (UTC)

>>My main concern is that implementations, especially server implementations, MUST implement migration support, even if they allow it to be configured "off".
There is unfortunately no way to enforce this. Regardless of server support or infra issue the server must be able to veto this. I also think since QUIC has very poor signaling for DDoS prevention, server infra may dynamically start rejection migrations when under attack.

NAT re-binding can be solved by keep-alives and existing load balancers allow 2-tuple load balancing for UDP which means a NAT-rebinding where the port changes will not suffer the same issue.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1271#issuecomment-379334910

---

From: Mike Bishop <[email-masked]>
Date: Fri, 06 Apr 2018 20:03:58 +0000 (UTC)

@tfpauly also hit on my biggest concern here:  What exactly gets turned off?  The point of the connection ID is the ability to survive NAT rebinds; ideally, the load balancer will be looking at the CID and not the 4-tuple at all.  After all, NAT rebinds can cross IP addresses (particularly in the Enterprise or mobile space), so 2-tuple binding isn't going to be sufficient.  But perhaps it's an acceptable Band-Aid for initial deployments.

So what are the protocol mechanics that get turned off?  The server is still going to need the ability to do path validation during the handshake (three-packet limit?), so that machinery still needs to be there.  If you're saying that you'll disable support for NAT rebinding, you're potentially going to be in a painful situation.

Regardless, the text added in #1254 states the principle that clients are the controlling party for all migrations; this remains true even if we merge #1251, because the server requests the client to perform a migration.  I think the most this restriction could get us is that clients should never intentionally change their local IP address -- though I suspect many clients will just be connecting without specifying a local IP address, so even that might have some interesting quirks.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1271#issuecomment-379362183

---

From: Nick Banks <[email-masked]>
Date: Fri, 06 Apr 2018 20:22:17 +0000 (UTC)

@MikeBishop ideally the load balancers will be using CID, but that won't happen overnight. We don't want that to be a blocker for deploying QUIC. As the infrastructure starts supporting QUIC, then the server won't need to set this TP.

As for what actually gets turned off, it's really up to the client implementations. If they do nothing, then certain scenarios will just break, and it might cause them to execute unnecessary fallback logic, worst case, all the way back to TCP. In the case of long lived connections, the client can use this information to be more aggressive with keep alive timers if it wants to maintain the connection. Also, as you stated, the clients should not explicitly trigger any migration scenarios. In the case of a client application layer not specifying a local IP address, the transport could still prevent any extra IP/port address changes.

Also, one argument for possibly making the QUIC TP set from both client and server is the P2P scenario. A client behind a NAT wouldn't support the peer migrating unless it supported the necessary NAT punch through protocols.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1271#issuecomment-379366601

---

From: Praveen Balasubramanian <[email-masked]>
Date: Fri, 06 Apr 2018 20:50:21 +0000 (UTC)

Yes, incremental deployment is important. Until load balancers add QUIC support, you have two options for deployment:
1. Use two-tuple (src IP, dst IP) load balancing to continue working when NAT rebinding happens (port number change).
2. Or client can do keep-alives to keep the NAT mapping open for long running persistent connections. We already have precedence for this with long poll TCP "push notification" connections. 

Connection migration != NAT rebinding. Connection migration is a new scenario that needs opt-in and is driven by app and OS policy. For connection migration the only option for deployment is to upgrade the load balancers. Hence servers must be able to veto it to minimize traffic that looks like DoS attack and also prevent client from falling back to TCP unnecessarily when it could have simply opened a new QUIC connection and carried on. 

@nibanks good point about p2p where the negotiation must be vetoable by both sides. Let's also not discount scenarios like anycast and DoS prevention where knowing that the connection can migrate can help the server side. The traditional negotiation mechanism where initiator requests and acceptor agrees/vetoes is the most generic mechanism and will satisfy all requirements.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1271#issuecomment-379373542

---

From: ianswett <[email-masked]>
Date: Fri, 06 Apr 2018 13:58:35 -0700

Praveen's point is spot on.  Today's load balancers don't know about QUIC.  The initial gQUIC deployment got by without connection ID routing for a while, and as long as the idle timeout is short, it largely works and still delivered large enough gains to be worthwhile.

I'm fine with QUIC default enabling migration and then having a mechanism for disabling it, though.  I just want to make sure we do have a way to disable it in the handshake(or version if the WG goes in that direction).

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1271#issuecomment-379376369

---

From: Ryan Hamilton <[email-masked]>
Date: Fri, 06 Apr 2018 23:30:38 +0000 (UTC)

To make sure I'm following correctly, I assume you're talking about
disabling explicit peer-initiated migration. We obviously can't prevent
NAT-rebinding based migration, right?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1271#issuecomment-379410532

---

From: ekr <[email-masked]>
Date: Fri, 06 Apr 2018 23:46:34 +0000 (UTC)

> NAT-rebinding based migration, right?

Well, we could cause it to fail, one assumes.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1271#issuecomment-379412883

---

From: Praveen Balasubramanian <[email-masked]>
Date: Fri, 06 Apr 2018 23:50:02 +0000 (UTC)

@RyanAtGoogle, yes this is strictly about explicit peer-initiated migration. As I mentioned before we have two possible options for dealing with NAT-rebinding and we cannot negotiate that anyways since it happens outside the control of the peers. The two options are: use 2-tuple load balancing OR do keep-alives and use 4-tuple load balancing. Please see my response before Ian's above. We cannot incrementally deploy peer-initiated migration with existing infra as Ian also pointed out. 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1271#issuecomment-379413340

---

From: Martin Thomson <[email-masked]>
Date: Mon, 09 Apr 2018 01:00:05 +0000 (UTC)

If the concern is that the server can't manage migrations, the server can simply not send NEW_CONNECTION_ID.

I tend to think that' would be a real shame though, because the cases that need migration are pretty significant as @tfpauly says.

If we are going to build a system that relies on client opt-in, and servers that potentially don't support the feature, then perhaps we should entertain a system whereby clients request new connection IDs rather than have them provided spontaneously.

Again, that would be unfortunate because now we have created a means by which clients are segregated based on capabilities.  Worse, that segregation might be based on their ability to pay for access, if a "metered" flag could disable migration.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1271#issuecomment-379598503

---

From: Kazuho Oku <[email-masked]>
Date: Sun, 08 Apr 2018 23:08:06 -0700

I agree with what @martinthomson says.

A server can always decide to not send a `NEW_CONNECTION_ID` frame (it's not even a "SHOULD" even though some if not many of us anticipate that the server's sending it), and then the client would notice that there is no point in doing a gratuitous migration.

I also think that for the data-center use-case, nobody would object for QUIC connections not doing gratuitous migrations.

For the connections originating from the browser, I believe that a server that does not send a `NEW_CONNECTION_ID` frame should declare an idle timeout of somewhere below 30 seconds, so that the client would try to either keep the connection alive with no rebinding (by sending pings) or to reconnect in 0-RTT after quiescence that could have caused a rebinding.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1271#issuecomment-379643507

---

From: ianswett <[email-masked]>
Date: Mon, 09 Apr 2018 13:00:33 +0000 (UTC)

I'm ok with the "if you don't support migrations, don't send a NEW_CONNECTION_ID" recommendation.  I think it's worth adding one sentence about that to the transport draft, if that's our preferred way of indicating connection migration isn't supported.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1271#issuecomment-379743434

---

From: Praveen Balasubramanian <[email-masked]>
Date: Mon, 09 Apr 2018 15:58:30 +0000 (UTC)

"if you don't support migrations, don't send a NEW_CONNECTION_ID" mechanism can work as long as it stated that using a new connection ID for gratuitous migration is a MUST. I would add this requirement in addition to the sentence that @ianswett requested. 

The only downside I can think for this mechanism is that it will preclude use of multiple connection IDs on the same path without migration. Do we foresee use of multiple connection IDs for the same connection for unreliable semantics? Is there any other use case for NEW_CONNECTION_ID outside of gratuitous migration that will get affected?

>we should entertain a system whereby clients request new connection IDs rather than have them provided spontaneously
One problem with requesting connection IDs is that this will anyways have to be really early in the connection because if client loses cellular connectivity it will not have any means to contact the server over wifi. So for the parking lot problem, up front supply of at least one connection ID seems to be necessary?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1271#issuecomment-379802605

---

From: Nick Banks <[email-masked]>
Date: Tue, 17 Apr 2018 17:17:54 +0000 (UTC)

One down side of implicitly stating (explicit) migration is not supported, just by the server not providing NEW_CONNECTION_IDs, is that the connection can't use different connection IDs for the same path, even if the server and load balancer support that scenario. Recently, the spec was updated to allow for changing the CID after some period of quiescence, with the expectation that NAT rebinding will likely happen. That wouldn't work with this model.

My personal preference here is still an explicit indication (most likely via TP) that the server cannot support explicit migration / client IP address changes.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1271#issuecomment-382072800