Subject: [quicwg/base-drafts] Server's preferred address (#1251)

From: Mike Bishop <[email-masked]>
Date: Tue, 27 Mar 2018 18:24:02 +0000 (UTC)

Fixes #560; this is a specific post-handshake handoff by the server to a new IP address.  A more general ability to hand off to a different server address is left for the future, though it would probably take the form of a frame which is substantially similar to the new transport option.

One new requirement which isn't really specific to this but which this makes more obvious:  If you're trying to do connection migration, but you get a Stateless Reset on the new path, that doesn't mean the connection gets dropped -- it just means that IP from your new address took you to the wrong server, and your new address can't be used to continue the connection.

Based on `migration3`; **DO NOT MERGE** until #1249 has landed and the target of this PR has been changed.
You can view, comment on, or merge this pull request online at:

  https://github.com/quicwg/base-drafts/pull/1251

-- Commit Summary --

  * First pass at server preferred address

-- File Changes --

    M draft-ietf-quic-transport.md (132)

-- Patch Links --

https://github.com/quicwg/base-drafts/pull/1251.patch
https://github.com/quicwg/base-drafts/pull/1251.diff

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1251

---

From: ianswett <[email-masked]>
Date: Tue, 27 Mar 2018 12:00:44 -0700

ianswett approved this pull request.

Looks good, though I'm still a bit confused on the definition of probing/non-probing packets.

> +ID provided in the preferred_address transport parameter.
+
+If path validation succeeds, the client SHOULD immediately begin sending all
+non-probe packets to the new server address using the new connection ID and
+discontinue use of the old server address.  If path validation fails, the client
+MUST continue sending all future packets to the server's original IP address.
+
+### Responding to Connection Migration
+
+A server might receive a packet addressed to its preferred IP address at any
+time during the connection after the handshake is completed.  If this packet
+contains a PATH_CHALLENGE frame, the server sends a PATH_RESPONSE frame as per
+{{migrate-validate}}, but the server MUST continue sending all other packets
+from its original IP address.
+
+Once the server has received a non-probing packet on its preferred address which

I'm still confused on what probing and non-probing packets are.  Did I just miss the definition?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1251#pullrequestreview-107424497

---

From: Mike Bishop <[email-masked]>
Date: Tue, 27 Mar 2018 19:03:48 +0000 (UTC)

MikeBishop commented on this pull request.

+

Line 1612 defines the "probing frames" and talks about "packets containing other frames," but you're correct that there's not as clear a definition as I'd thought.  Let's add that in the top-level migration PR.  @janaiyengar?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1251#discussion_r177537419

---

From: ianswett <[email-masked]>
Date: Tue, 27 Mar 2018 12:04:56 -0700

ianswett commented on this pull request.

+

@janaiyengar Already merged that, so I'd suggest we clarify it in this PR?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1251#discussion_r177537706

---

From: ianswett <[email-masked]>
Date: Tue, 27 Mar 2018 19:06:47 +0000 (UTC)

And thanks for writing this up Mike.  This is a great traffic management feature, and it's great to have this in an early draft, so different groups can ensure it works as intended.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1251#issuecomment-376639650

---

From: janaiyengar <[email-masked]>
Date: Tue, 27 Mar 2018 18:41:12 -0700

janaiyengar commented on this pull request.

High-order comment: I think this should be called something else, since the parallel with Client Connection Migration might be misleading.

> @@ -1485,6 +1505,11 @@ path validation with other frames.  For instance, an endpoint may pad a packet
 carrying a PATH_CHALLENGE for PMTU discovery, or an endpoint may bundle a
 PATH_RESPONSE with its own PATH_CHALLENGE.
 
+Differences in routing on the Internet might cause the same destination address
+and connection ID to reach a different server instance which does not posses the
+necessary connection state. Receiving a Stateless Reset in response to a probing
+packet SHOULD NOT terminate the connection, but MUST cause the endpoint to
+consider path validation to have failed.

nice catch.

> +QUIC allows servers to accept connections on one IP address and attempt to
+transfer these connections to a more preferred address shortly after the
+handshake.  This section describes the protocol for migrating a connection to a
+new server address.
+
+Migrating a connection to a new server address mid-connection is left for future
+work. If a client receives packets from a new server address not indicated by
+the preferred_address transport parameter, the client SHOULD discard these
+packets.
+
+### Initiating Connection Migration
+
+A server initiates connection migration by including the preferred_address
+transport parameter in the TLS handshake.
+
+Once the handshake is finished, the client MUST initiate path validation (see

Requiring the client to use the server's preferred address seems unnecessary.

> +Migrating a connection to a new server address mid-connection is left for future
+work. If a client receives packets from a new server address not indicated by
+the preferred_address transport parameter, the client SHOULD discard these
+packets.
+
+### Initiating Connection Migration
+
+A server initiates connection migration by including the preferred_address
+transport parameter in the TLS handshake.
+
+Once the handshake is finished, the client MUST initiate path validation (see
+{{migrate-validate}}) of the server's preferred IP address using the connection
+ID provided in the preferred_address transport parameter.
+
+If path validation succeeds, the client SHOULD immediately begin sending all
+non-probe packets to the new server address using the new connection ID and

s/non-probe/non-probing/

+

Fixed in #1253 

+is the largest packet number seen so far, the server begins sending to the
+client exclusively from its preferred IP address.

What should the server do if a subsequent packet from the client appears on the old address?

+contains a PATH_CHALLENGE frame, the server sends a PATH_RESPONSE frame as per
+{{migrate-validate}}, but the server MUST continue sending all other packets
+from its original IP address.
+
+Once the server has received a non-probing packet on its preferred address which
+is the largest packet number seen so far, the server begins sending to the
+client exclusively from its preferred IP address.
+
+
+## Interaction of Client and Server Migration
+
+A client might need to perform a connection migration before the server's
+connection migration has completed.  In this case, the client SHOULD perform
+path validation to both the original and preferred server address from the
+client's new address concurrently.

This is tricky. Where should the client send data if its old address is unusable?
(I want to spend more cycles thinking about this, but I'll be out. Do make sure that all corner cases are covered; I worry that there are more here.)

> @@ -1791,6 +1811,64 @@ number. "packet_number_secret" is derived from the TLS key exchange,
 as described in Section 5.6 of {{QUIC-TLS}}.

+## Server Connection Migration {#migration-server}

I don't think this should be called Server Connection Migration. There's a symmetry with Client Connection Migration that this name suggests, but that's not true, since the assumptions are quite different. How about "Using a Preferred Server Address"?

Along the same lines, I think you should state any assumptions here clearly, since these have implications. I can think of at least one: The server is not actually changing network attachment points. I believe this assumption is implicit in this design.

> @@ -1791,6 +1811,64 @@ number. "packet_number_secret" is derived from the TLS key exchange,
 as described in Section 5.6 of {{QUIC-TLS}}.

+## Server Connection Migration {#migration-server}
+

I would add a short description of where this is expected to be used.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1251#pullrequestreview-107516729

---

From: Martin Thomson <[email-masked]>
Date: Wed, 28 Mar 2018 02:47:42 +0000 (UTC)

martinthomson requested changes on this pull request.

This needs working group review and feedback.  Especially the part where support of this feature is mandatory.  We need consensus that forcing clients to do this is a good thing.

(For the record, I don't think that the changes needed are big, but I'll request changes until it's clear that there is consensus to adopt the change.)

> @@ -1105,6 +1107,14 @@ language from Section 3 of {{!I-D.ietf-tls-tls13}}.
       };
       TransportParameter parameters<22..2^16-1>;
    } TransportParameters;
+
+   struct {
+     enum { IPv4(4), IPv6(6), (15)} ip_version;

Please be consistent about snake_case and camelCase.

> @@ -1105,6 +1107,14 @@ language from Section 3 of {{!I-D.ietf-tls-tls13}}.
       };
       TransportParameter parameters<22..2^16-1>;
    } TransportParameters;
+
+   struct {
+     enum { IPv4(4), IPv6(6), (15)} ip_version;
+     opaque ip_address<4..2^8-1>;
+     uint16 port;
+     opaque connectionId<4..18>;

To be clear, this only works if the server chooses to use a connection ID.

Can we make this change in a separate PR?

drop "during the connection", it's redundant with "after the handshake"

I think that this is problematic.  Managing two migrations concurrently is just hard.  Though your stated design works (effectively trialing all three new paths), it's unnecessarily complicated.

I would revise the client migration and forbid it (or state that it isn't going to work) until the server has finished its migration.  That is, the client has to wait until after the server's migration occurs.  The odds that a client is forced to migrate in this period is low enough that starting over might be better than taking on the combinatorial mess.  That way leads to ICE.

A potential problem here is that the client concludes that the handshake is finished before the server does.  That leads to retransmissions of the client Finished getting confused.  I can't see any concrete issues (other than the ones we already have), but you should convince yourself that it's OK.

+connection migration has completed.  In this case, the client SHOULD perform
+path validation to both the original and preferred server address from the
+client's new address concurrently.
+
+If path validation of the server's preferred address succeeds, the client MUST
+abandon validation of the original address and migrate to using the server's
+preferred address.  If path validation of the server's preferred address fails,
+but validation of the server's original address succeeds, the client MAY migrate
+to using the original address from the client's new address.
+
+If the connection to the server's preferred address is not from the same client
+address, the server MUST protect against potential attacks as described in
+{{address-spoofing}} and {{on-path-spoofing}}.  In addition to intentional
+simultaneous migration, this might also occur because the client's access
+network used a different NAT binding for the server's preferred address.

This could be clearer: the server probes toward the client when?  After receiving a probe (no?), or after receiving non-probing packets (yes?).

> @@ -1558,7 +1583,7 @@ failure. Primarily, this happens if a connection migration to a new path is
 initiated while a path validation on the old path is in progress.

-## Connection Migration {#migration}
+## Client Connection Migration {#migration-client}

I don't think that we should make this "client" connection migration.  This is the generic description, and the addition you made below is just a special case for the server that can be initiated during the handshake.

I like (and prefer) the suggested name change.

When you say that not changing network attachment is a necessary assumption, why is that important?  The new address will be validated and congestion control rebooted in the same way that any other migration would be.  So even if this points to a completely different server instance (with keys being shipped across as needed), it still works.

This feature is only effective if it can be relied upon.  That said, it's a big deal, so we should talk about it.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1251#pullrequestreview-107519251

---

From: MikkelFJ <[email-masked]>
Date: Tue, 27 Mar 2018 23:06:29 -0700

This proposal is sort of acting like ones own DNS server. I'm not against doing this, but I think are corner cases to consider.
- The same server can use multiple load balancers and it is normal (and slow) to have clients fail over when one load balancer goes belly up. A server that wants to move the client may want to list multiple IP addresses on the new destination. It could be resolved by randomly picking one per connection, only the server might not know that one is down.
- Is it always the case that a server wends on the same address as it receives (via translation in path if necessary)?
- Certs are tied an IP and changing IP is an attack vector on the servers configuration system - though being able to present a cert on the original addres gives some level comfort.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1251#issuecomment-376772197

---

From: Christian Huitema <[email-masked]>
Date: Fri, 30 Mar 2018 10:26:07 -0700

huitema commented on this pull request.

Let's keep it simple. This PR is about triggering a migration at the end of the handshake. Given the prevalence of NAT, the path challenge sent by the client to a new server address will very often arrive at the server with a different source address than the source address of the client initial packet, and there is not much the client can do about that. The default source address of the client might also change, but from a server point of view that's just the same as any client migration. So in most cases we will see "challenge from client, response and challenge from server, response from client." Both server and clients will have to consider whether sending data before the verification completes, but that's what they always do.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1251#discussion_r178332770

---

From: Christian Huitema <[email-masked]>
Date: Fri, 30 Mar 2018 10:32:10 -0700

Really? The use case is that the server sends both a unicast address and a connection ID. In theory, the unicast address guarantees that the packet will reach the specific server farm that sent the redirect, and the connection ID guarantees that the load balancer will send the packet to the right server in the farm. Receiving a stateless reset is an indication that something is seriously wrong.

Or maybe you are not addressing this specific PR, but rather making a generic statement about probes. In which case I would suggest moving this to a different PR.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1251#discussion_r178333923