Subject: [quicwg/base-drafts] Handshake Flow Control can result in hung connections (#1764)

From: martinduke <[email-masked]>
Date: Tue, 18 Sep 2018 08:10:16 -0700

Section 4.7 of the handshake draft states:

>    Servers MUST NOT send more than three datagrams including Initial and
>    Handshake packets without receiving a packet from a verified source
>    address.  Source addresses can be verified through an address
>    validation token (delivered via a Retry packet or a NEW_TOKEN frame)
>    or by receiving any message from the client encrypted using the
>    Handshake keys.

As written, there are two scenarios where no one can transmit. If the client successfully sends the initial ACK to the server, and *either* the server's handshake flight or the client's handshake acks are lost, then (1) the client has nothing to do, and (2) the server does not have the flow control credit to retransmit the lost packets.

It would be nice to minimize the resource and amplification implications for the server. Ekr proposed the following conditions that apply when the path has not been verified:
1) The client initial must be padded to 1200 bytes, as today.
2) The client must retransmit its initial until it has a Handshake Crypto frame to send (i.e. it has received the entire server's first flight.)
3) Until the server gets an hs pkt (the path has been verified), it only retransmits in response to the Client Initial; this response is all server packets that have not been acked. With no retransmissions, the server will need to have some kind of timer to clean up the state if the connection is dead.

There are many other ways to solve this problem; this is an example.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1764

---

From: ianswett <[email-masked]>
Date: Tue, 18 Sep 2018 18:43:42 +0000 (UTC)

I would prefer to call this a limit on handshake amplification, not flow control FWIW, but the outlined solution seems reasonable.

Another option might be if once the client has handshake keys, if there is nothing to send immediately, send a PING in a HANDSHAKE packet to prove the client is on path.  And until the client has handshake keys, it needs to retransmit the Initial, as you outlined above.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1764#issuecomment-422503904

---

From: martinduke <[email-masked]>
Date: Fri, 21 Sep 2018 17:30:44 +0000 (UTC)

One related issue for the P-R: We need to allow PING frames in Handshake and Initial Packets.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1764#issuecomment-423614073

---

From: Martin Thomson <[email-masked]>
Date: Fri, 21 Sep 2018 18:38:14 -0700

Why?  It should be sufficient to send PADDING for this case.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1764#issuecomment-423707851

---

From: martinduke <[email-masked]>
Date: Fri, 21 Sep 2018 20:20:53 -0700

I was under the impression we'd agreed to that.

It can certainly work with just PADDING, but that's more complicated. If the client simply sends a PING right when it gets the Handshake Keys, it is able to use the normal retransmission logic without any odd special cases around handshake timers.

(Setting aside the multiple server-hello case)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1764#issuecomment-423712859

---

From: martinduke <[email-masked]>
Date: Fri, 21 Sep 2018 22:02:07 -0700

(Or the case where the server's ack gets through, but not the server hello - that wl require a timer to trigger a ping or padding)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1764#issuecomment-423717091

---

From: Martin Thomson <[email-masked]>
Date: Sat, 22 Sep 2018 09:41:51 -0700

I had a different conception of this.  That is, the client runs the Initial timer fairly aggressively for longer.  That is, you set a timer for retransmitting Initial+ClientHello and rather than stopping that timer when you get a packet from the server, you stop it when either the handshake completes (or you receive an ACK in a Handshake packet).

The tweak being that you can send Initial with the ClientHello in it, but then switch to sending anything else when you've received proof that the server got your message (basically, any response other than Retry).

At that point, it doesn't matter what you put in the packet.  I was thinking that triggering standard retransmission logic would be bad in this case because you don't really have anything to send, it's just a matter of giving the server primed enough incoming bytes that it can send enough outgoing.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1764#issuecomment-423756834

---

From: ianswett <[email-masked]>
Date: Sat, 22 Sep 2018 17:42:43 -0700

Martin, I think that works out fine, and I think it gives us a very specific piece of guidance
If the handshake timer goes off on the client side, and you don't have Handshake keys or anything outstanding, send a full-sized padding packet.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1764#issuecomment-423783240

---

From: martinduke <[email-masked]>
Date: Sat, 22 Sep 2018 18:31:01 -0700

That works for me.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1764#issuecomment-423785014

---

From: MikkelFJ <[email-masked]>
Date: Sat, 22 Sep 2018 23:04:39 -0700

What if the server is just counting packets? Then some clients would not be inclined to send a full initial packet, but the smallest possible that will convince the server to proceed. The server would just increment the count and ignore duplicates.

What if the server incorrectly fails to count packets that are considered duplicates - so handshakes still hang - i.e. specific advice is needed to prevent this?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1764#issuecomment-423794180

---

From: ianswett <[email-masked]>
Date: Sun, 23 Sep 2018 04:46:36 -0700

Yes, this text is wrong:

"Servers MUST NOT send more than three datagrams including Initial and
Handshake packets without receiving a packet from a verified source
address"

I believe it should be "Server MUST NOT sent more than 3 times as many bytes as the number of bytes received before receiving a packet from a verified source address."?

If a server decides to implement in packets instead of bytes, then it's opening itself up to an attack which isn't in it's own best interest, so I think the above MUST NOT is likely fine?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1764#issuecomment-423810608

---

From: Martin Thomson <[email-masked]>
Date: Sun, 23 Sep 2018 09:23:52 -0700

We can fix the byte-counting vs. packet-counting thing at the same time.

The datagram thing was OK if you assumed that the inbound datagrams were full-sized, but that assumption no longer holds.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1764#issuecomment-423828511