Subject: [quicwg/base-drafts] WIP: Begin lightly abstracting over the use of UDP as the underlying transport (#4043)

From: John Ericson <[email-masked]>
Date: Thu, 20 Aug 2020 22:07:37 -0700

Based on recent mailing list feedback that this might possibly be OK, I
decided to take a crack at reducing the &quot;hard coding&quot; of UDP in the
spirit of other RFCs which strive to be agnostic to the underlying
protocol.

Due to the controversy over this, I imposed these constraints on myself
to try to be as conservative as possible.

- Absolutely no design changes on things which are already standardized,
  just editorial changes, and very simple extrapolation of the existing
  design. (I never ever considered the former at this time, just
  documenting this for posterity.

- Anything that is not UDP (over IP) is deemed experimental. The size
  limitations I figure may not make sense for other transports / during
  other experiments, so I call that out.

- When UDP was mentioned in some interesting way that generalized (e.g.
  something that separately mentions IP address and UDP port), I provide
  the general language (plain &quot;address&quot;, but also provide the UDP common
  case as before so no specificity is lost.

The purpose of this exercise is *not* to get QUIC over UDP &quot;more ready
for production&quot;, as that would slow down the WG for little gain, but
rather to call out the dependencies that currently do exist so as to be
sure there is no unintentional coupling. I fully leave to others to
decide what coupling is intenional or unintentional, and whether
anything at all is cause for concern.

----

In this first commit, I just audited all the occurrence of &quot;UDP&quot;. If
this looks good, I would then go back and likewise scan for &quot;port&quot;
&quot;IPv4&quot;, and &quot;IPv6&quot;.

My one regret is that in keeping the line length the same, I made the
diff more complicated than it would otherwise be. Do you all have a
process for that?
You can view, comment on, or merge this pull request online at:

  https://github.com/quicwg/base-drafts/pull/4043

-- Commit Summary --

  * WIP: Begin lightly abstracting over the use of UDP as the underlying transport

-- File Changes --

    M draft-ietf-quic-transport.md (280)

-- Patch Links --

https://github.com/quicwg/base-drafts/pull/4043.patch
https://github.com/quicwg/base-drafts/pull/4043.diff

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/4043

---

From: Lucas Pardue <[email-masked]>
Date: Fri, 21 Aug 2020 04:32:06 -0700

Hi @Ericson2314. Would you mind creating an accompanying issue for this PR so that we can track continued discussion on the concept of abstracting UDP. The PR can then stay focused on discussion of the file changes.

Wrt line length, that's just something we have grown to deal with. Well done for keeping the CI checks happy on your first attempt 👍 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/4043#issuecomment-678244643

---

From: Lucas Pardue <[email-masked]>
Date: Fri, 21 Aug 2020 04:38:08 -0700

@LPardue commented on this pull request.

> @@ -263,6 +266,17 @@ Application:
 
 : An entity that uses QUIC to send and receive data.
 
+Underlying transport protocol:
+
+: The underlying protocol with which QUIC packet are sent.  The underlying
+  transport protocol is conventionally UDP over IP.
+
+Underlying transport datagram:
+
+: A message of the underlying transport protocol, which when used by QUIC will
+  contain multiple QUIC packets. Calling this a "datagram" and not a "packet"
+  emphasizes the message-oriented nature of the underlying transport, and helps
+  distinguish these from QUIC packets.

Are you suggesting that that the underling transport has to be message based? That seems to be a large caveat that restricts the value of these changes.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/4043#pullrequestreview-472418488

---

From: Lucas Pardue <[email-masked]>
Date: Fri, 21 Aug 2020 04:44:28 -0700

> @@ -2725,7 +2747,7 @@ longer than the time negotiated using the max_idle_timeout transport parameter;
 see {{termination}}.  However, state in middleboxes might time out earlier than
 that.  Though REQ-5 in {{?RFC4787}} recommends a 2 minute timeout interval,
 experience shows that sending packets every 30 seconds is necessary to prevent
-the majority of middleboxes from losing state for UDP flows
+the majority of middleboxes from losing state for underlying-transport flows

I don't believe we can make this claim generically, since the cited work does not.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/4043#pullrequestreview-472421926

---

From: Lucas Pardue <[email-masked]>
Date: Fri, 21 Aug 2020 04:52:48 -0700

> @@ -3205,6 +3230,29 @@ Packets with the short header are designed for minimal overhead and are used
 after a connection is established and 1-RTT keys are available; see
 {{short-header}}.
 
+## Underlying Transport Protocol {#underlying-transport-protocol}
+
+The underlying protocol is the protocol with which QUIC packet are sent.
+
+This protocol SHOULD NOT have automatic retransmission, ordering, congestion
+control, or other such mechanisms that would overlap with those provided by QUIC
+and therefore not actually enhance the overall quality of service.  This
+protocol SHOULD be message-oriented.

This answers my earlier point (sorry for not reading in completeness). Are there many examples of IP-based, message-based, Internet transport protocols? My sense is that the scope of these changes are putting it in the "Yah Ain't Gonna Need It" bracket.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/4043#pullrequestreview-472426103

---

From: Mike Bishop <[email-masked]>
Date: Fri, 21 Aug 2020 07:39:08 -0700

@MikeBishop commented on this pull request.

An interesting foray.  I think this generalizes some pieces that really are UDP-specific, and would need to be redefined (or at least reconsidered) in other aspects.

> @@ -1902,10 +1917,12 @@ that are uniquely attributed to a single connection. This includes datagrams
 that contain packets that are successfully processed and datagrams that contain
 packets that are all discarded.
 
-Clients MUST ensure that UDP datagrams containing Initial packets have UDP
-payloads of at least 1200 bytes, adding padding to packets in the datagram as
-necessary. A client that sends padded datagrams allows the server to send more
-data prior to completing address validation.
+Clients MUST ensure that underlying transports datagrams containing Initial
+packets have payloads of at least 1200 bytes, adding padding to packets in the
+datagram as necessary. (The payload is the assembled QUIC packets, and does not
+include any headers associated with the underlying protocol that are also part
+of the datagram.) A client that sends padded datagrams allows the server to send
+more data prior to completing address validation.

I feel like the anti-amplification features -- minimum initial size, the 3x limit, etc. -- really are UDP-focused because they mitigate attacks we know are common with UDP.  A different transport might have such concerns, or might not.  Perhaps, rather than generalizing these requirements, we should instead condition them on UDP.

> @@ -2526,13 +2543,16 @@ routing of packets across multiple network paths will also allow activity on
 those paths to be linked by entities other than the peer.
 
 A client might wish to reduce linkability by employing a new connection ID and
-source UDP port when sending traffic after a period of inactivity.  Changing the
-UDP port from which it sends packets at the same time might cause the packet to
-appear as a connection migration. This ensures that the mechanisms that support
-migration are exercised even for clients that do not experience NAT rebindings
-or genuine migrations.  Changing port number can cause a peer to reset its
-congestion state (see {{migration-cc}}), so the port SHOULD only be changed
-infrequently.
+source address when sending traffic after a period of inactivity.  (In
+conventional situations using UDP, the client cannot freely vary the IP address,

With IPv6 privacy addresses, this might not be true.

> +conventional situations using UDP, the client cannot freely vary the IP address,
+but can much more readily change the UDP port.  Since an address for QUIC when
+using UDP is the tuple of the IP address and UDP port, changing the source port
+along constitutes changing the QUIC address.) So changing the address from which
+it sends packets at the same time might cause the packet to appear as a
+connection migration.  This ensures that the mechanisms that support migration

This seems really wordy explaining why this is useful for UDP while still trying to generalize.  You already changed the specific of UDP port to "source address"; probably all you need to add is that "When the underlying transport protocol is UDP, changing the source port is sufficient."

It might also be the case that changing the source address isn't possible with the alternate transport, so this is an "if supported by the underlying protocol" thing.  (For that matter, the CID itself might be irrelevant on a transport where addresses can't change.)

```suggestion
The underlying protocol is the protocol over which QUIC packets are sent.
```

> +The underlying protocol MUST contain both a source and destination address:
+while the underlying protocol is free to route datagrams by whichever means it
+chooses, and futhermore a source address isn't strictly necessary to route a
+datagram, the QUIC implementation needs to be able to inspect both addresses for
+to implement connection migration.

```suggestion
The underlying protocol MUST contain both a source and destination address,
as the QUIC implementation needs to be able to inspect both addresses in
order to support connection migration.
```

> +An underlying datagram MUST correspond to a single IP packet, if it is sent over
+IP.  That means IP fragmentation MUST NOT be used.  When using IPv4
+({{!IPv4=RFC0791}}), the DF bit MUST be set to prevent fragmentation on the
+path.

I'm not certain QUIC generically has this requirement across all underlying transports.  The requirement is that datagrams be delivered whole to QUIC; if they were fragmented and reassembled along the way, that's irrelevant.  For UDP, we don't fragment because we want to be able to assess the maximum packet size; for some other transport, we might be able to query the maximum size directly or have effectively-infinite maximum size.

In fact, given the way encryption overhead decreases with larger units of encryption, we might find that very large datagrams perform really well in certain environments with certain transports, even in the presence of fragmentation.

> +The QUIC packet size is just the total size of the QUIC header and protected
+payload. It follows that the sizes of any other headers that are part of the
+underlying datagram and associated with underlying protocols are not counted.

```suggestion
The QUIC packet size includes the QUIC header and protected payload, but not the
headers of any underlying protocol.
```

> @@ -4030,12 +4086,16 @@ address of the client; see {{address-validation}}.
 
 ## Path Maximum Transmission Unit

Much of the work referenced here is specific to UDP.  I'd leave this as UDP-specific and merely note that a QUIC implementation on a different transport might need to use similar techniques to learn the properties of the path.

>  : The maximum UDP payload size parameter is an integer value that limits the
-  size of UDP payloads that the endpoint is willing to receive.  UDP packets
-  with payloads larger than this limit are not likely to be processed by the
-  receiver.
+  size of underlying datagram payloads that the endpoint is willing to receive.
+  Datagrams with payloads larger than this limit are not likely to be processed

A parameter that's specifically named UDP doesn't tell you anything about other underlying protocols.  Either the parameter becomes generic (i.e. renamed, perhaps `max_datagram_size`) or it remains specific to UDP and other parameters might be defined for other contexts.

> @@ -6669,7 +6736,7 @@ attacker to establish connectivity on a given path.
 An on-path attacker can:
 
 - Inspect packets
-- Modify IP and UDP packet headers
+- Modify underlying transport headers (e.g. UDP and IP headers)

Here again, it's really going to depend on what that underlying transport is.  This security analysis assumes UDP/IP; I think the safest statement is that QUIC over another transport needs its own security analysis to see what attackers might be able to do.

Personally, I'm reviewing this through the mental model of two additional transports:
- Message-based Websockets across an HTTP proxy
- Local datagram sockets for IPC

Websockets are clearly going to provide all those services which an underlying transport SHOULD NOT have, but that's just the suck you live with if that's your only way to speak QUIC to an endpoint.  Local sockets, on the other hand, mean that a lot of the machinery within QUIC simply won't be needed -- but if you're porting over a protocol that's already built atop QUIC, that might be okay.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/4043#pullrequestreview-472519287

---

From: John Ericson <[email-masked]>
Date: Thu, 27 Aug 2020 20:12:35 -0700

@Ericson2314 pushed 1 commit.

1a5c504b42132e98a57d023d8b5d2aa84f43e91b  Apply suggestions from code review

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/4043/files/ca022d8267deeb9e52ac3d35fea34fd3d959bc4c..1a5c504b42132e98a57d023d8b5d2aa84f43e91b

---

From: John Ericson <[email-masked]>
Date: Thu, 27 Aug 2020 20:28:03 -0700

c1151bc750814f8a0df75af5b1530ea4fef440fa  Merge remote-tracking branch 'upstream/master' into transport-udp-unhardcode

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/4043/files/1a5c504b42132e98a57d023d8b5d2aa84f43e91b..c1151bc750814f8a0df75af5b1530ea4fef440fa

---

From: John Ericson <[email-masked]>
Date: Thu, 27 Aug 2020 20:47:00 -0700

@Ericson2314 commented on this pull request.

 

Based on the feedback thread, I was trying to be super conservative, even if that risks an "any color as long as it's black" situation where UDP is basically the only protocol that meets the requirements.

If you think this is too cautious, I'm perfectly fine dropping it.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/4043#discussion_r478814673

---

From: John Ericson <[email-masked]>
Date: Thu, 27 Aug 2020 20:52:30 -0700

56d0b98f406c1e2de1905f456508d77f66da04fd  Don't abstract over UDP for this specific bit of empiracal evidence

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/4043/files/c1151bc750814f8a0df75af5b1530ea4fef440fa..56d0b98f406c1e2de1905f456508d77f66da04fd

---

From: John Ericson <[email-masked]>
Date: Thu, 27 Aug 2020 20:52:37 -0700

Good catch, fixed.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/4043#discussion_r478815920

---

From: John Ericson <[email-masked]>
Date: Thu, 27 Aug 2020 20:58:17 -0700

367347bec405504389addc60b280f5b60d7dce6b  Don't imply UDP without "real migrations" means constant IP.

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/4043/files/56d0b98f406c1e2de1905f456508d77f66da04fd..367347bec405504389addc60b280f5b60d7dce6b

---

From: John Ericson <[email-masked]>
Date: Thu, 27 Aug 2020 21:05:15 -0700

Thanks for the feedback so far, just knocked off the easy stuff. I'll next switch to conditioning a lot of this stuff on UDP in particular as suggested.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/4043#issuecomment-682312706

---

From: Lars Eggert <[email-masked]>
Date: Thu, 27 Aug 2020 23:33:24 -0700

FYI, I converted this to a draft PR for now so we can track it better.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/4043#issuecomment-682354072

---

From: David Schinazi <[email-masked]>
Date: Fri, 28 Aug 2020 14:03:55 -0700

I think that this PR makes the spec harder to read, with no obvious benefit. If someone wants to standardize QUIC over another medium, they'll have to write some sort of document for that, and simply saying "replace UDP with X" in that document should be good enough. I'd strongly prefer we didn't make the spec more generic at the cost of readability.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/4043#issuecomment-683145496

---

From: John Ericson <[email-masked]>
Date: Fri, 28 Aug 2020 20:05:36 -0700

@LPardue @MikeBishop So, going the basic thrust of both your comments so far, We'd end up with fewer constraints on the underlying protocol (e.g. need not be message-origin) and a good bit of information being conditioned on UDP only (ECN, MTU discovery and associated limits and restrictions, etc.). I see very different conclusions one might make based on that, such as:

1. Negative: Meaningfully generalizing QUIC is too difficult because so much is UDP specific, and we'd need to make sure the remainder stands on its own.

2. Neutral: Fine to just continue conditioning sections on being UDP-specific, and just leave fleshing out the remainder for other transports as a future exercise.

3. Positive: It's great there are so many UDP-specific details we can cut away. It would be good to reorder or even split the document to make UDP-specific vs UDP-agnostic a prominent axis around which the standard is organized.

I think before I put too much more work with it, would be good to confirm whether you agree with this analysis, and if so, which outlook you both take. Most of the broad feedback is still negative, so I don't want to just stir emotions while beating a dead horse.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/4043#issuecomment-683226997

---

From: Lars Eggert <[email-masked]>
Date: Mon, 31 Aug 2020 06:13:41 -0700

I'm seeing quite a bit of pushback on the issue and #4061. People who are in favor of making this change should please state their case soon.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/4043#issuecomment-683768828