Subject: [quicwg/base-drafts] Reserve Long Header Packet Type for QUIC-LB (#1980)

From: Nick Banks <[email-masked]>
Date: Wed, 07 Nov 2018 07:04:34 -0800

This issue is directly related to @martinthomson forthcoming changes to the first octet.

The [QUIC-LB](https://github.com/martinduke/draft-duke-quic-load-balancers/blob/master/draft-duke-quic-load-balancers.md) doc uses the QUIC packet framing to communicate between the load balancer and backend servers, because it is the easiest way for the backend server to have built in support. It already has to handle QUIC packets obviously. In order to differentiate QUIC-LB traffic from normal connection flows, QUIC-LB [defines a new QUIC packet type](https://github.com/martinduke/draft-duke-quic-load-balancers/blob/master/draft-duke-quic-load-balancers.md).

If the first octet is changed to only allow for the currently defined packet types in the transport spec, this will cause problems for QUIC-LB. So, we'd like to reserve 1 packet type in the transport spec.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1980

---

From: Kazuho Oku <[email-masked]>
Date: Wed, 07 Nov 2018 10:21:38 -0800

Can’t you use a different version number instead?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1980#issuecomment-436725889

---

From: Nick Banks <[email-masked]>
Date: Wed, 07 Nov 2018 10:23:03 -0800

The goal was to match the QUIC-LB version with the transport version.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1980#issuecomment-436726415

---

From: Kazuho Oku <[email-masked]>
Date: Wed, 07 Nov 2018 10:38:44 -0800

Yeah. While that sounds like a logical point of view from QUIC-LB's perspective, I wonder if it is possible to have such a property guaranteed, because IMO it sounds like requesting an invariant for (at this moment) an individual draft.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1980#issuecomment-436731614

---

From: Nick Banks <[email-masked]>
Date: Wed, 07 Nov 2018 18:49:43 +0000 (UTC)

We're not asking for an invariant. The reserved packet type in each version of QUIC-LB could be different.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1980#issuecomment-436735280

---

From: Subodh Iyengar <[email-masked]>
Date: Wed, 07 Nov 2018 19:08:13 -0800

I've spoken about this offline with @martinduke. I think that using a QUIC packet for this is not necessary at all and might be detrimental in some cases. For example, there is a circular dependency to talk to a particular server to assign it a connid when it doesn't currently have a connid assigned to it. 

I think this will open up a can of worms though. When we add a packet type for a lb packet, we might want to add a packet type for a health check packet, etc. and other use cases which adds up to several bits. 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1980#issuecomment-436857959

---

From: Nick Banks <[email-masked]>
Date: Wed, 07 Nov 2018 19:28:02 -0800

@siyengar I disagree with both of your points. For one, the QUIC-LB explicitly doesn't use CIDs. The SCID/DCID length fields are zero. Second, the QUIC-LB is itself an extensible packet. It has subtypes that can be used for configuration or things like the health check. That kind of stuff is exactly what this new packet type would allow. Because of migration, QUIC is a lot more complicated that TCP or plain UDP when it comes to load balancing so we need a standard way to do this, ideally built into the QUIC server implementations that plan to run behind a load balancer. The path of least resistance is to use the same framing logic that those server already have.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1980#issuecomment-436861077

---

From: Kazuho Oku <[email-masked]>
Date: Wed, 07 Nov 2018 19:36:16 -0800

@nibanks 
> We're not asking for an invariant. The reserved packet type in each version of QUIC-LB could be different.

Yeah. My point is that requiring each QUIC version to follow a particular design constraint (in this case reserve a code point) is what Invariants is.

As pointed out in 
https://github.com/quicwg/base-drafts/issues/1980#issuecomment-436731614, the Invariants draft provides you a way to coalesce other types packets; that is to use a private version number. Would you mind elaborating why you think it's illogical for QUIC-LB to use that mechanism?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1980#issuecomment-436862337

---

From: Nick Banks <[email-masked]>
Date: Wed, 07 Nov 2018 20:18:40 -0800

@kazuho the QUIC-LB packet would be no more invariant than the Initial long header packet. I don't see how you could call it invariant. Perhaps in V2, we do something different, maybe just a different value, maybe removed all together, maybe request more packet types. But bottom line, we are requesting 1 QUIC v1 packet type right now. Nothing invariant.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1980#issuecomment-436868423

---

From: Marten Seemann <[email-masked]>
Date: Wed, 07 Nov 2018 21:00:43 -0800

I don't like the idea of using an additional packet type, especially since the current mapping has the nice property that we're actually using all 4 encodable packet types.
Using a new packet type, which is not an invariant, has the additional unpleasant property that you'll potentially have update QUIC-LB every time we're releasing a new QUIC version.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1980#issuecomment-436873970

---

From: Nick Banks <[email-masked]>
Date: Wed, 07 Nov 2018 22:04:09 -0800

Having the QUIC-LB be a versioned interface is a pro in my mind, not a con. We would only updated version numbers if necessary, but having the ability to do so is a good thing.

Get Outlook for Android<https://aka.ms/ghei36>

________________________________
From: Marten Seemann <[email-masked]>
Sent: Wednesday, November 7, 2018 9:00:43 PM
To: quicwg/base-drafts
Cc: Nick Banks; Mention
Subject: Re: [quicwg/base-drafts] Reserve Long Header Packet Type for QUIC-LB (#1980)

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fquicwg%2Fbase-drafts%2Fissues%2F1980%23issuecomment-436873970&data=02%7C01%7Cnibanks%40microsoft.com%7C12f9f04c75a04e8fc31608d64537233b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636772500449656410&sdata=rF4itPQvy05OsEYjaWOGqowf7ztieLzM5Fw60LhumkY%3D&reserved=0>, or mute the thread<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FATtNBVDNSvOFaRmvHrboM_IjNg6ncJ3Uks5us7p7gaJpZM4YSmjs&data=02%7C01%7Cnibanks%40microsoft.com%7C12f9f04c75a04e8fc31608d64537233b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636772500449656410&sdata=Hq3mLBmLrN7IUfE7bdHOaQmtgmFxgdatRkSJIz0wlls%3D&reserved=0>.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1980#issuecomment-436883385

---

From: martinduke <[email-masked]>
Date: Thu, 08 Nov 2018 20:06:27 -0800

While I agree with @nibanks that an additional codepoint would be desirable, if the WG likes it better QUIC-LB might simply use the RETRY codepoint. It will likely be straightforward to disambiguate this, if we take a little care.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1980#issuecomment-437243485

---

From: Martin Thomson <[email-masked]>
Date: Sun, 11 Nov 2018 17:13:57 -0800

I think that we probably want to have a deeper conversation about this draft.  I look at it and ask why you need to create a bespoke protocol here at all.  If it is going to look like QUIC proper, why not make it be QUIC proper?

But even assuming that you do need that, why are you concerned about making this look like QUIC proper?  I can't see why you would need that for routing purposes, because it seems like you should be back to 5-tuples for routing on this bit.  I can't see why you would concern yourself with ossification on a link that you presumably have considerable influence over.  I can see why you might want to disambiguate this connection from "real" connections, but you could use a separate port, ALPN, or even a new QUIC version if it came to that (though I would advise against a version).

On response to @marten-seemann's comment, which I agree with, you might also like to consider the effect of having 5 types in the long header and what that does for protocol ossification.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1980#issuecomment-437724235

---

From: janaiyengar <[email-masked]>
Date: Mon, 12 Nov 2018 03:29:24 -0800

Reserving a codepoint would be putting the horse before the cart. Let's
first get the QUIC-LB draft to a point where it is a wg activity, then
discuss the designs in there, and agree on a need for a packet-type in the
wg. I don't think it makes sense to reserve a codepoint before we've agreed
on the mechanisms in the draft first.

On Mon, Nov 12, 2018 at 8:13 AM Martin Thomson <[email-masked]>
wrote:

> I think that we probably want to have a deeper conversation about this
> draft. I look at it and ask why you need to create a bespoke protocol here
> at all. If it is going to look like QUIC proper, why not make it be QUIC
> proper?
>
> But even assuming that you do need that, why are you concerned about
> making this look like QUIC proper? I can't see why you would need that for
> routing purposes, because it seems like you should be back to 5-tuples for
> routing on this bit. I can't see why you would concern yourself with
> ossification on a link that you presumably have considerable influence
> over. I can see why you might want to disambiguate this connection from
> "real" connections, but you could use a separate port, ALPN, or even a new
> QUIC version if it came to that (though I would advise against a version).
>
> On response to @marten-seemann <https://github.com/marten-seemann>'s
> comment, which I agree with, you might also like to consider the effect of
> having 5 types in the long header and what that does for protocol
> ossification.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/quicwg/base-drafts/issues/1980#issuecomment-437724235>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AKjg1LcqhgGp84gRP6L8Rf5GRvLZPx1Cks5uuMtVgaJpZM4YSmjs>
> .
>

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1980#issuecomment-437846835

---

From: Nick Banks <[email-masked]>
Date: Mon, 12 Nov 2018 07:28:23 -0800

@janaiyengar my understanding is that the current goal for QUIC-LB is to get it adopted by the WG ASAP (@martinduke please correct me if I'm wrong). Also, I'm not explicitly asking for a new value to be reserved JUST for QUIC-LB, just that we have a provision in the transport to allow something else to use another value.

@martinthomson the entire goal of the QUIC-LB draft is to define a protocol to allow load balancers to configure backend servers, with the primary goal focused on configuration for CID routing (though it may be used for other things in the future: RETRY token encoding, 0-RTT ticket keys, ...). We aren't worried about how the configuration packets are going to be routed to the backend servers. Obviously they can be sent directly. As I stated before, they don't even have CIDs in them anyways. These packets don't create connections between the backend server and load balancer. It would be possible to not use special QUIC packets, but instead to design the protocol on top of QUIC connections, but that would add a LOT of extra complexity. Right now, no TLS is required. Another goal for this protocol is to have it easily implementable in hardware. 

As to the possible ossification downside of this, I agree it could be a concern. As QUIC-LB's intended use of this is just between the backend server and load balancers, you'd never see this value on the open internet. Perhaps it can be reserved and greased on the open internet. IDK...

@martinduke I agree that we have a last resort of repurposing the RETRY code point to our own purposes, but that seems really ugly from a design and an implementation perspective. I'd really prefer to only do that as a last resort.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1980#issuecomment-437922256

---

From: Kazuho Oku <[email-masked]>
Date: Mon, 12 Nov 2018 07:59:23 -0800

As previously stated, QUIC already has an extension point, which is the version number field of a long header packet.

Any protocol including QUIC-LB can define it's own 4 octets to distinguish itself from other traffic.

For example, QUIC-LB can declare it self to use 0x1b000000 to identify itself. Then, if you need to have a version number field within QUIC-LB, you can create your own LB version field in the payload area.

I do not think we need any other extension mechanism, for QUIC LB or whatever.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1980#issuecomment-437933598

---

From: Nick Banks <[email-masked]>
Date: Mon, 12 Nov 2018 08:03:07 -0800

@kazuho I agree that is another solution. I would prefer that over repurposing the RETRY packet type. It would possibly add some complexity on how VN should work. Should servers advertise support of that version? If so, should QUIC-LB 'sanitize' it when sent out over the open internet?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1980#issuecomment-437934967

---

From: Kazuho Oku <[email-masked]>
Date: Mon, 12 Nov 2018 16:07:47 +0000 (UTC)

@nibanks 
> It would possibly add some complexity on how VN should work. Should servers advertise support of that version? If so, should QUIC-LB 'sanitize' it when sent out over the open internet?

IIUC the QUIC-LB protocol (or whatever we call it) is not a variant of QUIC. It's a completely different protocol that happen to have the need to be demuxed from ordinary QUIC protocol.

Assuming that is true, I think you do not need to include it in VN. All you need is to use it for demuxing.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1980#issuecomment-437936653

---

From: ianswett <[email-masked]>
Date: Mon, 12 Nov 2018 08:42:57 -0800

I'd prefer to reserve a version for QUIC-LB instead of changing the core QUIC transport or abusing RETRY.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1980#issuecomment-437949036

---

From: martinduke <[email-masked]>
Date: Mon, 12 Nov 2018 17:19:43 +0000 (UTC)

The objective we are trying to meet here is to support cases where the load balancer to server path is not trusted. To do this, we believe we need to pass through middleboxes that are dropping things that don't like QUIC.

At the same time, we are reluctant to make load balancers support a full implementation of QUIC.

I am fairly agnostic as to the solution as long as it looks "enough" like QUIC without requiring a full implementation. When there were more available codepoints, it was simple enough to just take 0xfb. I see the ossification problems with going to 8 codepoints instead of 4.

Whatever the design team comes up with will have to get ratified by the WG anyway. If the least objectionable choice is a different version, we can proceed with that.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1980#issuecomment-437961591

---

From: MikkelFJ <[email-masked]>
Date: Mon, 12 Nov 2018 11:42:14 -0800

A subset of QUIC sounds like a new version with implicit ties to an existing version. LB could take advantage of version negotiation and get encryption, handshake, and version stability without going full QUIC and without inventing its own protocol.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1980#issuecomment-438004702

---

From: janaiyengar <[email-masked]>
Date: Mon, 12 Nov 2018 22:45:48 -0800

@nibanks -- it's still cart before the horse. You want people to discuss the mechanisms in QUIC LB first (which is now happening a bit on this thread) before reserving a codepoint for one particular design.

FWIW, I am with @kazuho and @ianswett -- let's use the Version field for demuxing.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1980#issuecomment-438155849

---

From: MikkelFJ <[email-masked]>
Date: Mon, 12 Nov 2018 23:57:26 -0800

With VN an LB should detect a requested LB version and only offer LB versions in response. It should not offer LB versions normally. It would probably also filter on IP range and use a different port when possible.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1980#issuecomment-438170813

---

From: Nick Banks <[email-masked]>
Date: Tue, 13 Nov 2018 06:37:01 -0800

Thanks folks for all the input. I think we can work reasonably well with a different version number.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1980#issuecomment-438287974

---

From: Nick Banks <[email-masked]>
Date: Tue, 13 Nov 2018 06:37:01 -0800

Closed #1980.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1980#event-1963194227

---

From: MikkelFJ <[email-masked]>
Date: Tue, 13 Nov 2018 15:50:24 +0000 (UTC)

Note that LB to backend encryption is of interest to this protocol: Some cloud providers offer a fast internal network but the network is not invisible to other customers and at the very least injection attacks are a concern. For this reason a basic QUIC handshake does make sense. If cloud providers can agree on a standard protocol it would be simpler to deploy cloud services in a generic fashion.

I kind of hope something similar could be made for updating name servers.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1980#issuecomment-438315896