Subject: Re: [quicwg/base-drafts] HTTP header compression: static table fit for QUIC (#904)

From: Mike Bishop <[email-masked]>
Date: Wed, 09 May 2018 12:52:05 -0700

I'm going to propose we divide this into two issues.  It seems like we have agreement to replace QPACK's static table; I've received a bit of feedback on the draft that hasn't been incorporated yet, but will probably turn that into a PR soon.

If there's appetite to put in a table negotiation mechanism, let's do that separately.  I've opened #1343 to track that.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/904#issuecomment-387855415

---

From: afrind <[email-masked]>
Date: Wed, 09 May 2018 17:45:43 -0700

I ran a simulation of QPACK with the static table in the wiki, and found it was worse for request performance, because :path moved from 4 to 60.  Depends on your encoding strategy of course, but I think shuffling the table a bit could help improve compression.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/904#issuecomment-387916884

---

From: Mike Bishop <[email-masked]>
Date: Thu, 10 May 2018 09:51:21 -0700

Yes -- I'd been looking at putting things "above the fold" for indexing purposes, which is <64, but for name references, above the fold is <16 now.  Things which are especially common and unlikely to get indexed need to move to the very top, especially client-sent fields.  At a first pass,

- `:path` obviously
- `date`
- `referrer`
- `content-length` probably
- `:authority` and/or `host` perhaps, though it's likely to get indexed after the first use
- `cookie` in case they're not being indexed
- `etag` probably
- `alt-svc` maybe, but likely to get indexed after the first use
- `location` probably
- `link` probably
- `if-*` probably

...and that probably fills out the top 16.  Would be interesting to have statistics of which headers are the most commonly name-referenced, but that's likely to be implementation-dependent.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/904#issuecomment-388113768

---

From: Lucas Pardue <[email-masked]>
Date: Thu, 10 May 2018 13:15:54 -0700

We (the BBC) may be able to gather statistics on header names for our services, across all user agents. Is this of value to this issue.

aside: header values are more problematic due to sensitivities to various confidentialites.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/904#issuecomment-388172227

---

From: Mike Bishop <[email-masked]>
Date: Thu, 10 May 2018 20:50:32 +0000 (UTC)

It seems like we have the header names (and values) from the HTTP Archive data, though that obviously produces single-client, single-scenario bias that I've tried to remove manually.  Any data that you generate would have a different bias, so it would help to confirm commonalities.

If you were able to instrument your library to see what header names get name-referenced most often, that would specifically be useful to fix this most recent issue; that doesn't need any value disclosure.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/904#issuecomment-388181582

---

From: Lucas Pardue <[email-masked]>
Date: Thu, 10 May 2018 13:59:04 -0700

I think I misunderstood your earlier request. This is not about filling out the table, but ordering of entries based on frequency of name-reference usage experienced in QPACK implementation. Is that right?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/904#issuecomment-388183998

---

From: Martin Thomson <[email-masked]>
Date: Thu, 10 May 2018 17:58:46 -0700

You should prioritize mandatory pseudo header fields.  That includes `:scheme` and `:authority`.  I have no issue punting `host` down the list (it only exists for compatibility with some intermediation cases).

Favouring client header fields is probably wise as well.  That's where we see the biggest constraints.  Also, servers use those too (if you believe that server push has any life left in it :p ).

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/904#issuecomment-388228486

---

From: afrind <[email-masked]>
Date: Mon, 14 May 2018 13:13:16 -0700

:scheme is almost never a name reference, since the static table contains http and https, so there's not much compression value in having it < 16.  As Mike points out, :authority is very likely to be added to the dynamic table after the first request (or first RTT, if a blocking encoder), so there may be more bang for the buck for putting something above the fold.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/904#issuecomment-388947189

---

From: Mike Bishop <[email-masked]>
Date: Mon, 14 May 2018 16:21:21 -0700

I've incorporated some feedback as well as reordered (mostly via judgement call) the important name-reference entries into the top 16.  Because slots are limited and the top-16 threshold doesn't matter for indexing, I've only brought one instance of each header up top; this means in some cases instances are split between rungs.

We can quibble (hopefully with data) about the tiering choices, but my hope is that this one will perform better for typical browsing.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/904#issuecomment-388993271

---

From: Lucas Pardue <[email-masked]>
Date: Tue, 15 May 2018 11:23:37 -0700

A thought I just had, should we chose to not include `X-` based headers in the name of waving the Httpbis flag?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/904#issuecomment-389265714

---

From: Mike Bishop <[email-masked]>
Date: Tue, 15 May 2018 11:41:56 -0700

Martin and I have been having that discussion offline.  I'm inclined to stick to reality, which includes their existence.  I'm trying to only remove vendor-specific headers or values in the interests of not tilting the table toward a particular client or server implementation.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/904#issuecomment-389271407

---

From: Lucas Pardue <[email-masked]>
Date: Tue, 15 May 2018 18:49:49 +0000 (UTC)

So this is the "pragmatic-static" table. Perhaps an incentive would be to keep the entries but remove the X prefix. Servers interested in the performance ride can get a cheap ticket by jumping in the conformance cart ;)

* Aware this is more easily said than done

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/904#issuecomment-389273638

---

From: Martin Thomson <[email-masked]>
Date: Tue, 15 May 2018 18:49:52 -0700

If the header field without the `X-` had any meaning, sure.  But I don't see any point in using this to beat people up about not following RFC 6648.  The performance degradation will be there, but I don't see it changing anything.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/904#issuecomment-389368927