Subject: Re: [httpwg/http-extensions] base64 dialect (#539)

From: HTTP issue updates <[email-masked]>
Date: Mon, 02 Apr 2018 23:09:20 -0700

There are good reasons to use a single, less permissive syntax in this context.  On the assumption that the forgiving parser is mostly an unfortunate consequence of having poor specifications in the past, the move to base64url has several reasons:

1. it's simpler
2. it's smaller
3. it's more secure

The last one is the one I care about.  If we are going to put secrets in header fields, as is our wont, then the encoding and decoding processes need to be constant time.  Now, we don't really know how to do that perfectly, but a limited grammar makes it more likely that we get it right.

Now, the forgiving algorithms aren't actually all that forgiving.  As far as I can tell, they are lax about padding and trailing bits. Given that, they would be perfectly good in this context.

FWIW, I generally prefer the URL-safe variants because you get the following sort of thing less often, which is not good from a security perspective:

```js
base64.encode(x).replace(/\+/g, '-').replace(/\//g, '_')
```

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/httpwg/http-extensions/issues/539#issuecomment-378138687

---

From: HTTP issue updates <[email-masked]>
Date: Mon, 09 Apr 2018 16:40:17 -0700

A quick look (JavaScript, Python, Ruby) shows that most "url-safe base64" implementations _in the standard libraries_ do exactly that. Also, it appears to be less available than straight base64.

So I'm not sure that specifying `base64url` is going to have the intended effect. I'm more inclined to leave the spec as-is and assure that the test cases (coming any day!) explore the dark corners here.

Is that sufficient?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/httpwg/http-extensions/issues/539#issuecomment-379926549

---

From: HTTP issue updates <[email-masked]>
Date: Tue, 10 Apr 2018 00:52:43 +0000 (UTC)

That is a fine answer.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/httpwg/http-extensions/issues/539#issuecomment-379939075

---

From: HTTP issue updates <[email-masked]>
Date: Tue, 10 Apr 2018 01:10:36 +0000 (UTC)

Closed #539.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/httpwg/http-extensions/issues/539#event-1565097350

---

From: HTTP issue updates <[email-masked]>
Date: Mon, 09 Apr 2018 23:27:14 -0700

I'd rather browsers only have a single base64 implementation and a single base64url implementation. Having two different base64 variants would be slightly annoying.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/httpwg/http-extensions/issues/539#issuecomment-379988407

---

From: HTTP issue updates <[email-masked]>
Date: Mon, 09 Apr 2018 23:44:57 -0700

@annevk where is the browser implementation of base64url specified? I.e., how to do we know that it has the properties we're looking for?

Also, given that there are a small number of browser vendors and they're highly motivated, and there are many people using many different server-side implementation libraries, I'm not sure we should be optimising for the browser makers.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/httpwg/http-extensions/issues/539#issuecomment-379991876

---

From: HTTP issue updates <[email-masked]>
Date: Tue, 10 Apr 2018 06:44:58 +0000 (UTC)

Reopened #539.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/httpwg/http-extensions/issues/539#event-1565406770

---

From: HTTP issue updates <[email-masked]>
Date: Tue, 10 Apr 2018 06:49:39 +0000 (UTC)

@mnot if I'm to believe @martinthomson browsers match the base64url definition defined by the RFCs. It's base64 that's divergent.

(I'll note that the argument you present there was also frequently cited for URLs and other data types. Unfortunately it doesn't really work since nobody is that on the ball and pays attention to all the details, so implementations leak and get copy-and-pasted and end up in the wrong spot affecting the larger ecosystem.)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/httpwg/http-extensions/issues/539#issuecomment-379992913

---

From: HTTP issue updates <[email-masked]>
Date: Wed, 11 Apr 2018 01:45:22 +0000 (UTC)

I don't think that is right, or I apologize if that is an impression I gave.  My understanding matches that @mnot gained.  base64 is widely implemented, base64url less so and often just wrapping base64 in a hacky fashion.  But it will depend on context as to what is good and what isn't.

(Not sure what is implied by "divergent".)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/httpwg/http-extensions/issues/539#issuecomment-380299891

---

From: HTTP issue updates <[email-masked]>
Date: Tue, 10 Apr 2018 19:53:14 -0700

@annevk does that address your concern?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/httpwg/http-extensions/issues/539#issuecomment-380310187

---

From: HTTP issue updates <[email-masked]>
Date: Wed, 11 Apr 2018 03:33:06 +0000 (UTC)

@martinthomson see https://github.com/w3c/push-api/issues/280.

@mnot I'm just more confused now.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/httpwg/http-extensions/issues/539#issuecomment-380316080

---

From: HTTP issue updates <[email-masked]>
Date: Wed, 11 Apr 2018 19:10:08 -0700

@annevk as I read it, we can close this with no action. I understand your confusion, but I don't think it currently affects this spec (at least, until we get more information).

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/httpwg/http-extensions/issues/539#issuecomment-380652048

---

From: HTTP issue updates <[email-masked]>
Date: Thu, 12 Apr 2018 04:31:23 +0000 (UTC)

I guess as long as you write tests that demonstrate how it works differently from `data:` URLs you might be okay, but I'm still doubtful implementations have multiple base64 variants (other than base64url) and it's rather questionable that they should have. (I disagree with your rationale that implementers with a large code base should be able to tolerate shoddy architecture. That just hinders competition, refactoring, etc.)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/httpwg/http-extensions/issues/539#issuecomment-380674770

---

From: HTTP issue updates <[email-masked]>
Date: Wed, 11 Apr 2018 23:31:26 -0700

> I disagree with your rationale that implementers with a large code base should be able to tolerate shoddy architecture

That is not my rationale, and I'm not sure how emotive words like "shoddy architecture" help.

If `base64url` is a genuinely different codebase that has the properties we want, that's great; however, from what I saw, it's generally a wrapper around `base64` implementations.

> I'm still doubtful implementations have multiple base64 variants (other than base64url)

Quick spot-check shows broad support for "variants" along the lines of what we're looking for:

* [Python](https://docs.python.org/3.5/library/base64.html#base64.b64decode) - `validate` flag
* [Ruby](https://ruby-doc.org/stdlib-2.5.0/libdoc/base64/rdoc/Base64.html) -  `strict_decode64` function
* [Go](https://golang.org/pkg/encoding/base64/#Encoding.Decode) - raises `CorruptInputError` on non-b64 characters.
* [PHP](https://secure.php.net/manual/en/function.base64-decode.php) - `$strict` argument
* [Swift](https://developer.apple.com/documentation/foundation/nsdata.base64decodingoptions) - `ignoreUknownCharacters` constant
* [Java](https://docs.oracle.com/javase/8/docs/api/java/util/Base64.html) - rejects characters outside the b64 alphabet by default

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/httpwg/http-extensions/issues/539#issuecomment-380693736

---

From: HTTP issue updates <[email-masked]>
Date: Thu, 19 Apr 2018 00:29:35 -0700

Closing for now, but happy to reopen if there's more data / thoughts.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/httpwg/http-extensions/issues/539#issuecomment-382638398

---

From: HTTP issue updates <[email-masked]>
Date: Thu, 19 Apr 2018 00:29:35 -0700

Closed #539.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/httpwg/http-extensions/issues/539#event-1583105949

---

From: HTTP issue updates <[email-masked]>
Date: Thu, 19 Apr 2018 07:53:16 +0000 (UTC)

I think the main thing you want to do here is make sure the testsuite really covers all the various angles. In particular the differences in processing with https://github.com/w3c/web-platform-tests/blob/master/fetch/data-urls/resources/base64.json.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/httpwg/http-extensions/issues/539#issuecomment-382644344