Subject: [OAUTH-WG] FW: Duplicating request component in an HTTP authentication scheme

From: Eran Hammer-Lahav <[email-masked]>
Date: Thu, 27 May 2010 16:50:01 -0700

-----Original Message-----
From: Roy T. Fielding [mailto:[email-masked]] 
Sent: Thursday, May 27, 2010 4:48 PM
To: Eran Hammer-Lahav
Cc: HTTP Working Group ([email-masked])
Subject: Re: Duplicating request component in an HTTP authentication scheme

On May 27, 2010, at 4:11 PM, Eran Hammer-Lahav wrote:

> The OAuth working group is debating how to make signed authenticated requests. The two main questions is what do sign, and what to sign it with. On the 'what to sign part' we know we want to sign the request URI, HTTP method, and host name (among other cryptographic attributes such as timestamp and nonce).
> 
> OAuth 1.0 includes a normalization process taking the request and serializing it into a base string (which is then signed). For OAuth 2.0, we have two competing proposals:
> 
> 1. Normalize the request and send just the signature 2. Normalize the 
> request and send both the signature and the signed string
> 
> The advantage of #2 is that is moves the complexity from the client to the server. It is up to the server to figure out if the signed blob matches the request itself. The issue is that by doing so, the HTTP request is duplicated (HTTP request and authentication blob). Server will be required to compare the request with the blob to make sure they match. In practice, servers can ignore the request and just respond to the blob because it's the authenticated copy.
> 
> The ramification of #2 is that it moves the HTTP request bits to another layer, creating a sort of wrapper.

And a security hole that you can drive a truck through.  If you are going to sign the routing bits, then you have to verify that the route is the same in both the blob and the message so that the blob is not being used to bypass route security (e.g., firewalls) to access internal paths.

If the server doesn't check the actual message, then don't bother to sign anything.  In other words, I say #1, or at least I would if this were not heading down the exact same road as S-HTTP.

> However, this approach should make it easier for client developers (who proved themselves to be extremely incompetent with the 1.0 specification) to get signatures right. It also allows for signing arbitrary data in addition to the HTTP request.
> 
> Thoughts?

The same client developers will have the same competence issues with regard to any new spec.  Just make sure that the protocol breaks visibly if anything is done wrong, and then they can fix their own protocol stacks if they want to implement this signing.

....Roy

---

From: Brian Eaton <[email-masked]>
Date: Thu, 27 May 2010 18:21:28 -0700

Umm, needless to say, I disagree with Roy on this one. =)

To respond to the main point:

> And a security hole that you can drive a truck through.  If you
> are going to sign the routing bits, then you have to verify that
> the route is the same in both the blob and the message so that
> the blob is not being used to bypass route security (e.g., firewalls)
> to access internal paths.

Yes, the server needs to check that the URL it received and the URL
that was signed match.  This kind of check is extremely common in
authentication protocols.  It happens in SAML, OpenID, etc... Servers
in all of those protocols verify that the "audience" field in the
signed message matches the expected audience.  All of those protocols
are insecure if this check is left out.

NTLM leaves this check-out, and this vulnerability can be used to
launch relay attacks.

OAuth 1.0 was unusual in that it required that the server match a hash
of the URL, rather than the real URL.  It's an extra layer of
indirection and complexity.  It doesn't improve security.

Cheers,
Brian

On Thu, May 27, 2010 at 4:50 PM, Eran Hammer-Lahav <[email-masked]> wrote:
> _______________________________________________
> OAuth mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/oauth
>

---

From: Blaine Cook <[email-masked]>
Date: Fri, 28 May 2010 02:48:51 +0100

> indirection and complexity.  It doesn't improve security.

To be more precise, OAuth 1.0 required that the server match a
normalised form of the URL. You're absolutely correct that it doesn't
improve security [over matching the URL], but it *is* more secure than
either not proving that the token bearer provided the URL in the first
place or having the client and server match potentially different
versions of the URL.

This is a problem of leaky abstractions: if HTTP was used in a way
such that the client unequivocally asserted "This: {x} is the
unabridged HTTP URL that I am requesting", and such that {x} was
presented untouched to the service handling the request, then we
wouldn't have to worry about normalisation.

As it stands, getting access to the raw request URL is relatively
difficult in many environments that handle HTTP requests, and even
more difficult to obtain from HTTP client libraries, since the actual
request URI is often constructed in a private method at the last
moment before a request is actually made.

Which is all to say that it is indeed complex, but much of that
complexity is a result of HTTP libraries trying to hide complexity
from users. I'd echo Roy's assertion that as library support improves,
approaches to URL normalisation will become hidden behind the same
layers of abstraction as constructing query strings and request URIs
are today.

b.

---

From: Eran Hammer-Lahav <[email-masked]>
Date: Thu, 27 May 2010 20:04:02 -0700

> From: Brian Eaton [mailto:[email-masked]]
> Sent: Thursday, May 27, 2010 6:21 PM

> OAuth 1.0 was unusual in that it required that the server match a hash of the
> URL, rather than the real URL.  It's an extra layer of indirection and
> complexity.  It doesn't improve security.

The current draft signs the real URL.

EHL

---

From: Brian Eaton <[email-masked]>
Date: Thu, 27 May 2010 20:58:55 -0700

> place or having the client and server match potentially different

Cool.  Glad we can put Roy's security concern to rest, at least.

Bearer tokens vs signed requests is a separate issue entirely.

> layers of abstraction as constructing query strings and request URIs

I think we're going to get some real data on which approach is easier soon. =)

---

From: Eran Hammer-Lahav <[email-masked]>
Date: Thu, 27 May 2010 22:18:21 -0700

> From: Blaine Cook [mailto:[email-masked]]
> Sent: Thursday, May 27, 2010 6:49 PM
> To: Brian Eaton
> Cc: Eran Hammer-Lahav; [email-masked]; OAuth WG ([email-masked])
> Subject: Re: [OAUTH-WG] FW: Duplicating request component in an HTTP
> authentication scheme
> 
> To be more precise, OAuth 1.0 required that the server match a normalised
> form of the URL.

No it doesn’t.

OAuth 1.0 requires the server to *independently* normalize the received request. There is no duplication of data and there is nothing to match. There are limitation in the way the URI is normalized (parameter order is not kept for example), but the server works with the actual HTTP request bits.

> You're absolutely correct that it doesn't improve security
> [over matching the URL]

Not sure I get what you mean, but having to compare what was signed with what was received adds a potential security risk and complexity. There is very little room for mismatch in 1.0 and in the current 2.0 text.

>, but it *is* more secure than either not proving that
> the token bearer provided the URL in the first place or having the client and
> server match potentially different versions of the URL.
> 
> This is a problem of leaky abstractions: if HTTP was used in a way such that
> the client unequivocally asserted "This: {x} is the unabridged HTTP URL that I
> am requesting", and such that {x} was presented untouched to the service
> handling the request, then we wouldn't have to worry about normalisation.

Between the request URI and Host header, you got pretty much everything you need. The only thing missing is the scheme which can probably be ignored since the port number is being signed. Some poorly written platforms might not expose the raw HTTP request, but that doesn't mean HTTP doesn't give you everything you need to construct the absolute request URI.

> As it stands, getting access to the raw request URL is relatively difficult in
> many environments that handle HTTP requests, and even more difficult to
> obtain from HTTP client libraries, since the actual request URI is often
> constructed in a private method at the last moment before a request is
> actually made.
> 
> Which is all to say that it is indeed complex, but much of that complexity is a
> result of HTTP libraries trying to hide complexity from users. I'd echo Roy's
> assertion that as library support improves, approaches to URL normalisation
> will become hidden behind the same layers of abstraction as constructing
> query strings and request URIs are today.

I'm willing to limit signatures to platforms that expose the raw bits on the client and server sides, or that implement native OAuth signature support. This is, after all, being labeled an advance feature...

EHL

---

From: Eran Hammer-Lahav <[email-masked]>
Date: Thu, 27 May 2010 22:51:53 -0700

> Sent: Thursday, May 27, 2010 8:59 PM
> To: Blaine Cook
> >> OAuth 1.0 was unusual in that it required that the server match a
> >> hash of the URL, rather than the real URL.  It's an extra layer of

I disagree. Your signature proposal makes matching worse, but moves the "canonicalization problem" to the server side. You just flipped the problem. The client gets much simpler but the server gets potentially less secure (as a likely result of poor implementation).

The server has to compare the HTTP request method, as well as the host, port, scheme, and request URI with the signed blob. In the request URI, the host has to compare the path and query parameters, which means it needs some rules about how to perform this comparison. Does parameter order matters? Case sensitivity? Duplicated parameter names? Percent encoding?

Starting to sound familiar?

Comparing two URIs is notoriously tricky which means servers will need to either allow lose comparison and risk security holes, or require a very strict comparison (such as case sensitive string comparison of the raw request URI to the signed request URI minus the scheme, host, and port). In such a case, the client request will often fail because the URI doesn't match (for the same reasons 1.0 goes through hoops).

The only way to avoid all of this is for the server to ignore the HTTP request and only care about what is signed. This means that a client can make an HTTP GET request with a signed POST request in the authentication header, bypassing routing security (Roy's point).

Your proposal makes it extremely tempting to cut corners (and when you do, yes, it is significantly simpler than 1.0).

> I think we're going to get some real data on which approach is easier soon. =)

I didn't know 'easier' is a guiding principal in designing security. Someone should tell the TLS/SSL folks. :-)

---

To be clear, I am not completely against your proposal.

But it has a fundamental design flaw in its duplication of HTTP request data. Take the HTTP bits out of the JSON structure and I'm completely supportive (i.e. the signature base string is a simple combination of the HTTP bits + the provided base64 encoded JSON blob or something like that). Otherwise, it would need to include specific rules on how the server MUST validate the request by comparing the signed bits with the actual request (and it has to be simpler than the OAuth 1.0 signature base string and current text in 2.0).

Your proposal has parts I really like such as the ability to sign arbitrary data, beyond just the HTTP request bits. This makes it useful for signing identity assertions, server responses, etc. But before we get to that, we need to address the duplication problem.

Everything we do is a tradeoff between security and simplicity, but it only works when we make accurate analysis. I haven't seen anything in your response to Roy to justify your conclusions that we can put it aside.

EHL

---

From: "William Mills" <[email-masked]>
Date: Fri, 28 May 2010 09:21:41 -0700

I thought one of the fundamental ugly problems is that the client
doesn't actually know the full URL authoritatively in all frameworks,
because variables get appended to the query string in an unknown order
in some cases?

Solving that problem seems key.  Oauth 1.0 had one solution, which it
turns out people tend to get wrong.  Brian's proposal solves it a
different way with the problem that it makes for data duplication with
those associated risks/problems.

What other options do we have?

> From: [email-masked] [mailto:[email-masked]] 
> On Behalf Of Eran Hammer-Lahav
> Sent: Thursday, May 27, 2010 8:04 PM
> To: Brian Eaton; [email-masked]
> Cc: OAuth WG ([email-masked])
> Subject: Re: [OAUTH-WG] FW: Duplicating request component in 
> an HTTP authentication scheme
> 
> 
>

---

From: Brian Eaton <[email-masked]>
Date: Fri, 28 May 2010 11:02:05 -0700

On Thu, May 27, 2010 at 10:51 PM, Eran Hammer-Lahav <[email-masked]> wrote:
> I disagree. Your signature proposal makes matching worse, but moves the
> "canonicalization problem" to the server side. You just flipped the problem.
> The client gets much simpler but the server gets potentially less secure
> (as a likely result of poor implementation).

You raise a bunch of really good points in your response, and I'm
going to ignore almost all of them for now.  =)

I want to get to the heart of the "potentially less secure" statement.

In OAuth 1.0, in order to be secure, the server had to check a signature. [1]

Under the proposal I made earlier, the same server will need to check
a signature, and it will also need to check the intended target of the
signed message.

That is *not unusual* in authentication protocols.  OpenID, SAML, and
others all have the exact same requirement.  It's documented in their
security recommendations.  And there are compliance tests that verify
that servers do check this.  If someone fails to make this check, it
will be revealed as soon as anyone looks at their code or tests their
server.

Cheers,
Brian

[1] Aside: secure servers actually need to do lots more than check
signatures.  They need good user management, and key management, and
XSS-prevention, and XSRF-prevention, and patch management, and
physical security, and so on.  But that's out of scope for an
authentication protocol.

---

From: "Roy T. Fielding" <[email-masked]>
Date: Fri, 28 May 2010 11:28:18 -0700

> because variables get appended to the query string in an unknown order

If the client doesn't know exactly what it is sending, then it isn't
a secure client and any signature it might provide is bogus, by
definition.  Why do you bother to consider such a case?  Let the
clients fix their own technology.

....Roy

---

From: Kris Selden <[email-masked]>
Date: Fri, 28 May 2010 15:21:25 -0700

Can't you always have the front end web server preserve the original URI in a header or something before the rewrite engine or application framework mangle it, right?

For example, this Nginx conf that preserves original parts of the request that get changed when the request is proxied to the upstream web server:
  proxy_set_header  X-Real-IP         $remote_addr;
  proxy_set_header  X-Forwarded-For   $proxy_add_x_forwarded_for;
  proxy_set_header  X-Forwarded-Proto https;
  proxy_set_header  Host              $http_host;

Or even writing a module that validated the signature and intended target directly in the web server before the app framework.

I guess this makes in hard to write a complete drop-in solution just on the app framework but that doesn't seem like a good idea anyways. Seems like libraries that do different modular pieces OAuth are a lot more useful rather than "plugin" library deciding something like you should write a nonce to a MySQL DB table every request.

---

From: Eran Hammer-Lahav <[email-masked]>
Date: Fri, 28 May 2010 15:35:37 -0700

That's the same thing (in a header or in the attached signed blob).

EHL

From: Kris Selden [mailto:[email-masked]]
Sent: Friday, May 28, 2010 3:21 PM
To: Brian Eaton
Cc: Eran Hammer-Lahav; [email-masked]; OAuth WG ([email-masked])
Subject: Re: [OAUTH-WG] FW: Duplicating request component in an HTTP authentication scheme

On Thu, May 27, 2010 at 10:51 PM, Eran Hammer-Lahav <[email-masked]<mailto:[email-masked]>> wrote:

Cheers,
Brian

[email-masked]<mailto:[email-masked]>

---

From: Kris Selden <[email-masked]>
Date: Fri, 28 May 2010 15:45:32 -0700

Except you are in control of your web server so you know you put it there and where it came from right?

On May 28, 2010, at 3:35 PM, Eran Hammer-Lahav wrote:

> That’s the same thing (in a header or in the attached signed blob).
>  
> EHL
>

---

From: "William Mills" <[email-masked]>
Date: Fri, 28 May 2010 16:26:50 -0700

So we can define something much simpler than Oauth 1.0, where the
complexity was around normalizing the parameters, and if you want to
implement signing you have to be able to know what you are actually
sending, yes?

I'm OK with having some frameworks not capable of signing without being
fixed.

> Sent: Friday, May 28, 2010 11:28 AM
> To: William Mills
> Cc: Eran Hammer-Lahav; Brian Eaton; [email-masked]
> If the client doesn't know exactly what it is sending, then 
> it isn't a secure client and any signature it might provide 
> is bogus, by definition.  Why do you bother to consider such 
> a case?  Let the clients fix their own technology.
> 
> ....Roy
> 
>

---

From: Eran Hammer-Lahav <[email-masked]>
Date: Fri, 28 May 2010 18:47:29 -0700

How is that different than using the authorization header?

EHL

From: Kris Selden [mailto:[email-masked]]
Sent: Friday, May 28, 2010 3:46 PM
To: Eran Hammer-Lahav
Cc: Brian Eaton; [email-masked]; OAuth WG ([email-masked])
Subject: Re: [OAUTH-WG] FW: Duplicating request component in an HTTP authentication scheme

EHL

From: Kris Selden [mailto:[email-masked]]
Sent: Friday, May 28, 2010 3:21 PM
To: Brian Eaton
Cc: Eran Hammer-Lahav; [email-masked]<mailto:[email-masked]>; OAuth WG ([email-masked]<mailto:[email-masked]>)
Subject: Re: [OAUTH-WG] FW: Duplicating request component in an HTTP authentication scheme

Cheers,
Brian

---

From: Kris Selden <[email-masked]>
Date: Fri, 28 May 2010 19:28:54 -0700

Because the authorization header is from the client which isn't  
trusted like your web front end server.

All I was trying to say is the duplication is not necessary because  
you can workaround it on the server side before it gets mangled by  
rewrite rules or the app framework. You only need that workaround if  
you don't have access to the original uri.

On May 28, 2010, at 6:47 PM, Eran Hammer-Lahav <[email-masked]>  
wrote:

> Can't you always have the front end web server preserve the original  
> URI in a header or something before the rewrite engine or  
> application framework mangle it, right?
>
>
>
> For example, this Nginx conf that preserves original parts of the  
> request that get changed when the request is proxied to the upstream  
> web server:
>
> Or even writing a module that validated the signature and intended  
> target directly in the web server before the app framework.
>
>
>
>
>
> I guess this makes in hard to write a complete drop-in solution just  
> on the app framework but that doesn't seem like a good idea anyways.  
> Seems like libraries that do different modular pieces OAuth are a  
> lot more useful rather than "plugin" library deciding something like  
> you should write a nonce to a MySQL DB table every request.
>
>
>
> On Thu, May 27, 2010 at 10:51 PM, Eran Hammer-Lahav <[email-masked] 
> > wrote:
>
>
>