Subject: [websec] HPKP & different encodings of the same public key

From: Jesse Wilson <[email-masked]>
Date: Sat, 14 May 2016 22:46:22 -0400

I work on OkHttp, an Android HTTP client that implements HPKP-like
certificate pinning. We recently saw a problem where different versions of
Android returned different bytes for the ASN.1 encoding of the same
certificate. This is bad: the pins don’t match!

The certificate of interest uses a named curve (secp521r1) with ECC. I used
der2ascii <https://github.com/google/der-ascii> to view the SPKI ASN.1
bytes.

Older versions of Android (which use BouncyCastle) embed the curve:

SEQUENCE {
  SEQUENCE {
    # ecPublicKey
    OBJECT_IDENTIFIER { 1.2.840.10045.2.1 }
    SEQUENCE {
      INTEGER { 1 }
      SEQUENCE {
        # prime-field
        OBJECT_IDENTIFIER { 1.2.840.10045.1.1 }
        INTEGER { `01ffffffffffffff...` }
      }
      SEQUENCE {
        OCTET_STRING { `01ffffffffffffff...` }
        OCTET_STRING { `0051953eb9618e1c...` }
      }
      OCTET_STRING { `0400c6858e06b704...` }
      INTEGER { `01ffffffffffffff...` }
      INTEGER { 1 }
    }
  }
  BIT_STRING { `0004019519de800d...` }
}

But new versions of Android (which use Conscrypt/OpenSSL) reference the
curve by name:

SEQUENCE {
  SEQUENCE {
    # ecPublicKey
    OBJECT_IDENTIFIER { 1.2.840.10045.2.1 }
    # secp521r1
    OBJECT_IDENTIFIER { 1.3.132.0.35 }
  }
  BIT_STRING { `0004019519de800d...` }
}

The original certificate embeds the curve.

I believe my problem is that the Java APIs I’m using don’t return raw bytes
from the certificate. Instead Java decodes the certificate into a model and
re-encodes that when the bytes are requested. The original and roundtripped
SPKI bytes are not identical.

Does anyone else do ASN.1 encoding in order to compute a certificate’s pin?
Or is it a uniquely Java problem?!

The spec should warn that a single SPKI may have multiple conflicting
encodings. I suggest that only encoding in the certificate should be
pinned. TLS server administrators should also be careful to not
inadvertently re-encode their SPKIs when doing maintenance.

Thanks!
​

---

From: Yoav Nir <[email-masked]>
Date: Sun, 15 May 2016 10:22:39 +0300

That’s interesting. With HPKP you can pin keys from existing certificates, or keys that are not (yet) in certificates.

One of the early deployment scenarios (which got de-emphasized later on) was that you include two pins: your current production key and a spare key that you will certify if something bad happens to the production key (like the private key leaking out).

So because there is more than one way to encode the public key in SPKI you would need to predict how your CA is going to encode the SPKI of the spare key. There’s a couple of obvious heuristics: assume that the SPKI will be encoded the same way the SPKI is encoded in the certificate request, or assume that the SPKI in the next certificate will be encoded the same way as in the current certificate. Both of these heuristics could work most of the time, but if you miss, your pin is invalid and your site is bricked if you move to the backup public key.

It gets worse. Some sites re-certify the same key over and over. But again, nothing is preventing the CA from encoding the SPKI differently on the next certificate Maybe they’ve upgraded their software.

So there could be more caveats here for using HPKP in addition to needing to use the SPKI bytes from your actual DER-encoded certificate.

As a procedural note, there is no longer a WebSec working group, and RFCs, once published, are set in stone. Someone (you?) could publish a document with a title like “Additional Operational Considerations in Deploying HPKP”. That could prove interesting.

Yoav

> On 15 May 2016, at 5:46 AM, Jesse Wilson <[email-masked]> wrote:
> 
> I work on OkHttp, an Android HTTP client that implements HPKP-like certificate pinning. We recently saw a problem where different versions of Android returned different bytes for the ASN.1 encoding of the same certificate. This is bad: the pins don’t match!
> 
> The certificate of interest uses a named curve (secp521r1) with ECC. I used der2ascii <https://github.com/google/der-ascii> to view the SPKI ASN.1 bytes.
> 
> But new versions of Android (which use Conscrypt/OpenSSL) reference the curve by name:
> 
> SEQUENCE {
>   SEQUENCE {
>     # ecPublicKey
> I believe my problem is that the Java APIs I’m using don’t return raw bytes from the certificate. Instead Java decodes the certificate into a model and re-encodes that when the bytes are requested. The original and roundtripped SPKI bytes are not identical.
> 
> Does anyone else do ASN.1 encoding in order to compute a certificate’s pin? Or is it a uniquely Java problem?!
> 
> The spec should warn that a single SPKI may have multiple conflicting encodings. I suggest that only encoding in the certificate should be pinned. TLS server administrators should also be careful to not inadvertently re-encode their SPKIs when doing maintenance.
> 
> Thanks!
> 
> _______________________________________________
> websec mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/websec

---

From: Yaron Sheffer <[email-masked]>
Date: Sun, 15 May 2016 20:10:15 +0300

On 15/05/16 10:22, Yoav Nir wrote:
> That’s interesting. With HPKP you can pin keys from existing 
> certificates, or keys that are not (yet) in certificates.
>
> One of the early deployment scenarios (which got de-emphasized later 
> on) was that you include two pins: your current production key and a 
> spare key that you will certify if something bad happens to the 
> production key (like the private key leaking out).
>
>
Hi Yoav,

I had assumed this *is* the main deployment scenario. If it was 
de-emphasized, what do you consider as the "classic" HPKP usage scenario?

Thanks,
     Yaron

---

From: Jeffrey Walton <[email-masked]>
Date: Sun, 15 May 2016 13:22:18 -0400

On Sat, May 14, 2016 at 10:46 PM, Jesse Wilson <[email-masked]> wrote:
> der2ascii to view the SPKI ASN.1 bytes.
>
> encodings. I suggest that only encoding in the certificate should be pinned.
> TLS server administrators should also be careful to not inadvertently
> re-encode their SPKIs when doing maintenance.

This is kind of expected. The problem is because the first certificate
(older version of Android) uses domain parameters; while the second
certificate (Conscrypt/OpenSSL) uses a named curve rather than domain
parameters.

I've experienced similar problems with OpenSSL servers and clients. In
fact, if a named curve is *not used*, then you will encounter an error
"no shared ciphers" under some circumstances. Also see
http://wiki.openssl.org/index.php/Elliptic_Curve_Cryptography#Named_Curves.

A certificate simply binds a public key to an identity through a
trusted party's signature. Once you verify the signature on the
certificate, you should canonicalize the certificate data and then pin
the normalizedd data.

Jeff

---

From: Jesse Wilson <[email-masked]>
Date: Sun, 15 May 2016 17:31:09 +0000

I definitely like the idea of a canonical form. That makes everything easy!
But which format is the canonical one?

---

From: Jeffrey Walton <[email-masked]>
Date: Sun, 15 May 2016 13:42:01 -0400

> But which format is the canonical one?

It depends. One way to do it is to expand to domain parameters. All
named curves can be expanded to its constituent domain parameters.
Another way to do it is by comparing public points after you determine
the named curve or domain parameters are equivalent.

What you can't do is take domain parameters, and map _all_ of them
back to named curves. Custom curves likely won't have an OID
associated with them. Additionally, not all named curves are
recognized by all libraries. For example, the old 1998 X9.62 curves
are mostly no longer supported; and the WTLS curves are usually not
supported because of weaker parameters even though they have well
known names.

For completeness, the public point is the coordinate Q=(x,y), and its
created by raising the base point G to the private exponent x (i.e.,
Q=G^x). But for the Q=G^x machinery to work, the domain parameters
need to be the same for both parties,

Jeff

---

From: Yoav Nir <[email-masked]>
Date: Sun, 15 May 2016 21:24:26 +0300

> On 15 May 2016, at 8:10 PM, Yaron Sheffer <[email-masked]> wrote:
> 
> 
> 
> I had assumed this *is* the main deployment scenario. If it was de-emphasized, what do you consider as the "classic" HPKP usage scenario?

Current certificate plus some CA certificate that you are likely to use to certify your next certificate.

Yoav

---

From: Yoav Nir <[email-masked]>
Date: Sun, 15 May 2016 21:24:26 +0300

Yoav

---

From: Yaron Sheffer <[email-masked]>
Date: Sun, 15 May 2016 22:54:43 +0300

> Current certificate plus some CA certificate that you are likely to use to certify your next certificate.

But this too means that you're guessing how the CA will behave in the 
future. If your current cert is expiring in a month and you generate the 
new one, you can be surprised by the CA using a new intermediate cert.

And of course, some people would never pin to a CA cert. To me, the 
whole idea of certificate pinning is to reduce the need to trust the PKI 
industry, and that includes my own friendly CA.

Thanks,
	Yaron

---

From: Yoav Nir <[email-masked]>
Date: Sun, 15 May 2016 23:32:11 +0300

> On 15 May 2016, at 10:54 PM, Yaron Sheffer <[email-masked]> wrote:
> 
> But this too means that you're guessing how the CA will behave in the future. If your current cert is expiring in a month and you generate the new one, you can be surprised by the CA using a new intermediate cert.
> 
> And of course, some people would never pin to a CA cert. To me, the whole idea of certificate pinning is to reduce the need to trust the PKI industry, and that includes my own friendly CA.

I agree. Appendix B of RFC 7469 recommends to have the backup pin signed by a different CA and ready to use. That solves the issue of not knowing how a CA might encode the SPKI. It used to be a costly solution, but these days a certificate is pretty much free, so you really can have a backup certificate ready to use.

Yoav

---

From: Brian Smith <[email-masked]>
Date: Sun, 15 May 2016 10:53:59 -1000

Jesse Wilson <[email-masked]> wrote:

> This is invalid. See https://tools.ietf.org/html/rfc5480#section-2.1.1: "implicitCurve
and specifiedCurve MUST NOT be used in PKIX."

> The original certificate embeds the curve.
This is the only correct encoding.

> The original and roundtripped SPKI bytes are not identical.
>
1. I recommend that you don't decode and then re-encode. Instead, I
recommend that you only verify that the curve is specified using the valid
encoding, and then compute the result using the single valid encoding
directly, without using any encoder.

> Or is it a uniquely Java problem?!
IIRC, older versions of OpenSSL have this problem too.

> encodings.
No. There's only one valid way to encode each SPKI. This makes sense if you
think about the primary goal of the DER encoding of ASN.1: It's supposed to
allow round-trip decode-encode where the output is the same as the input.

> I suggest that only encoding in the certificate should be pinned. TLS
> server administrators should also be careful to not inadvertently re-encode
> their SPKIs when doing maintenance.
>
Instead, certificates with invalid encodings should be rejected by
verifiers and nobody should produce certificates with invalid encodings.

Cheers,
Brian
-- 
https://briansmith.org/

---

From: Jesse Wilson <[email-masked]>
Date: Sun, 15 May 2016 21:19:24 +0000

Thanks Brian! I’m happy to hear that this is an implementation bug (that I
can petition to get fixed), rather than spec bug (that we all have to
workaround).

---

From: Jeffrey Walton <[email-masked]>
Date: Sun, 15 May 2016 18:23:46 -0400

> can petition to get fixed), rather than spec bug (that we all have to

It depends on the issuing policies.

The IETF has no way to specify that a certificate was created or
issued under PKIX, so its a moot point. (It creates a vaccum like the
EV mess, except for standard certificates rather than EV
certificates).

Jeff

---

From: Brian Smith <[email-masked]>
Date: Sun, 15 May 2016 12:34:04 -1000

> EV mess, except for standard certificates rather than EV

HPKP is specified in terms of RFC 5280, so we can assume only PKIX
certificates are used for HPKP. In particular, HPKP (RFC7469) defers to
RFC5280 for the specification of SPKI. RFC 5280 then defers to other specs
for defining SPKI: "Conforming  implementations that use the algorithms
identified in [RFC3279 <http://tools.ietf.org/html/rfc3279>], [RFC4055
<http://tools.ietf.org/html/rfc4055>], and [RFC4491
<http://tools.ietf.org/html/rfc4491>] MUST identify and encode the public
key materials and digital signatures as described in those specifications."
RFC 5480 updates RFC 3279.

So, yes, a CA can issue a certificate that's not RFC 5480 but nobody should
expect HPKP to work with such certificates.

Cheers,
Brian
-- 
https://briansmith.org/

---

From: Jeffrey Walton <[email-masked]>
Date: Sun, 15 May 2016 19:04:11 -0400

> certificates are used for HPKP....

In that case, the IETF provides a document on path building and
validation (RFC 4158), but not certificate validation (modulo RFC
6125). As far as I can tell, its still the wild, wild west with no
guidance on end-entity certificate validation.

Jeff

---

From: Alice Wonder <[email-masked]>
Date: Sun, 15 May 2016 16:23:17 -0700

> guidance on end-entity certificate validation.

DANE is better anyway.

HPKP is trust on first use, and even worse, trust on first use without
   user interaction.
DANE is validate on every use

HPKP only works with https
DANE works with anything tcp or udp

HPKP can be used for a DOS attack. Simple DNS cache poison and send the
   header with a bogus set of keypins and long TTL
DANE validates via DNSSEC avoiding that issue.

I do not understand browser resistance to DANE validation.