Subject: [secdir] JWK member names, was: [jose] SECDIR review of draft-ietf-jose-json-web-key-31

From: Kathleen Moriarty <[email-masked]>
Date: Fri, 12 Sep 2014 07:59:49 -0400

Hi Mike,

The text Steve called out below has been very problematic as you know.
 Could you call out some options here as the last time this came up, we
didn't resolve it.  The working group was asked for suggestions, but none
came through.  If you could provide some options and then have the working
group weigh in, I think that would be good.

I snipped away the rest of the review and changed the subject as not to get
in the way of the current dialog.

Thanks.

On Wed, Sep 10, 2014 at 8:57 PM, Mike Jones <[email-masked]>
wrote:

>  Hi Stephen.  Thanks for your detailed and useful review.  I’ve cc’ed the
> working group in my reply so they’re aware of the contents of your review.
> Replies are inline below…
>
>
>
> *From:* Stephen Kent [mailto:[email-masked] <[email-masked]>]
> *Sent:* Tuesday, September 02, 2014 1:09 PM
> *To:* [email-masked]; Mike Jones; [email-masked]; Moriarty,
> Kathleen
> *Subject:* SECDIR review of draft-ietf-jose-json-web-key-31
>
>
>
> [snip]

>
>
> This section imposes a rather wimpy constraint on parameter names:
>
>
>
>    The member names within a JWK MUST be unique; recipients MUST either
>
>    reject JWKs with duplicate member names or use a JSON parser that
>
>    returns only the lexically last duplicate member name, as specified
>
>    in Section 15.12 (The JSON Object) of ECMAScript 5.1 [ECMAScript].
>
>
>
> This text says that member names MUST be unique, but if they are not,
> that’s OK too; just use the last instance of a member with a duplicate
> name. This seems like a terrible design principle. It imposes what appears
> to be a requirement, then says how to accommodate data structures that fail
> to meet the requirement. This would seem to encourage sloppy
> implementations (for JWK generation). I’d like to see the rationale for
> this.
>
>
>
>
>
> Unfortunately, the intentional laxness in the spec in this regard is a
> reflection of the semantics of the actual JSON specifications and
> implementations.  For instance,
> http://tools.ietf.org/html/rfc7159#section-4 says:
>
>
>
>    An object whose names are all unique is interoperable in the sense
>
>    that all software implementations receiving that object will agree on
>
>    the name-value mappings.  When the names within an object are not
>
>    unique, the behavior of software that receives such an object is
>
>    unpredictable.  Many implementations report the last name/value pair
>
>    only.  Other implementations report an error or fail to parse the
>
>    object, and some implementations report all of the name/value pairs,
>
>    including duplicates.
>
>
>
> This topic has been heavily discussed by the working group, and while the
> specs used to just say that objects with duplicate member names MUST be
> rejected, working group members, including Tim Bray (the editor of the JSON
> spec), prevailed on us to weaken this so that parsers that implement the
> ECMAscript behavior of returning only the last member name may be legally
> used.  (The argument was made that there was more security downside in
> effectively requiring people to write and debug their own strict parsers
> than in using laxer, but well-supported and debugged parsers.)
>
>
>
> However, we also intentionally require that producers use only one
> instance of each member name, so that legally produced objects will never
> exercise the ambiguities that are present in real JSON parsers.  That
> seemed to be the most practical solution to the working group.
>
>
>
> [snip]
>
>                                                             Thanks again,
> Stephen,
>
>                                                             -- Mike
>
>
>
> _______________________________________________
> jose mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/jose
>
>

-- 

Best regards,
Kathleen

-- 

Best regards,
Kathleen

---

From: Mike Jones <[email-masked]>
Date: Fri, 12 Sep 2014 15:18:42 +0000

Sure.  Here’s an analysis of the requirements about duplicate member names.

There could be two very different kinds of objections to the present text:
A.  People think we have the semantics for duplicate identifiers wrong.
B.  People think we should explain the current semantics for duplicate identifiers more clearly.

I sure hope that we’re dealing with B and not A.  Stephen, which is the nature of your critique of this text?

If we’re in the realm of A, I think there are three possibilities for the semantics:

1.  Require producers not use duplicate members and require consumers to reject inputs with duplicate member names.
Pros:  This is the most locked-down, consistent, and alternative.
Cons:  Real JSON parsers don’t all reject duplicate inputs.  If we force people to write custom parsers, this will result in exploitable bugs (and some just won’t do it and won’t be conformant).

2.  Require producers not to use duplicate members but allow consumers to accept inputs with duplicate member names in the manner defined in ECMAscript.  (This is the current choice.)
Pros:  People can use all standard JSON parsers.  Producers are still required to produce well-formed data structures.
Cons:  The requirements on producers are stricter than those on consumers.

3.  Allow both producers and consumers to use duplicate members, with duplicate member names handled in the manner defined in ECMAscript.
Pros:  People can use ECMAscript parsers (which are more liberal than some JSON parsers).  The requirements on producers and consumers are the same.
Cons:  Hidden content can be inserted into duplicate member names.  This could give attackers a way to manipulate the inputs to crypto operations.  Strict JSON parsers (that reject inputs with duplicate members) can’t be used.

Practically, I think were we presently are (2) is the best compromise between implementability, consistency, and security.  The working group put a lot of discussion into this, including changing from (1) to (2) and it’s my sense that most agree we’ve landed in the right place.  From a security perspective, I don’t think that (3) is a viable option.

If people think that the current semantics are right but are not sufficiently clearly explained or motivated, I’d certainly welcome proposed text to clarify the explanation.

                                                            -- Mike

From: Kathleen Moriarty [mailto:[email-masked]]
Sent: Friday, September 12, 2014 5:00 AM
To: [email-masked]; [email-masked]; [email-masked]; [email-masked]; Stephen Kent; Mike Jones
Subject: JWK member names, was: [jose] SECDIR review of draft-ietf-jose-json-web-key-31

Hi Mike,

The text Steve called out below has been very problematic as you know.  Could you call out some options here as the last time this came up, we didn't resolve it.  The working group was asked for suggestions, but none came through.  If you could provide some options and then have the working group weigh in, I think that would be good.

I snipped away the rest of the review and changed the subject as not to get in the way of the current dialog.

Thanks.

On Wed, Sep 10, 2014 at 8:57 PM, Mike Jones <[email-masked]<mailto:[email-masked]>> wrote:
Hi Stephen.  Thanks for your detailed and useful review.  I’ve cc’ed the working group in my reply so they’re aware of the contents of your review.  Replies are inline below…

From: Stephen Kent [mailto:[email-masked]]
Sent: Tuesday, September 02, 2014 1:09 PM
To: [email-masked]<mailto:[email-masked]>; Mike Jones; [email-masked]<mailto:[email-masked]>; Moriarty, Kathleen
Subject: SECDIR review of draft-ietf-jose-json-web-key-31

[snip]

This text says that member names MUST be unique, but if they are not, that’s OK too; just use the last instance of a member with a duplicate name. This seems like a terrible design principle. It imposes what appears to be a requirement, then says how to accommodate data structures that fail to meet the requirement. This would seem to encourage sloppy implementations (for JWK generation). I’d like to see the rationale for this.

Unfortunately, the intentional laxness in the spec in this regard is a reflection of the semantics of the actual JSON specifications and implementations.  For instance, http://tools.ietf.org/html/rfc7159#section-4 says:

This topic has been heavily discussed by the working group, and while the specs used to just say that objects with duplicate member names MUST be rejected, working group members, including Tim Bray (the editor of the JSON spec), prevailed on us to weaken this so that parsers that implement the ECMAscript behavior of returning only the last member name may be legally used.  (The argument was made that there was more security downside in effectively requiring people to write and debug their own strict parsers than in using laxer, but well-supported and debugged parsers.)

However, we also intentionally require that producers use only one instance of each member name, so that legally produced objects will never exercise the ambiguities that are present in real JSON parsers.  That seemed to be the most practical solution to the working group.

[snip]
                                                            Thanks again, Stephen,
                                                            -- Mike

_______________________________________________
jose mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/jose

--

Best regards,
Kathleen

--

Best regards,
Kathleen

---

From: Mike Jones <[email-masked]>
Date: Fri, 12 Sep 2014 16:20:15 +0000

I understand that that’s an ideal long-term solution, once I-JSON parsers are common/ubiquitous, but I don’t expect that to be the case for a number of years and people are using JOSE now.  Tim, you were one of the primary advocates for the position that implementers need to be able to use JSON parsers as they actually exist – hence the change from (1) to (2) in draft -12 in July 2013 after extensive working group discussion on the topic.

Unless you’ve changed your tune, I doubt that you’re actually saying that we should move back to (1) now, which would mean that people using today’s JSON parsers would be nonconformant?

Once I-JSON is ubiquitous, I’d be fine doing bis versions of the specs to tighten the requirement in a few years.  But until then, keeping the requirement that producers must not use duplicate member names would mean that if we do update to use I-JSON in the future, nothing would then break.

                                                            -- Mike

From: Tim Bray [mailto:[email-masked]]
Sent: Friday, September 12, 2014 8:55 AM
To: Mike Jones
Cc: Kathleen Moriarty; [email-masked]; [email-masked]; [email-masked]; [email-masked]; Stephen Kent
Subject: Re: [jose] JWK member names, was: SECDIR review of draft-ietf-jose-json-web-key-31

I’m pretty sure that at some point before the end of the year the IETF will publish I-JSON (see https://datatracker.ietf.org/doc/draft-ietf-json-i-json/) which simply specifies that there MUST NOT be dupe keys.  Depending on the timeframe, referring to this might be a good solution.

On Fri, Sep 12, 2014 at 8:18 AM, Mike Jones <[email-masked]<mailto:[email-masked]>> wrote:
Sure.  Here’s an analysis of the requirements about duplicate member names.

                                                            -- Mike

From: Kathleen Moriarty [mailto:[email-masked]<mailto:[email-masked]>]
Sent: Friday, September 12, 2014 5:00 AM
To: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; Stephen Kent; Mike Jones
Subject: JWK member names, was: [jose] SECDIR review of draft-ietf-jose-json-web-key-31

Hi Mike,

Thanks.

[snip]

[snip]
                                                            Thanks again, Stephen,
                                                            -- Mike

--

Best regards,
Kathleen

--

Best regards,
Kathleen

--
- Tim Bray (If you’d like to send me a private message, see https://keybase.io/timbray)

---

From: Tim Bray <[email-masked]>
Date: Fri, 12 Sep 2014 08:55:27 -0700

I’m pretty sure that at some point before the end of the year the IETF will
publish I-JSON (see https://datatracker.ietf.org/doc/draft-ietf-json-i-json/)
which simply specifies that there MUST NOT be dupe keys.  Depending on the
timeframe, referring to this might be a good solution.

On Fri, Sep 12, 2014 at 8:18 AM, Mike Jones <[email-masked]>
wrote:

> 1.  Require producers not use duplicate members and require consumers to
> reject inputs with duplicate member names.
>
> Cons:  Real JSON parsers don’t all reject duplicate inputs.  If we force
> people to write custom parsers, this will result in exploitable bugs (and
> some just won’t do it and won’t be conformant).
>
>
>
> 2.  Require producers not to use duplicate members but allow consumers to
> accept inputs with duplicate member names in the manner defined in
> ECMAscript.  (This is the current choice.)
>
> Pros:  People can use all standard JSON parsers.  Producers are still
> required to produce well-formed data structures.
>
> 3.  Allow both producers and consumers to use duplicate members, with
> duplicate member names handled in the manner defined in ECMAscript.
>
> Pros:  People can use ECMAscript parsers (which are more liberal than some
> JSON parsers).  The requirements on producers and consumers are the same.
>
> Cons:  Hidden content can be inserted into duplicate member names.  This
> could give attackers a way to manipulate the inputs to crypto operations.
> Strict JSON parsers (that reject inputs with duplicate members) can’t be
> used.
>
>
>
> Practically, I think were we presently are (2) is the best compromise
> between implementability, consistency, and security.  The working group put
> a lot of discussion into this, including changing from (1) to (2) and it’s
> my sense that most agree we’ve landed in the right place.  From a security
> perspective, I don’t think that (3) is a viable option.
>
>
>
> If people think that the current semantics are right but are not
> sufficiently clearly explained or motivated, I’d certainly welcome proposed
> text to clarify the explanation.
>
>
>
>                                                             -- Mike
>
>
>
> *From:* Kathleen Moriarty [mailto:[email-masked]]
> *Sent:* Friday, September 12, 2014 5:00 AM
> *To:* [email-masked]; [email-masked];
> [email-masked]; [email-masked]; Stephen
> Kent; Mike Jones
> *Subject:* JWK member names, was: [jose] SECDIR review of
> draft-ietf-jose-json-web-key-31
>
>
>
> Hi Mike,
>
>
>

-- 
- Tim Bray (If you’d like to send me a private message, see
https://keybase.io/timbray)

---

From: Tim Bray <[email-masked]>
Date: Fri, 12 Sep 2014 10:29:30 -0700

Well, quoting from your option (2):

“2.  Require producers not to use duplicate members ”

Another way to express this would be to require that producers emit I-JSON.
 This would also have benefits such as forbidding malformed Unicode,
forbidding dependency on object member ordering, and staying within the
bounds of IEEE double precision.  I agree that at this point in time,
REQUIRING consumers to reject data with dupes is problematic, but as long
as they’re allowed to do so, that means they can deploy I-JSON parsers when
such things arrive.

If the goal is to allow the use of off-the-shelf parsers, there’s really
not much you can say about what consumers should do about dupe keys,
because the behavior of such parsers is inconsistent in practice.

In any practical programming scenario, what the recipient wants is to take
the JSON and stuff objects into hashes or dicts or whatever so they can
pull out fields by name.  They’d really rather avoid thinking about dealing
with malformed messages - and I claim that any message with dupe keys is
malformed.  The one way to be sure that consumers don’t have to deal with
this crap is to require that producers don’t emit broken messages.  I-JSON
messages have way less scope for breakage.

On Fri, Sep 12, 2014 at 9:20 AM, Mike Jones <[email-masked]>
wrote:

>  I understand that that’s an ideal long-term solution, once I-JSON
> parsers are common/ubiquitous, but I don’t expect that to be the case for a
> number of years and people are using JOSE now.  Tim, you were one of the
> primary advocates for the position that implementers need to be able to use
> JSON parsers as they actually exist – hence the change from (1) to (2) in
> draft -12 in July 2013 after extensive working group discussion on the
> topic.
>
>
>
> Unless you’ve changed your tune, I doubt that you’re actually saying that
> we should move back to (1) now, which would mean that people using today’s
> JSON parsers would be nonconformant?
>
>
>
> Once I-JSON is ubiquitous, I’d be fine doing bis versions of the specs to
> tighten the requirement in a few years.  But until then, keeping the
> requirement that producers must not use duplicate member names would mean
> that if we do update to use I-JSON in the future, nothing would then break.
>
>
>
>                                                             -- Mike
>
>
>
> *From:* Tim Bray [mailto:[email-masked]]
> *Sent:* Friday, September 12, 2014 8:55 AM
> *To:* Mike Jones
> *Cc:* Kathleen Moriarty; [email-masked]; [email-masked];
> Kent
> *Subject:* Re: [jose] JWK member names, was: SECDIR review of
> I’m pretty sure that at some point before the end of the year the IETF
> will publish I-JSON (see
> https://datatracker.ietf.org/doc/draft-ietf-json-i-json/) which simply
> specifies that there MUST NOT be dupe keys.  Depending on the timeframe,
> referring to this might be a good solution.
>
>
>

---

From: Tim Bray <[email-masked]>
Date: Fri, 12 Sep 2014 10:48:21 -0700

It also seems that the Security Considerations should call out malformed
messages, specifically including dupe keys, as an attack vector.  The best
defense against such attacks is for clients to reject malformed messages.

On Fri, Sep 12, 2014 at 10:29 AM, Tim Bray <[email-masked]> wrote:

> Another way to express this would be to require that producers emit
> I-JSON.  This would also have benefits such as forbidding malformed
> Unicode, forbidding dependency on object member ordering, and staying
> within the bounds of IEEE double precision.  I agree that at this point in
> time, REQUIRING consumers to reject data with dupes is problematic, but as
> long as they’re allowed to do so, that means they can deploy I-JSON parsers
> when such things arrive.
>
>> [email-masked]; [email-masked];
>> Stephen Kent
>> Pros:  People can use ECMAscript parsers (which are more liberal than
>> some JSON parsers).  The requirements on producers and consumers are the
>> same.
>>
>> [email-masked]; [email-masked];
>> Stephen Kent; Mike Jones

---

From: Stephen Kent <[email-masked]>
Date: Mon, 15 Sep 2014 10:56:16 -0400

Mike,

> Sure. Here's an analysis of the requirements about duplicate member names.
>
> I sure hope that we're dealing with B and not A.  Stephen, which is 
> the nature of your critique of this text?
>
C- don't accept duplicate IDs, is what I was hoping for.

I noted why allowing a recipient to accept a dup name, and use just the 
last instance, will
likely lead to such behavior being perpetuated, based on PKIX experience.

Also, in a reply to Tim, I think you argued that people have already 
implemented JOSE and so
we ought not make any changes at this late stage. If that's what you 
said, I disagree emphatically.
The IETF always warns implementers that specs may change until an RFC is 
published, and thus
one implements a pre-RFC spec at risk.

Steve

---

From: Tim Bray <[email-masked]>
Date: Mon, 15 Sep 2014 09:13:13 -0700

On Mon, Sep 15, 2014 at 7:56 AM, Stephen Kent <[email-masked]> wrote:

> we ought not make any changes at this late stage. If that's what you said,
> I disagree emphatically.

​No; In theory I would entirely support requiring receivers of malformed
messages to reject them.

In practice, it’s problematic to say that the format is JSON, and then to
require any particular policy concerning duplicate keys, because existing
software generally doesn’t handle them in a consistent manner, and in
particular may not even inform receiving software that dupes existed.

---

From: Mike Jones <[email-masked]>
Date: Mon, 15 Sep 2014 16:17:37 +0000

Replies inline below...

From: Stephen Kent [mailto:[email-masked]]
Sent: Monday, September 15, 2014 7:56 AM
To: Mike Jones; Kathleen Moriarty; [email-masked]; [email-masked]; [email-masked]; [email-masked]
Subject: Re: JWK member names, was: [jose] SECDIR review of draft-ietf-jose-json-web-key-31

Mike,

I sure hope that we're dealing with B and not A.  Stephen, which is the nature of your critique of this text?
C- don't accept duplicate IDs, is what I was hoping for.

I noted why allowing a recipient to accept a dup name, and use just the last instance, will
likely lead to such behavior being perpetuated, based on PKIX experience.

So you're advocating this alternative from the analysis, correct?
1.  Require producers not use duplicate members and require consumers to reject inputs with duplicate member names.
Pros:  This is the most locked-down, consistent, and alternative.
Cons:  Real JSON parsers don't all reject duplicate inputs.  If we force people to write custom parsers, this will result in exploitable bugs (and some just won't do it and won't be conformant).

The primary reason that the working group changed from this alternative in draft -12 in July 2013 to the present one is that it allows JSON parsers as actually deployed and supported to be used.  The argument made then was that forcing people to write a custom parser to reject duplicate member names is likely to create security bugs, since insufficiently locked down parsers are a fertile area for security vulnerabilities.  It was thought that it was better to have people using well-maintained, standard parsers and count on producers to not emit duplicate member names in the first place.  Is there some aspect of this reasoning that doesn't seem convincing to you, Stephen?

As for perpetuating the behavior, it's not clear to me that the behavior will ever start, since producers can't use duplicate member names and consumers are allowed to reject them.  Thus, producers can't count on consumers accepting the illegal inputs, and so have a strong disincentive against ever producing them.  Is there something I'm missing here?

Also, in a reply to Tim, I think you argued that people have already implemented JOSE and so
we ought not make any changes at this late stage. If that's what you said, I disagree emphatically.
The IETF always warns implementers that specs may change until an RFC is published, and thus
one implements a pre-RFC spec at risk.

You apparently misunderstood the point of my reply to Tim.  I wasn't saying that changes aren't possible.  I was saying that we want JOSE to be usable by people now - not just several years in the future when parsers for a more locked-down JSON dialect may become available.

Steve

                                                            -- Mike

---

From: Mike Jones <[email-masked]>
Date: Mon, 15 Sep 2014 16:20:03 +0000

Thanks Tim – that was exactly the point that caused the working group to change to the current behavior.

From: Tim Bray [mailto:[email-masked]]
Sent: Monday, September 15, 2014 9:13 AM
To: Stephen Kent
Cc: Mike Jones; Kathleen Moriarty; [email-masked]; [email-masked]; [email-masked]; [email-masked]
Subject: Re: [jose] JWK member names, was: SECDIR review of draft-ietf-jose-json-web-key-31

On Mon, Sep 15, 2014 at 7:56 AM, Stephen Kent <[email-masked]<mailto:[email-masked]>> wrote:

​No; In theory I would entirely support requiring receivers of malformed messages to reject them.

In practice, it’s problematic to say that the format is JSON, and then to require any particular policy concerning duplicate keys, because existing software generally doesn’t handle them in a consistent manner, and in particular may not even inform receiving software that dupes existed.

Steve

---

From: Stephen Kent <[email-masked]>
Date: Mon, 15 Sep 2014 14:51:11 -0400

Mike,

> ...
> 1. Require producers not use duplicate members and require consumers 
> to reject inputs with duplicate member names.
>
> Cons: Real JSON parsers don't all reject duplicate inputs.  If we 
> force people to write custom parsers, this will result in exploitable 
> bugs (and some just won't do it and won't be conformant).
>
>
> The primary reason that the working group changed from this 
> alternative in draft -12 in July 2013 to the present one is that it 
> allows JSON parsers as actually deployed and supported to be used.  
> The argument made then was that forcing people to write a custom 
> parser to reject duplicate member names is likely to create security 
> bugs, since insufficiently locked down parsers are a fertile area for 
> security vulnerabilities.  It was thought that it was better to have 
> people using well-maintained, standard parsers and count on producers 
> to not emit duplicate member names in the first place.  Is there some 
> aspect of this reasoning that doesn't seem convincing to you, Stephen?
>
I read the rationale. Is there a good baseline of experience showing 
that JSON parsers are
not very exploitable today? We have seen problems with ASN.1 parsers, 
problems that have
taken years to surface, despite extensive use of ASN.1 in both certs and 
in SNMP.  I suggest,
without substantiation, that bugs in ASN.1 parsers were not exploited in 
the SNMP context
because there was little to gain. Use of ASN.1 for certs, where there 
can be financial incentives
for exploiting flaws, motivated more analysis by attackers. If JOSE 
represents an initial
step into an arena where crypto is to be applied to JSON in a widespread 
fashion, it also
may represent a new motivation for attackers to try to exploit any bugs 
in parsers. If so, then
the lack of identified vulnerabilities in JSON parsers so far may not be 
indicative of their
security.

> ...
>
>
> You apparently misunderstood the point of my reply to Tim.  I wasn't 
> saying that changes aren't possible.  I was saying that we want JOSE 
> to be usable by people now -- not just several years in the future 
> when parsers for a more locked-down JSON dialect may become available.

I did misunderstand your point, perhaps because you said:

"I understand that that's an ideal long-term solution, once I-JSON 
parsers are common/ubiquitous, but I don't expect that to be the case 
for a number of years and _*people are using JOSE now.*_"

Steve

---

From: Stephen Kent <[email-masked]>
Date: Mon, 15 Sep 2014 14:51:50 -0400

OK, I'm a bit confused.

I thought the JOSE specs were intended to create standards for transport 
of keys, and for sigs,
MACs, and encryption of JSON objects.

What is the existing software to which you and Tim refer, when referring 
to keys (vs.
JSON parsing in general)?

Steve

---

From: Tim Bray <[email-masked]>
Date: Mon, 15 Sep 2014 11:54:28 -0700

​When I talk about existing software I’m referring to generic JSON parsers
such as are included in the basic library set of every programming language
now, and which are unfortunately idiosyncratic and inconsistent in their
handling of dupe keys, but in almost no cases actually inform the calling
software whether or not dupe keys were encountered.

---

From: John Bradley <[email-masked]>
Date: Mon, 15 Sep 2014 16:18:59 -0300

Tim,

To clarify.

Are you recommending that:
That receivers MUST reject JOSE objects with duplicate keys.  

This would require compliant implementations to write there own parsers (perhaps not a good idea), or wait for I-JSON parsers (perhaps sometime soonish)

Or that JOSE require producers not to send dup keys, and receivers SHOULD reject them if possible based on the parser.

For JWE and JWS the header is integrity protected so we are talking about duplicate keys inserted by a bad producer rather than an attacker modifying the message after signing..

The concern is if something at the application layer is tricked into inserting a parameter with a duplicate name or one that otherwise changes the message verification.

I suspect the important issue is taking care that when producing a JWE/JWS you are not accepting arbitrary elements for the header without verifying that they are not JOSE parameters.

John B.

On Sep 15, 2014, at 3:54 PM, Tim Bray <[email-masked]> wrote:

> ​When I talk about existing software I’m referring to generic JSON parsers such as are included in the basic library set of every programming language now, and which are unfortunately idiosyncratic and inconsistent in their handling of dupe keys, but in almost no cases actually inform the calling software whether or not dupe keys were encountered.
> 
> I thought the JOSE specs were intended to create standards for transport of keys, and for sigs,
> What is the existing software to which you and Tim refer, when referring to keys (vs.

---

From: Mike Jones <[email-masked]>
Date: Mon, 15 Sep 2014 20:47:43 +0000

Replies inline below…

From: Tim Bray [mailto:[email-masked]]
Sent: Monday, September 15, 2014 11:54 AM
To: Stephen Kent
Cc: Mike Jones; Kathleen Moriarty; [email-masked]; [email-masked]; [email-masked]; [email-masked]
Subject: Re: [jose] JWK member names, was: SECDIR review of draft-ietf-jose-json-web-key-31

On Mon, Sep 15, 2014 at 11:51 AM, Stephen Kent <[email-masked]<mailto:[email-masked]>> wrote:
OK, I'm a bit confused.

Actually, the payloads of JWS and JWE objects can be any octet sequence – not just those representing JSON objects.

JWK objects are already used in production to distribute public keys.  For instance, the keys for Salesforce’s identity services are in JWK format at https://login.salesforce.com/id/keys.  (Note that I’m not saying that just because the current specs are in use, that no changes are possible.)
The existing JSON parsers include every existing JavaScript implementation, for starters, plus the JSON implementations in Java, PHP, Ruby, Python, Perl, .NET, Objective C, Scala, etc.  There’s a pretty large list at http://json.org/.  Any of these could be used to consume these keys.

Steve

                                                            -- Mike

---

From: Tim Bray <[email-masked]>
Date: Mon, 15 Sep 2014 14:21:24 -0700

On Mon, Sep 15, 2014 at 12:18 PM, John Bradley <[email-masked]> wrote:

> This would require compliant implementations to write there own parsers
> (perhaps not a good idea), or wait for I-JSON parsers (perhaps sometime
> soonish)
>

​Agree, sigh.​

> Or that JOSE require producers not to send dup keys, and receivers SHOULD
> reject them if possible based on the parser.
>

​Yeah… MUST NOT send malformed JSON (I argue that an economical way to do
that would be to reference I-JSON if the schedule works).  As for the
receiver, I like SHOULD reject, but I’m wondering if we can get away with
that given that a high proportion of receiving software will have no way to
detect the error.​

> For JWE and JWS the header is integrity protected so we are talking about
> duplicate keys inserted by a bad producer rather than an attacker modifying
> the message after signing..
>

​Whatever. If you get a busted message, something is wrong somewhere
upstream.​

> I suspect the important issue is taking care that when producing a JWE/JWS
> you are not accepting arbitrary elements for the header without verifying
> that they are not JOSE parameters.
>

​Hm, for ID Tokens, didn’t we do exactly the opposite, and end up with a
Must-Ignore policy?​

---

From: Tim Bray <[email-masked]>
Date: Mon, 15 Sep 2014 14:25:55 -0700

I read the rationale. Is there a good baseline of experience showing that
> JSON parsers are
> not very exploitable today?
>

​Well, increasingly, more or less every Internet facing API is
JSON-over-HTTP; the amount of JSON in circulation is huge. So, nothing in
the world is 100% debugged, but production JSON parsers are among the
better-tested software suites in the world.  It also helps that the JSON
format is very stripped-down and doesn’t provide a very rich attack
surface.​

---

From: John Bradley <[email-masked]>
Date: Mon, 15 Sep 2014 18:40:58 -0300

Inline
On Sep 15, 2014, at 6:21 PM, Tim Bray <[email-masked]> wrote:

> ​Yeah… MUST NOT send malformed JSON (I argue that an economical way to do that would be to reference I-JSON if the schedule works).  As for the receiver, I like SHOULD reject, but I’m wondering if we can get away with that given that a high proportion of receiving software will have no way to detect the error.​
> 
> 
> ​Whatever. If you get a busted message, something is wrong somewhere upstream.​
> 
> 
> ​Hm, for ID Tokens, didn’t we do exactly the opposite, and end up with a Must-Ignore policy?​

Yes the receiver ignores unknown elements in the body of the JWT.  

I think that is different than a JWS/JWE library accepting additional parameters from a calling application to go into the envelope.   In that case the sender lib may want to check that they are not JW* reserved names that might cause some interoperability problems.

The lib producing the JWE/S is probably not going to normally duplicate elements on it's own.

Where I see it perhaps going bas is if someone is counting on the receiver taking the last one of a duplicate element and trying to override some header parameter by inserting a duplicate after the lib generated one.   This would work sometime and fail others depending on the receivers parser.

We probably want to be clear that anything extra going into an envelope needs to be checked to avoid reserved elements.

John B.

---

From: "Jim Schaad" <[email-masked]>
Date: Mon, 15 Sep 2014 15:48:41 -0700

My problem is that MUST NOT send means nothing to an attacker.  Esp. if the attacker is the message originator.  The only meaningful enforcement of any of this is going to be by the recipient.   If the recipient cannot enforce this restriction, then it will always be open to some type of attack from duplicate keys.

I would note that the MUST NOT for reordering of fields is not a JSON requirement but only a JavaScript requirement.  So an intermediate agent can read in and re-order the fields if it writes them out and not have a problem with the JSON specifications.

Jim

From: jose [mailto:[email-masked]] On Behalf Of Tim Bray
Sent: Monday, September 15, 2014 2:21 PM
To: John Bradley
Cc: [email-masked]; Stephen Kent; [email-masked]; [email-masked]; Kathleen Moriarty; Michael Jones; [email-masked]
Subject: Re: [jose] JWK member names, was: SECDIR review of draft-ietf-jose-json-web-key-31

​Agree, sigh.​

 

 

 

John B.

OK, I'm a bit confused.

Steve

-- 

 

--

---

From: Tero Kivinen <[email-masked]>
Date: Tue, 16 Sep 2014 11:02:28 +0300

> (perhaps not a good idea), or wait for I-JSON parsers (perhaps sometime

I do not understand this. Why would they need to write their own
parsers? I would expect they would require the existing parsers to be
fixed to reject objetcs with duplicate keys. If I understand correctly
that would already be allowed, as you said some implementations do
that, some return the last key.

If we make it MUST reject JW* items with duplicate keys, that would
make changes to parsers high priority items for the users of the JW*,
thus that would most likely get those changes done quite soon.

Of course there would still be some broken implementations allowing
broken JW* items to be parsed, but the only thing would been that
those implementations cannot claim to be complient with the RFC we are
publishing now. If they want to clam to be complient with RFC xxxx,
they would have to make sure their parser is also fixed...

I mean how many lines of code it would be in the parser to reject the
object if there is duplicate keys? I would expect the change to be
quite small (in order of few to few tens of lines). 

If we do not mandate this now, then nothing is going to change. There
is no demand to the existing parsers to be fixed, which means they
will not get fixed, and we will be stuck with bad parsers forever...

Ps. Following and participating this thread has been almost impossible
to me because some people write text inline without any indication
which is quoted text and which is original text. Only after going to
the secdir archives I noticed there are some color differences to
indicate quoted text, and on my text only screen session all that
information is lost... It would be nice to people actually do
something else than rely on some html-formatted colors to indidate
quoted text vs new text.
-- 
[email-masked]

---

From: Stephen Kent <[email-masked]>
Date: Tue, 16 Sep 2014 10:48:20 -0400

Tim,

> When I talk about existing software I'm referring to generic JSON 
> parsers such as are included in the basic library set of every 
> programming language now, and which are unfortunately idiosyncratic 
> and inconsistent in their handling of dupe keys, but in almost no 
> cases actually inform the calling software whether or not dupe keys 
> were encountered.
Thanks for the clarifying comments. I am still a bit puzzled, though. I 
thought JWK was
a proposal to establish JSON formats for key transport. Are you saying 
that the formats we are
about to standardize have been in use in JSON for a while and that's why 
parsers are not
prepared to reject dupe keys?  Or is this a lower layer, JS issue re 
partsing?

Steve

---

From: Stephen Kent <[email-masked]>
Date: Tue, 16 Sep 2014 10:52:42 -0400

Mike,

> Actually, the payloads of JWS and JWE objects can be any octet 
> sequence -- not just those representing JSON objects.
>
OK, thanks for correcting my mis-characterization.
> JWK objects are already used in production to distribute public keys.  
> For instance, the keys for Salesforce's identity services are in JWK 
> format at https://login.salesforce.com/id/keys. (Note that I'm not 
> saying that just because the current specs are in use, that no changes 
> are possible.)
>
if not that, what is the point of this comment?

Steve

---

From: John Bradley <[email-masked]>
Date: Tue, 16 Sep 2014 12:35:49 -0300

I was mostly trying to dig out what Tim was advocating.

I make no claims to be a expert on JSON parsers.  
That most of them accept duplicate keys is a reflection of the current JSON specs.

As Tim stated in a later message most of them don't reject or report duplicate keys.
He is proposing a new JSON profile I-JSON that changes that.

Changing things that are currently in wide deployment in many environments will not be a overnight thing.
It may be that in some environments ignoring duplicate keys may be seen by some (not me) as a feature.

I think we all agree that producers must be prohibited from emitting JSON envelopes with duplicate keys.

As Jim points out that won't stop bad people from doing it.

The only real protection is if receivers check and reject.

I agree that making it a MUST will encourage parsers behaviour to change more quickly. 

In the interim there are going to be a lot of non compliant implementations.

My only concern is to get people to fix the parsers properly rather than JOSE libraries building there own,
 parsers from scratch that may not be well tested and have other issues.

This did receive a lot of debate in the WG and the current wording was changed from MUST reject duplicate elements, based
on feed back from the JSON community.

This is not just a JWK issue, it also applies to JWE and JWS.

I am open to changing it if there is support for it in the WG.

John B.

---

From: Tim Bray <[email-masked]>
Date: Tue, 16 Sep 2014 08:47:32 -0700

On Tue, Sep 16, 2014 at 7:48 AM, Stephen Kent <[email-masked]> wrote:

> prepared to reject dupe keys?  Or is this a lower layer, JS issue re

​I’m not clear on the history, but I have heard that dupe keys are
sometimes generated by software that’s producing output on a streaming
basis, that can’t afford to keep track of every key it’s already generated.
 It is also my impression that for essentially all software that receives
and parses JSON, dupe keys are useless and perhaps damaging, since JSON
objects are invariably stuffed into hash-table-flavored things that don’t
support dupe keys.

It’s just that in the JOSE context, there is (justified) concern​ that
there are attack vectors based on the use of dupe keys.  Everyone agrees (I
think) that it would be desirable for such messages to be rejected.  It’s
just that current production software doesn’t make this easy.

---

From: Tim Bray <[email-masked]>
Date: Tue, 16 Sep 2014 08:51:27 -0700

On Tue, Sep 16, 2014 at 8:35 AM, John Bradley <[email-masked]> wrote:

> ​​
> ​​
> ​​

​Actually, the JSON working group is under no illusion that shipping I-JSON
as an RFC is going to magically cause existing JSON software to fix this
issue (although I’m optimistic that implementations will pop up pretty
quickly, because it’s not hard).

The value of I-JSON is that it takes all the things that we’ve observed to
cause interop problems in practical JSON, that in some cases have had to be
explicitly argued-over in other contexts (like we’re doing here), and in a
short simple document says “don’t do any of these things”.​  So it’ll be
handy as a spec-writer’s tool even before the software catches up.
​​

---

From: Mike Jones <[email-masked]>
Date: Tue, 16 Sep 2014 16:57:53 +0000

Actually, the payloads of JWS and JWE objects can be any octet sequence - not just those representing JSON objects.
OK, thanks for correcting my mis-characterization.

JWK objects are already used in production to distribute public keys.  For instance, the keys for Salesforce's identity services are in JWK format at https://login.salesforce.com/id/keys.  (Note that I'm not saying that just because the current specs are in use, that no changes are possible.)
if not that, what is the point of this comment?

The point of the comment was simply to answer your question "What is the existing software to which you and Tim refer...?".

                                                                Cheers,
                                                                -- Mike

---

From: Stephen Kent <[email-masked]>
Date: Tue, 16 Sep 2014 16:07:26 -0400

Mike,
> The point of the comment was simply to answer your question "What is 
> the existing software to which you and Tim refer...?"
>
And I have not yet received an answer to that question, in terms I can 
understand.

Let me try again.

What is the impediment to requiring a receiver of a JWK object to reject 
the object if
it contains more than one instance of a key?

Is it a limitation of a parser that are completely independent of the 
JOSE work that defines
the JWK objects, or is it the result of how folks have written code to 
parse such objects?

If the answer is the first clause, then I understand the reluctance to 
impose that requirement.

If the answer is the latter, then this is an argument based on early 
implementation
of an IETF spec, and that is not an good reason to accommodate such 
sloppiness.

Steve

---

From: John Bradley <[email-masked]>
Date: Tue, 16 Sep 2014 17:12:56 -0300

It is the first case.

JOSE libraries are typically using the built in JSON parsers in their environments at the moment.  Not surprisingly as the parsers are built in to most languages now.
It is those that don't reject duplicate members, not the JOSE libraries as the parser makes all but one of the duplicates disappear in most if not all cases.

John B.

On Sep 16, 2014, at 5:07 PM, Stephen Kent <[email-masked]> wrote:

>> The point of the comment was simply to answer your question “What is the existing software to which you and Tim refer…?”
> And I have not yet received an answer to that question, in terms I can understand.
> 
> Let me try again.
> 
> What is the impediment to requiring a receiver of a JWK object to reject the object if
> Is it a limitation of a parser that are completely independent of the JOSE work that defines
> the JWK objects, or is it the result of how folks have written code to parse such objects?
> 
> If the answer is the first clause, then I understand the reluctance to impose that requirement.
> 
> If the answer is the latter, then this is an argument based on early implementation
> of an IETF spec, and that is not an good reason to accommodate such sloppiness.
> 
> Steve

---

From: Tim Bray <[email-masked]>
Date: Tue, 16 Sep 2014 13:13:05 -0700

> the JWK objects, or is it the result of how folks have written code to

​Yes and no, respectively.  Existing parsers which are being used all the
time on every computing device within your reach to generate and parse JSON
for purposes which have nothing to with JOSE.  JSON has been the dominant
message format for HTTP for some time now.

---

From: Stephen Kent <[email-masked]>
Date: Wed, 17 Sep 2014 10:43:29 -0400

OK, now I have a clear answer to the question I posed earlier, i.e., this
is a JSON parser problem, not specific to the JOSE-defined formats.

I still believe it makes sense for the RFC(s) to mandate rejection of 
duplicates,
as a way to "encourage" transition to better parsers, as others have 
noted. And
I rely on Tero's judgement that the required changes are not onerous.

Steve

---

From: Tim Bray <[email-masked]>
Date: Wed, 17 Sep 2014 09:42:48 -0700

The chance  of the JOSE working group moving the vast world of deployed
JSON infrastructure round to 0.00.   Thus putting a MUST reject in here
would essentially say you can’t use well-debugged production software, and
would be a really bad idea.

On the other hand, if JOSE specified that producers’ messages MUST conform
to I-JSON, and a couple other WGs climbed on that bandwagon, and the word
started to get around, I wouldn’t be surprised if a few of the popular JSON
implementations added an I-JSON mode.  That would be a good thing and
lessen the attack surface of all JSON-based protocols (which these days, is
a whole lot of them).

---

From: Richard Barnes <[email-masked]>
Date: Wed, 17 Sep 2014 12:52:41 -0400

People are going to use off-the-shelf JSON parsers because they're the
obvious tool for the job.  It seems like a general maxim that if we
prohibit using the obvious tools without a really good reason (which I
don't really see here), people are just going to ignore the prohibition.

--Richard

---

From: Stephen Kent <[email-masked]>
Date: Wed, 17 Sep 2014 13:58:01 -0400

Tim,
> The chance  of the JOSE working group moving the vast world of 
> deployed JSON infrastructure round to 0.00.   Thus putting a MUST 
> reject in here would essentially say you can't use well-debugged 
> production software, and would be a really bad idea.
So, JSON is not easily changed, but adopting I-JSON will easier. OK, 
I'll take your word on that.
> On the other hand, if JOSE specified that producers' messages MUST 
> conform to I-JSON, and a couple other WGs climbed on that bandwagon, 
> and the word started to get around, I wouldn't be surprised if a few 
> of the popular JSON implementations added an I-JSON mode.  That would 
> be a good thing and lessen the attack surface of all JSON-based 
> protocols (which these days, is a whole lot of them).

I am comfortable with mandating I-JSON if you believe that will be a 
more effective way to
encourage change.

Steve

---

From: Tero Kivinen <[email-masked]>
Date: Thu, 18 Sep 2014 11:06:06 +0300

Tim Bray writes:
> reject in here would essentially say you can’t use well-debugged

And are none of those jose parsers open source? If any of them is open
source, then someone who wants to use jose, could take one, fix it to
reject duplicates, and use that still well-debugged production
software, with small patch, and he just need to add regression test
case for the new patch, and rerun the normal regression tests to know
everything else still works.

If all of them are closed source software which you cannot patch, then
it might be better that people write proper open source parser which
actually tries to be secure.

> On the other hand, if JOSE specified that producers’ messages MUST
> and the word started to get around, I wouldn’t be surprised if a few
> of the popular JSON implementations added an I-JSON mode.  That
> would be a good thing and lessen the attack surface of all
> JSON-based protocols (which these days, is a whole lot of them).

And if we say MUST reject structures with duplicate keys, that would
perhaps force them even more, especially as those vendors really
wanting to be conformant would start asking that.

On the other hand, I think most of the vendors would just issue
request for the fix, but still continue using the relaxed parser,
regardless what we write in the specification here. At least if we say
MUST then they hopefully will put the feature request in. If we say
SHOULD, they will not...
-- 
[email-masked]

---

From: Richard Barnes <[email-masked]>
Date: Thu, 18 Sep 2014 11:03:35 -0400

We had a whole working group on JSON, which I was hoping would solve the
problem of duplicate members (by forbidding them).  That would have caused
the parser community to get more strict.

Unfortunately, that didn't happen, largely due to concern over streaming
generators/parsers that can't keep track of what keys they've seen.
Whether you agree with that or not, that's the IETF consensus in the
current JSON RFC.

The JOSE documents are not the place to solve JSON's duplicate-key
problems.  We tried to that problem and decided not to.

So we have two choices: Either deal with the limitations of the tool, or
throw it away and use another.  The limitations here are really not bad.
There are no attacks that would be enabled by duplicate keys, and there are
no other tools that provide what we need.  Regardless  of the merits of
I-JSON, the software base doesn't exist, and this spec isn't going to make
it exist (but if it does come to be, then implementations can use it!).
The current text basically says, "use an off-the-shelf parser that works
the way most off-the-shelf parsers work."  It accommodates the limitations
of reality.

Continuing to tilt at this windmill is just going to result in either
killing JOSE deployment or creating a portion of the spec that is
universally ignored.  And moreover, it will demonstrate further how
out-of-touch the IETF is with reality.

--Richard

On Thu, Sep 18, 2014 at 4:06 AM, Tero Kivinen <[email-masked]> wrote:

> secdir mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/secdir
> wiki: http://tools.ietf.org/area/sec/trac/wiki/SecDirReview
>

---

From: Mike Jones <[email-masked]>
Date: Tue, 23 Sep 2014 23:40:39 +0000

FYI, I did not change the language about duplicate member names in the JOSE -32 and JWT -26 drafts at this time because it seems that there remains substantial working group support for the current semantics, including by Tim Bray (the JSON spec editor) and Richard Barnes.  I did not yet add an I-JSON reference to impose a requirement on producers because it seemed imprudent to take a normative dependency on an unfinished specification.  However, if I-JSON does finish before these specs are RFCs, we could easily do that when it finishes, if the working group, etc. concurs with that action.

My focus for this round of edits was to resolve all the review comments for which the proposed resolutions appeared to be uncontroversial.  I understand that the working group and others may continue discussing this issue.

                                                                -- Mike

From: Stephen Kent [mailto:[email-masked]]
Sent: Wednesday, September 17, 2014 10:58 AM
To: Tim Bray
Cc: John Bradley; Mike Jones; [email-masked]; Kathleen Moriarty; [email-masked]; [email-masked]; [email-masked]
Subject: Re: [jose] JWK member names, was: SECDIR review of draft-ietf-jose-json-web-key-31

Tim,

The chance  of the JOSE working group moving the vast world of deployed JSON infrastructure round to 0.00.   Thus putting a MUST reject in here would essentially say you can't use well-debugged production software, and would be a really bad idea.
So, JSON is not easily changed, but adopting I-JSON will easier. OK, I'll take your word on that.

On the other hand, if JOSE specified that producers' messages MUST conform to I-JSON, and a couple other WGs climbed on that bandwagon, and the word started to get around, I wouldn't be surprised if a few of the popular JSON implementations added an I-JSON mode.  That would be a good thing and lessen the attack surface of all JSON-based protocols (which these days, is a whole lot of them).

I am comfortable with mandating I-JSON if you believe that will be a more effective way to
encourage change.

Steve

---

From: Kathleen Moriarty <[email-masked]>
Date: Thu, 25 Sep 2014 10:18:12 -0400

I think it is fine to leave this issue as open through the IESG review.
The discussion and further explanation in this thread has been helpful,
thank you.  This could get handled in a number of ways, leave it as-is,
address it with the I-JSON reference in this draft, or in an update to the
published RFC.  We'll see if there are strong opinions in the IESG.  I tend
to go for stricter options to prevent issues, but do recognize that there
are some challenges with that option and would like to see the IESG
opinions.

Thank you.

On Tue, Sep 23, 2014 at 7:40 PM, Mike Jones <[email-masked]>
wrote:

>  FYI, I did not change the language about duplicate member names in the
> JOSE -32 and JWT -26 drafts at this time because it seems that there
> remains substantial working group support for the current semantics,
> including by Tim Bray (the JSON spec editor) and Richard Barnes.  I did not
> yet add an I-JSON reference to impose a requirement on producers because it
> seemed imprudent to take a normative dependency on an unfinished
> specification.  However, if I-JSON does finish before these specs are RFCs,
> we could easily do that when it finishes, if the working group, etc.
> concurs with that action.
>
>
>
> My focus for this round of edits was to resolve all the review comments
> for which the proposed resolutions appeared to be uncontroversial.  I
> understand that the working group and others may continue discussing this
> issue.
>
>
>
>                                                                 -- Mike
>
>
>
> *From:* Stephen Kent [mailto:[email-masked]]
> *Sent:* Wednesday, September 17, 2014 10:58 AM
> *To:* Tim Bray
> *Cc:* John Bradley; Mike Jones;
> [email-masked]; Kathleen Moriarty;
> [email-masked]; [email-masked]; [email-masked]
> conform to I-JSON, and a couple other WGs climbed on that bandwagon, and
> the word started to get around, I wouldn’t be surprised if a few of the
> popular JSON implementations added an I-JSON mode.  That would be a good
> thing and lessen the attack surface of all JSON-based protocols (which
> these days, is a whole lot of them).
>
>

-- 

Best regards,
Kathleen