Subject: [Json] Counterproposal on work items

From: Tim Bray <[email-masked]>
Date: Wed, 20 Feb 2013 09:27:12 -0800

My proposal is: do nothing.

I use JSON for protocol design and work all the time, and have not observed
any interop problems in the wild which originate at the JSON parson or
construction level.  I give the incoming text to the library and it Just
Works or reliably reports a syntax botch.  I give my data structures to the
JSON serializer and cheerfully send off whatever comes out. I read specs
and build clients and servers and, when things break, it’s because I’m
stupidly using a bogus name or value in some field, not because of the
serialization.

I suggest that there is not a problem here that needs the investment of
precious IETF time.

 -T

---

From: Paul Hoffman <[email-masked]>
Date: Wed, 20 Feb 2013 09:38:19 -0800

On Feb 20, 2013, at 9:27 AM, Tim Bray <[email-masked]> wrote:

> I use JSON for protocol design and work all the time, and have not observed any interop problems in the wild which originate at the JSON parson or construction level.  I give the incoming text to the library and it Just Works or reliably reports a syntax botch.  I give my data structures to the JSON serializer and cheerfully send off whatever comes out. I read specs and build clients and servers and, when things break, it’s because I’m stupidly using a bogus name or value in some field, not because of the serialization.
> 
> I suggest that there is not a problem here that needs the investment of precious IETF time.

-1.

There are places where RFC 4627 has SHOULDs where some processors do one thing and others do something different. That should be cleaned up in a standards-track RFC, and it should be done with lots of JSON developers and users having a discussion that comes to rough consensus.

The other stuff mentioned so far will probably happen as experimental RFCs even if there is no WG; a WG might help prevent those from being silly, but doing so might take a lot of effort (more than the value).

--Paul Hoffman

---

From: "Matt Miller (mamille2)" <[email-masked]>
Date: Wed, 20 Feb 2013 17:43:19 +0000

> There are places where RFC 4627 has SHOULDs where some processors do one thing and others do something different. That should be cleaned up in a standards-track RFC, and it should be done with lots…

Just to reinforce Paul's point; while things seem to more-or-less work out, there are serious questions occasionally asked about the appropriateness of JSON for use in security protocols given those differences.  It might seem like re-arranging the deck chairs, but that arrangement can be important for acceptance.

- m&m

Matt Miller < [email-masked] >
Cisco Systems, Inc.

---

From: Gonzalo Salgueiro <[email-masked]>
Date: Wed, 20 Feb 2013 12:43:21 -0500

I don't hink doing anything is the proper course.  At *minimum* clean up of 4627 and moving it to Standards-Track is needed IMO.

--G

On Feb 20, 2013, at 12:27 PM, Tim Bray <[email-masked]> wrote:

> _______________________________________________
> json mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/json

---

From: Mike Jones <[email-masked]>
Date: Wed, 20 Feb 2013 18:03:58 +0000

The "SHOULD" versus "MUST" caused the JOSE specs to have to include this language in http://tools.ietf.org/html/draft-ietf-jose-json-web-signature-08#section-4:

   The Header Parameter Names within this object MUST be unique;
   JWSs with duplicate Header Parameter Names MUST be rejected.

Whereas, if we were referencing a RFC4627bis with the MUST, this language could be softened to be a caution about older JSON parsers, and probably moved to the Security Considerations section.

				-- Mike

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Matt Miller (mamille2)
Sent: Wednesday, February 20, 2013 9:43 AM
To: [email-masked]
Subject: Re: [Json] Counterproposal on work items

On Feb 20, 2013, at 10:38 AM, Paul Hoffman <[email-masked]> wrote:

>> I use JSON for protocol design and work all the time, and have not observed any interop problems in the wild which originate at the JSON parson or construction level.  I give the incoming text to the library and it Just Works or reliably reports a syntax botch.  I give my data structures to the JSON serializer and cheerfully send off whatever comes out. I read specs and build clients and servers and, when things break, it's because I'm stupidly using a bogus name or value in some field, not because of the serialization.
>> 

- m&m

---

From: Tatu Saloranta <[email-masked]>
Date: Wed, 20 Feb 2013 10:22:30 -0800

+1

-+ Tatu +-

On Wed, Feb 20, 2013 at 9:27 AM, Tim Bray <[email-masked]> wrote:
> JSON serializer and cheerfully send off whatever comes out. I read specs and
> build clients and servers and, when things break, it’s because I’m stupidly
> using a bogus name or value in some field, not because of the serialization.
>

---

From: Nico Williams <[email-masked]>
Date: Wed, 20 Feb 2013 12:43:25 -0600

> There are places where RFC 4627 has SHOULDs where some processors do one thing and others do something different. That should be cleaned up in a standards-track RFC, and it should be done with lots…

One I-D as simple as this hardly justifies a WG.

---

From: Tony Hansen <[email-masked]>
Date: Wed, 20 Feb 2013 13:55:38 -0500

> One I-D as simple as this hardly justifies a WG.

Countercase in point: the imapmove working group.  It had one document, 
was chartered in 7/2012, and lived for a total of 6 months until it 
successfully shut down in 1/2013.

     Tony Hansen

---

From: Paul Hoffman <[email-masked]>
Date: Wed, 20 Feb 2013 11:02:14 -0800

> One I-D as simple as this hardly justifies a WG.

Getting broad consensus on changing a standard that is implemented widely outside the IETF justifies the effort to have the time and space for consensus. This is *not* just IETF work.

--Paul Hoffman

---

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Wed, 20 Feb 2013 20:24:13 +0000

On 2/20/13 12:02 PM, "Paul Hoffman" <[email-masked]> wrote:

>Getting broad consensus on changing a standard that is implemented widely
>outside the IETF justifies the effort to have the time and space for
>consensus. This is *not* just IETF work.

(individual)
That seems like a solid argument.

(BoF chair)
We should definitely spend some time on this question at the BoF.  I think
it's going to be the crux for whether we charter a working group or not.
As such, the apparea wg chairs and AD's opinions are likely to be
interesting from a management perspective.  Do they want the overhead of a
working group for something relatively small or not?

In my talks with Pete and Barry, I've been given to believe that they
would like to be able to charter and kill small working groups rapidly.

-- 
Joe Hildebrand

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 21 Feb 2013 09:30:58 +1100

> Getting broad consensus on changing a standard that is implemented widely outside the IETF justifies the effort to have the time and space for consensus. This is *not* just IETF work.

I don't know. I think I'd be fine if we just asked Crockford (perhaps helped by a willing editor) to do 4627bis and then have the AD sponsor it on Standards Track.

Cheers,

--
Mark Nottingham   http://www.mnot.net/

---

From: Tim Bray <[email-masked]>
Date: Wed, 20 Feb 2013 14:34:24 -0800

+1 (Not that I have any idea if he’s interested) -T

On Wed, Feb 20, 2013 at 2:30 PM, Mark Nottingham <[email-masked]> wrote:

> >>> There are places where RFC 4627 has SHOULDs where some processors do
> one thing and others do something different. That should be cleaned up in a
> standards-track RFC, and it should be done with lots of JSON developers and
> users having a discussion that comes to rough consensus.
> >>
> > Getting broad consensus on changing a standard that is implemented
> widely outside the IETF justifies the effort to have the time and space for
> I don't know. I think I'd be fine if we just asked Crockford (perhaps
> helped by a willing editor) to do 4627bis and then have the AD sponsor it
> on Standards Track.
>
> Cheers,
>
> --

---

From: Peter Saint-Andre <[email-masked]>
Date: Wed, 20 Feb 2013 18:44:22 -0700

On 2/20/13 3:30 PM, Mark Nottingham wrote:
>>>> There are places where RFC 4627 has SHOULDs where some 
>>>> processors do one thing and others do something different.
>>>> That should be cleaned up in a standards-track RFC, and it
>>>> should be done with lots of JSON developers and users having
>>>> a discussion that comes to rough consensus.
>>> 
>> Getting broad consensus on changing a standard that is
>> implemented widely outside the IETF justifies the effort to have
>> the time and space for consensus. This is *not* just IETF work.
> 
> 
> I don't know. I think I'd be fine if we just asked Crockford
> (perhaps helped by a willing editor) to do 4627bis and then have
> the AD sponsor it on Standards Track.

That does seem reasonable, and it carefully sidesteps the
all-too-common desire to "improve" a spec...

Peter

- -- 
Peter Saint-Andre
https://stpeter.im/

---

From: Robert Sayre <[email-masked]>
Date: Wed, 20 Feb 2013 18:03:19 -0800

> all-too-common desire to "improve" a spec...

Yes. The RFC is out of date now, in the sense that you can expect to
have interoperability problems if you follow it to the letter. But, as
Bjoern points out, there are aspects of JSON defined only in the RFC.

In other words, I think a new RFC is needed, but Crockford has already
documented almost all of the changes needed in ES5.

- Rob

---

From: "Paul E. Jones" <[email-masked]>
Date: Wed, 27 Feb 2013 11:44:53 -0500

Mark,

> helped by a willing editor) to do 4627bis and then have the AD sponsor
> it on Standards Track.

I did ask him.  I actually asked him about updating it even before I had
heard about the JSON BoF.  He indicated that he is willing to work on it "as
long as it does not break compatibility with existing implementations."

As editor, I think he would be in a good position to ensure that his
requirement is met.

Paul

---

From: Barry Leiba <[email-masked]>
Date: Wed, 27 Feb 2013 13:15:20 -0500

> As editor, I think he would be in a good position to ensure that his

Indeed; I had talked with him back in August, as well, with the same
response.  And as the likely responsible AD for any WG that forms, I
can say that I strongly agree with the "doesn't break compatibility"
point.  My view is that the charter has to make it clear that the
"4627 to Standards Track" work is essentially a re-classification in
place, rolling in errata and clarifications, but *not* changing the
language.  Any proposals that actually make changes would have to come
later, and be considered as separate work proposals.

Barry, Applications AD

---

From: Francis Galiegue <[email-masked]>
Date: Wed, 27 Feb 2013 19:21:18 +0100

On Wed, Feb 27, 2013 at 7:15 PM, Barry Leiba <[email-masked]> wrote:
[...]
> point.

Which means there will always be the sour point of specs like JSON
Patch having to specify that an operation must have one and only one
member named "op". Because "doesn't break compatibility" means the "no
duplicate member names" will remain a SHOULD, right?

I personally hate this wording in JSON Patch because it doesn't really
impose constraints on the format of JSON Patch itself, it also imposes
constraints on _parsers_. And this is a problem. Many parsers will
just _not_ yell on duplicate member names, and RFC 4627 currently
allows them to do so.

Just my 2e-42 cents,
-- 
Francis Galiegue, [email-masked]
JSON Schema in Java: http://json-schema-validator.herokuapp.com

---

From: Peter Saint-Andre <[email-masked]>
Date: Wed, 27 Feb 2013 11:29:36 -0700

On 2/27/13 11:15 AM, Barry Leiba wrote:
>>> (perhaps helped by a willing editor) to do 4627bis and then
>>> have the AD sponsor it on Standards Track.
>> 
>> I did ask him.  I actually asked him about updating it even
>> before I had heard about the JSON BoF.  He indicated that he is
>> willing to work on it "as long as it does not break compatibility
>> with existing implementations."
>> 
>> As editor, I think he would be in a good position to ensure that
>> his requirement is met.
> 
> Indeed; I had talked with him back in August, as well, with the
> same response.  And as the likely responsible AD for any WG that
> forms, I can say that I strongly agree with the "doesn't break
> compatibility" point.  My view is that the charter has to make it
> clear that the "4627 to Standards Track" work is essentially a
> re-classification in place, rolling in errata and clarifications,
> but *not* changing the language.  Any proposals that actually make
> changes would have to come later, and be considered as separate
> work proposals.

+1 for sure.

Peter

- -- 
Peter Saint-Andre
https://stpeter.im/

---

From: "Paul E. Jones" <[email-masked]>
Date: Wed, 27 Feb 2013 13:40:40 -0500

> Which means there will always be the sour point of specs like JSON Patch
> having to specify that an operation must have one and only one member
> named "op". Because "doesn't break compatibility" means the "no

If you want more than one member named "op", why not just use an array?

Paul

---

From: Francis Galiegue <[email-masked]>
Date: Wed, 27 Feb 2013 19:45:27 +0100

> If you want more than one member named "op", why not just use an array?

That is not the problem here.

The problem is that one JSON Patch operation is an object, and this
object must have a member named "op", so far so good.

BUT due to RFC 4627 being lax here, the draft says that there should
be "one and only one member" by that name. It should not have to say
that. RFC 4627 should make duplicate member names outright illegal.
Well, not 4627, but whatever will supersede/complement it.

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 27 Feb 2013 19:45:29 +0100

On Feb 27, 2013, at 19:15, Barry Leiba <[email-masked]> wrote:

> doesn't break compatibility

Being embroiled in a similar discussion right now in a different place, I'd sure like to know what that means:

-- we can't change anything substantive at all;

-- we're not going to extend it (I sure hope that is consensus);

-- we can't take out things that shouldn't be in there (UTF-32, anyone?);

-- we can't clarify ambiguities because there are implementations that solve them either way (duplicate keys?);

-- we can't do often-asked-for things like allow numbers on top (well, that is not quite consensus, maybe).

Grüße, Carsten

---

From: Francis Galiegue <[email-masked]>
Date: Wed, 27 Feb 2013 19:49:06 +0100

On Wed, Feb 27, 2013 at 7:45 PM, Francis Galiegue <[email-masked]> wrote:
[...]
> that.

And, I'll repeat myself, it especially should not (have to) say that
since it imposes constraints on the parser. Which means it tramples on
RFC 4627 territory. And that is not a good thing.

---

From: "Markus Lanthaler" <[email-masked]>
Date: Wed, 27 Feb 2013 21:24:11 +0100

> > changes would have to come later, and be considered as separate

Huge +1

--
Markus Lanthaler
@markuslanthaler

---

From: Paul Hoffman <[email-masked]>
Date: Thu, 28 Feb 2013 11:46:01 -0800

> can say that I strongly agree with the "doesn't break compatibility"

As we discussed, RFC 4627 says "The names within an object SHOULD be unique." So, what does "break compatibility" mean for this specific issue? If we make the SHOULD a MUST, there are emitters that will become incompatible. If change the sentence to follow the ECMAScript spec ("In the case where there are duplicate name Strings within an object, lexically preceding values for the same key shall be overwritten."), there are parsers that will become incompatible. 

> My view is that the charter has to make it clear that the
> language.  

Is the ECMAScript text a clarification or a change?

> Any proposals that actually make changes would have to come

That certainly sounds reasonable. The question is whether you will allow clarifications that might make something incompatible.

--Paul Hoffman

---

From: Tim Bray <[email-masked]>
Date: Thu, 28 Feb 2013 12:06:06 -0800

It’s clear to me that, *for the purposes of the IETF*, someone needs to say
“People sending JSON MUST NOT use duplicate keys.”  The fact that certain
libraries allow less-clueful developers to do this, and that parsing
software is observed to behave unpredictably when they do, should only
encourage us.
-T

> can say that I strongly agree with the "doesn't break compatibility"

As we discussed, RFC 4627 says "The names within an object SHOULD be
unique." So, what does "break compatibility" mean for this specific issue?
If we make the SHOULD a MUST, there are emitters that will become
incompatible. If change the sentence to follow the ECMAScript spec ("In the
case where there are duplicate name Strings within an object, lexically
preceding values for the same key shall be overwritten."), there are
parsers that will become incompatible.

> later, and be considered as separate work proposals.

That certainly sounds reasonable. The question is whether you will allow
clarifications that might make something incompatible.

--Paul Hoffman

---

From: Paul Hoffman <[email-masked]>
Date: Thu, 28 Feb 2013 12:43:14 -0800

On Feb 28, 2013, at 12:06 PM, Tim Bray <[email-masked]> wrote:

> It’s clear to me that, *for the purposes of the IETF*, someone needs to say “People sending JSON MUST NOT use duplicate keys.”  The fact that certain libraries allow less-clueful developers to do this, and that parsing software is observed to behave unpredictably when they do, should only encourage us. 

That doesn't answer the question in the heading, and the question was aimed at our esteemed Area Director. :-)

--Paul Hoffman

---

From: Nico Williams <[email-masked]>
Date: Thu, 28 Feb 2013 14:46:54 -0600

> software is observed to behave unpredictably when they do, should only

The obvious compromise is to say senders MUST NOT send dup object keys
and receivers MUST take the last key value pair of any dup keys,
per-ECMAScript.  The latter preserves compatibility.

Nico
--

---

From: Barry Leiba <[email-masked]>
Date: Thu, 28 Feb 2013 16:46:39 -0500

> unique." So, what does "break compatibility" mean for this specific issue?
...
> clarifications that might make something incompatible.

Yes, I guess I did speak too soon: that specific change is one we've
talked about, and that's likely to be on the charter.

But we do need to scope this very tightly.

Barry

---

From: Paul Hoffman <[email-masked]>
Date: Thu, 28 Feb 2013 13:51:17 -0800

> per-ECMAScript.  The latter preserves compatibility.

...but the former ("say senders MUST NOT send dup object keys") breaks it, given that the RFC had a SHOULD not a MUST. Thus the question in this thread.

--Paul Hoffman

---

From: Tim Bray <[email-masked]>
Date: Thu, 28 Feb 2013 13:59:37 -0800

If there were a significant number of protocols, or deployed software
components, that relied on the use of multiple keys, then changing the
SHOULD to a MUST would be a breaking change. I do not believe that there
are any, and thus this is not, *de facto*, a breaking change.  That
hypothesis is easily falsified, if in the course of the IETF work someone
sticks their hand up and says “this is going to break me”.  But I’m not
holding my breath.

 -T

On Thu, Feb 28, 2013 at 1:51 PM, Paul Hoffman <[email-masked]> wrote:

> ...but the former ("say senders MUST NOT send dup object keys") breaks it,
> given that the RFC had a SHOULD not a MUST. Thus the question in this
> thread.
>
> --Paul Hoffman

---

From: Nico Williams <[email-masked]>
Date: Thu, 28 Feb 2013 16:06:05 -0600

> ...but the former ("say senders MUST NOT send dup object keys") breaks it, given that the RFC had a SHOULD not a MUST. Thus the question in this thread.

Depends on your definition of "break".  If you mean that some senders
are now "non-compliant", you're right, but there's no compliance
police here so what matters is whether *receivers* change behavior in
ways that a) are different from the old spec *and* b) break actual
interop.  It's the interop that matters.

Nico
--

---

From: Tatu Saloranta <[email-masked]>
Date: Thu, 28 Feb 2013 14:14:26 -0800

On Thu, Feb 28, 2013 at 1:59 PM, Tim Bray <[email-masked]> wrote:
> SHOULD to a MUST would be a breaking change. I do not believe that there are
> any, and thus this is not, *de facto*, a breaking change.  That hypothesis
> is easily falsified, if in the course of the IETF work someone sticks their
> hand up and says “this is going to break me”.  But I’m not holding my
> breath.

+1 on this.

One minor comment on latter part -- in cases where parsers do not
retain more state than absolutely required for matching open scopes
(aka "streaming parsers"), this would presumably mean no change, as
all key/value pairs would still be exposed as is. It would then be up
to higher layers, such as tree builder or data-binder, to ensure that
of possible duplicates the last one sticks. This is how many tools
already work.
I just want to mention this as it is one case where wording could
still be debated ("but parser SHOULD remove other values, only expose
last one"), depending on wording.

-+ Tatu +-