Subject: [Json] [Technical Errata Reported] RFC7158 (3907)

From: RFC Errata System <[email-masked]>
Date: Sun, 2 Mar 2014 12:34:13 -0800 (PST)

The following errata report has been submitted for RFC7158,
"The JavaScript Object Notation (JSON) Data Interchange Format".

--------------------------------------
You may review the report below and at:
http://www.rfc-editor.org/errata_search.php?rfc=7158&eid=3907

--------------------------------------
Type: Technical
Reported by: Marc Lehmann <[email-masked]>

Section: 1.3, 2.0, 3.

Original Text
-------------
   This document's goal is to apply the errata, remove inconsistencies
   with other specifications of JSON, and highlight practices that can
   lead to interoperability problems.

   A JSON value MUST be an object, array, number, or string, or one of
   the following three literal names:

Corrected Text
--------------

Notes
-----
Since RFC7158 breaks compatibility with the specifications, this should be
duly noted.

The original specification (both in RFC4627 and on www.json.org, I know of
no other JSON "specifications") specify JSON as a self-delimited format,
that is, each JSON text can be parsed without extra delimiters or length
wrapping, and some network protocols came to rely on this by asking for
back-to-back JSON texts.

RFC7158 changes this, so protocols that formerly consumed JSON texts
without any extra wrapping are no longer compliant to this new
specification. These protocols are now in violation of RFC7158, and they
can't be fixed in a backwards compatible way.

This introduces a big incompatibility between RFC7158 implementations and
older ones, and this is not at all reflected in the introduction nor, as
far as I could see, anywhere else in the new rfc.

Thus it is likely that this is a specification error - the new RFC
should either warn that the new RFC is not compatible with the existign
specifications, or be fixed to be compatible.

I think changing JSON in an incompatible way at this stage would be an
extremely bad idea.

Instructions:
-------------
This errata is currently posted as "Reported". If necessary, please
use "Reply All" to discuss whether it should be verified or
rejected. When a decision is reached, the verifying party (IESG)
can log in to change the status and edit the report, if necessary. 

--------------------------------------
RFC7158 (draft-ietf-json-rfc4627bis-10)
--------------------------------------
Title               : The JavaScript Object Notation (JSON) Data Interchange Format
Publication Date    : March 2014
Author(s)           : T. Bray, Ed.
Category            : PROPOSED STANDARD
Source              : JavaScript Object Notation
Area                : Applications
Stream              : IETF
Verifying Party     : IESG

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Sun, 02 Mar 2014 22:31:07 +0100

>Since RFC7158 breaks compatibility with the specifications, this should be

It is in Appendix A., "Changes from RFC 4627":

   o  Changed the definition of "JSON text" so that it can be any JSON
      value, removing the constraint that it be an object or array.

Failing to note this in the Introduction is not ideal, but the errata
system is not a good place to record that (holding it for document up-
date probably does not make sense, if and when the document does get
updated, this particular issue will not require prominent notice).
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Sun, 02 Mar 2014 23:29:49 +0100

* Marc Lehmann wrote:
>Just for additional info, I followed the instructions given in the
>RFC to give feedback (the page linked to by the rfc for feedback,
>http://www.rfc-editor.org/info/rfc7158, didn't seem to list any other
>options to give feedback than the errata system).
>
>It was merely my intent to give feedback on this issue (which is a big
>issue for me as an implementerm not necessarily an errata).

The document has gone through a series of drafts with various calls for
review and many comments and has now been published as RFC, meaning it's
impossible to change it; it can only be replaced through the heavyweight
process of going through the IETF consensus process again. RFC Errata
https://www.ietf.org/iesg/statement/errata-processing.html allow us to
track some information about errors in the document, but people who miss
this change from RFC 4627 in their use of the document are not likely to
consult the errata pages.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Pete Resnick <[email-masked]>
Date: Mon, 3 Mar 2014 11:07:51 +0000

On 3/3/14 11:03 AM, Ladislav Lhotka wrote:
> Hmm, according to the RFC 7158 text, the month of publication appears to be “March 2013”.
>    

Yes, we noticed that. The RFC Editor is going to publish a corrected 
version.

pr

-- 
Pete Resnick<http://www.qualcomm.com/~presnick/>
Qualcomm Technologies, Inc. - +1 (858)651-4478

---

From: "Martin J. Dürst" <[email-masked]>
Date: Tue, 04 Mar 2014 18:01:05 +0900

I saw the announcement for the new RFC (7159). For the record, although 
I understand the basic idea behind the policy that RFCs are never 
changed, I think it would have been much wiser to just make the change 
in place.

In the long run (in this case, my guess would be that this means more 
than one week), the implications would have been much smaller than 
having to live with two virtually identical RFCs with numbers differing 
only by 1 for a long time.

Regards,   Martin.

P.S.: Needless to say that the best would have to be to avoid the 
mistake in the first place.

P.P.S.: W3C has a policy for very limited in-place fixes
(http://www.w3.org/2003/01/republishing/), but it doesn't include this case.

---

From: Paul Hoffman <[email-masked]>
Date: Tue, 4 Mar 2014 10:29:28 +0000

On Mar 4, 2014, at 9:01 AM, Martin J. Dürst <[email-masked]> wrote:

> I saw the announcement for the new RFC (7159). For the record, although I understand the basic idea behind the policy that RFCs are never changed, I think it would have been much wiser to just make the change in place.

This comment is relevant to [email-masked], not here. This WG doesn't make rules for how RFCs are published.

--Paul Hoffman

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Tue, 04 Mar 2014 15:43:50 +0100

* Paul E. Jones wrote:
>I've not been following the list closely, as I thought this was more of 
>an editorial exercise than anything else.  This is definitely more than 
>editorial.  This could definitely break things and the commenter is 
>right that there are instances where there could be misinterpretation.
>
>Why was it decided to change this:
>
>     JSON-text = object / array
>
>to
>
>     JSON-text = ws value ws
>
>Was there some misunderstanding of what 4627 said?

http://www.ietf.org/mail-archive/web/json/current/msg02040.html is the
decision, you can find relevant discussion nearby; most arguments I've
seen revolved around various notions of consistency.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: "Paul E. Jones" <[email-masked]>
Date: Tue, 04 Mar 2014 14:17:37 +0000

to

     JSON-text = ws value ws

Paul

------ Original Message ------
From: "Bjoern Hoehrmann" <[email-masked]>
To: "RFC Errata System" <[email-masked]>
Cc: [email-masked]; [email-masked]; [email-masked]; 
[email-masked]; [email-masked]; [email-masked]; 
[email-masked]; [email-masked]
Sent: 3/2/2014 4:31:07 PM
Subject: Re: [Json] [Technical Errata Reported] RFC7158 (3907)

>_______________________________________________
>json mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/json

---

From: Pete Resnick <[email-masked]>
Date: Tue, 4 Mar 2014 15:18:33 +0000

Hi Marc,

As you may have surmised from some of the discussion, the note on the 
RFC Editor page is misleading regarding where to report things, and we 
(the IETF leadership) are working with the RFC Editor to update their 
page to make that clearer. What you have reported is actually input to 
the JSON Working Group, which the working group should be discussing on 
the WG mailing list, not errata, which are really for simple textual 
corrections to published RFCs. Certainly your input is appreciated, and 
it doesn't matter that it landed in the wrong place through no fault of 
your own.

So here's what I would suggest: Your comments have been Cc'ed to the 
JSON WG list from the errata system. I'll leave it to the chairs (Paul 
and Matt) to say whether they want you to resend them directly to the WG 
mailing list with a new Subject so they can track it, or whether the Cc 
of the errata messages will suffice. Either way, please remove the RFC 
Editor and the Area Directors (Barry and I) from the Cc list if you 
continue the discussion of the issue. Meanwhile, I'll have the RFC 
Editor remove these entries from the Errata system; there's no need to 
keep the RFC Editor in the loop on these topics.

Again, I appreciate your input, and I hope we can get the RFC Editor 
pages updated so it's clear where to send different kinds of comments.

pr

---

From: Paul Hoffman <[email-masked]>
Date: Tue, 4 Mar 2014 16:43:17 +0000

On Mar 4, 2014, at 3:18 PM, Pete Resnick <[email-masked]> wrote:

> Your comments have been Cc'ed to the JSON WG list from the errata system. I'll leave it to the chairs (Paul and Matt) to say whether they want you to resend them directly to the WG mailing list with a new Subject so they can track it, or whether the Cc of the errata messages will suffice.

The former. That is, please send a message to the list with a relevant Subject: line.

--Paul Hoffman

---

From: Tim Bray <[email-masked]>
Date: Tue, 4 Mar 2014 09:27:51 -0800

On this issue, also please read the interoperability note in the paragraph
before the new production.

On Tue, Mar 4, 2014 at 6:17 AM, Paul E. Jones <[email-masked]> wrote:

> I've not been following the list closely, as I thought this was more of an
> editorial exercise than anything else.  This is definitely more than
> editorial.  This could definitely break things and the commenter is right
> that there are instances where there could be misinterpretation.
>

---

From: "Paul E. Jones" <[email-masked]>
Date: Tue, 04 Mar 2014 20:12:45 +0000

Tim,

Yeah, I saw that.  However, I still think this only opens the door for 
future implementations to produce things that do not work reliably.  As 
Marc Lehmann rightfully pointed out (and, BTW, his Errata text appears 
to have disappeared), a serializer that is requested to serialize a 
series of integers like 1 and 999 might produce 1999.  Reading that back 
in, the single value 1999 would be read.  Likewise, a serializer might 
produce truefalse as output, but then fail when one tries to read that 
back.  One might argue to start all serialization with whitespace to 
avoid having values serialized adjacent to each other.  That would work, 
but unfortunate.

Maybe I missed it, but I could not see _why_ the change was made to this 
production.  What was wrong with allowing only an object or array?

Looking at some of the messages hanging off of the link Björn provided, 
I can see people are worried, too, about the BOM.  The previous text 
made it very clear how to determine whether the text is UTF-16LE or 
UTF-16BE, for example.  That text seems to have disappeared.  Is there a 
reason?  I think the same approach spelled out in Section 3 of RFC 4627 
would still work, no?

Paul

------ Original Message ------
From: "Tim Bray" <[email-masked]>
To: "Paul E. Jones" <[email-masked]>
Cc: "Bjoern Hoehrmann" <[email-masked]>; "RFC Errata System" 
<[email-masked]>; "Pete Resnick" <[email-masked]>; 
"Paul Hoffman" <[email-masked]>; "[email-masked]" <[email-masked]>; 
"Matthew Miller" <[email-masked]>; "Barry Leiba" 
<[email-masked]>; [email-masked]
Sent: 3/4/2014 12:27:51 PM
Subject: Re: [Json] [Technical Errata Reported] RFC7158 (3907)

>On this issue, also please read the interoperability note in the 
>paragraph before the new production.
>
>
>>I've not been following the list closely, as I thought this was more 
>>of an editorial exercise than anything else.  This is definitely more 
>>than editorial.  This could definitely break things and the commenter 
>>is right that there are instances where there could be 
>>misinterpretation.
>>

---

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Fri, 7 Mar 2014 13:09:30 +0000

Paul: ECMA's version of JSON has allowed anything at the top level for
quite a while.  The WG explicitly decided that having the two specs in
sync was more important to us than the backward-compatility issue, which
we documented.  This isn't a "mistake" in the sense that we didn't know it
was happening - it was the consensus of the people who were participating.

On 3/4/14 8:12 PM, "Paul E. Jones" <[email-masked]> wrote:

>Marc Lehmann rightfully pointed out (and, BTW, his Errata text appears to
>have disappeared), a serializer that
> is requested to serialize a series of integers like 1 and 999 might
>produce 1999.  Reading that back in, the single value 1999 would be read.
> Likewise, a serializer might produce truefalse as output, but then fail
>when one tries to read that back.  One might
> argue to start all serialization with whitespace to avoid having values
>serialized adjacent to each other.  That would work, but unfortunate.
> 
>Looking at some of the messages hanging off of the link Björn provided, I
>can see people are worried, too, about the BOM.  The previous text made
>it very clear how to determine whether the text is UTF-16LE or UTF-16BE,
>for example.  That text seems to
> have disappeared.  Is there a reason?  I think the same approach spelled
>out in Section 3 of RFC 4627 would still work, no?
> 
>Paul
> 
>Cc: [email-masked];
>[email-masked] <mailto:[email-masked]>; [email-masked];
>[email-masked] <mailto:[email-masked]>; [email-masked];
>[email-masked];
>[email-masked] <mailto:[email-masked]>; [email-masked]
>Björn Höhrmann · mailto:[email-masked] ·
>http://bjoern.hoehrmann.de <http://bjoern.hoehrmann.de/>
>Am Badedeich 7 · Telefon:
>+49(0)160/4415681 <tel:%2B49%280%29160%2F4415681> ·
>http://www.bjoernsworld.de <http://www.bjoernsworld.de/>
>25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 ·
>http://www.websitedev.de/ <http://www.websitedev.de/>
>
>
>

-- 
Joe Hildebrand

---

From: "Paul E. Jones" <[email-masked]>
Date: Fri, 07 Mar 2014 21:09:39 +0000

Joe,

But if my understanding is correct, it's not just a matter of 
backward-compatibility.  There is the question of "how does a serializer 
write out data now"?

As Marc pointed out, if one has two integers and requests a stateless 
serializing function to write those out to a file or on the wire, it 
will write out "1" and "999", using his examples.  Reading it back in, 
it would look like a single integer ("1999").  This means the serializer 
would have to be explicitly told whether to insert space before or after 
writing out data.  Alternatively, the application would have to ask the 
serializer to serialize it to a string and the application deal with 
whitespace itself.

Paul

------ Original Message ------
From: "Joe Hildebrand (jhildebr)" <[email-masked]>
To: "Paul E. Jones" <[email-masked]>; "Tim Bray" 
<[email-masked]>
Cc: "Pete Resnick" <[email-masked]>; "Bjoern Hoehrmann" 
<[email-masked]>; "Paul Hoffman" <[email-masked]>; 
"[email-masked]" <[email-masked]>; "Matt Miller (mamille2)" 
<[email-masked]>; "Barry Leiba" <[email-masked]>; 
"[email-masked]" <[email-masked]>; "RFC Errata System" 
<[email-masked]>
Sent: 3/7/2014 8:09:30 AM
Subject: Re: [Json] [Technical Errata Reported] RFC7158 (3907)

>>of
>>right

---

From: R S <[email-masked]>
Date: Sun, 9 Mar 2014 20:37:59 -0700

The new RFC is backward compatible:

"In telecommunications and computing, a product or technology is backward
or downward compatible if it can work with input generated by an older
product or technology"
<http://en.wikipedia.org/wiki/Backward_compatibility>

> this only opens the door for
> future implementations to produce things that do not work reliably.

Nope. This door has been wide open for quite some time, as both the new and
old RFC state:

"A JSON parser MAY accept non-JSON forms or extensions."

- Rob