Subject: Re: [Json] Differences between RFC 4627 or the current ECMAScript specification

From: Tony Hansen <[email-masked]>
Date: Tue, 01 Oct 2013 09:32:27 -0400

Thank you Rob, I appreciate that additional bit of information. In that
case, you were indeed required to remove your private extension in order
to be compliant with the ECMAScript specification. My point remains
though: this was a case of removing a private extension you had
previously allowed in your parser; it was not something that was done
because the JSON language itself was different.

This may seem like it's splitting hairs to you, but knowing where we're
both coming from is important so that we understand each other better.

    Tony Hansen

On 10/1/2013 12:38 AM, R S wrote:
> my implementation was used for ECMAScript JSON.parse.
>
> - Rob
>
> On Saturday, September 28, 2013, Tony Hansen wrote:
>
>     On 9/27/2013 12:34 PM, R S wrote:
>>     On Fri, Sep 27, 2013 at 8:34 AM, Tony Hansen <[email-masked]
>>     <javascript:_e({}, 'cvml', [email-masked]');>> wrote:
>>
>>         RFC 4627 does not allow trailing commas in objects. Nor does
>>         4627-bis. 
>>
>>
>>     RFC 4627 allows parsers to accept non-JSON if they wish, while
>>     ECMAScript does not allow this.
>
>     Your claim was that you had to change your parser of JSON to no
>     longer allow trailing commas because of converting to parsing
>     ECMAScript. I disagree with that assessment.
>
>     You had added your own non-standard extension to JSON (accepting
>     trailing commas) to your parser. Note that ECMA's restrictions on
>     parsers are only on implementations of ECMAScript. ECMA does NOT
>     say that *your* parser may not continue to accept trailing commas,
>     except in the specific case of your parser being used in an
>     implementation of ECMAScript's JSON.parse() function. That is, if
>     your function is not used in an implementation of JSON.parse(),
>     ECMA places no restrictions on it; it certainly is not restricted
>     in the same ways that ECMAScript's JSON.parse() is restricted. You
>     chose to remove that non-standard extension, but only because you
>     chose to *follow* the same restrictions as ECMAScript's JSON.parse().
>
>     Did the JSON spec say you could add your own extension to your
>     parser? Yes it did. However, if someone wrote JSON using a
>     trailing comma, they're going into non-interoperable territory.
>
>     ECMA also extends 4627-JSON by accepting additional top-level
>     primitives. ECMA's JSON.parse() is still a 4627-JSON-compliant
>     parser because it properly interprets 4627-JSON. However, someone
>     writing JSON using ECMA's extensions will only safely be
>     interoperable with someone using ECMAScript's JSON.parse() or
>     another parser that's been similarly extended. But they aren't
>     interoperable with the many other JSON libraries that haven't been
>     similarly extended.
>
>         Tony Hansen
>
>
>
> _______________________________________________
> json mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/json

---

From: Tim Bray <[email-masked]>
Date: Wed, 2 Oct 2013 09:52:28 -0700

On Mon, Sep 30, 2013 at 6:55 PM, R S <[email-masked]> wrote:

> It is true for RFC 4627. That makes claims that JSON has nothing to do
> with ECMAScript rather baseless, at least if one plays the part of spec
> nerd.
>
> I think we should have a rationale for changing this, since we're not
> supposed be changing things without being very careful.
>

Quoting from earlier in the thread:

On Mon, Sep 30, 2013 at 7:47 AM, Carsten Bormann <[email-masked]> wrote:

> Normative references specify documents that must be read to understand or
implement the technology in the new RFC, or whose technology must be
present for the technology in the new RFC to work.

Sounds like a good rationale to me.

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 2 Oct 2013 18:58:30 +0200

On Oct 2, 2013, at 18:52, Tim Bray <[email-masked]> wrote:

> > Normative references specify documents that must be read to understand or implement the technology in the new RFC, or whose technology must be present for the technology in the new RFC to work.

Just to re-inject a bit more context: I didn't write that.
The IESG did, in their official statement of what references are normative.

Grüße, Carsten

---

From: Tim Bray <[email-masked]>
Date: Wed, 2 Oct 2013 10:20:50 -0700

So, I (blush) confess to never having read ECMA-262, which is what 4627
references.  I thought I’d have a glance; I'm assuming that what I want is:
http://www.ecma-international.org/publications/files/ECMA-ST-ARCH/ECMA-262,%203rd%20edition,%20December%201999.pdf

It turns out to be nontrivial to isolate the diffs between it and 4627; to
start with, 262 does not contain any instances of either "JSON" nor "object
notation”.  I guess if you look at 11.1 and squint your eyes and ignore the
“this”, Identifier, and (Expression) variants, what’s left over might be
JSON more or less. Maybe.

So, since the spec is >150 pages, and I’m a JavaScript journeyman at best,
if there’s going to be a concise list of differences, someone who
understands that spec is going to have to write it, with references into
the spec.  I suggest that if we don’t get such a proposal, this is not a
fatal flaw in RFC4627bis because actually nobody cares.  If someone is
inclined to do the work, it would improve our charter conformance but I’m
not sure it would really add much practical value to the spec.

On Thu, Sep 26, 2013 at 11:36 AM, Paul Hoffman <[email-masked]>wrote:

> On Sep 26, 2013, at 10:31 AM, R S <[email-masked]> wrote:
>
> > Charter:
> > >
> > > All differences between RFC 4627 or the current ECMAScript
> specification > will be documented in the new RFC.
> >
> > The ECMAScript specification allows primitives at the root level,
> specifies exactly how to interpret numbers, and can handle " bit sequences
> which cannot encode Unicode characters" just fine.
>
> <no hat>
>
> Based on what we have learned in the last six months, it might be better
> for this RFC *not* to do what the charter says.
>
> - TC39 is actively revising ECMAScript and it is not clear whether the
> -bis draft of their version will be out first.
>
> - Some of what ECMAscript says about JSON is intertwingled with the
> definition of ECMAscript, such as "exactly how to interpret numbers"
>
> I'm no longer sure that a long-lasting RFC interpreting parts of another
> SDO's spec is a good idea.
>
> --Paul Hoffman

---

From: R S <[email-masked]>
Date: Wed, 2 Oct 2013 10:59:31 -0700

ECMAScript 5.1 contains JSON:

http://www.ecma-international.org/ecma-262/5.1/#sec-15.12

http://www.ecma-international.org/ecma-262/5.1/#sec-A.8'

- Rob

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Wed, 02 Oct 2013 20:06:33 +0200

>http://www.ecma-international.org/publications/files/ECMA-ST-ARCH/ECMA-262,%203rd%20edition,%20December%201999.pdf

The JSON object was introduced in the 5th edition of the specification,
while you have the third edition from 1999. The current edition is 5.1,
http://www.ecma-international.org/ecma-262/5.1/#sec-15.12 with the JSON
object in section 15.12 as linked.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Tim Bray <[email-masked]>
Date: Wed, 2 Oct 2013 11:26:14 -0700

Hah, so at least the reference in the -bis needs to be fixed.

---

From: Tim Bray <[email-masked]>
Date: Wed, 2 Oct 2013 11:31:19 -0700

So, would it be enough to:
- update the informative reference to point to 5.1
- Have a note that there are differences, with a pointer to section 15.12?

I think 15.12 does a good job of describing them.  The other stuff Rob
mentioned gives details of how javascript implementations have to parse
JSON instances, which I don’t think the -bis needs to care about.

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 2 Oct 2013 20:34:28 +0200

On Oct 2, 2013, at 19:20, Tim Bray <[email-masked]> wrote:

> So, I (blush) confess to never having read ECMA-262, which is what 4627 references.  I thought I’d have a glance; I'm assuming that what I want is: http://www.ecma-international.org/publications/files/ECMA-ST-ARCH/ECMA-262,%203rd%20edition,%20December%201999.pdf

That version defines a language that we usually call JavaScript, which includes a literal notation for JavaScript objects and arrays.  The latter notation can be called JavaScript Object Notation.

JSON used that literal notation as an inspiration (and as a way to minimize bikeshedding), but is otherwise not dependent on that standard.

The fact that the acronym JSON was built from the long form "JavaScript Object Notation" causes no end of confusion, including the impression that JSON is still somehow tied to JavaScript.

That's why it is important to add an RFC editor note that explains that we don't want to expand "JSON" in the title of the document, which otherwise inevitably will happen.

Of course, the 1999 edition of ECMAScript is still the edition that should be cited to explain the historical JavaScript roots of JSON.

If a comparison between JSON and the format implemented by the "JSON" object in newer versions of ECMAScript is desired, we first need to choose a stable version of ECMAScript that already includes the JavaScript "JSON" object.  I have no opinion which one that should be.

Grüße, Carsten

---

From: Paul Hoffman <[email-masked]>
Date: Wed, 2 Oct 2013 11:50:25 -0700

On Oct 2, 2013, at 10:59 AM, R S <[email-masked]> wrote:

> ECMAScript 5.1 contains JSON

Are you saying we should change the reference that was in RFC 4627? Or that we should leave the old reference but create a section on differences that points to 5.1?

--Paul Hoffman

---

From: Tony Hansen <[email-masked]>
Date: Wed, 02 Oct 2013 15:27:51 -0400

> Are you saying we should change the reference that was in RFC 4627? Or that we should leave the old reference but create a section on differences that points to 5.1?

It's clear to me that the current ECMAScript reference is a mixed bag
and does not make sense internally. Nor does it match the thrust of the
text around its reference.

I'm wondering if it would be useful to have the first paragraph of
section 1 cut back. Then expand on JSON's relationship with JavaScript
and ECMAScript into another short section that can be a bit more
historical in nature.

******
1. Introduction

JavaScript Object Notation (JSON) is a text format for the serialization
of structured data.  [text cut out here] JSON can represent ....

[the rest of section 1 is unchanged]

1.1 Relationship with ECMAScript

The definition of JSON is based on the object literals of the JavaScript
language, as defined in the ECMAScript Programming Language Standard,
Third Edition [ECMA3]. Note that JSON purposely restricts its definition
to the representation of structured data, that is an object or array.

Subsequent releases of the ECMAScript spec [ECMA5] have introduced
support for an extended version of JSON. In particular, their
JSON.parse() function accepts a superset of JSON as specified in this
document.

******

   [ECMA3]     European Computer Manufacturers Association, "ECMAScript
              Language Specification 3rd Edition ", December 1999,
             
<http://www.ecma-international.org/publications/files/ECMA-ST-ARCH/ECMA-262,%203rd%20edition,%20December%201999.pdf>.

   [ECMA5]     European Computer Manufacturers Association, "ECMAScript
              Language Specification 5th Edition ", June 2011,
              <http://www.ecma-international.org/publications/files/
              ecma-st/ECMA-262.pdf>.

******

I'm going to discuss the second paragraph of section 1 in a separate email.

    Tony

---

From: R S <[email-masked]>
Date: Wed, 2 Oct 2013 12:32:58 -0700

I think we pretty clearly need to refer to 5.1 to do the comparison. We can
leave the old reference for the "derived from" text if we want.

- Rob

On Wed, Oct 2, 2013 at 11:50 AM, Paul Hoffman <[email-masked]> wrote:

> Are you saying we should change the reference that was in RFC 4627? Or
> that we should leave the old reference but create a section on differences
> that points to 5.1?
>
> --Paul Hoffman

---

From: R S <[email-masked]>
Date: Wed, 2 Oct 2013 12:36:23 -0700

> So, would it be enough to:
[snip ...]

> I think 15.12 does a good job of describing them.
>

That works for me.

- Rob

>
> On Wed, Oct 2, 2013 at 11:06 AM, Bjoern Hoehrmann <[email-masked]>wrote:
>
>> * Tim Bray wrote:

---

From: Tony Hansen <[email-masked]>
Date: Wed, 02 Oct 2013 15:40:32 -0400

The text in section 1 paragraph 2 on what JSON can represent is not
really accurate. I know that this text was in the original 4627 and is
unchanged from there. However, this is really one of the areas where
4627-JSON and ECMA-JSON differ, and is incorrect for 4627-JSON. This
text currently reads:

   JSON can represent four primitive types (strings, numbers, booleans,
   and null) and two structured types (objects and arrays).

In 4627-JSON, Section 2 says that JSON-text is an object / array.
In ECMA-JSON, Section 15.2 says that JSONtext is an object / array /
string / number / boolean-literal / null-literal.

4627-JSON cannot truly represent one of the four primitive types, except
when it has been encapsulated within an object or array. However,
ECMA-JSON can represent any of the four primitive types or two
structured types.

Since we've as a WG rejected extending 4627-JSON to be compatible with
ECMA-JSON, a better statement for 4627bis would be something like this:

   JSON can represent two structured types (objects and arrays) that are
created from
   four primitive types (strings, numbers, booleans, and null) and
recursive objects and
   arrays.

- Tony Hansen

---

From: Paul Hoffman <[email-masked]>
Date: Wed, 2 Oct 2013 12:43:51 -0700

<no hat>

> - Have a note that there are differences, with a pointer to section 15.12?  

This works for me. In specific, if we try to list the differences, we are in some sense saying what ECMAScript means, and that feels kind of rude (just as it would be rude for them to tell us what RFC 4627 meant to say).

--Paul Hoffman

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 2 Oct 2013 21:53:54 +0200

On Oct 2, 2013, at 21:40, Tony Hansen <[email-masked]> wrote:

> recursive

Nested?

We can't do recursive, as we only do trees.

(maybe "nested objects and
  arrays of similar construction".)

(Really nitpicky people will note that arrays and objects can also be empty; s/are/can be/?.)

Grüße, Carsten

---

From: Jorge Chamorro <[email-masked]>
Date: Wed, 2 Oct 2013 22:22:25 +0200

On 02/10/2013, at 21:27, Tony Hansen wrote:
> ******
> 1. Introduction
> 
> JavaScript Object Notation (JSON)

No, "JavaScript Object Notation" is *not* "JSON".

JSON is based on, and a subset of, the JavaScript Object Notation.

-- 
( Jorge )();

---

From: Tim Bray <[email-masked]>
Date: Wed, 2 Oct 2013 13:32:27 -0700

The “JSON can represent...” text is perfectly accurate.  It may not be able
to represent them in a standalone top-level way, but the parts of a JSON
text do in fact represent those things.

---

From: Tony Hansen <[email-masked]>
Date: Wed, 02 Oct 2013 16:40:32 -0400

> JSON is based on, and a subset of, the JavaScript Object Notation.

I did not change that wording. It is what is in the current draft, and
in turn was copied directly from RFC 4627.

Feel free to argue, preferably in a different thread, for a change to
that sentence.

    Tony Hansen

---

From: John Cowan <[email-masked]>
Date: Wed, 2 Oct 2013 16:43:41 -0400

> text do in fact represent those things.

+1.  If it read "JSON texts can represent" it would indeed be inaccurate,
but it doesn't.

-- 
John Cowan    <[email-masked]>     http://www.ccil.org/~cowan
But no living man am I!  You look upon a woman.  Eowyn I am, Eomund's daughter.
You stand between me and my lord and kin.  Begone, if you be not deathless.
For living or dark undead, I will smite you if you touch him.

---

From: John Cowan <[email-masked]>
Date: Wed, 2 Oct 2013 16:44:35 -0400

R S scripsit:

> I think we pretty clearly need to refer to 5.1 to do the comparison. We
> can leave the old reference for the "derived from" text if we want.

+1 on both points:  JSON is derived from the 3rd edition, but should be
compared to the 5.1th (5.1st?) edition.

-- 
Principles.  You can't say A is         John Cowan <[email-masked]>
made of B or vice versa.  All mass      http://www.ccil.org/~cowan
is interaction.  --Richard Feynman

---

From: Tim Bray <[email-masked]>
Date: Wed, 2 Oct 2013 13:47:09 -0700

Does it matter in the slightest to anyone on the planet what it was derived
from?

---

From: Tony Hansen <[email-masked]>
Date: Wed, 02 Oct 2013 16:49:20 -0400

> +1.  If it read "JSON texts can represent" it would indeed be inaccurate,

Am I the only one who read that statement to be referring to the
top-level value and *not* specifically to internal values?

At the least, I think the current statement is ambiguous. It *can* be
read either way.

    Tony Hansen

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 2 Oct 2013 22:49:45 +0200

On Oct 2, 2013, at 22:47, Tim Bray <[email-masked]> wrote:

> Does it matter in the slightest to anyone on the planet what it was derived from?

It matters to those who want to understand the confusion between JSON and JavaScript object notation.

Grüße, Carsten

---

From: John Cowan <[email-masked]>
Date: Wed, 2 Oct 2013 16:51:56 -0400

> Does it matter in the slightest to anyone on the planet what it was

Did the etymology of "XML" matter to anyone?  Yet there it is.

-- 
You annoy me, Rattray!  You disgust me!         John Cowan
You irritate me unspeakably!  Thank Heaven,     [email-masked]
I am a man of equable temper, or I should       http://www.ccil.org/~cowan
scarcely be able to contain myself before
your mocking visage.            --Stalky imitating Macrea

---

From: Jorge Chamorro <[email-masked]>
Date: Wed, 2 Oct 2013 23:03:46 +0200

> Feel free to argue, preferably in a different thread, for a change to

Ooops, my apologies. For some reason I was reading object literal instead of object notation, I'm sorry.

JSON is based on, and a subset of, the JavaScript object literal notation.

-- 
( Jorge )();

---

From: Jorge Chamorro <[email-masked]>
Date: Wed, 2 Oct 2013 23:06:25 +0200

> Does it matter in the slightest to anyone on the planet what it was derived from?

Yes, of course.

-- 
( Jorge )();

---

From: Paul Hoffman <[email-masked]>
Date: Wed, 2 Oct 2013 14:34:57 -0700

<no hat>

> top-level value and *not* specifically to internal values?

Maybe. When you hit the definition of JSON-text, it sounds like you said "these disagree" instead of "these are describing different things".

> At the least, I think the current statement is ambiguous. It *can* be

The question of what is a JSON-text seems important, but maybe should not be made to the detriment of the clarity of the first sentence.

--Paul Hoffman

---

From: Tim Bray <[email-masked]>
Date: Wed, 2 Oct 2013 16:05:43 -0700

First para of 1 says “JavaScript Object Notation (JSON) is a text format
for the serialization of structured data. It is derived from the object
literals of JavaScript, as defined in the ECMAScript Programming Language
Standard, Third Edition [ECMA].”

I think that on balance we don’t want to screw with the words, and I’d be
OK with leading the reference to the 1998 version as it is in the original.
We’ll need another reference to 5.1 to put the pointer to their list of
differences, which I’ll find a home for somewhere.

---

From: "Martin J. Dürst" <[email-masked]>
Date: Thu, 03 Oct 2013 10:40:03 +0900

On 2013/10/03 5:49, Carsten Bormann wrote:
> On Oct 2, 2013, at 22:47, Tim Bray<[email-masked]>  wrote:
>

And will help even those who don't necessarily want to understand this 
confusion, but are confused anyhow.

In other words, such explanations help because they tend to spread, via 
books, web pages, and so on. Having the correctly worded explanation in 
the original document improves this spread.

Regards,   Martin.

---

From: "Manger, James H" <[email-masked]>
Date: Thu, 3 Oct 2013 11:55:06 +1000

> ...
> ECMA-JSON,

Did we reject that? Lets un-reject it then.

There was a substantial discussion <http://www.ietf.org/mail-archive/web/json/current/msg00553.html>; half-a-dozen people strongly supported extending 4627-JSON to be compatible with ECMA-JSON; about 3 were strongly against; and a handful seemed to be in the middle.

There is clearly no major problem with allowing any JSON value at the top level as ECMAScript allows this and many implementations support it. It doesn't break backwards-compatibility as all existing texts are still valid. Objects and arrays are still self-delimiting if any systems rely on that implied property; and separating items with a space (or newline) trivially allows any JSON values to be self-delimiting. Auto-detecting charsets can still work. Detecting truncation (but not other varieties of modification) seems like a very marginal benefit, and would still apply to everything except numbers which are unlikely to be truncated as they are invariably short (eg 10s of bytes): noticing truncation of long items (objects, arrays, strings) is unchanged.

A note in 4627bis could still say you get more interoperability when sticking to objects and arrays at the top level.

A note in a subsequent JSON-best-practises doc could recommend using objects for messages in general, as they easily allows ignorable extensions via new elements.

--
James Manger

---

From: R S <[email-masked]>
Date: Wed, 2 Oct 2013 19:41:42 -0700

> Does it matter in the slightest to anyone on the planet what it was

Well, let's see.

The text passes the first test we should use: it is true.

Second, the text uses this section to appropriately credit the work on
which it is based.

Third, the text is a concise explanation of what one might expect from the
JSON format.

Frankly, it's hard to for me to see the desire to remove that reference as
anything more than ideological.

- Rob

---

From: "Martin J. Dürst" <[email-masked]>
Date: Thu, 03 Oct 2013 13:21:48 +0900

> compared to the 5.1th (5.1st?) edition.

Edition 5.1, hopefully.

Regards,   Martin.

---

From: John Cowan <[email-masked]>
Date: Thu, 3 Oct 2013 00:53:09 -0400

Manger, James H scripsit:

> There was a substantial discussion
> <http://www.ietf.org/mail-archive/web/json/current/msg00553.html>;
> half-a-dozen people strongly supported extending 4627-JSON to be
> compatible with ECMA-JSON; about 3 were strongly against; and a handful
> seemed to be in the middle.

It doesn't seem to fit with the current mission, which is not to extend
JSON but to clarify the spec and add warnings about compatibility issues.
There are a whole mess of common extensions I'd rather do before I do
this.  For one thing, JSON numbers aren't self-delimiting, though strings
and true/false/null are.

-- 
It was dreary and wearisome.  Cold clammy winter still held sway in this
forsaken country.  The only green was the scum of livid weed on the dark
greasy surfaces of the sullen waters.  Dead grasses and rotting reeds loomed
up in the mists like ragged shadows of long-forgotten summers.
        --"The Passage of the Marshes"          http://www.ccil.org/~cowan

---

From: "Manger, James H" <[email-masked]>
Date: Thu, 3 Oct 2013 16:32:09 +1000

> JSON but to clarify the spec and add warnings about compatibility

This issue fits the charter perfectly. It is a significant inconsistency in the JSON community.

  It is acknowledged that there are differences between RFC 4627 and the
  ECMAScript specification in the rules for parsing JSON. Any changes that
  break compatibility with existing implementations of either RFC 4627 or
  the ECMAScript specification will need to have very strong justification
  and broad support. All differences between RFC 4627 or the current
  ECMAScript specification will be documented in the new RFC.

The last sentence above says 4627-bis needs to document the text=obj/array vs text=any-value difference. The choice is to document it as a difference to ECMAScript or as a difference to 4627. However it is documented, I hope JSON library developers get the message not to bother enforcing the text=obj/array restriction; and JSON message designers notice that they don’t need to add an extra wrapping layer (a la SOAP) when they want to exchange arbitrary JSON values.

> this.

Well this is the primary extension that is consistently supported by ECMAScript and many implementations so I suggest this should be a priority before a "whole mess of common extensions".

>  For one thing, JSON numbers aren't self-delimiting, though strings
> and true/false/null are.

So what? It is hard to imagine how numbers not being self-delimiting will break anything that wasn’t already dodgy.

--
James Manger