Subject: Re: [Json] Proposed document set from this WG

From: R S <[email-masked]>
Date: Tue, 19 Feb 2013 22:47:13 -0800

Paul Hoffman wrote:
> 1) JSON base, 4627bis
>   Target: current JSON users
>   Current: grammar, encoding, parser rules, generator rules, MIME type registration)
>   New: Internet transfer rules, list of changes from 4627

My suggestion is for the WG to target this use case only, and to treat
the JSON processing rules in ECMAScript 5, Section 15.12 (The JSON
Object) as the baseline for rules regarding encoding and decoding,
rather than RFC4627.

The motivation for this recommendation is that it's impractical to
make breaking change to this algorithm on a reasonable time scale.
But JSON RFC readers will expect to interoperate with web browsers
and other JavaScript embeddings.

The most significant delta concerns this line from RFC4627:
> A JSON parser MAY accept non-JSON forms or extensions.

ES5 parsers must raise an error when they encounter deviations from
the grammar defined in ES5:
"It is not permitted for a conforming implementation of JSON.parse to
extend the JSON grammars."

Tim Bray wrote:
> \uxxxx is only allowed for chars that must be escaped per the JSON spec
> \uxxxx is freely allowed, but the \uxxxx is considered to represent a single codepoint, and all comparison/hashing
> operations have to be conducted codepoint-by-codepoint

ES5 mandates \uxxxx for certain control characters and allows it for
all (so does RFC4627 "Any character may be escaped.").  JavaScript
strings are composed of code units. Consider JSONKit's behavior when
serializing escaped Unicode:

"When JKSerializeOptionEscapeUnicode is enabled, JSONKit will encode
Unicode code points that can be encoded as a single UTF16 code unit as
\uXXXX, and will encode Unicode code points that require UTF16
surrogate pairs as \uhigh\ulow."
<https://github.com/johnezang/JSONKit>

json ⊄ js: <https://medium.com/joys-of-javascript/42a28471221d>
> The problem comes down to two unicode characters that are considered line terminators
> in JavaScript: the line separator \u2028 and the paragraph separator \u2029.

Recommend that encoders escape these characters.

Paul Hoffman wrote:
> If we are supposed to be keeping backwards compatibility, then yes, it's too late.
> The same is true for changing SHOULD not have duplicate names in objects.

ES5 mandates behavior here:
"In the case where there are duplicate name Strings within an object,
lexically preceding values for the same key shall be overwritten."

- Rob

---

From: Paul Hoffman <[email-masked]>
Date: Wed, 20 Feb 2013 08:20:08 -0800

> Object) as the baseline for rules regarding encoding and decoding,

It feels weird to be agreeing with Rob so early in the discusson, but I agree on both parts.

- Charter 4627bis *and nothing else*, so that the 4627bis work is done without distraction. It is really clear that canonicalization and schema/description have the *high* potential for distraction.

- Say that the base for that one charter item is RFC 4627 *and* ECMAScript 5, which seems to be widely-deployed.

- Say that the WG is likely to recharter while 4627bis is under review by the IESG to discuss schema/description.

- Given the recent consensus, don't mention canonicalization in the charter at all unless another WG or SDO has specifically asked us to work on it.

--Paul Hoffman

---

From: Mike Jones <[email-masked]>
Date: Wed, 20 Feb 2013 16:29:29 +0000

+1

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Paul Hoffman
Sent: Wednesday, February 20, 2013 8:20 AM
To: R S
Cc: [email-masked]
Subject: Re: [Json] Proposed document set from this WG

--Paul Hoffman
_______________________________________________
json mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/json

---

From: Tony Hansen <[email-masked]>
Date: Wed, 20 Feb 2013 11:36:04 -0500

+1

---

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Wed, 20 Feb 2013 16:51:05 +0000

+1, with one caveat.  What if we find ECMAScript 5 and 4627 to be in
conflict in some way?

In particular, I find:

15.12: "Conforming implementations of JSON.parse and JSON.stringify must
support the exact interchange format described in this specification
without any deletions or extensions to the format. This differs from RFC
4627 which permits a JSON parser to accept non-JSON forms and extensions."

15.12.1.1: The ABNF is constructed differently.  Hopefully in a completely
compatible way (looks that way with a couple of minutes analysis), but we
all know how finicky even small changes to ABNF can be.

15.12.1.2: JSONText can be any value, not just object or array.  I think
this would be a positive change for 4627bis, perhaps with a note that
older processors are likely to reject any top level that is not an object
or array.

On 2/20/13 9:20 AM, "Paul Hoffman" <[email-masked]> wrote:

>- Charter 4627bis *and nothing else*, so that the 4627bis work is done
>without distraction. It is really clear that canonicalization and
>schema/description have the *high* potential for distraction.
>
>- Say that the base for that one charter item is RFC 4627 *and*
>ECMAScript 5, which seems to be widely-deployed.
>
>- Given the recent consensus, don't mention canonicalization in the
>charter at all unless another WG or SDO has specifically asked us to work
>on it.
>
>--Paul Hoffman

-- 
Joe Hildebrand

---

From: Tim Bray <[email-masked]>
Date: Wed, 20 Feb 2013 08:57:46 -0800

+1 to Rob/Paul’s proposal, and Joe’s right to be worried, but this is a
problem we have to deal with.

> older processors are likely to reject any top level that is not an object

Eeek, that scares me.  That is a huge liberalization; there’s a lot of code
out there that assumes that when you parse JSON you get a hash.

Thinking off the top of my head: I’ve seen protocols defined where the
payload looks like

{
  "values" : [ ... the array that I actually wanted ...]
}

But you know, I really like the top-level-is-a-hash assumption; it buys a
lot of extensibility.  Write your protocol with a MUST-IGNORE policy and
people can later enrich it by adding new goodies... but only if the top
level is a hash.

 -T

---

From: Paul Hoffman <[email-masked]>
Date: Wed, 20 Feb 2013 08:59:00 -0800

> older processors are likely to reject any top level that is not an object

Proposal:
- The WG picks one of the two ways
- 4627bis says MUST for that way
- In the Introduction (not an appendix), there is a subsection that lists every change from either 4627 or ECMAScript 5

--Paul Hoffman

---

From: Francis Galiegue <[email-masked]>
Date: Wed, 20 Feb 2013 18:00:28 +0100

On Wed, Feb 20, 2013 at 5:57 PM, Tim Bray <[email-masked]> wrote:
[...]
> out there that assumes that when you parse JSON you get a hash.

Well, RFC 4627 also "allows" arrays as JSON texts...

I agree about any JSON value being possible. In practice, there are a
lot of parsers which don't obey RFC 4627 in that they do parse any
JSON value whereas they shouldn't...

--
Francis Galiegue, [email-masked]
Try out your JSON Schemas: http://json-schema-validator.herokuapp.com

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Wed, 20 Feb 2013 18:09:45 +0100

>Object) as the baseline for rules regarding encoding and decoding,

The ecmascript specification defines a grammar for JSON to explain the
behavior of JSON.stringify and JSON.parse and notes that the on-the-wire
JSON format is defined in RFC 4627. The only difference between the two
grammars is that ecmascript allows values like `null` at the top level,
while RFC 4627 is more restrictive.

The Working Group might decide to remove this restriction in RFC4627bis,
but I see no reason to take the ecmascript specification as a baseline.
Also note that since the ecmascript specification does not define an on-
the-wire format, it does not address encoding and decoding issues like
handling character encodings at all, only RFC 4627 addresses those.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Gonzalo Salgueiro <[email-masked]>
Date: Wed, 20 Feb 2013 12:41:06 -0500

+1

I can get behind this.

--G

---

From: Robert Sayre <[email-masked]>
Date: Wed, 20 Feb 2013 10:27:42 -0800

> older processors are likely to reject any top level that is not an object

Ah, yes. I had forgotten about that large change. I think most JSON
implementations support it at this point.

- Rob