Subject: Re: [Json] Kicking Off JSONbis

From: Martin J. Dürst <[email-masked]>
Date: Thu, 5 Nov 2015 17:39:09 +0900

I like this very much too. The approach is right, and the wording looks 
great.

Regards,   Martin.

On 2015/10/31 11:57, Carsten Bormann wrote:
> I like that approach.
>
> Grüße, Carsten
>
>
> Joe Hildebrand (jhildebr) wrote:
>> "This document intends to document the exact same data model as described in ECMA-404, which is the main document used by ECMAscript implementors.  Implementors that want to interoperate with ECMAscript implementations (such as web browsers) are advised to also read ECMA-404 to understand any potential interoperability challenges. Any differences in the model described in this document with that described in ECMA-404 are not intentional, and will require the IETF and ECMA to work together to resolve the difference."
>>
>>
>> That makes the normative reference understandable, and makes clear that the IETF and ECMA are partners in maintaining JSON going forward.
>>
>
> _______________________________________________
> json mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/json
>

---

From: Anders Rundgren <[email-masked]>
Date: Thu, 5 Nov 2015 09:49:03 +0100

On 2015-11-05 09:39, Martin J. Dürst wrote:
>>> "This document intends to document the exact same data model as described in ECMA-404,
 >>> which is the main document used by ECMAscript implementors.  Implementors that want to
 >>> interoperate with ECMAscript implementations (such as web browsers) are advised to also
 >>> read ECMA-404 to understand any potential interoperability challenges.

Since EMCAScript introduces predictable property order, the reference to EMCA-404 doesn't
really tell the whole interoperability story.

Anders

Any differences in the model described in this document with that described in ECMA-404 are not intentional, and will require the IETF and ECMA to work together to resolve the difference."

---

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Mon, 9 Nov 2015 21:20:04 +0000

>really tell the whole interoperability story.

That predictability has nothing to do with the wire format, though.  It's specific to the ECMAscript serializer and does not affect the semantics of the generated document.  It's not even terribly useful for canonicalization, without going to the trouble of creating a new ES object built in the correct order, which seems error-prone.  On the parse side, I *guess* you might want to know that there is an information signal retained by the order of properties.  That could be used as a steganographic channel, for instance.  It doesn't seem specific to ES, though.

I think there maybe some other reason why ES's property order logic might be interesting to you though - can you say more, please?

-- 
Joe Hildebrand

---

From: Anders Rundgren <[email-masked]>
Date: Tue, 10 Nov 2015 17:19:14 +0100

On 2015-11-09 22:20, Joe Hildebrand (jhildebr) wrote:
> That predictability has nothing to do with the wire format, though.  It's specific to the ECMAscript serializer and does not affect the semantics of the generated document.  It's not even terribly useful for canonicalization, without going to the trouble of creating a new ES object built in the correct order, which seems error-prone.

On the creation side there is nothing special to do in an ES6 environment.  In a non-ES6 JSON toolbox, you only need to use an ordered dictionary like LinkedHashMap in Java to get things right. Well, numeric properties would need something else but that appears to be an edge-case, not worth bothering with.   For those who disagree with that, strict creation order does in no way rule out numeric property order, it only requires a bit more care by the programmer :-)

>   On the parse side, I *guess* you might want to know that there is an information signal retained by the order of properties.  That could be used as a steganographic channel, for instance.  It doesn't seem specific to ES, though.
>

My target application has since mid-2013 been a JSONesque counterpart to XML's enveloped signatures but without getting bogged down by complex, stand-alone canonicalizers.  The latest developments seem to verify that the most difficult part of JSON (Numbers), also are properly canonicalized by the majority (but not all) ES6 implementations.  This enables very simple and efficient "in-object" JavaScript and JSON signatures which only rely on standard JavaScript for creating and manipulating objects and the built-in JSON parse/stringify methods for parsing and serialization respectively:

https://mailarchive.ietf.org/arch/msg/jose/s5TdEtfOdQ0z3Qmtij2yXemZW-A

Anders

---

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Wed, 11 Nov 2015 21:31:09 +0000

On 11/10/15, 9:19 AM, "Anders Rundgren" <[email-masked]> wrote:

>On the creation side there is nothing special to do in an ES6 environment.  In a non-ES6 JSON toolbox, you only need to use an ordered dictionary like LinkedHashMap in Java to get things right. 

I understand that I *could* write code to generate keys in the same order, but I don't understand why I would bother, when the specified order is:

(from: http://www.ecma-international.org/ecma-262/6.0/#sec-ordinary-object-internal-methods-and-internal-slots-ownpropertykeys)

- Integer keys in numeric order (ok, makes sense for array-like things, but seems mostly moot for JSON)
- String keys in *creation* order

I don't yet see how creation order is useful to preserve.

>Well, numeric properties would need something else but that appears to be an edge-case, not worth bothering with.   For those who disagree with that, strict creation order does in no way rule out numeric property order, it only requires a bit more care by the programmer :-)

Please focus on the interoperability benefits.

>https://mailarchive.ietf.org/arch/msg/jose/s5TdEtfOdQ0z3Qmtij2yXemZW-A

Oh!  You're relying on the parsing and serializing code to be parallel, such that the parser maintains order, and the serializer repeats that order on output.

Interesting.  However, that would be a major change to the JSON spec (that would necessitate changes to LOTS of existing code, at a performance or memory penalty). It also took me a good amount of time to track down even in the ECMAscript doc - so I don't think you can expect most folks reading ECMA-404 to figure this out either.  If you really want interoperability, there's nothing stopping you from doing something like I-JSON that says an ordered-json is JSON, except for some other constraints.

-- 
Joe Hildebrand

---

From: "Peter F. Patel-Schneider" <[email-masked]>
Date: Wed, 11 Nov 2015 13:58:09 -0800

Also apologies for the delayed response.

On 10/30/2015 07:42 PM, Joe Hildebrand (jhildebr) wrote:
> (Sorry I didn't get to this until now)
> 
> On 10/18/15, 4:23 PM, "json on behalf of Carsten Bormann" <[email-masked] on behalf of [email-masked]> wrote:
[...]
> "This document intends to document the exact same data model as described in ECMA-404, which is the main document used by ECMAscript implementors.  Implementors that want to interoperate with ECMAs…
[...]

I don't think that ECMA-404 describes a data model at all, unless you mean
unbounded-precision floating point numbers, collections of name/value pairs
with possible repeating names, and strings as sequences of Unicode code
points.  I don't think that many JSON libraries implement this data model.

peter

---

From: Anders Rundgren <[email-masked]>
Date: Thu, 12 Nov 2015 06:22:30 +0100

> On 11/10/15, 9:19 AM, "Anders Rundgren" <[email-masked]> wrote:
<snip>
>>
>> My target application has since mid-2013 been a JSONesque counterpart
 > to XML's enveloped signatures but without getting bogged down by complex,
 > stand-alone canonicalizers.  The latest developments seem to verify that
 > the most difficult part of JSON (Numbers), also are properly canonicalized
 > by the majority (but not all) ES6 implementations.  This enables very simple
 > and efficient "in-object" JavaScript and JSON signatures which only rely on
 > standard JavaScript for creating and manipulating objects and the built-in
 > JSON parse/stringify methods for parsing and serialization respectively:
>>
> Oh!  You're relying on the parsing and serializing code to be parallel,
 > such that the parser maintains order, and the serializer repeats that order on output.
>
> Interesting.  However, that would be a major change to the JSON spec (that would
 > necessitate changes to LOTS of existing code, at a performance or memory penalty).
 > It also took me a good amount of time to track down even in the ECMAscript
 > doc - so I don't think you can expect most folks reading ECMA-404 to figure this
 > out either.  If you really want interoperability, there's nothing stopping you
 > from doing something like I-JSON that says an ordered-json is JSON, except for
 > some other constraints.

The depicted signature runs without hiccups in Chrome, Firefox, Safari, and of course
in the Java reference implementation which generated it. The current alternative
is using JWS which encapsulates (=hides) data in Base64 which is perfectly OK for
some applications but (IMO) slightly less appetizing for things like Web Payments.

The genie is out of the bottle and now it is time to wish something :-)

Anders

---

From: Tim Bray <[email-masked]>
Date: Sat, 14 Nov 2015 10:55:33 -0800

On Fri, Oct 30, 2015 at 7:42 PM, Joe Hildebrand (jhildebr) <
[email-masked]> wrote:
​​

> ​​
> There's an argument to be made that if you want to interoperate with JSON
> implementations in browsers, you should probably read ECMA-404.  Several of
> the places where 7159 says "interoperable in
> ​​
> the sense" have specific choices in ECMA-404.
>

​I just re-read ECMA-404 (I recommend this, it doesn’t take long) and I
couldn't find examples of such specific choices, could you be more specific?
​​

> ​​
> Conversely, serious implementors of ECMA-404 really ought to read 7159bis
> in order to understand some of the edge cases of interoperability a little
> better.
>

​Agreed.​
​

> The argument that we shouldn't bother with 7159 at all was discussed the
> last time through.  ECMA-404 only specifies what to do in an ECMAscript
> environment, and someone needed to specify interoperability for other
> environments, such as those that don't use UTF-16 code units or 64bit
> floats as their implementation substrate.
>

​Actually ECMA-404 says exactly nothing about what any software should do.​
(Maybe I missed it?)

Whether one should bother with ECMA-404 is out of our control, and overcome
> by events.  Both documents exist, and we need to ensure that they stay in
> sync going forward.  We could add a statement to section 1.2 such as:
>

​I'm not sure about the premise.  It's not obvious to me that any changes
are required in either 404 or 7159 for the foreseeable future.  404
explicitly says “Because it is so simple, it is not expected that the JSON
grammar will ever change.”​

"This document intends to document the exact same data model as described
> in ECMA-404, which is the main document used by ECMAscript implementors.
> Implementors that want to interoperate with ECMAscript implementations
> (such as web browsers) are advised to also read ECMA-404 to understand any
> potential interoperability challenges. Any differences in the model
> described in this document with that described in ECMA-404 are not
> intentional, and will require the IETF and ECMA to work together to resolve
> the difference."
>

1. ​First, ECMA-404 absolutely does not describe any data model.
2. I don't like this sort of nudge-nudge, wink-wink language. If we want to
provide this language, don't tell them to go search 404 for
“interoperability challenges”. Rather, a simple statement along these lines
would be appropriate.  “This document contains several recommendations for
best practices to avoid interoperability problems.   ECMA-404 enforces none
of these, thus implementations based on it might potentially emit JON texts
which exhibit these problems.“
3. Having re-read 404 closely, I am pretty sure that there are not in fact
any differences in the set of data objects which qualify as “JSON texts”
aside from 404's greater permissiveness already noted. So I'm dubious of
the need for any future work.
​​

> ​​
> That makes the normative reference understandable, and makes
> ​​
> clear that the IETF and ECMA are partners in maintaining JSON going
> forward.
>

​No, it still falls massively short of the very explicit and clear
definition of the term “normative reference”.

I’m sticking with my suggestion that if it is absolutely necessary to
insert a normative reference, we need language acknowledging that, while
this reference does not meet the usual criteria for that sort of reference,
we are inserting one anyhow because [X].   I think Joe was starting to
suggest a value for [X], but I'm pretty sure we can do better.  I'll write
a note with some suggestions.
​​

​​

-- 
- Tim Bray (If you’d like to send me a private message, see
https://keybase.io/timbray)

---

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Tue, 17 Nov 2015 09:06:32 +0000

On 11/14/15, 7:55 PM, "Tim Bray" <[email-masked]> wrote:

>​I just re-read ECMA-404 (I recommend this, it doesn’t take long) and I couldn't find examples of such specific choices, could you be more specific?

They're not as much in 404 as they are in 262, which is referenced from 404.  For example, the double implementation for all numbers, leading to the 2^53-1 issue.  Also, some of the Unicode issues creep in through the same door.

>​Actually ECMA-404 says exactly nothing about what any software should do.​ (Maybe I missed it?)

I think it's implicit in 404's ties to 262.

>​I'm not sure about the premise.  It's not obvious to me that any changes are required in either 404 or 7159 for the foreseeable future.  404 explicitly says “Because it is so simple, it is not expected that
> the JSON grammar will ever change.”​

Nod.  This is a statement of political détente, that we hope is never actually used in practice.

>1. ​First, ECMA-404 absolutely does not describe any data model.  

Pick any other words than "data model".  Syntax?

>2. I don't like this sort of nudge-nudge, wink-wink language. If we want to provide this language, don't tell them to go search 404 for “interoperability challenges”. Rather, a simple statement along these
> lines would be appropriate.  “This document contains several recommendations for best practices to avoid interoperability problems.   ECMA-404 enforces none of these, thus implementations based on it might potentially emit JON texts which exhibit these problems.“

Perhaps we need to reference 262 explicitly as well, then.

>3. Having re-read 404 closely, I am pretty sure that there are not in fact any differences in the set of data objects which qualify as “JSON texts” aside from 404's greater permissiveness already noted. So
> I'm dubious of the need for any future work.

I hope we never need to do any future work.

>​No, it still falls massively short of the very explicit and clear definition of the term “normative reference”.  
>
>
>I’m sticking with my suggestion that if it is absolutely necessary to insert a normative reference, we need language acknowledging that, while this reference does not meet the usual criteria for that sort of
> reference, we are inserting one anyhow because [X].   I think Joe was starting to suggest a value for [X], but I'm pretty sure we can do better.  I'll write a note with some suggestions.

I'm completely ok with language to this effect.  And yes, the text I provided was meant entirely as a starting point, to be picked apart and replaced with something good.  Sometimes a strawman is nice to have so that *something* can be set on fire.

-- 
Joe Hildebrand

---

From: Julian Reschke <[email-masked]>
Date: Tue, 17 Nov 2015 10:16:34 +0100

On 2015-11-17 10:06, Joe Hildebrand (jhildebr) wrote:
> ...
> ...

So one needs ECMA 262 (Javascript) to implement JSON?

Best regards, Julian

---

From: "Paul Hoffman" <[email-masked]>
Date: Tue, 17 Nov 2015 07:45:22 -0800

On 17 Nov 2015, at 1:16, Julian Reschke wrote:

>>> 2. I don't like this sort of nudge-nudge, wink-wink language. If we 
>>> want to provide this language, don't tell them to go search 404 for 
>>> “interoperability challenges”. Rather, a simple statement along 
>>> these
>>> lines would be appropriate.  “This document contains several 
>>> recommendations for best practices to avoid interoperability 
>>> problems.   ECMA-404 enforces none of these, thus implementations 
>>> based on it might potentially emit JON texts which exhibit these 
>>> problems.“
>>

In the ECMA world, yes. I would hope that we would not require that for 
JSONbis other than by informational reference.

--Paul Hoffman

---

From: "Peter F. Patel-Schneider" <[email-masked]>
Date: Tue, 17 Nov 2015 09:15:22 -0800

On 11/17/2015 01:06 AM, Joe Hildebrand (jhildebr) wrote:
> They're not as much in 404 as they are in 262, which is referenced from
> 404.  For example, the double implementation for all numbers, leading to
> the 2^53-1 issue.  Also, some of the Unicode issues creep in through the
> same door.
> 

The ties between ECMA-404 and ECMA-262 are very weak. There is even explicit
wording in ECMA-404 that JSON "does not attempt to impose ECMAScript’s
internal data representations on other programming languages."

[...]

>> 2. I don't like this sort of nudge-nudge, wink-wink language. If we want
>> to provide this language, don't tell them to go search 404 for
>> “interoperability challenges”. Rather, a simple statement along these 
>> recommendations for best practices to avoid interoperability problems.
>> ECMA-404 enforces none of these, thus implementations based on it might
>> potentially emit JON texts which exhibit these problems.“
> 

Why?  If so, then why not reference other definitions of how JSON is
interpreted, for example how JSON is interpreted in Fortran?  I thought that
the whole idea of JSON was to not impose EcmaScript data structures or data
models or serializations or parsing on producers or consumers of JSON text.

[...]

peter