Subject: Re: [Json] JSON: encodings

From: Paul Hoffman <[email-masked]>
Date: Tue, 12 Nov 2013 07:28:29 -0800

[[ Adding the JSON WG to this thread ]]

On Nov 11, 2013, at 10:58 PM, Anne van Kesteren <[email-masked]> wrote:

> Supporting encodings other than UTF-8 in new formats is not good.
> 
> Supporting UTF-32 is actively harmful as support for it has been
> removed or is being removed from clients. You ought to actively
> recommend against it.
> 
> In general ASCII incompatible encodings have very bad security
> characteristics, the IETF would do well to steer away from them, just
> like the W3C has.
> 
> 
> -- 
> http://annevankesteren.nl/
>

---

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Wed, 13 Nov 2013 20:38:08 +0000

>> characteristics, the IETF would do well to steer away from them, just

Although I hate UTF-32 with the heat of a several moderately-sized stars
and completely agree that UTF-8 is the one true path, I don't think we can
completely remove UTF-32 from the bis draft.  There may be existing
conformant JSON documents stored in UTF-32 that would be made unparseable
by this change.

What I think we *could* do is put a stronger recommendation for UTF-8 in
section 8.1, rather than just saying it's the default.

-- 
Joe Hildebrand

---

From: Julian Reschke <[email-masked]>
Date: Wed, 13 Nov 2013 21:42:47 +0100

> conformant JSON documents stored in UTF-32 that would be made unparseable

+0.5

> section 8.1, rather than just saying it's the default.

+10

---

From: John Cowan <[email-masked]>
Date: Wed, 13 Nov 2013 17:48:32 -0500

> conformant JSON documents stored in UTF-32 that would be made unparseable

Well, no, they would just be non-portable but legitimate extensions.

-- 
Using RELAX NG compact syntax to        John Cowan <[email-masked]>
develop schemas is one of the simple    http://www.ccil.org/~cowan
pleasures in life....
        --Jeni Tennison                 <[email-masked]>

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Wed, 13 Nov 2013 20:19:52 -0500

> conformant JSON documents stored in UTF-32 that would be made unparseable

Nothing the IETF does is going to make documents unparseable. All the IETF
can do is to tell people what the standard form of JSON is.

99.99 % of JSON code out there will reject UTF-32. Much of that code will
reject it in horrible, broken ways. So I don't see any problem at all in
telling people that such encoding is not valid JSON.

All that closing off this option does is to limit the number of test cases
that JSON code has to handle.

-- 
Website: http://hallambaker.com/

---

From: "Martin J. Dürst" <[email-masked]>
Date: Thu, 14 Nov 2013 14:19:01 +0900

>> section 8.1, rather than just saying it's the default.

What about something like:

OLD

    JSON text SHALL be encoded in Unicode.  The default encoding is
    UTF-8.

NEW

    JSON text SHALL be encoded in Unicode.  The default encoding is
    UTF-8.  The vast majority of JSON text is encoded in UTF-8, and
    UTF-8 is the preferred encoding when creating JSON text.  UTF-32
    is not widely supported, and JSON texts encoded in UTF-32 are
    very difficult to find if they exist at all.

Regards,   Martin.

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Thu, 14 Nov 2013 13:47:59 -0500

> Well, no, they would just be non-portable but legitimate extensions.

I like that approach a lot.

>From an interop point of view it is FAR more likely that a reader supports
BSON than JSON-UTF32 and that is hardly likely to change.

Standards are read by people who implement encoders and decoders. What I
would like the spec to tell writers of encoders is that they MUST NOT use
UTF-32 encoding and call the result JSON. This is so that the spec can then
tell decoders that they have absolutely no need to support UTF-32 to comply
with the JSON spec.

Standards is about making choices that don't matter to the semantics or
performance of an operation. Taking an unnecessary option out is an
important and useful improvement.

This is incidentally why I was so peeved by the CBOR fiasco. First I am
told that my requirements don't matter, then the authors are throwing in
unnecessary complexity like definite length encodings because they can.

Any time a specification changes it is likely to cause some existing code
to stop being compliant. That is quite OK. The whole point of going through
this process was to reduce the probability that differences in
implementations would break interoperability.

UTF-32 is a difference in interoperability that adds no expressive value,
adds a great deal of complexity and damages performance. It has negligible
deployed base.

So kill it.