Subject: [Json] Naked surrogates already banned?

From: Tim Bray <[email-masked]>
Date: Thu, 17 Oct 2013 18:36:55 -0700

I was looking closely at the 4627 text and just about convinced myself that
it already bans

{"a": "\uDEAD"}

It says Any character may be escaped. If the character is in the Basic
Multilingual Plane (U+0000 through U+FFFF), then it may be represented as a
six-character sequence: a reverse solidus, followed by the lowercase letter
u,... etc

But U+DEAD is not a character.  So a narrow reading would make that
illegal.  Not that any implementation I’ve encountered enforces that. -T

---

From: Stephen Dolan <[email-masked]>
Date: Fri, 18 Oct 2013 02:59:54 +0100

On Fri, Oct 18, 2013 at 2:36 AM, Tim Bray <[email-masked]> wrote:
> But U+DEAD is not a character.  So a narrow reading would make that illegal.
> Not that any implementation I’ve encountered enforces that. -T

Several do, including libjansson, Python and Perl. In fact, I suspect
that most implementations on platforms that haven't baked in UTF16 as
the One True Unicode (javascript, JVM) reject your example.

Stephen

---

From: R S <[email-masked]>
Date: Thu, 17 Oct 2013 19:26:17 -0700

$ python
Python 2.7.4 (default, Oct  6 2013, 14:58:34)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import json
>>> json.loads('{"a": "\uDEAD"}')
{u'a': u'\udead'}

On Thu, Oct 17, 2013 at 6:59 PM, Stephen Dolan
<[email-masked]>wrote:

> illegal.
> _______________________________________________
> json mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/json
>

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 18 Oct 2013 08:39:42 +0200

On Oct 18, 2013, at 03:36, Tim Bray <[email-masked]> wrote:

> It says Any character may be escaped. If the character is in the Basic Multilingual Plane (U+0000 through U+FFFF), then it may be represented as a six-character sequence: a reverse solidus, followed by the lowercase letter u,... etc

It already says in section 1:    
A string is a sequence of zero or more Unicode characters [UNICODE].

I think we had that discussion already.
Count me on the side of the people who don't think UTF-16 artifacts are, or have ever been, a part of JSON.
ECMA-404 is on the side of "any Unicode code point", but that is just one of the extensions 404 makes over JSON.

Now there is a problem that the definition in 4627 ties JSON to a specific version of Unicode.
(The reference is nicely confusing in which version is meant, but that is an artifact of the way Unicode versions are documented.)
I think a robust interpretation of the intent here will add all code points that are available to be characters in future versions of Unicode.
That is a change the WG SHOULD make, the predictable noise from the surrogate faction notwithstanding.

Grüße, Carsten