Subject: [Json] [Technical Errata Reported] RFC7159 (4220)

From: RFC Errata System <[email-masked]>
Date: Mon, 5 Jan 2015 14:03:44 -0800 (PST)

The following errata report has been submitted for RFC7159,
"The JavaScript Object Notation (JSON) Data Interchange Format".

--------------------------------------
You may review the report below and at:
http://www.rfc-editor.org/errata_search.php?rfc=7159&eid=4220

--------------------------------------
Type: Technical
Reported by: mirabilos <[email-masked]>

Section: 7

Original Text
-------------
   […] All Unicode characters may be placed within the
   quotation marks, except for the characters that must be escaped:
   quotation mark, reverse solidus, and the control characters (U+0000
   through U+001F).

[…]

      unescaped = %x20-21 / %x23-5B / %x5D-10FFFF

Corrected Text
--------------
   […] All Unicode characters may be placed within the
   quotation marks, except for the characters that must be escaped:
   quotation mark, reverse solidus, the control characters (U+0000
   through U+001F), and codepoints beyond the BMP (U-00010000
   through U-0010FFFF).

[…]

      unescaped = %x20-21 / %x23-5B / %x5D-FFFF

Notes
-----
ECMA-262 states that "ECMAScript source text is assumed to be a sequence
of 16-bit code units", and everything must be converted to UTF-16 first.
This stems from the fact that Java™, like Win32, used UCS-2, which only
later got extended to allow UTF-16. This means that SMP codepoints must
always be escaped into their twelve-character form.

This is also an interoperability issue: implementations may wish to parse
(or generate) JSON by using a wchar_t data type (in C) which, depending on
the platform, may be only 16 bits wide. ECMAscript allows for this.

Instructions:
-------------
This erratum is currently posted as "Reported". If necessary, please
use "Reply All" to discuss whether it should be verified or
rejected. When a decision is reached, the verifying party (IESG)
can log in to change the status and edit the report, if necessary. 

--------------------------------------
RFC7159 (draft-ietf-json-rfc4627bis-rfc7159bis)
--------------------------------------
Title               : The JavaScript Object Notation (JSON) Data Interchange Format
Publication Date    : March 2014
Author(s)           : T. Bray, Ed.
Category            : PROPOSED STANDARD
Source              : JavaScript Object Notation
Area                : Applications
Stream              : IETF
Verifying Party     : IESG

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Tue, 06 Jan 2015 03:19:25 +0100

* RFC Errata System wrote:
>Original Text
>-------------
>   [â€¦] All Unicode characters may be placed within the
>[â€¦]

>This stems from the fact that Javaâ„¢, like Win32, used UCS-2, which only

What assumptions ECMA-262 makes when defining the ECMAScript language is
not really relevant to the construction of `application/json` entities.
Likewise, how implementations might internally represent parts of JSON
documents does not affects the bits-on-the-wire format. I also do not
see what difference it would make from either perspective when a UTF-8
encoded `application/json` entity encoded a non-BMP character literally
or as escaped surrogate pair, it's just a lexical difference, you can
use wchar_t the same either way. In any case, I am quite sure allowing
unescaped non-BMP characters is a deliberate decision and not an error.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
D-10243 Berlin · PGP Pub. KeyID: 0xA4357E78 · http://www.bjoernsworld.de
 Available for hire in Berlin (early 2015)  · http://www.websitedev.de/

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 6 Jan 2015 09:46:35 +0100

> of 16-bit code units", and everything must be converted to UTF-16 first.

… which is a source of a common misconception about JavaScript that sometimes carries over into a common misconception about JSON.

While it is true that a number of programming languages that were created at a time when Unicode was thought to be a 16-bit character set are now stuck with the workarounds of UTF-16, there is no problem with non-BMP characters in JavaScript source code or in JavaScript strings.
JavaScript will just process them as two code units (a source of bugs, but something a JavaScript coder has to contend with in any case):

> "🐭".length
< 2

As Björn said, any limitations of ECMA-262 or JavaScript are of no import to JSON anyway, as would be limitations of specific C implementations.

More importantly, this errata is not really reporting an error, but trying to change a conscious decision of the WG (here: not to deviate from RFC 4627 in what is allowed unescaped in strings); it thus needs to be rejected.

Grüße, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 6 Jan 2015 10:54:48 +0100

< 2

Grüße, Carsten

---

From: Paul Hoffman <[email-masked]>
Date: Tue, 6 Jan 2015 07:46:31 -0800

Please reject this errata. It stems from a disagreement with a WG decision, not as a report of a  error.

--Paul Hoffman, JSON WG co-chair

> On Jan 5, 2015, at 2:03 PM, RFC Errata System <[email-masked]> wrote:
> 
> _______________________________________________
> json mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/json