Subject: [Json] Proposal: the minimal edit

From: R S <[email-masked]>
Date: Sat, 22 Jun 2013 09:57:58 -0700

>From charter:

"The work is essentially a reclassification in place, with minimal changes.
The working group will review errata and update the document as needed
to incorporate
those, and will correct significant errors and inconsistencies, but will
keep changes to a minimum."

I propose a new section, to appear after "Generators" and before "IANA
Considerations"

+ N. Differences from ECMAScript
+
+ ECMAScript implementations produce and consume primitive JSON values at
the
+ root level of JSON documents.

It's my opinion that none of the errors or inconsistencies present in the
document are significant, since JSON works pretty well. I'll probably
oppose any edits that aren't to the introduction or the security
considerations, including updates to references such as ABNF and Unicode.
Note that the IETF does not update all standards track documents with
obsolete references, which demonstrates that leaving the current references
in place is tractable.

- Rob

---

From: Tim Bray <[email-masked]>
Date: Sat, 22 Jun 2013 10:31:12 -0700

If we’re doing an ECMAdiff section... Since 4627 says “A string is a
sequence of zero or more Unicode characters [UNICODE]” by definition this
cannot include unpaired surrogates, since they’re not .

So in Rob’s proposed section:

ECMAScript implementations generate and consume unpaired Unicode surrogate
code points in JSON documents.

On Sat, Jun 22, 2013 at 9:57 AM, R S <[email-masked]> wrote:

> From charter:
>
> "The work is essentially a reclassification in place, with minimal
> changes. The working group will review errata and update the document as
> needed to incorporate those, and will correct significant errors and inconsistencies,
> but will keep changes to a minimum."
>
> _______________________________________________
> json mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/json
>
>

---

From: Tim Bray <[email-masked]>
Date: Sat, 22 Jun 2013 10:48:12 -0700

BTW I should say that I have some sympathy with Rob’s viewpoint here.  It
looks like it might be hard to get consensus about the nits we want to pick
here on 4627, and JSON in the wild works just fine, and the real de facto
irritant is that ECMAScript does some non-4627-blessed things.  I’m willing
to put some more work into the struggle for consensus, but if we just
decided to give up, just add a “Stupid ECMAScript Tricks” section then
declare victory, that wouldn’t be a terrible outcome. -T

---

From: Paul Hoffman <[email-masked]>
Date: Sat, 22 Jun 2013 10:51:27 -0700

<no hat>

On Jun 22, 2013, at 10:48 AM, Tim Bray <[email-masked]> wrote:

> BTW I should say that I have some sympathy with Rob’s viewpoint here.  It looks like it might be hard to get consensus about the nits we want to pick here on 4627, and JSON in the wild works just fine, and the real de facto irritant is that ECMAScript does some non-4627-blessed things.  I’m willing to put some more work into the struggle for consensus, but if we just decided to give up, just add a “Stupid ECMAScript Tricks” section then declare victory, that wouldn’t be a terrible outcome. -T

Certainly +1 to being willing to do the struggle for consensus, but also +1 to a reconsideration of paying more attention to the part of the charter that says "essentially a reclassification in place, with minimal changes".

--Paul Hoffman

---

From: R S <[email-masked]>
Date: Sat, 22 Jun 2013 11:17:23 -0700

> code points in JSON documents.

Yeah, that's fair. I'm not opposed to any text in that section that
reflects reality and refrains from editorializing.

It might be good to focus the WG on producing a document that can be said
to meet the requirements in the charter, and postpone changes that aren't
necessary for that. I think documenting these ECMAScript irritants is the
only thing we're absolutely required to do.

- Rob

---

From: Paul Hoffman <[email-masked]>
Date: Sat, 22 Jun 2013 11:30:05 -0700

<no hat>

On Jun 22, 2013, at 11:17 AM, R S <[email-masked]> wrote:

> ECMAScript implementations generate and consume unpaired Unicode surrogate code points in JSON documents.
> 
> Yeah, that's fair. I'm not opposed to any text in that section that reflects reality and refrains from editorializing.

Well, Tim wanted to call the section “Stupid ECMAScript Tricks”, which seems kinda editorial. :-)

The second thing such a section would contain would be a sentence about how parsers react when they encounter an object with duplicate names. But, beside that, I'm not sure there is anything else that this section would need to cover.

If the WG went down this path, the document would need to deal with the errata for RFC 4627, but those are pretty easy.

--Paul Hoffman

---

From: Jorge <[email-masked]>
Date: Sat, 22 Jun 2013 20:42:34 +0200

On 22/06/2013, at 19:31, Tim Bray wrote:

> If we’re doing an ECMAdiff section... Since 4627 says “A string is a sequence of zero or more Unicode characters [UNICODE]” by definition this cannot include unpaired surrogates, since they’re not .  

Yes they are if they are escaped.
-- 
( Jorge )();

---

From: John Cowan <[email-masked]>
Date: Sat, 22 Jun 2013 14:52:01 -0400

R S scripsit:

> It's my opinion that none of the errors or inconsistencies present in
> the document are significant, since JSON works pretty well.

It does at the level of an informational protocol, but not at the level of
standards track.  Would you be willing to sign a contract, arm's-length,
with someone whose only constraint was that their code passed *some*
interpretation of the RFC?  I certainly would not.  As I pointed out
before, the parser conformance clause (first paragraph of Section 4)
is satisfied by any program that accepts arbitrary input and produces
arbitrary output distinct from the input (or indeed no output at all,
given the over-free wording in the second paragraph).

> obsolete references, which demonstrates that leaving the current
> references in place is tractable.

I don't think that applies to documents entering the standards track in
the first place.

-- 
You are a child of the universe no less         John Cowan
than the trees and all other acyclic            http://www.ccil.org/~cowan
graphs; you have a right to be here.            [email-masked]
  --DeXiderata by Sean McGrath

---

From: R S <[email-masked]>
Date: Sat, 22 Jun 2013 13:42:33 -0700

Incorporating updates from Paul and Tim:

+ N. Differences from ECMAScript
+
+ ECMAScript implementations produce and consume primitive JSON values at
the
+ root level of JSON documents.
+
+ ECMAScript implementations generate and consume unpaired Unicode
surrogate code points in JSON documents.
+
+ When there are duplicate names within an object, ECMAScript parsers
overwrite the value corresponding to
+ such names with the value that appears last in the serialization.

Maybe this proposal should also deal with the errata for RFC 4627, since
they are said to be easy. That proposal would produce a document for -03 or
-04 that meets the requirements of the charter, and further proposals could
be evaluated with cost/benefit analysis in mind.

- Rob

---

From: John Cowan <[email-masked]>
Date: Sat, 22 Jun 2013 16:59:36 -0400

R S scripsit:

> + the root level of JSON documents.
> +
> + surrogate code points in JSON documents.
> +
> + overwrite the value corresponding to such names with the value that
> + appears last in the serialization.

To which I add:

+ ECMAScript parsers only accept, and ECMAScript generators only generate,
+ numbers that correspond to the finite values of binary 64-bit IEEE
+ 754:2008 floating point numbers.

We'll need a non-normative reference, too:

    IEEE Computer Society (August 29, 2008). IEEE Standard for
    Floating-Point Arithmetic. IEEE.  doi:10.1109/IEEESTD.2008.4610935.
    ISBN 978-0-7381-5753-5. IEEE Std 754-2008.

-- 
Time alone is real                      John Cowan <[email-masked]>
  the rest imaginary
like a quaternion       --phma          http://www.ccil.org/~cowan

---

From: R S <[email-masked]>
Date: Sat, 22 Jun 2013 14:13:23 -0700

On Sat, Jun 22, 2013 at 1:59 PM, John Cowan <[email-masked]> wrote:

> + ECMAScript parsers only accept,

This bit is not quite right. I think they'll accept anything that conforms
to the JSON grammar, and then produce a number that is not what the sender
intended if it's out of range.

> and ECMAScript generators only generate,

As I said in the other thread, I wouldn't oppose including text along these
lines. It is a real problem and people do get bitten by it. For example:

<https://dev.twitter.com/docs/twitter-ids-json-and-snowflake>

But I don't think it's required for the minimal edit I'm proposing here.
It's one of the things we should look at as a follow on.

- Rob

---

From: John Cowan <[email-masked]>
Date: Sat, 22 Jun 2013 17:22:20 -0400

> intended if it's out of range.

Fair enough.  "Only interpret correctly", perhaps, then.

> <https://dev.twitter.com/docs/twitter-ids-json-and-snowflake>

This says that some parsers may throw exceptions if the number is too
big, though it doesn't mention any such.

-- 
Clear?  Huh!  Why a four-year-old child         John Cowan
could understand this report.  Run out          [email-masked]
and find me a four-year-old child.  I           http://www.ccil.org/~cowan
can't make head or tail out of it.
        --Rufus T. Firefly on government reports

---

From: R S <[email-masked]>
Date: Sat, 22 Jun 2013 14:38:03 -0700

> big, though it doesn't mention any such.

That document is not exclusively targeted at ECMAScript programmers, and it
is targeted at programmers with widely varying levels of skill. It's very
easy to encounter such an exception in languages where integers are assumed
to be 32 bits, even though that exception is rarely produced by a JSON
library itself.

- Rob

---

From: Tim Bray <[email-masked]>
Date: Sat, 22 Jun 2013 15:02:30 -0700

Since there are those who (gasp!) are not native speakers of IEEE Floats,
it’s probably helpful to say something like “which have 53 bits of
precision (2**53 is roughly 9.007 * 10**15)”

---

From: Carsten Bormann <[email-masked]>
Date: Sun, 23 Jun 2013 00:03:46 +0200

> Fair enough.  "Only interpret correctly", perhaps, then.

Trying to understand this discussion:

Is the IEEE 754 number a JavaScript parser generates from [1.1] "interpreted correctly"?
There is no way to "correctly" represent 1.1 in IEEE 754 double precision binary.

So maybe it is more useful to point out that parsers (and that observation is not limited to JavaScript) might convert JSON's decimal numbers to a (binary or other) form, losing precision and range in the process.

Section 4 now says:

   An implementation may set limits on the size of texts that it
   accepts.  An implementation may set limits on the maximum depth of
   nesting.  --> An implementation may set limits on the range of numbers. <--
   An implementation may set limits on the length and character contents
   of strings.

So maybe it is useful to point out that parsers (again, not just the ones that try to conform to a version of ECMAscript) are also likely to set limits on the precision of numbers (maybe that even was meant to be included by the term "range" here, although I'd prefer the stricter interpretation of that term).

Grüße, Carsten

---

From: John Cowan <[email-masked]>
Date: Sat, 22 Jun 2013 18:39:06 -0400

Carsten Bormann scripsit:

> So maybe it is more useful to point out that parsers (and that
> observation is not limited to JavaScript) might convert JSON's decimal
> numbers to a (binary or other) form, losing precision and range in
> the process.

I had range rather than precision in mind: if you parse "[1, 10, 1E400]",
you are likely to get either an exception or an array with 1.0, 10.0,
and Inf in it.  But my point is that ECMAScript implementations MUST
NOT represent 1E400 other than as Inf, nor 1.9999999999999999999999999
as other than 2.0.  The same is not true in other languages, notably
various Lisps.

> So maybe it is useful to point out that parsers (again, not just the
> ones that try to conform to a version of ECMAscript) are also likely
> to set limits on the precision of numbers

+1

-- 
John Cowan          http://www.ccil.org/~cowan        [email-masked]
To say that Bilbo's breath was taken away is no description at all.  There are
no words left to express his staggerment, since Men changed the language that
they learned of elves in the days when all the world was wonderful. --The Hobbit

---

From: Paul Hoffman <[email-masked]>
Date: Sat, 22 Jun 2013 16:54:43 -0700

> as other than 2.0.  The same is not true in other languages, notably

I would propose that this proposal is in the wrong thread. Robert started this thread as "really minimal changes", and this is clearly an addition, or possibly a note for the likely implementation guidance document.

--Paul Hoffman

---

From: Eliot Lear <[email-masked]>
Date: Mon, 24 Jun 2013 07:25:39 +0200

Robert,

I like the idea for a minimal edit.  I would be interested in what ends
up in such a section.  When it is thought to be complete I will want
want to know what implementations should do to avoid interoperability
problems.  Once you've written that out in normative language, if we can
get consensus on THAT I wonder if there will be ANY differences with
ECMAscript.

Interoperability is the issue.

Eliot

On 6/22/13 6:57 PM, R S wrote:
> changes. The working group will review errata and update the document
> as needed to incorporate those, and will correct significant errors
> and inconsistencies, but will keep changes to a minimum."
>
> + ECMAScript implementations produce and consume primitive JSON values
> at the 
> the document are significant, since JSON works pretty well. I'll
> probably oppose any edits that aren't to the introduction or the
> security considerations, including updates to references such as ABNF
> and Unicode. Note that the IETF does not update all standards track
> documents with obsolete references, which demonstrates that leaving
> the current references in place is tractable.
>
> - Rob
>
>

---

From: Norbert Lindenberg <[email-masked]>
Date: Mon, 24 Jun 2013 11:07:08 -0700

> ECMAScript implementations generate and consume unpaired Unicode surrogate code points in JSON documents.

What are "unpaired Unicode surrogate code points"? I'm familiar with surrogate code points, and with unpaired surrogate code units in UTF-16.

If you mean one of those, then this behavior is conformant with the JSON grammar, specifically the "unescaped" production. The behavior likely also exists in several other JSON implementations - for example, the source code of the string parsing functions in the Java org.json library and the C# fastJSON library doesn't seem to do anything to prevent unpaired surrogate code units from passing through and escape sequences from being unescaped to unpaired surrogate code units.

Norbert

---

From: Norbert Lindenberg <[email-masked]>
Date: Mon, 24 Jun 2013 11:28:21 -0700

> given the over-free wording in the second paragraph).

I agree with John - for a standard, some of the errors, undefined terminology, and inconsistencies seem unacceptable. The ones that worry me in particular (I tend to worry about internationalization):

- The RFC uses the term "Unicode character", which isn't defined in the Unicode standard as far as I can tell. If it means "Unicode assigned character", the most likely candidate, then it's inconsistent with the "unescaped" production in the grammar, which allows all Unicode code points.

- The RFC says "JSON text SHALL be encoded in Unicode", which is obviously meant as a normative statement, but, as John pointed out, is meaningless because Unicode is not an encoding. Other text in the RFC seems to assume that "encoded in Unicode" means "encoded in UTF-8, UTF-16BE, UTF-16LE, UTF-32BE, or UTF-32-LE".

Norbert

---

From: Norbert Lindenberg <[email-masked]>
Date: Mon, 24 Jun 2013 11:41:26 -0700

> BTW I should say that I have some sympathy with Rob’s viewpoint here.  It looks like it might be hard to get consensus about the nits we want to pick here on 4627, and JSON in the wild works just f…

Which of these "non-4627-blessed things" is done only by ECMAScript? The one Rob mentioned, producing and consuming primitive JSON values at the root level, can also be found, e.g., in the JSON implementation in Python:
http://docs.python.org/3/library/json.html#top-level-non-object-non-array-values

A section about ECMAScript may be useful because of the central position of JavaScript in browsers in the web eco system. But the assumption that all other JSON implementations are in agreement and only ECMAScript does "stupid tricks" doesn't match reality.

Norbert

---

From: John Cowan <[email-masked]>
Date: Mon, 24 Jun 2013 14:53:14 -0400

Norbert Lindenberg scripsit:

> Which of these "non-4627-blessed things" is done only by ECMAScript? The
> one Rob mentioned, producing and consuming primitive JSON values at
> the root level, can also be found, e.g., in the JSON implementation
> in Python:

Yes, but that's just the *implementation* in Python.  The Python *standard*
(insofar as there is such a thing) has nothing to say about JSON, whereas
the ECMAScript standard does.  It's standard vs. standard that needs to be
called out, not standard vs. implementation.

-- 
Knowledge studies others / Wisdom is self-known;      John Cowan
Muscle masters brothers / Self-mastery is bone;       [email-masked]
Content need never borrow / Ambition wanders blind;   http://ccil.org/~cowan
Vitality cleaves to the marrow / Leaving death behind.    --Tao 33 (Bynner)

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 24 Jun 2013 22:04:26 +0200

On Jun 24, 2013, at 20:53, John Cowan <[email-masked]> wrote:

> It's standard vs. standard that needs to be

Actually, I'm more interested in interoperability standards reflecting interoperability reality than reflecting other standards.

There is a fine line to walk here, of course.

In particular, "allowing" more and more deviations from a normative reference encourages implementors to turn that normative reference into soup *).  So it is quite understandable that the community behind that normative reference will be outraged each time that happens.

Documenting reality is not enough.  Behavior that creates interoperability problems needs to be discouraged, even if it is widespread.  

Conformity is not enough.  Behavior that creates interoperability problems needs to be discouraged, even if it is condoned by a normative reference.

Grüße, Carsten

*) I'm using that word as a technical term here.
Quick usage example:
HTML was based on SGML, but turned it into soup.

---

From: Norbert Lindenberg <[email-masked]>
Date: Mon, 24 Jun 2013 13:05:46 -0700

> called out, not standard vs. implementation.

If that's the intent of the proposed new section, it should start with "The ECMAScript Language Specification [ECMA262] requires implementations to ..." rather than "ECMAScript implementations ...".

[ECMA262] Ecma International, "ECMAScript Language Specification", Standard ECMA-262, 5.1 Edition, June 2011,
<http://www.ecma-international.org/ecma-262/5.1/Ecma-262.pdf>.

Norbert

---

From: John Cowan <[email-masked]>
Date: Mon, 24 Jun 2013 16:40:16 -0400

Norbert Lindenberg scripsit:

> If that's the intent of the proposed new section, it should start
> with "The ECMAScript Language Specification [ECMA262] requires
> implementations to ..." rather than "ECMAScript implementations ...".

+1

-- 
John Cowan <[email-masked]>             http://www.ccil.org/~cowan
The internet is a web of tiny tyrannies giving an illusion of anarchy.
                --David Rush

---

From: Paul Hoffman <[email-masked]>
Date: Mon, 24 Jun 2013 14:28:21 -0700

<no hat>

A few people on this thread have said they might support Rob's idea if they knew all the changes, and a few changes were thrown back and forth. I propose the following is a full set of what is needed. Please note the wording at the beginning of the proposed Section 1.3: this is about what ECMAScript actually says about JSON, not what that means to ECMAScript implementations.

Thoughts?

--Paul Hoffman

Add Section 1.2, "Changes from RFC 4627"

   This section lists all changes between this document and the text in
   RFC 4627.

   - Applied errata #607 from RFC 4627 to correctly align the artwork
     for the definition of "object".

   - Applied errata #3607 from RFC 4627 by removing the security
     consideration that begins "A JSON text can be safely passed"
     and the JavaScript code that went with that consideration.

   - Added Section 1.3, "Differences from the JSON Definition in ECMAScript". 

   - Changed the [ECMA] and [UNICODE] references to be non-version-specific.

Add Section 1.3, "Differences from the JSON Definition in ECMAScript"

   The following lists the known major differences between this document and the definition of JSON
   in Section 15.12 of [ECMA].

   - ECMAScript implementations produce and consume primitive JSON values at the root level of JSON
     documents.

   - ECMAScript implementations can generate and consume code points in JSON strings that are not
     Unicode characters.

   - When there are duplicate names within an object, ECMAScript JSON parsers overwrite the value
     corresponding to such names with the value that appears last in the serialization.

In Section 6, remove "A JSON text can be safely passed" and the JavaScript code in the following
paragraph.

In Section 9, change the title in the reference to [ECMA] to be be non-version-specific:

   [ECMA]    European Computer Manufacturers Association, "ECMAScript
             Language Specification",
             <http://www.ecma-international.org/publications/files/
             ecma-st/ECMA-262.pdf>.

In Section 9, change the reference to [UNICODE] to be be non-version-specific:

   [UNICODE]  The Unicode Consortium, "The Unicode Standard",
              <http://www.unicode.org/versions/latest/>.

---

From: Tim Bray <[email-masked]>
Date: Mon, 24 Jun 2013 14:42:55 -0700

In general I’d support this, with one nit: s/that are not Unicode
characters/that can never be Unicode characters/ (just to make it clear
we’re not talking about unassigned code points, which don't seem to
distress anyone).

Strongest argument in favor of this approach: JSON works. -T

On Mon, Jun 24, 2013 at 2:28 PM, Paul Hoffman <[email-masked]> wrote:

> <no hat>
>
> A few people on this thread have said they might support Rob's idea if
> they knew all the changes, and a few changes were thrown back and forth. I
> propose the following is a full set of what is needed. Please note the
> wording at the beginning of the proposed Section 1.3: this is about what
> ECMAScript actually says about JSON, not what that means to ECMAScript
> implementations.
>
> Thoughts?
>
> --Paul Hoffman
>

---

From: John Cowan <[email-masked]>
Date: Mon, 24 Jun 2013 19:35:21 -0400

> we’re not talking about unassigned code points, which don't seem to

Well, that excludes not only surrogate code points, which are the really
dangerous ones (because they have no standard representation) but non-character
code points as well, which shouldn't be interchanged but aren't actually
dangerous.  I don't know if you mean to do that or not.

-- 
The experiences of the past show                John Cowan
that there has always been a discrepancy        [email-masked]
between plans and performance.                  http://www.ccil.org/~cowan
        --Emperor Hirohito, August 1945

---

From: Paul Hoffman <[email-masked]>
Date: Mon, 24 Jun 2013 16:42:15 -0700

> dangerous.  I don't know if you mean to do that or not.

I'm not sure what you mean by "exclude". The sentence Tim was referring to is about inclusion:

--Paul Hoffman

---

From: Norbert Lindenberg <[email-masked]>
Date: Mon, 24 Jun 2013 16:58:14 -0700

>  - ECMAScript implementations can generate and consume code points in JSON strings that are not

Which definition of "Unicode character" are the three of you referring to?

Thanks,
Norbert

---

From: Tim Bray <[email-masked]>
Date: Mon, 24 Jun 2013 17:05:20 -0700

I meant Surrogates or Noncharacters (per Unicode section 2.4). -T

---

From: Paul Hoffman <[email-masked]>
Date: Mon, 24 Jun 2013 17:28:02 -0700

<no hat>

> Which definition of "Unicode character" are the three of you referring to?

Does it actually matter? This thread is about making minimal changes to the current spec. The current spec says "Unicode characters". The proposed addition to the spec indicates that, for some definition of "Unicode character" that is not in RFC 4627, ECMAScript can have strings with more than just Unicode characters.

Do you believe that proposed sentence is incorrect?

--Paul Hoffman

---

From: John Cowan <[email-masked]>
Date: Mon, 24 Jun 2013 21:28:46 -0400

> Do you believe that proposed sentence is incorrect?

No, nor is "Cats are members of the species _Felis catus_."  But
what business either one has in the proposed spec is unclear.

-- 
Real FORTRAN programmers can program FORTRAN    John Cowan
in any language.  --Ed Post                     [email-masked]

---

From: John Cowan <[email-masked]>
Date: Mon, 24 Jun 2013 21:19:05 -0400

Paul Hoffman scripsit:

>   - ECMAScript implementations can generate and consume code points
>     in JSON strings that are not Unicode characters.

There is no point in putting this item into a list of differences
unless it is not true of RFC-JSON.  If this language is to be added, it
follows that RFC-JSON excludes certain code points that are not Unicode
characters.  But exactly which code points?  As worded, presumably
all of them: the surrogate code points, the non-character code points
(a term of art in Unicode; it does not mean "code points that are not
characters", but refers to a specific list of 66 code points), and the
unassigned (as of some particular Unicode version) code points.

Tim's proposed revision replaced the last five words with "that can never
be Unicode characters", thus excluding from RFC-JSON only the surrogate
code points and the non-character code points, but not the unassigned
code points.  As it turns out, this is indeed what he meant to say.

My preference would be to exclude only the surrogates, not the
non-characters, which is yet a third position.  My suggested wording
would be simply "that are surrogates".

(Strictly speaking, code points are not characters; code points
*represent* characters.  This editorial point is orthogonal to the current
discussion.)

-- 
He made the Legislature meet at one-horse       John Cowan
tank-towns out in the alfalfa belt, so that     [email-masked]
hardly nobody could get there and most of       http://www.ccil.org/~cowan
the leaders would stay home and let him go      --H.L. Mencken's
to work and do things as he pleased.              Declaration of Independence

---

From: Norbert Lindenberg <[email-masked]>
Date: Mon, 24 Jun 2013 18:35:24 -0700

> Do you believe that proposed sentence is incorrect?

Without a definition of "Unicode character" we can't determine whether the sentence is correct or not.

On the other hand, all code points that an ECMAScript implementation can generate and consume are allowed by the JSON grammar, so by that measure there's no difference between the two specifications.

I think what Tim and others really want to say is "JSON over the wire must not use surrogate code points", with Tim adding "... or non-character code points". From that would follow that "ECMAScript implementations can generate and consume code points in JSON strings that must not be used in JSON over the wire". And I think that's the kind of information that actually matters from an interoperability point of view. It's not a minimal edit though.

Norbert

---

From: Norbert Lindenberg <[email-masked]>
Date: Mon, 24 Jun 2013 18:38:29 -0700

>   in Section 15.12 of [ECMA].

The JSON section didn't exist before the 5th edition of ECMA-262, and its number might change in the 6th or a later edition, so it would be better to refer to "section 15.12 of edition 5.1, or successor".

>             <http://www.ecma-international.org/publications/files/

The former ECMA changed its name to "Ecma International" in 1994.

Norbert

---

From: Paul Hoffman <[email-masked]>
Date: Mon, 24 Jun 2013 19:11:52 -0700

<no hat>

On Jun 24, 2013, at 6:19 PM, John Cowan <[email-masked]> wrote:

> unless it is not true of RFC-JSON.

Agree.

>  If this language is to be added, it
> characters.  

There are many people in the WG who believe that is the case.

> But exactly which code points?  

Why is that question important? As long as there is a difference, why does there have to be a specific list for this purpose?

> As worded, presumably

Errr, why make that presumption? The sentence stands on its own, without presumptions. Rob's suggestion to make this all simple is the basis for this thread.

> code points.  As it turns out, this is indeed what he meant to say.

The sentence above is more general. If it is still true, I thought it would probably be supported by more people who wanted simplicity in the -bis document.

> would be simply "that are surrogates".

Why are you wanting to exclude anything? This section is a list of differences. Do you believe you know exactly which non-characters are allowed by ECMAScript? I ask because there were differing opinions earlier.

--Paul Hoffman

---

From: R S <[email-masked]>
Date: Mon, 24 Jun 2013 19:13:03 -0700

This looks good to me. It can serve as a baseline for further proposals at
the very least.

- Rob

---

From: Paul Hoffman <[email-masked]>
Date: Mon, 24 Jun 2013 19:14:55 -0700

<no hat>

> Without a definition of "Unicode character" we can't determine whether the sentence is correct or not.

I believe you can. If you look at the mailing list, you will see that some people interpret ECMAScript as allowing every code point in strings. There are some code points that do not represent Unicode characters. Isn't that sufficient?

> On the other hand, all code points that an ECMAScript implementation can generate and consume are allowed by the JSON grammar, so by that measure there's no difference between the two specification…

There is wide disagreement on that statement in the WG so far.

> I think what Tim and others really want to say is "JSON over the wire must not use surrogate code points", with Tim adding "... or non-character code points". From that would follow that "ECMAScrip…

Exactly right. The attempt in this thread is *not* to list interoperability points, just cleanup.

--Paul Hoffman

---

From: Paul Hoffman <[email-masked]>
Date: Mon, 24 Jun 2013 19:19:20 -0700

Asked a different way:

Given a section titled "Differences from the JSON Definition in ECMAScript" that begins "The following lists the known major differences between this document and the definition of JSON in Section 15.12 of [ECMA]", what would you say about the perceived difference in the code points that ECMAScript implementations can generate and consume relative to RFC 4627?

--Paul Hoffman

---

From: John Cowan <[email-masked]>
Date: Mon, 24 Jun 2013 22:33:14 -0400

Paul Hoffman scripsit:

> Given a section titled "Differences from the JSON Definition in
> ECMAScript" that begins "The following lists the known major differences
> between this document and the definition of JSON in Section 15.12 of
> [ECMA]", what would you say about the perceived difference in the
> code points that ECMAScript implementations can generate and consume
> relative to RFC 4627?

Actually, I wouldn't say anything at this stage, now that I think
about it.  The time to write a list giving the differences between
ECMA-JSON and RFC-JSON is after we know what it is that RFC-JSON says.
Right now we don't.

-- 
John Cowan  [email-masked]  http://www.ccil.org/~cowan
Thor Heyerdahl recounts his attempt to prove Rudyard Kipling's theory
that the mongoose first came to India on a raft from Polynesia.
        --blurb for Rikki-Kon-Tiki-Tavi

---

From: Paul Hoffman <[email-masked]>
Date: Mon, 24 Jun 2013 19:43:29 -0700

> ECMA-JSON and RFC-JSON is after we know what it is that RFC-JSON says.

I apologize if I didn't make it clear, but the proposal that I made earlier says *exactly* what is in what you call RFC-JSON. It is the current draft, with only the changes that I listed. This is what Robert proposed at the beginning of this thread.

--Paul Hoffman

---

From: John Cowan <[email-masked]>
Date: Mon, 24 Jun 2013 22:43:24 -0400

Paul Hoffman scripsit:

> > On the other hand, all code points that an ECMAScript implementation
> > can generate and consume are allowed by the JSON grammar, so by that
> > measure there's no difference between the two specifications.
> 

I have seen no disagreement about the truth of it, but much about the
desirability of it.

-- 
John Cowan    [email-masked]    http://ccil.org/~cowan
The present impossibility of giving a scientific explanation is no proof
that there is no scientific explanation. The unexplained is not to be
identified with the unexplainable, and the strange and extraordinary
nature of a fact is not a justification for attributing it to powers
above nature.  --The Catholic Encyclopedia, s.v. "telepathy" (1913)

---

From: Tim Bray <[email-masked]>
Date: Mon, 24 Jun 2013 19:46:09 -0700

No, John, Paul provided a completely worked through proposal encompassing a
small but nonzero set of changes, the largest being the vs-ECMA section. -T

---

From: Norbert Lindenberg <[email-masked]>
Date: Mon, 24 Jun 2013 20:12:03 -0700

On Jun 24, 2013, at 19:14 , Paul Hoffman wrote:

> I believe you can. If you look at the mailing list, you will see that some people interpret ECMAScript as allowing every code point in strings.

I'm one of these people.

> There are some code points that do not represent Unicode characters.

Absent an agreed-upon definition of "Unicode character", I can simply define the term to be a synonym of "code point", which renders the sentence incorrect.

Code point, assigned code point, reserved code point, surrogate code point, and noncharacter code point are terms that the Unicode standard defines and that should be used in the RFC.

Norbert

---

From: R S <[email-masked]>
Date: Mon, 24 Jun 2013 20:15:22 -0700

On Mon, Jun 24, 2013 at 8:12 PM, Norbert Lindenberg <
[email-masked]> wrote:

>
> Code point, assigned code point, reserved code point, surrogate code
> point, and noncharacter code point are terms that the Unicode standard
> defines and that should be used in the RFC.
>

That sounds like a possible follow-on proposal. The group is chartered to
reclassify RFC 4627 in place, and the existing Unicode terminology just
isn't a deal breaker. If it were, JSON interoperability would be much worse.

- Rob

---

From: Norbert Lindenberg <[email-masked]>
Date: Mon, 24 Jun 2013 20:15:44 -0700

> I have seen no disagreement about the truth of it, but much about the

Thank you, John.

Norbert

---

From: "Martin J. Dürst" <[email-masked]>
Date: Tue, 25 Jun 2013 17:55:10 +0900

I generally agree with Tim Bray on this.

<NIT>
However, please change 'errata' to 'erratum' (two times)
</NIT>

Regards,    Martin.

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Tue, 25 Jun 2013 11:44:48 +0200

* Paul Hoffman wrote:
>I apologize if I didn't make it clear, but the proposal that I made 
>earlier says *exactly* what is in what you call RFC-JSON. It is the 
>current draft, with only the changes that I listed. This is what Robert 
>proposed at the beginning of this thread.

The Working Group has yet to decide whether it wants to, say, allow
strings at the top level of application/json resources, so proposals
for text detailing differences between "ECMA-JSON" and what this WG
is going to define are premature. People who think strings should be
allowed at the top level would not speak up in this thread, so the
thread is not useful for finding consensus around substantive issues.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/