Subject: [Json] Parser and encoder options (Re: The names within an object SHOULD be unique.)

From: Nico Williams <[email-masked]>
Date: Thu, 6 Jun 2013 17:54:01 -0500

On Thu, Jun 6, 2013 at 5:21 PM, Nico Williams <[email-masked]> wrote:
> Regarding comments, I thought we'd have to accept that JSON does not
> have comments, but since there's talk of extensions, and since we
> could (and many, many parsers do) have options that can default to off
> (unavailable), we might indeed want to consider a comments option.

I forgot to finish this thought, which is just as well as this one
should have a different Subject line, so now I've an excuse to change
it.

I could certainly imagine a parser that a) accepts parse options from
the application, b) has some of the following options regarding
duplicate names:

 - take first value
 - take last value
 - aggregate all the values into an array preserving the order in
which they were received
 - return an error on duplicate names
 - warn of duplicate names
 - detect duplicate names (for streaming parsers)
 - use probabilistic duplicate name detection (also for streaming parsers)

I think we should formalize the concept of options for parsers and encoders.

For example, it's common to have options for pretty printing JSON.
And there's been talk of extensions, so why not have some useful
extensions (like comments, or string join, like in C, for
pretty-printing purposes)?

We should also have a taxonomy for describing an implementation.
We've seen the need to distinguish between streaming and non-streaming
parsers.  (For now that's my only contribution to such a taxonomy.)

Nico
--

---

From: John Cowan <[email-masked]>
Date: Thu, 6 Jun 2013 18:59:06 -0400

> parsers.  (For now that's my only contribution to such a taxonomy.)

While all these things are nice-to-haves, they are why specs balloon up
to dozens or even hundreds of pages, and become impossible for anyone
but large, well-funded organizations to implement.  I do most strongly
urge this group NOT to go there.

-- 
We call nothing profound                        [email-masked]
that is not wittily expressed.                  John Cowan
        --Northrop Frye (improved)

---

From: Nico Williams <[email-masked]>
Date: Thu, 6 Jun 2013 18:06:37 -0500

> urge this group NOT to go there.

I agree with the sentiment.  Many parsers I've seen (hmm, *every*
parser I've seen) has encoding/parsing options.

One of the things we should want to do here is describe reality as it
relates to interoperability.  If that means adding text, so be it.
Not every option will be relevant to interoperability, and if we're
careful maybe none are (I've not made an inventory, so I don't yet
know myself).

---

From: "Markus Lanthaler" <[email-masked]>
Date: Fri, 7 Jun 2013 01:18:47 +0200

> I think we should formalize the concept of options for parsers and

I disagree. We are primarily defining a data format. Specific
implementations can support whatever options they want. It just doesn't
matter for the data format. What does matter is that different
implementations are able to talk to each other *without* knowing about the
other ends settings. Specifying options as the ones you list above would
thus be counterproductive.

> pretty-printing purposes)?

Anyone is free to do that in his implementation but the spec doesn't need to
talk about this.

--
Markus Lanthaler
@markuslanthaler

---

From: "Pete Cordell" <[email-masked]>
Date: Fri, 7 Jun 2013 09:41:25 +0100

Original Message From: "John Cowan"

> urge this group NOT to go there.

+1

Surely we don't want to go down the path of protocol specs say "This 
protocol requires a JSON streaming parser" or "This protocol requires a JSON 
non-streaming parser" etc.  We just want to use JSON.

Douglas said the text "The names within an object SHOULD be unique" should 
have been "The names within an object MUST be unique".  IMO by defining a 
fall-back mechanism we're justifying having non-unique names and we're 
actually weakening the requirement to something like "The names within an 
object MAY be unique".  Is that the direction we want to go in?

Pete Cordell
Codalogic Ltd
C++ tools for C++ programmers, http://codalogic.com
Easily read & write XML in C++, http://www.xml2cpp.com

---

From: Vinny A <[email-masked]>
Date: Fri, 7 Jun 2013 07:32:52 -0500

On Jun 7, 2013, at 3:41 AM, "Pete Cordell" <[email-masked]> wrote:
> Douglas said the text "The names within an object SHOULD be unique" should have been "The names within an object MUST be unique".  IMO by defining a fall-back mechanism we're justifying having non-unique names and we're actually weakening the requirement to something like "The names within an object MAY be unique".  Is that the direction we want to go in?

Unfortunately enough, yes. I agree with you that the best course of action would be to switch to a MUST; however Douglas has also indicated a need to preserve backward-compatibility as well. See this previous message:

> On Jun 5, 2013, at 4:56 PM, Douglas Crockford <[email-masked]<mailto:[email-masked]>> wrote:
> > There are people who interpreted the SHOULD as DON'T HAVE TO and intentionally duplicated the keys. My personal feeling is that we should change the SHOULD to MUST and let those applications break. But the consensus of ECMA TC39 is that the SHOULD must stand.

So since the SHOULD is standing, we absolutely must have a plan for those parsers and emitters that are