Subject: [Json] Counterproposal #2 on work items

From: Tim Bray <[email-masked]>
Date: Wed, 20 Feb 2013 10:01:41 -0800

JSON is finished and well-defined, and we have an immense amount of running
code in place as testimony.  Perhaps, though it would be worthwhile to do
an analogue of RFC3470 aka BCP70, “Guidelines for the Use of Extensible
Markup Language (XML) within IETF Protocols”, for JSON.

Rather than niggle about things that aren’t really actually problems, since
JSON is observed to pretty well Just Work, actually add some value by
giving good advice on how to get good mileage in the IETF-protocol context.
For example:

- always use UTF-8
- always include a charset parameter on the Media type anyhow
- always make the top-level construct an object
- never have duplicate keys in an object
- bear in mind that if you include U+0000 in a string value, you are making
life difficult for implementors in certain popular languages

... I’m sure the fertile minds here could add to the list.

 -T

---

From: Francis Galiegue <[email-masked]>
Date: Wed, 20 Feb 2013 20:41:38 +0100

On Wed, Feb 20, 2013 at 7:01 PM, Tim Bray <[email-masked]> wrote:
[...]
> - always make the top-level construct an object

Why?

I have the totally opposite view -- right now only arrays and objects
are dubbed as legal JSON texts. And I have never seen a parser raise
an error if it were fed with anything else -- nor fail to produce
anything else, for that matter.

Restricting the top level construct to objects and arrays is "bad"
enough, but restricting it even more? I think the _opposite_ should be
done. JSON is flexible, restricting its flexibility would be a mistake
imho.

-- 
Francis Galiegue, [email-masked]
Try out your JSON Schemas: http://json-schema-validator.herokuapp.com

---

From: Tim Bray <[email-masked]>
Date: Wed, 20 Feb 2013 12:14:37 -0800

I’m actually pretty sure about that one, for use in protocols.  If in some
protocol I'm going to send you 3 data items called “size”, “date”, and
“uid”, if I put them in a hash, then when for a later version we need to
add “ttl”, assuming a MUST-IGNORE policy, you can just do it.  If you’ve
put them in a tuple or some such, you have much less flexibility.  -T

---

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Wed, 20 Feb 2013 20:17:33 +0000

+1.  Keep in mind this BCP wouldn't be defining what the legal syntax is,
but would be helping protocol designers who use JSON make sane choices
based on what we have seen work in practice.

On 2/20/13 1:14 PM, "Tim Bray" <[email-masked]> wrote:

>I¹m actually pretty sure about that one, for use in protocols.  If in
>some protocol I'm going to send you 3 data items called ³size², ³date²,
>and ³uid², if I put them in a hash, then when for a later version we need
>to add ³ttl², assuming a MUST-IGNORE
> policy, you can just do it.  If you¹ve put them in a tuple or some such,
>you have much less flexibility.  -T
>
>
>
>On Wed, Feb 20, 2013 at 11:41 AM, Francis Galiegue
><[email-masked]> wrote:
>
>Try out your JSON Schemas:
>http://json-schema-validator.herokuapp.com
><http://json-schema-validator.herokuapp.com>
>
>
>
>
>
>

-- 
Joe Hildebrand

---

From: Francis Galiegue <[email-masked]>
Date: Wed, 20 Feb 2013 21:19:02 +0100

On Wed, Feb 20, 2013 at 9:14 PM, Tim Bray <[email-masked]> wrote:
> “uid”, if I put them in a hash, then when for a later version we need to add
> “ttl”, assuming a MUST-IGNORE policy, you can just do it.  If you’ve put
> them in a tuple or some such, you have much less flexibility.  -T
>

And what about JSON-RPC which uses an _array_ of requests/responses
for batch requests?

While an object may be suitable for 95+% of protocols, this is not a
reason for outlawing protocols which _do not_ use objects. And it
costs nothing to enlarge what is possible.

---

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Wed, 20 Feb 2013 20:28:39 +0000

>costs nothing to enlarge what is possible.

I would urge not focusing on specific issues for a potential BCP just yet.
 Let's focus first on the question: "Does it seem useful for us to work on
this sort of problem?"

Traditionally, we would want the charter to be pretty well-understood at
the time of the BoF, and this sort of scope definition is our most
important task this week.

-- 
Joe Hildebrand

---

From: Francis Galiegue <[email-masked]>
Date: Wed, 20 Feb 2013 21:39:36 +0100

On Wed, Feb 20, 2013 at 9:28 PM, Joe Hildebrand (jhildebr)
<[email-masked]> wrote:
[...]
> important task this week.

That makes sense and it does not seem that important of a question
indeed. It is just that, at least to my eyes, allowing any JSON value
over the wire looks like a no-brainer, but it seems that not everyone
agrees ;)

On the whole, I don't believe it is that important of an issue either.
Duplicate keys in an object, on the other hand, are indeed an
important subject -- again, at least to my eyes.

---

From: Philipp Kewisch <[email-masked]>
Date: Wed, 20 Feb 2013 22:20:24 +0100

I second Francis' opinion. While it might be more common to have an 
object as a root, this is by no means a best common practice without 
explaination. I'd be ok with hilighting that there is greater 
flexibility in adding new key-value type data when using an object as 
the root, but there is more than one protocol type where an array as 
root makes more sense. Specifically, any protocol that uses sequential 
data or where potential keys can have duplicate names should probably 
have an array as a root. We went through just this question when 
designing draft-kewisch-et-al-icalendar-in-json. While iCalendar may 
seem like key-value type data, these keys can show up more than once so 
using an object as the root is the wrong thing to do.

I know the question is first if its useful to work on a such problem, 
but since I might not be around to mention this when a BCP is being 
created I still wanted to mention it.

Philipp

---

From: Vinny A <[email-masked]>
Date: Wed, 20 Feb 2013 19:12:01 -0600

On Feb 20, 2013, at 2:39 PM, Francis Galiegue <[email-masked]> wrote:
> important subject

There's a lot of interesting discussion in this thread, but I'd like to see more on duplicate keys. Various parsers either throw errors or record the last key:value pair when encountering duplicate keys, and I'd like to see a recommendation consensus.

One parser I've seen took a fascinating approach: it copied all values of a shared key and added them to an array, keyed to the duplicate key.

---

From: Francis Galiegue <[email-masked]>
Date: Thu, 21 Feb 2013 02:20:34 +0100

> One parser I've seen took a fascinating approach: it copied all values of a shared key and added them to an array, keyed to the duplicate key.

Isn't that org.json? I recall having seen that as well... I know for
sure that Jackson picks the last key/value pair. But the problem is
indeed that the "append to an array" parser does nothing illegal per
se -- basically, as RFC 4627 stands today, the behaviour of parsers
with regards to duplicate object member names is undefined.

And some existing I-Ds, such as JSON Patch, as a result, have felt
compelled to say that a JSON Patch operation, for instance, must have
one and only one member named "op". Which imposes constraints on
_parsing_ that RFC 4627 does not...

---

From: Robert Sayre <[email-masked]>
Date: Wed, 20 Feb 2013 17:29:51 -0800

> _parsing_ that RFC 4627 does not...

I can't think of a reason not to specify what ES5 does here (always
use the last one). This is a perfectly reliable way to write into a
hash table during decoding. And while you can't depend on a single
hash table's iteration order during encoding, you can count on unique
keys, so the appearance of duplicate keys in a JSON dictionary implies
a sequence of sets.

- Rob

---

From: Vinny A <[email-masked]>
Date: Wed, 20 Feb 2013 20:02:13 -0800 (PST)

>Isn't that org.json? I recall having seen that as well... I know for sure that Jackson picks the last key/value pair. But the problem is indeed that the "append to an array" parser does nothing illegal per se -- basically, as RFC 4627 stands today, the behaviour of parsers with regards to duplicate object member names is undefined.
 
I believe org.json throws a exception if it hits a duplicated key; at least the documentation [ http://json.org/javadoc/org/json/JSONObject.html#JSONObject(java.lang.String) ] suggests that to be the case. I don't remember where I saw the append to array parser - but it's such a good idea it's worth bringing up in this conversation.
 
>And some existing I-Ds, such as JSON Patch, as a result, have felt compelled to say that a JSON Patch operation, for instance, must have one and only one member named "op". Which imposes constraints on _parsing_ that RFC 4627 does not...
 
Which is precisely why I'd like to see a recommendation regarding duplicated keys. To be honest, I like the idea of array conversion, but I can understand if others dislike it. I work with JSON quite a bit, and it would be nice to be able to point to a documented, recommended spec.
 
-----------
-Vinny A

---

From: Cyrus Daboo <[email-masked]>
Date: Wed, 20 Feb 2013 23:14:44 -0500

Hi Vinny,

--On February 20, 2013 8:02:13 PM -0800 Vinny A <[email-masked]> wrote:

> I believe org.json throws a exception if it hits a duplicated key; at
> least the documentation [
> http://json.org/javadoc/org/json/JSONObject.html#JSONObject(java.lang.Str
> ing) ] suggests that to be the case. I don't remember where I saw the
> append to array parser - but it's such a good idea it's worth bringing up
> in this conversation.

Another data point: Python 2.7.2 -

>>> import json
>>> print json.loads('{"a":"a", "a":"b"}')
{u'a': u'b'}

So Python will accept duplicate keys, but one value is lost. This makes 
sense since keys in Python dicts are unique. At this point I favor changing 
the SHOULD to a MUST in RFC 4627bis to make it clear that there is lack of 
interoperability here.

-- 
Cyrus Daboo

---

From: Robert Sayre <[email-masked]>
Date: Wed, 20 Feb 2013 21:15:37 -0800

On Wednesday, February 20, 2013, Cyrus Daboo wrote:

>> http://json.org/javadoc/org/**json/JSONObject.html#**
>> JSONObject(java.lang.Str<http://json.org/javadoc/org/json/JSONObject.html#JSONObject(java.lang.Str>

There is a lack of interoperability, and this issue is one of the smaller
points. The larger point is that the JS implementations (and many smaller
ones) have no incentive to change their behavior, and what they do is
reasonable. Let's just do that.

- Rob

---

From: Tim Bray <[email-masked]>
Date: Wed, 20 Feb 2013 22:33:45 -0800

Which was why I was interested in a best-practices spec.  Because it's
basically crazy, in a network protocol, to issue duplicate keys, whether or
not it's legal JSON.  Somewhere this should be stated.

---

From: Robert Sayre <[email-masked]>
Date: Wed, 20 Feb 2013 22:46:17 -0800

On Wed, Feb 20, 2013 at 10:33 PM, Tim Bray <[email-masked]> wrote:
> Which was why I was interested in a best-practices spec.

I agree that such a document would be the right place to address the
duplicate-key issue.

- Rob

---

From: Paul Hoffman <[email-masked]>
Date: Thu, 21 Feb 2013 06:48:06 -0800

> I agree that such a document would be the right place to address the

If we felt that best-practices RFCs were ever read, I would agree. They're not, and even when they are, people see "SHOULD" and say "great, I don't *have* to do that".

The standard should be clear and not allow bendy-ness.

--Paul Hoffman

---

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Thu, 21 Feb 2013 19:05:04 +0000

>ones) have no incentive to change their behavior, and what they do is

How about text like this for section 2.2:

"The names within an object SHOULD be unique.  Applications and protocols
built on top of JSON SHOULD require names within an object to be unique,
specifying an error condition when this constraint is broken.  However, if
multiple instances of a name are allowed by a particular usage of JSON,
the value for the last member in the input serialization with the same
name MUST be used as the value for that name."

-- 
Joe Hildebrand

---

From: Paul Hoffman <[email-masked]>
Date: Thu, 21 Feb 2013 11:14:49 -0800

> How about text like this for section 2.2:

Wrong thread. This thread is for "only product a best practices guide".

I bring this up because there is a huge difference between:
- Changing the requirements in the format specification
- Adding new processing rules, but allowing bad-idea data, in the format document
- Not touching the format specification and creating a best practices guide

Further, as we have discovered in many other areas of the IETF, there is a huge difference between:
- A format specification that is unclear on how to process illegal input
- A format specification that says processors must reject illegal input
- A format specification that says how processors handle legal-but-bad-idea input

--Paul Hoffman

---

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Thu, 21 Feb 2013 20:40:24 +0000

Fine, but I'm more interested in an existence proof at this point than the
actual language.

On 2/21/13 12:14 PM, "Paul Hoffman" <[email-masked]> wrote:

>_______________________________________________
>json mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/json
>

-- 
Joe Hildebrand