Subject: Re: [Json] Proposal: JSON text sequence MIME type and Proposed Standard

From: "Manger, James" <[email-masked]>
Date: Wed, 12 Mar 2014 10:01:11 +1100

A short spec on JSON sequences is a good idea, Nico.

I agree that a newline is the best separator for creators to include, and always including it is best.

Some (most?) receivers will be more lenient and accept values separated by any whitespace, or with no separator (for objects, arrays, and/or strings). Perhaps it is best for interop to say receivers MUST be this lenient.

--
James Manger

On 12 Mar 2014 07:52, Nico Williams <[email-masked]> wrote:
On Tue, Mar 11, 2014 at 1:20 PM, Nico Williams <[email-masked]> wrote:
> Details:
>
>  - the newline is REQUIRED when following any of "true", "false",
> "null", or any numeric value, otherwise it is RECOMMENDED (but
> optional)

Just to clarify: it should be possible to build a JSON text sequence
parser given the above and any existing JSON parser, even when the
JSON texts do have embedded newlines (e.g., between array elements,
...).

The rationale for using a newline is twofold:

 - it's roughly how text-based log files tend to work;
 - there's always a "readline" primitive about that can be used to
build a JSON text sequence parser (in conjunction with a JSON parser,
of course).

And, of course, these are _texts_, so newlines help readability.

And this:

>  - it will be RECOMMENDED that JSON texts written to JSON text
> sequences be encoded with no newlines in the encoding.

would only help implementors who lack an incremental JSON parser, but
it would only help them if this were a requirement, which it shouldn't
be, thus there's no point to this recommendation, so I'll just remove
it.

Nico
--

_______________________________________________
json mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/json

---

From: Nico Williams <[email-masked]>
Date: Tue, 11 Mar 2014 18:04:54 -0500

> A short spec on JSON sequences is a good idea, Nico.

I'll submit an I-D soon.

> Some (most?) receivers will be more lenient and accept values separated by any whitespace, or with no separator (for objects, arrays, and/or strings). Perhaps it is best for interop to say receiver…

Good point.  Sequence parsers MUST be able to handle newline text
separators and SHOULD be able to handle other whitespace, as well as
no separator in the unambiguous cases.  Sequence encoders SHOULD use a
newline separator and MAY use other whitespace (with some guidance as
to why newline is best); no need to mention when the separator may be
omitted.

Nico
--

---

From: "Manger, James" <[email-masked]>
Date: Wed, 12 Mar 2014 14:12:19 +1100

> > Some (most?) receivers will be more lenient and accept values
> separated by any whitespace, or with no separator (for objects, arrays,
> and/or strings). Perhaps it is best for interop to say receivers MUST
> be this lenient.

> to why newline is best); no need to mention when the separator may be

You don't quite guarantee interop with "encoders MAY use other whitespace" and "parsers SHOULD be able to handle other whitespace".

The spec needs to say: either "encoders MUST use a newline separator"; or "decoders MUST be able to handle any whitespace (including none in the unambiguous cases)".

--
James Manger

---

From: John Cowan <[email-masked]>
Date: Tue, 11 Mar 2014 23:48:47 -0400

Manger, James scripsit:

> The spec needs to say: either "encoders MUST use a newline separator";
> or "decoders MUST be able to handle any whitespace (including none in
> the unambiguous cases)".

Or, as I suggest, "encoders MUST use whitespace".

-- 
They do not preach                              John Cowan
  that their God will rouse them                [email-masked]
    A little before the nuts work loose.        http://www.ccil.org/~cowan
They do not teach
  that His Pity allows them                         --Rudyard Kipling,
    to drop their job when they damn-well choose.   "The Sons of Martha"

---

From: Tim Bray <[email-masked]>
Date: Tue, 11 Mar 2014 20:49:04 -0700

Yeah, the options are:

1. zero or more whitespace characters
2. 1 or more whitespace characters
3. A specific required set of whitespace characters, for example \s*\n\s*
or just \n

On Tue, Mar 11, 2014 at 8:12 PM, Manger, James <
[email-masked]> wrote:

> You don't quite guarantee interop with "encoders MAY use other whitespace"
> and "parsers SHOULD be able to handle other whitespace".
>
> The spec needs to say: either "encoders MUST use a newline separator"; or
> "decoders MUST be able to handle any whitespace (including none in the
> unambiguous cases)".
>
> --
> James Manger

---

From: Nico Williams <[email-masked]>
Date: Tue, 11 Mar 2014 23:02:09 -0500

> 1. zero or more whitespace characters

Not an option for values where that would result in ambiguity.  (E.g.,
two top-level numeric values in a row.)

> 3. A specific required set of whitespace characters, for example \s*\n\s* or
> just \n

Given that there's no such thing as an empty JSON text, clearly any
number of JSON text separators is logically the same as one.  That
makes (2) and (3) OK.

jq allows any whitespace to separate JSON texts in input sequences,
but it always uses \n to separate texts when outputting JSON text
sequences.  So I'm inclined to go for (2).

Nico
--

---

From: Nico Williams <[email-masked]>
Date: Tue, 11 Mar 2014 23:02:42 -0500

> The spec needs to say: either "encoders MUST use a newline separator"; or "decoders MUST be able to handle any whitespace (including none in the unambiguous cases)".

Right you are.  I'll go with the latter.

Nico
--

---

From: Tim Bray <[email-masked]>
Date: Tue, 11 Mar 2014 21:25:46 -0700

Except for, in this kind of application, I think it’s really foolish to
write out anything but objects. In which case you could theoretically have
no separators.

I lean to \s+ because as soon as you see the first char, you’re at the
start of the next legal JSON text.

---

From: Nico Williams <[email-masked]>
Date: Wed, 12 Mar 2014 00:03:10 -0500

> write out anything but objects. In which case you could theoretically have

I don't think that's going to change running code.  Indeed, this is about
writing down what's actually been implemented and deployed that doesn't
suffer from ambiguities.  You also seem to want all JSON texts to be
objects, but wanting ain't getting :)

> start of the next legal JSON text.

The separator can be whitespace.  It can be any whitespace recognized as
such in the JSON ABNF.  It could be a subset of that.  It also could be any
character or string of them that can't start a JSON value's encoding.
 Since in practice existing apps use whitespace, we should use whitespace.
 If other apps can be found using ", " or "," or whatever, we might be able
to require parser support for all of them or else define multiple MIME
types.

Since any contiguous sequence of separators cannot yoeld an empty JSON text
(since we've not defined the meaning of such a thing), And given what apps
like jq apps do, I think any number of contiguous separators must be
equivalent to just one.

Also, it's much better to follow a text with a separator than to only
precede them with a separator.  This is so parsers needn't wait for the
next text or EOF before yielding a true/false/null/numeric top-level value.

Finally, I think we should require that sequence parsers be able to handle
missing separators where they aren't necessary to disambiguate texts.

Nico
--

---

From: Stefan Drees <[email-masked]>
Date: Wed, 12 Mar 2014 06:19:05 +0100

> Or, as I suggest, "encoders MUST use whitespace".

+
1

{"Greeting":"All the best"}
{"Stefan":true}

---

From: "Markus Lanthaler" <[email-masked]>
Date: Wed, 12 Mar 2014 09:36:11 +0100

On Wednesday, March 12, 2014 6:03 AM, Nico Williams wrote:
On Tuesday, March 11, 2014, Tim Bray wrote:
> > I lean to \s+ because as soon as you see the first char, you’re
> > at the start of the next legal JSON text. 
>
> The separator can be whitespace.  It can be any whitespace recognized
> as such in the JSON ABNF.  It could be a subset of that.  It also
> could be any character or string of them that can't start a JSON value's
> encoding.  Since in practice existing apps use whitespace, we should use
> whitespace.  If other apps can be found using ", " or "," or whatever,
> we might be able to require parser support for all of them or else define
> multiple MIME types.

Given that "," is already used as separator in objects and arrays, I think it would make sense to at least include that as additional separator (and not just whitespace). Please let's not create multiple media types.

--
Markus Lanthaler
@markuslanthaler

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Wed, 12 Mar 2014 07:26:53 -0400

> 3. A specific required set of whitespace characters, for example \s*\n\s*

or

4. use the JSON separator character, the comma

-- 
Website: http://hallambaker.com/

---

From: Sean Gillies <[email-masked]>
Date: Wed, 12 Mar 2014 09:28:47 -0600

> 4. use the JSON separator character, the comma

I've thought about multi-text documents in the past and RFC 1521 Multipart
immediately came to mind.

  http://tools.ietf.org/html/rfc1521#page-29

Couldn't a pair of hyphens be a fifth option?

-- 
Sean Gillies

---

From: Nico Williams <[email-masked]>
Date: Wed, 12 Mar 2014 10:43:47 -0500

On Wednesday, March 12, 2014, Sean Gillies <[email-masked]> wrote:

> On Wed, Mar 12, 2014 at 5:26 AM, Phillip Hallam-Baker <[email-masked]<javascript:_e(%7B%7D,'cvml',[email-masked]');>
> > wrote:
>
>>
>> On Tue, Mar 11, 2014 at 11:49 PM, Tim Bray <[email-masked]<javascript:_e(%7B%7D,'cvml',[email-masked]');>
>> > wrote:
>>
>>> Yeah, the options are:
>>>
>>> 3. A specific required set of whitespace characters, for example
>>> \s*\n\s* or just \n
>>>
>>
>> or
>>

Anything that isn't ambiguous as to the preceding or following text will
do.  The string "foobar" will do, for example.  Running code is using
whitespace though.  So at the very least the set of separators must include
whitespace.  Adding more than whitespace will mean modifying existing
sequence parsers.  How many separators do we really need, and what is the
benefit adding more?

Nico
--