Subject: [Json] Regarding JSON text sequence ambiguities (Re: serializing sequences of JSON values)

From: Nico Williams <[email-masked]>
Date: Mon, 10 Mar 2014 17:06:43 -0500

On Mon, Mar 10, 2014 at 3:19 PM, Paul Hoffman <[email-masked]> wrote:
> [ No hat ]
>
> On Mar 10, 2014, at 7:35 PM, Paul E. Jones <[email-masked]> wrote:
>> Why would sequences of objects not be preferred for logging.
>
> Because the log is always open for appending, so the sequence never terminates.

This.

Other use cases relate to transferring large sequences of texts.  For
example, serializing a database: it may be easier to arrange for the
encoding of such a serialization as multiple top-level values than as
one large array (because, e.g., the encoder may not support online
functionality).  Yes, yes, the application could emit a '[' to start,
',' between top-level values, and a ']' at the end.  But if the
sequence is indeterminate in length (because it is an event log
including future events as they happen) then the parser on the other
end had better be an online (at least as to top-level array elements)
parser!  Sure the application could specially handle top-level arrays
to "explode" them in an online manner, but this still requires the
parser to identify exactly where in the input stream each array
element ends.

IOW, some use cases are easier to code as sequences of top-level values.

> This, and many other related topics, were discussed at length earlier on the mailing list and the rough consensus was that the current wording was sufficient.

Yes, though it's worth going into this a bit further.

There are two ambiguities then.  The values true, false, null, and
numeric values are all ambiguous unless separated by whitespace or
other unambiguous values (strings, arrays, or objects).  The second
ambiguity is between true/false/null/numeric values and EOF: a parser
fed 'true' cannot produce that value (or a parse error) until one more
byte is read _or_ EOF is detected.

This means that top-level non-array/object/string values should be
immediately _followed_ by whitespace when sequences of of such
top-level values are emitted.  Note: _followed_, not _preceded_ --
it's OK to also precede them with whitespace, but it is necessary to
follow them with whitespace in order to avoid the EOF/error ambiguity.

Nico
--

---

From: Nico Williams <[email-masked]>
Date: Mon, 10 Mar 2014 17:11:56 -0500

Consider jq (https://stedolan.github.io/jq) as an example.

jq can and will read a sequence of JSON texts from stdin.  That
sequence isn't an array.  jq applies its program to each input.  jq
produces as its output all the values -encoded as JSON texts- output
by the jq program.  This is exceedingly convenient, including for log
files (for the reason that you give: log files are an indeterminate
length, unending append-only sequence).

jq handles the ambiguity by requiring top-level values to be
"terminated" by any of: unambiguous top-level values, whitespace, or
EOF.  Thus the sequence of null, true, false, 1, and 2, requires
whitespace between every value.  While the sequence of true, "foo",
false, "bar", 1, "foobar", and 2 doesn't.  An input that looks like
"truefalsenull12" elicits a parse error.

Note that the jq parser does NOT in fact parse multiple top-level
values; it parses at most one top-level value.  Instead the program
using the parser feeds input bytes to the parser until the parser
finds the end of a top-level value and outputs it.  Remaining unparsed
bytes are buffered, of course so that when the program adds more input
bytes the parser can be restarted to parse the next top-level value.

This does mean that the jq parser will not output null when fed 'null'
until one more byte _or_ EOF are fed to it.  But if fed '[1,2]' then
the parser emits the parsed array value immediately when the closing
bracket is parsed, without waiting for further inputs.

jq always outputs a newline (though it could be space or tab) after
outputting any top-level value's JSON text encoding.  It does so
precisely to avoid these ambiguities.

(This is also why jq must continue to output a newline or other
whitespace after every output text, except, perhaps, when in raw
output mode, in which case the outputs aren't JSON texts.)

Nico
--

---

From: John Cowan <[email-masked]>
Date: Mon, 10 Mar 2014 19:59:03 -0400

Nico Williams scripsit:

> whitespace between every value.  

Why is that?  The input "nulltruefalse1 2" would be unambiguous.

-- 
John Cowan  <[email-masked]>  http://www.ccil.org/~cowan
        Raffiniert ist der Herrgott, aber boshaft ist er nicht.
                --Albert Einstein

---

From: Nico Williams <[email-masked]>
Date: Mon, 10 Mar 2014 19:15:06 -0500

> Why is that?  The input "nulltruefalse1 2" would be unambiguous.

The input "[nulltrue]" is clearly an error.  The input "nulltrue"
could either be: an error, or null then true.  Since there's two
possible interpretations of "nulltrue" it is ambiguous.  But you're
right that one could take the position that the "e" in "true" and
"false" terminates those values, and the second "l" in "null
terminates that one, and that resolves that ambiguity.  But there's
still an ambiguity in an online parser: is online parsing of
"[nulltrue]" to produce an array element value of null followed by an
error, or is it to produce just the one error?

Given that we have to separate top-level numbers with whitespace
anyways, I think the safe position is that "nulltrue" and so on are
invalid and that when emitting top-level values other than
arrays/objects/strings (and even then anyways) the encoding
application should follow the encoded value with whitespace (a
newline, preferably).

(I say encoding application because I believe an encoder is not
defined in such a way that permits it to emit a sequence of more than
one top-level values in a single octet stream output.)

Nico
--

---

From: John Cowan <[email-masked]>
Date: Mon, 10 Mar 2014 23:06:49 -0400

Nico Williams scripsit:

> But there's

If it's an error, I don't care what junk it has already produced,
so I'm quite indifferent to that.

-- 
I Hope, Sir, that we are not                    John Cowan
mutually Un-friended by this                    [email-masked]
Difference which hath happened                  http://www.ccil.org/~cowan
betwixt us.     --Thomas Fuller, Appeal of Injured Innocence (1659)

---

From: Tatu Saloranta <[email-masked]>
Date: Tue, 11 Mar 2014 03:12:20 +0000

> so I'm quite indifferent to that.

Based on feature requests I received for handling this specific case (for
Jackson), I would suggest producing error as soon as practical; in this
case once "nullt" has been decoded from input.
At least this was the behavior that was considered appropriate by users;
and specifically NOT to allow intermediate production of valid token for
'null' (doing which was reported as a bug).

-+ Tatu +-

---

From: Matthew Morley <[email-masked]>
Date: Tue, 11 Mar 2014 11:31:04 -0400

This issue came up with json-rpc structures and streaming json, which while
not directly related in that we expect to encounter only objects at the top
level... having guidance on handling top-level streaming is important for
json.

https://groups.google.com/forum/#!msg/json-rpc/fd1_anYEnDs/nnvnU51bfCoJ
*https://groups.google.com/d/msg/json-rpc/cXlc8bD-a_M/cwfLYWSly54J
<https://groups.google.com/d/msg/json-rpc/cXlc8bD-a_M/cwfLYWSly54J>*
http://www.simple-is-better.org/json-rpc/transport_sockets.html

On Mon, Mar 10, 2014 at 6:11 PM, Nico Williams <[email-masked]>wrote:

> _______________________________________________
> json mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/json
>

-- 
Matthew P. C. Morley

---

From: Nico Williams <[email-masked]>
Date: Tue, 11 Mar 2014 11:45:12 -0500

> level... having guidance on handling top-level streaming is important for

Arguably JSON text sequences are a new format, deserving of its own
MIME type and so on.  The change in JSON top-level value syntax didn't
really break JSON text sequencing as a) there were plenty of encoders
and parsers that handled non-array/object top-level values anyways, so
b) the ambiguity already existed.

Maybe we should publish an RFC on JSON text sequences?

Nico
--

---

From: Tim Bray <[email-masked]>
Date: Tue, 11 Mar 2014 09:48:54 -0700

Heh, I wonder if there’d be any chance of getting consensus.  I can’t
imagine ever using anything but Object Object Object with optional
whitespace separator; unless we all agree on that going in I’d pessimistic
about anyone convincing anyone else...

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Tue, 11 Mar 2014 13:09:14 -0400

On Tue, Mar 11, 2014 at 12:48 PM, Tim Bray <[email-masked]> wrote:

> Heh, I wonder if there'd be any chance of getting consensus.  I can't
> whitespace separator; unless we all agree on that going in I'd pessimistic

But JSON has comma separators, so {..}, {..}, {..} makes far more sense.

The array production in the grammar already specifies this syntax so JSON
parsers already recognize it. Your proposal is something new.

-- 
Website: http://hallambaker.com/

---

From: Nico Williams <[email-masked]>
Date: Tue, 11 Mar 2014 12:10:19 -0500

> about anyone convincing anyone else...

As I recall you opposed as strenuously the top-level value type change
in the RFC you edited.  And yet look where we are...

Alright, let's talk examples.  Suppose I have a JSON text sequence,
all of them objects.  And I want to extract a number from each, to be
output in a sequence, natch.  Why would I?  Because I'm processing
data in a sequence of filters, and one stage will be applying some
statistical analysis to just those numbers.  What's wrong with that?
It's perfectly natural to do that with -you see it coming, I know- jq.

Anyone who is adept/experienced with the Unix shell pipeline concept
will tend to like this approach.  (It's one reason I like jq.)

But more importantly, what do you care?!  Why should your lack of a
use for anything other than a sequence of objects constrain me to the
same?  On what principle?  Once you agree to the optional whitespace
and make it required when following an otherwise ambiguous value
(true, false, null, numeric), we're there, we have agreement.  You can
stick to object sequences, and I can use sequences of numbers if it be
convenient for me.

Perhaps you might say that the separator whitespace ought to be
optional always, and that fundamentally limits sequences to arrays,
objects, and strings.  That'd be a principle we could argue about
(whether to adopt it or not), but even that wouldn't justify limiting
sequences to objects.

I'm struggling to think of a principle that would justify your
favorite constraint, and I can't think of one.  What am I missing?

Nico
--

---

From: Nico Williams <[email-masked]>
Date: Tue, 11 Mar 2014 13:08:11 -0500

> But JSON has comma separators, so {..}, {..}, {..} makes far more sense.

JSON text sequences would be a new Proposed Standard (if we go there)
but like JSON, there exist uses of this "new" thing already -- that
is, before we get to writing the RFC.

The uses of JSON text sequences that I know of use newlines, not
commas nor comma-and-newline.  The reason for this is that these use
cases are text logfile-like: the entries are lines, lines containing
JSON texts -- usually compact texts, i.e., with no newlines in the
text, and never more than one text per-line.

For me other uses of JSON text sequences generally result from my use
of jq, which also effectively separates texts with a newline.  Note
that jq doesn't need texts to be written compactly when parsing JSON
text sequences.  It happens though that if you write texts compactly
followed by a newline then you can implement JSON text sequences with
all existing JSON parsers.

Switching to using a comma-and-newline would require significant
retooling.  Therefore I don't see it happening.  Whereas just
separating JSON texts with newlines is in use because it's always been
the obvious thing to do.

Nico
--

---

From: Matthew Morley <[email-masked]>
Date: Tue, 11 Mar 2014 14:27:02 -0400

I'm not advocating for comma separators...

But having multiple top level JSON elements separated by a coma is
equivalent to processing an array structure. The initial [ and the closing
] are implicitly mapped to the connection/stream/etc. start and end events.

It is just a minor token replacement at the top level between elements,
which could be layered into some existing tooling. From this point of view,
I would imagine the retooling is minor for either use case. It does mean
tools need to be depth aware.

-- 
Matthew P. C. Morley

---

From: Tim Bray <[email-masked]>
Date: Tue, 11 Mar 2014 11:35:37 -0700

My assumption is that many (most? all?) JSON parsers can be altered to,
when they encounter a JSON text containing an object, just stop reading
when they hit the trailing “}”.  So my notion is that using such a parser
you have a loop like

while !eof
   obj = parser.ReadObjectAndStop() // parse error handling left as an
exercise for the reader
   doWhateverWith(obj)
   eatZeroOrMoreWhiteSpaceCharacters()  // assumes some sort of peekChar()
end

So yeah, requiring a SINGLE NEWLINE AND NOTHING ELSE would be simpler.  But
if you’re going to allow any other non-significant white space characters,
 then why bother requiring that one of them be a newline?

---

From: Nico Williams <[email-masked]>
Date: Tue, 11 Mar 2014 13:59:22 -0500

On Tue, Mar 11, 2014 at 1:35 PM, Tim Bray <[email-masked]> wrote:
> My assumption is that many (most? all?) JSON parsers can be altered to, when
> they encounter a JSON text containing an object, just stop reading when they
> hit the trailing “}”.  So my notion is that using such a parser you have a
> loop like

That's true of arrays as well.

It's also true of strings for any parser that permits them at the top-level.

It's only true, false, null, and numeric top-level values that require
a disambiguation separator.

Using a newline (or any other separator character or string) permits
the parsing of any JSON text sequences with unmodified parsers.  This
is especially true when the JSON texts are encoded with no newlines
and the separator is or has a newline.

>    obj = parser.ReadObjectAndStop() // parse error handling left as an

The key is being able to parse one text, then the next.  If the texts
self-delimit (e.g., they contain no embedded newlines and are
terminated by a newline) then any JSON parser can be used to build a
JSON text sequence parser.

Even if a text contains a new line (e.g., after every element of an
array or after every name or value in an object), one could parse a
JSON text sequence even with the simplest JSON parsers (ones that have
no incremental parsing capability).

Thus I'd RECOMMEND that JSON text sequences consist of sequences of
JSON texts encoded with no embedded newlines, and I'd REQUIRE that
each text followed by a newline.

Then the general case (any parser, incremental or not) is as simple as:

    while read line; do
        parse "$line" ...
        ...
    done

> So yeah, requiring a SINGLE NEWLINE AND NOTHING ELSE would be simpler.  But

Indeed, it's the simplest: because every language/runtime has a "read
a line at a time" primitive.  If your texts have embedded newlines and
you have an incremental parser, you're still OK.

Heck, even if you don't have an incremental parser, you'd be OK,
though the result would be inefficient:

    buffer=
    while read line; do
        last_is_partial=true
        if [[ -z $buffer ]] && ! parse "$line"...; then
            buffer=$line
            continue
        elif [[ -n $buffer ]]; then
            buffer+=$line
            if ! parse "$buffer"...; then
                continue
            fi
        fi
        last_is_partial=false
        ...
    done
    if $last_is_partial; then
        # Report error
        ...
    fi

> then why bother requiring that one of them be a newline?

Because reading lines is a common (universal?) primitive.  Sure, if
you have an evented app and an incremental JSON parser then you'll be
reading bytes till you get a complete entry, newline or not.

Nico
--

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Tue, 11 Mar 2014 15:19:57 -0400

> separating JSON texts with newlines is in use because it's always been

The constraint that the objects can't have newlines inside the object will
require rather more retooling than a requirement that objects be separated
by commas which is the existing JSON approach.

I think finding a standard JSON way to meet this requirement is
appropriate. Blessing a particular approach that isn't in the existing
grammar is not AFIK.

---

From: Nico Williams <[email-masked]>
Date: Tue, 11 Mar 2014 14:42:21 -0500

> by commas which is the existing JSON approach.

Actually, that isn't a constraint.  It does help when you have parsers
that don't work incrementally, but you can make do even if the
objects/arrays have embedded newlines.  See my reply to Tim about
that.

Nico
--

---

From: "Martin J. Dürst" <[email-masked]>
Date: Wed, 12 Mar 2014 18:44:32 +0900

> appropriate. Blessing a particular approach that isn't in the existing

For the record, I strongly agree.

Seeing claims that we need a separate spec just because an initial '[', 
interleaving ','s, and a potential(1) final ']' is too much to deal with 
really surprises me.

At the utmost, this might become an Informational RFC if some people 
really, really want it, but it should NOT become a working item of this 
group, nor should it go to standards track.

Why? JSON is great, but the more variation we get, the more we end up 
with a *big mess*, even if every step may be small and look good.

As far as I have heard (no first hand experience, fortunately), CORBA 
ended up that way. Many people, in particular in the JSON community, see 
XML that way. Similar things have been said about ASN-1, although that 
may have been complicated from the start (again, fortunately no first 
hand experience).

Maybe JSON is just next in line.

Regards,    Martin.

(1) When you write out a log, there's no need to actually write the 
final ']' and then replace it by a ','. XMPP did something very similar 
for XML streaming, please see
http://tools.ietf.org/html/rfc6120#section-4. In JSON, it's slightly 
trickier because you need an explicit separating comma, and a 'dummy' 
element before the closing ']', but that's not rocket science and well 
worth the effort to avoid format splits.

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Wed, 12 Mar 2014 07:35:05 -0400

On Wed, Mar 12, 2014 at 5:44 AM, "Martin J. Dürst"
<[email-masked]>wrote:

>> On Tue, Mar 11, 2014 at 2:08 PM, Nico Williams <[email-masked]
>> >wrote:
>>
>

The final ']' completely kills the use of append only logs. Which means
that you can't rely on the O/S append operation to give atomicity.

So there is a performance and a security issue here.

> Why? JSON is great, but the more variation we get, the more we end up with
> a *big mess*, even if every step may be small and look good.
>

I don't think that is what happened to XML or CORBA.

The CORBA people had no idea of how to communicate what they were
attempting to do. Plus they were trying to exchange C++ objects that
weren't C++ objects at a time when C++ was not functional or stable.

XML is the way it is because it is trying to be a document markup not a
data markup.

> XML that way. Similar things have been said about ASN-1, although that may
> have been complicated from the start (again, fortunately no first hand
> experience).
>

If we stick to commas then the production already exists in the spec:

json-text = ws-value

json-texts = ws-value (',' ws-value)

---

From: Larry Masinter <[email-masked]>
Date: Thu, 13 Mar 2014 21:39:43 +0000

> Arguably JSON text sequences are a new format,

yes

> deserving of its own MIME type and so on.

no, not all formats need MIME types. There are infinitely many formats. 
MIME is one way of noting a file format and distinguishing it from others, but most applications don't NEED to name the types they accept and in general, they can't.

I think to justify a new MIME type, you need to provide a concrete use case that would actually use it and need it.  In most of the content-negotiation scenarios, MIME types are far too course-grained to express content-features.

---

From: Tim Bray <[email-masked]>
Date: Thu, 13 Mar 2014 14:43:25 -0700

I’m halfway convinced that it might be useful to say that if you are
persisting something as chunks of JSON, use X as a separator.  I wouldn’t
be surprised to see the advent of standardized log-as-JSON libraries. I’d
go further and say that if you are doing this, use objects too.  Having
said that, it’s not obvious that this is the kind of format that you’d
interchange much, so the case for a media type is weak.

On Thu, Mar 13, 2014 at 2:39 PM, Larry Masinter <[email-masked]> wrote:

> MIME is one way of noting a file format and distinguishing it from others,
> but most applications don't NEED to name the types they accept and in
> general, they can't.
>
> I think to justify a new MIME type, you need to provide a concrete use
> case that would actually use it and need it.  In most of the
> content-negotiation scenarios, MIME types are far too course-grained to
> express content-features.
>

---

From: Larry Masinter <[email-masked]>
Date: Thu, 13 Mar 2014 21:44:38 +0000

I think most of i-json and also this discussion on JSON sequences would fit into a "guidelines" document. "Guidelines for use of JSON in Internet protocols".
Then it could be BCP, could cover \u2028 \u2029, string length limits, floating point, separators in sequences, and other advice.

much more straightforward. It could even cover "JSON vs alternatives" and pros and cons comparison to XML, CBOR, ASN.1 and whatever.

From: json [mailto:[email-masked]] On Behalf Of Matthew Morley
Sent: Tuesday, March 11, 2014 11:27 AM
To: Nico Williams
Cc: Tim Bray; Phillip Hallam-Baker; Paul Hoffman; [email-masked]
Subject: Re: [Json] Regarding JSON text sequence ambiguities (Re: serializing sequences of JSON values)

But having multiple top level JSON elements separated by a coma is equivalent to processing an array structure. The initial [ and the closing ] are implicitly mapped to the connection/stream/etc. start and end events.

It is just a minor token replacement at the top level between elements, which could be layered into some existing tooling. From this point of view, I would imagine the retooling is minor for either use case. It does mean tools need to be depth aware.

On Tue, Mar 11, 2014 at 2:08 PM, Nico Williams <[email-masked]<mailto:[email-masked]>> wrote:
On Tue, Mar 11, 2014 at 12:09 PM, Phillip Hallam-Baker <[email-masked]<mailto:[email-masked]>> wrote:
> On Tue, Mar 11, 2014 at 12:48 PM, Tim Bray <[email-masked]<mailto:[email-masked]>> wrote:
>>

Nico
--

--
Matthew P. C. Morley

---

From: Tim Bray <[email-masked]>
Date: Thu, 13 Mar 2014 14:52:09 -0700

Except for, a spec that just rules out the interoperability gotchas in 7159
is such a low-hanging fruit, and avoids so many slippery slopes.  Damn, I
wish that stupid 2028/2029 thing had come across the radar before we
finished 7159.

On Thu, Mar 13, 2014 at 2:44 PM, Larry Masinter <[email-masked]> wrote:

>  I think most of i-json and also this discussion on JSON sequences would
> fit into a “guidelines” document. “Guidelines for use of JSON in Internet
> protocols”.
>
> Then it could be BCP, could cover \u2028 \u2029, string length limits,
> floating point, separators in sequences, and other advice.
>
>
>
> much more straightforward. It could even cover “JSON vs alternatives” and
> pros and cons comparison to XML, CBOR, ASN.1 and whatever.
>
>
>
>
>
>
>
>
>
>
>
> *From:* json [mailto:[email-masked]] *On Behalf Of *Matthew Morley
> *Sent:* Tuesday, March 11, 2014 11:27 AM
> *To:* Nico Williams
> *Cc:* Tim Bray; Phillip Hallam-Baker; Paul Hoffman; [email-masked]
> *Subject:* Re: [Json] Regarding JSON text sequence ambiguities (Re:
> serializing sequences of JSON values)
>
>
>

---

From: Nico Williams <[email-masked]>
Date: Thu, 13 Mar 2014 17:58:20 -0500

>> Arguably JSON text sequences are a new format,

It seems we're getting consensus on that much.

> MIME is one way of noting a file format and distinguishing it from others, but most applications don't NEED to name the types they accept and in general, they can't.

The same argument leads to not needing application/json either.  And
yet there it is.

For HTTP applications an application/json-text-sequence type might
mean that a server could generate N objects one at a time instead of
generating them all into a top-level array then sending that once
built.  Sure, it's not hard to build a top-level array in an online
manner, but if you've ever run into an application that doesn't, you
know that there is value in this.

> I think to justify a new MIME type, you need to provide a concrete use case that would actually use it and need it.  In most of the content-negotiation scenarios, MIME types are far too course-grai…

A proprietary app I work could use this.  It's a database, queries
return results, as expected, specifically as a top-level array or
results -- each result being roughly like a SQL row.  Actually, it's
an app built out of Rails and Postgresql, of which there are many out
there.  Who knows, it might be trivial to deploy this new format type.

Nico
--

---

From: "Manger, James" <[email-masked]>
Date: Fri, 14 Mar 2014 10:27:54 +1100

> > MIME is one way of noting a file format and distinguishing it from
> others, but most applications don't NEED to name the types they accept
> and in general, they can't.
> 
> > I think to justify a new MIME type, you need to provide a concrete
> use case that would actually use it and need it.  In most of the

Media types are cheap aren’t they? I cannot see any reason not to define one while specifying a sequence of JSON values.

How about
  application/jsons
it is a bit shorter than application/json-text-sequence; and we can use +jsons as a media type extension if required.

--
James Manger

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Fri, 14 Mar 2014 01:29:30 +0100

* Manger, James wrote:
>How about
>  application/jsons
>it is a bit shorter than application/json-text-sequence; and we can use 
>+jsons as a media type extension if required.

Single character differences are not a very good pattern and this one
may also have the wrong connotations (e.g., is 'jsons' to 'json' as
'https' is to 'http').
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: "Manger, James" <[email-masked]>
Date: Fri, 14 Mar 2014 12:16:07 +1100

> may also have the wrong connotations (e.g., is 'jsons' to 'json' as

It has the perfect connotation: jsons looks like the plural of json, which is perfect for a format holding any number of JSON values.

--
James Manger

---

From: Nico Williams <[email-masked]>
Date: Thu, 13 Mar 2014 20:25:11 -0500

> It has the perfect connotation: jsons looks like the plural of json, which is perfect for a format holding any number of JSON values.

I'm sympathetic to a "single character differences" lead to
typos/errors argument.  I'm also sympathetic to an argument about
wasting bandwidth.

I happen to like application/jsons, but would settle for
application/json-sequence.

Nico
--

---

From: "Martin J. Dürst" <[email-masked]>
Date: Fri, 14 Mar 2014 11:08:22 +0900

> application/json-sequence.

The fact that something like application/json-text-sequence is lengthy 
also has the advantage that it may give a hint that this isn't a widely 
used format, whereas JSON itself is.

And yes, MIME types, like any other sort of identifiers, are cheap. The 
cost isn't in the identifier itself, but in the confusion and loss of 
network effects that may result.

Regards,   Martin.

---

From: John Cowan <[email-masked]>
Date: Thu, 13 Mar 2014 22:27:11 -0400

"Martin J. Dürst" scripsit:

> The fact that something like application/json-text-sequence is
> lengthy also has the advantage that it may give a hint that this
> isn't a widely used format, whereas JSON itself is.

No format is widely used when first defined.

An advantage of a simple identifier is that it can be used as a "+"
suffix.

-- 
With techies, I've generally found              John Cowan
If your arguments lose the first round          http://www.ccil.org/~cowan
    Make it rhyme, make it scan                 [email-masked]
    Then you generally can
Make the same stupid point seem profound!           --Jonathan Robie

---

From: "Martin J. Dürst" <[email-masked]>
Date: Fri, 14 Mar 2014 12:57:43 +0900

> No format is widely used when first defined.

Yes. But many if not most formats are defined but then never widely used.

> An advantage of a simple identifier is that it can be used as a "+"

As far  as I know, there are no limits on the theoretical number of 
characters in Mime types, subtypes, or "+" suffixes.

And I think creating a pleothra of json-related formats, and even giving 
them "+" suffixes, goes just way overboard.

The JSON text sequence idea comes from a minor inconvenience of JSON 
arrays. The benefit of JSON is that it is simple and straightforward, 
but that's apparently lost on most people in this discussion ;-(.

Regards,   Martin.

---

From: Nico Williams <[email-masked]>
Date: Thu, 13 Mar 2014 23:17:21 -0500

On Thu, Mar 13, 2014 at 10:57 PM, "Martin J. Dürst"
<[email-masked]> wrote:
> The JSON text sequence idea comes from a minor inconvenience of JSON arrays.

Hmm?  For me it comes from jq, and there it's just... natural, nothing
to do with inconveniences of JSON arrays, and everything to do with
the need to have indeterminate-length streams of values.

Exploring that further there's the fact that non-online JSON parsers
can be used to construct JSON text sequence parsers that handle each
text in a non-online manner, but the sequence in an online manner --
this makes a world of a difference when processing large amounts of
data encoded in JSON:

 - One big array + not-online parser -> can't process more than fits
in memory parsed, and can't start until it's all parsed.

 - JSON text sequence + not-online parser -> no limit based on number
of texts, just size of each text.

Needless to say, online parsers can handle huge texts just fine, but
they are very difficult to use.  JSON text sequences are just the
sweet spot for easy of use and performance.

> The benefit of JSON is that it is simple and straightforward, but that's
> apparently lost on most people in this discussion ;-(.

Which is why people want to use it more and more, even for large
datasets, and then you run into the need for an online parser.  Or
JSON text sequencing.

Nico
--