Subject: Re: [Json] Proposed rechartering for the JSON WG

From: Paul Hoffman <[email-masked]>
Date: Fri, 7 Feb 2014 14:32:44 -0800

First off: I dropped the ball on this one. I told Matt I would follow up, and now I am doing so, but terribly late.

People felt that our two work items didn't match what we had discussed over the past few months. Below is another run at the proposed wording.

--Paul Hoffman

-----BEGIN CHARTER-----

Javascript Object Notation (JSON) is a lightweight, text-based,
language-independent data interchange format. It was derived from the
ECMAScript Programming Language Standard and was published in RFC 4627,
an Informational document. JSON has come into very broad use, often
instead of or in addition to XML. The JSON WG has already published
a standards-track update to RFC 4627.

The WG will work on two items:

- a restricted profile of JSON designed to maximize interoperability;
  the starting point will be draft-bray-i-json

- a standardized nomenclature for referring to the constructs in a
  JSON text, for use by those specifying protocols with JSON payloads

Milestones:

May 2014:    WG LC on a restricted profile of JSON
May 2014:    WG LC on a standardized nomenclature for JSON
June 2014:   IETF LC on a restricted profile of JSON
June 2014:   IETF LC on a standardized nomenclature for JSON

-----END CHARTER-----

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Fri, 7 Feb 2014 17:53:26 -0500

On Fri, Feb 7, 2014 at 5:32 PM, Paul Hoffman <[email-masked]> wrote:

> First off: I dropped the ball on this one. I told Matt I would follow up,
> and now I am doing so, but terribly late.
>
> People felt that our two work items didn't match what we had discussed
> over the past few months. Below is another run at the proposed wording.
>
> --Paul Hoffman
>
> -----BEGIN CHARTER-----
>

Does a 'standardized nomenclature for JSON' mean formal languages are in or
out of scope?

If schemas are in scope then I can't see us completing by May 2014.

My schema tool now allows the specification to be defined in a very clean
syntax that is similar to python (no braces, indentation denotes block
structure).

It generates xml2rfc, C# and now C code to implement a client and server
stub library. Runs on all major platforms.

OK the C library needs more work, the parser is fine but its not got the
Web service wrapper yet.

If all we are doing is specifying how to describe the JSON elements in a
spec then I will simply wait for the output and then make my specification
generator write conforming text.

Some issues that have come up recently are how to manage NaN values for
numeric fields. I think the only clean way to do this in JSON is with the
null lexeme

-- 
Website: http://hallambaker.com/

---

From: Paul Hoffman <[email-masked]>
Date: Fri, 7 Feb 2014 15:13:08 -0800

On Feb 7, 2014, at 2:53 PM, Phillip Hallam-Baker <[email-masked]> wrote:

> Does a 'standardized nomenclature for JSON' mean formal languages are in or out of scope?

Neither in or out of scope yet. There has only been lightweight discussion in the WG so far.

> If schemas are in scope then I can't see us completing by May 2014.

Schemas are *not* in scope. In Vancouver, I proposed that the WG look at requirements for schema, and the WG was nearly-universally against it.

> My schema tool now allows the specification to be defined in a very clean syntax that is similar to python (no braces, indentation denotes block structure).
> 
> It generates xml2rfc, C# and now C code to implement a client and server stub library. Runs on all major platforms.
> 
> OK the C library needs more work, the parser is fine but its not got the Web service wrapper yet.
> 
> 
> If all we are doing is specifying how to describe the JSON elements in a spec then I will simply wait for the output and then make my specification generator write conforming text.

Maybe add the nomenclature to your tool as the WG develops the nomenclature. Your tool will be a good sanity check about whether the nomenclature is sane and complete. Edge cases will certainly be welcome.

--Paul Hoffman

---

From: Tim Bray <[email-masked]>
Date: Fri, 7 Feb 2014 15:16:56 -0800

Works for me
On 7 Feb 2014 17:32, "Paul Hoffman" <[email-masked]> wrote:

> _______________________________________________
> json mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/json
>

---

From: Tim Bray <[email-masked]>
Date: Fri, 7 Feb 2014 15:20:02 -0800

> Does a 'standardized nomenclature for JSON' mean formal languages are in
or out of scope?

Don't know what you mean by Formal Languages in this context, but schemas
are clearly out of scope.

Web service wrapper yet.

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Fri, 7 Feb 2014 19:57:52 -0500

> are clearly out of scope.
By formal language I mean a machine readable description of the data
structure that specifies the tag names and the set of corresponding values.

http://sourceforge.net/p/prismproof/code/ci/master/tree/PrismProof/Registration/KeyFile.protocol

Protocol Goedel.KeyFile KeyStore

	Structure KeyEntry

	Structure Key
		Inherits			KeyEntry
		String				KeyID
			Description
				|Key Identifier	in text form.			
		Struct Certificate	SelfCertificates
			Multiple
			Description

---

From: Nico Williams <[email-masked]>
Date: Fri, 7 Feb 2014 19:03:02 -0600

> structure that specifies the tag names and the set of corresponding values.

That's schema.  One (or more) schema representations for JSON would be
nice, yes.

Note that ABNF is a formal language, but not what you're after, which
is why I would have asked the question Tim asked.

Nico
--

---

From: Tim Bray <[email-masked]>
Date: Fri, 7 Feb 2014 17:14:49 -0800

I really want to stay away from the slippery schema slope. So I do not
think we're trying to be ABNF for JSON, because that is in fact a simple
lexical schema.  In a spec sentence like this:

The value of the "size" member of the top level "buffer" item MUST be a
non-negative integer.

I think nomenclature means having a uniform standardised way to express the
meaning of the words between "value of" and "MUST".  Perhaps something like
XPath for JSON.

Disclosure: I'm not convinced this is necessary or even useful.
On 7 Feb 2014 20:03, "Nico Williams" <[email-masked]> wrote:

> >> > Does a 'standardized nomenclature for JSON' mean formal languages are
> in
> >> > or out of scope?
> >>

---

From: Stefan Drees <[email-masked]>
Date: Sat, 08 Feb 2014 12:41:33 +0100

> June 2014:   IETF LC on a standardized nomenclature for JSON

+1 from me as long as we stay away far enough from any schema "death 
star" and most of us interpret "standardized nomenclature" as
"""In a spec sentence like this:

I think nomenclature means having a uniform standardised way to express 
the meaning of the words between "value of" and "MUST".  Perhaps 
something like XPath for JSON.
""" (citing Tim Bray, who put it that way)

{"Stefan": true}

---

From: Nico Williams <[email-masked]>
Date: Sat, 8 Feb 2014 16:50:13 -0600

On Fri, Feb 7, 2014 at 7:14 PM, Tim Bray <[email-masked]> wrote:
> I really want to stay away from the slippery schema slope. [...]
>

An XPath/XSLT for JSON would be good.  There's a great candidate for
that: http://stedolan.github.io/jq .   But I don't propose
standardizing such a thing.

> Disclosure: I'm not convinced this is necessary or even useful.

Schemas help document protocols.  They are as useful as ABNF, ASN.1,
and so on.  In practice we can't get to where we have 100% formal
descriptions of everything relevant to interop, but a total lack of
formality isn't exactly a good thing either.  In absence of a JSON
schema language (or three) we'll have to resort to showing examples
and English prose for the rest.

Nico
--

---

From: Tim Bray <[email-masked]>
Date: Sat, 8 Feb 2014 15:10:26 -0800

On Sat, Feb 8, 2014 at 2:50 PM, Nico Williams <[email-masked]> wrote:

>  In absence of a JSON

Another excellent reason not to have a JSON schema language.  I think that
the very best protocol specs center around clear English prose and
well-chosen illustrative examples, and schema cleverness is often a
distraction from getting that right.

---

From: Nico Williams <[email-masked]>
Date: Sat, 8 Feb 2014 17:52:23 -0600

> distraction from getting that right.

It depends on whether the schema language is easy to build tools for
or not.  ASN.1 (with all its complexity, including its encoding rules,
the IOS and the related SDL) is an example of a language which is not
easy to build tools for.

We're in a much simpler space here, not because we lack schema for
JSON but because we'd not port all the complexity of ASN.1 to a JSON
schema.  We know better.  Also, for a great many implementors the JSON
tools at their disposal (e.g., ECMAScript) make it easy to write
schema-based tools, such as code generators and schema validators.

Besides that, English prose is difficult to write in such a way that
it will be understood clearly by the many readers for whom English is
not a native/first language.  I don't ever stop to think about
non-native English readers of my prose and I'm not a native English
speaker.  I can see that English convenient for those of us who've
mastered it well enough, and it is today's lingua franca, but I'd
rather have a bit of formalism to go with the text.  Also, I
personally find some entirely-prose RFCs (like RFC4422) to be
difficult to read -- think of formalism as a crutch if you like, but
it's a very convenient crutch, helping readers, implementors, and the
authors themselves.

Nico
--

---

From: Tim Bray <[email-masked]>
Date: Sat, 8 Feb 2014 15:52:47 -0800

On Sat, Feb 8, 2014 at 2:50 PM, Nico Williams <[email-masked]> wrote:

> that: http://stedolan.github.io/jq .

jq seems like overkill. Wow, would it ever be easy to do a JSON path
selector.   They practically read themselves

"buffer"/"size"
//"size"
//"buffers"/./"size"
//"buffers"/0/"size"
//"buffers"/-1/.
"buffer"/./"key"

But I don't propose

---

From: Nico Williams <[email-masked]>
Date: Sat, 8 Feb 2014 18:22:23 -0600

> selector.   They practically read themselves

jq let's you do exactly that, and more too (just like XPath, and
especially XSLT).

It is missing an operator that trivially allows writing things like
//"buffer"/"size", but it's not hard to write a utility function that
allows it, and the manual has examples of this sort of thing using the
recurse built-in.  (I'll probably end up contributing such a thing;
it'd make sense to parse ..buffer.size and turning it into a search
for objects with a buffer name whose value is an object with a size
name.)

Nico
--

---

From: Mark Nottingham <[email-masked]>
Date: Sun, 9 Feb 2014 13:37:41 +1100

+1

I’d be more comfortable if the charter were more explicit about this; e.g., “A set of natural-language terms and/or phrases for use in future specifications that use JSON. This explicitly excludes schema languages and similar formalisms.” 

On 8 Feb 2014, at 12:14 pm, Tim Bray <[email-masked]> wrote:

> I really want to stay away from the slippery schema slope. So I do not think we're trying to be ABNF for JSON, because that is in fact a simple lexical schema.  In a spec sentence like this:
> 
> The value of the "size" member of the top level "buffer" item MUST be a non-negative integer.
> 
> I think nomenclature means having a uniform standardised way to express the meaning of the words between "value of" and "MUST".  Perhaps something like XPath for JSON.
> 

--
Mark Nottingham   http://www.mnot.net/

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Sat, 8 Feb 2014 22:01:02 -0500

Me too.

I was asking if schema was in scope. I have no particular interest in
having a committee redesign a tool that works perfectly well as it is.

On Sat, Feb 8, 2014 at 9:37 PM, Mark Nottingham <[email-masked]> wrote:

> +1
>
> I'd be more comfortable if the charter were more explicit about this;
> e.g., "A set of natural-language terms and/or phrases for use in future
> specifications that use JSON. This explicitly excludes schema languages and
> similar formalisms."
>
>
> the meaning of the words between "value of" and "MUST".  Perhaps something
> like XPath for JSON.
> >
> > >> > Does a 'standardized nomenclature for JSON' mean formal languages
> are in
> > >> > or out of scope?
> > >>

---

From: Paul Hoffman <[email-masked]>
Date: Sat, 8 Feb 2014 19:13:16 -0800

> I’d be more comfortable if the charter were more explicit about this; e.g., “A set of natural-language terms and/or phrases for use in future specifications that use JSON. This explicitly excludes…

Do the folks who earlier +1'd the proposed text like Mark's reformulation?

--Paul Hoffman

---

From: Tim Bray <[email-masked]>
Date: Sat, 8 Feb 2014 19:40:13 -0800

Partly. I don’t see why we have to write natural-language into the charter,
maybe we should do XPath-ish.  So how about leaving it the way it was with
“nomenclature” and add on Mark’s “This explicitly excludes schema languages
and similar formalisms.”

On Sat, Feb 8, 2014 at 7:13 PM, Paul Hoffman <[email-masked]> wrote:

> > I’d be more comfortable if the charter were more explicit about this;
> e.g., “A set of natural-language terms and/or phrases for use in future
> similar formalisms.”

---

From: Cyrus Daboo <[email-masked]>
Date: Sun, 09 Feb 2014 11:09:47 -0800

Hi Paul,

--On February 8, 2014 at 7:13:16 PM -0800 Paul Hoffman 
<[email-masked]> wrote:

>> e.g., “A set of natural-language terms and/or phrases for use in
>> future specifications that use JSON. This explicitly excludes schema
>> languages and similar formalisms.”
>

Would that rule out something like: 
<http://tools.ietf.org/html/draft-newton-json-content-rules-01>? I have 
found that extremely useful in a couple of specs I have used. It focusses 
the spec developer on enumerating the structure and set of members etc in a 
JSON document. It helps developers by making clear all the possible options 
(something examples are not good at).

I am not sure a "natural-language" formalism is sufficient because that is 
not as expressive as a "structured" formalism for describing things like 
hierarchy, e.g.:

A contains B contains C

vs

A : {
  B : {
    C
  }
}

Even with "natural language" I suspect people will introduce lists, indents 
etc leading to something more structured.

-- 
Cyrus Daboo

---

From: Carsten Bormann <[email-masked]>
Date: Sun, 9 Feb 2014 17:32:23 +0100

Those schema scars must be deep, because Cyrus of course is right.

Having a minimal parseable description technique for JSON-based protocols would save us from tedium such as in RFC 7071 section 6.2.2.
(Not needing something like that ever again*) in the future is my personal benchmark for what we need to achieve here.)

The important thing**) here is the 80-20 rule — we don’t need to make this thing powerful or perfect, because we can always fall back to English.
But we shouldn’t always have to, even for the most basic purposes.

(draft-newton looks like an 85 % solution to me; maybe this can be even simpler.)

Grüße, Carsten

*) Yes, I know why it was done, and I agree that it had to be done at the time.

**) The other important thing is that a JSON description technique doesn’t have to be parse as JSON, just as an XML schema does not have to be parse as XML.
Why are people falling into this trap again and again...

---

From: Stefan Drees <[email-masked]>
Date: Sun, 09 Feb 2014 18:01:16 +0100

Am 09.02.14 04:13, schrieb Paul Hoffman:
>> specifications that use JSON. This explicitly excludes schema languages
>> and similar formalisms.”
>
> Do the folks who earlier +1'd the proposed text like Mark's reformulation? ...

+1 and yes, absolutely - from my side!

{"Stefan": true}

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Sun, 9 Feb 2014 13:15:38 -0500

On Sun, Feb 9, 2014 at 11:32 AM, Carsten Bormann <[email-masked]> wrote:

> Having a minimal parseable description technique for JSON-based protocols
> would save us from tedium such as in RFC 7071 section 6.2.2.
> (Not needing something like that ever again*) in the future is my personal
> benchmark for what we need to achieve here.)
>

A machine readable specification supported by tools is very useful. But how
many people are writing tools? Are we at a stage where it is helpful to
converge on one tool?

https://sourceforge.net/projects/jsonschema/

My view is that it would be a very bad idea to design such a tool in
committee and that there would be negligible value from having the format
be declared a standard prematurely.

A better way forward would be to publish proposals for schema formats as
INFORMATIONAL RFCs and see which get used.

**) The other important thing is that a JSON description technique doesn't
> have to be parse as JSON, just as an XML schema does not have to be parse
> as XML.

For SAML I generated all my XML schemas using another tool written using
Goedel. If I was going to write another XML spec I would extend the
generator to dump XML schema.

I have an ASN.1 parser as well so I could provide a tool that generated
code for ASN.1, XML and JSON.

---

From: Tim Bray <[email-masked]>
Date: Sun, 9 Feb 2014 11:10:24 -0800

> benchmark for what we need to achieve here.)

Hm, I just went and looked at
http://tools.ietf.org/html/rfc7071#section-6.2.2 - it reads like a
perfectly sensible piece of specification-ware to me. Looks to me like it
leaves implementors very little uncertainty, and it’s easy to understand.
 It all fits on a couple of screens on my laptop. The dorky ALL_CAPS makes
it typographically ugly, but it’s hard to imagine a formalism that would
make it dramatically shorter, or more useful.

I actually thought what we were talking about was something to replace
6.2.1 in that spec, so people don’t ever need to write an “Imported JSON
terms” section again.

>
> The important thing**) here is the 80-20 rule — we don’t need to make this
> thing powerful or perfect, because we can always fall back to English.
> **) The other important thing is that a JSON description technique doesn’t

---

From: Andrew Newton <[email-masked]>
Date: Sun, 9 Feb 2014 14:11:09 -0500

On Sun, Feb 9, 2014 at 11:32 AM, Carsten Bormann <[email-masked]> wrote:
> The important thing**) here is the 80-20 rule -- we don't need to make this thing powerful or perfect, because we can always fall back to English.
> But we shouldn't always have to, even for the most basic purposes.
>

The intent of the draft was to find a way to document JSON. It
probably got too formal and could use some thoughtful reduction plus
use of english words instead of symbols to ease readability.

As for the backlash against schema languages, there are some lessons
to be learned. After all, DTDs and XSDs do not a standard make. The
sweet spot is probably not just a common notation or language to
describe the use of JSON in a standard but also some guidelines
regarding the other information necessary for a good specification.

-andy

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 10 Feb 2014 09:28:55 +1100

On 10 Feb 2014, at 5:15 am, Phillip Hallam-Baker <[email-masked]> wrote:

> A machine readable specification supported by tools is very useful. But how many people are writing tools? Are we at a stage where it is helpful to converge on one tool?
> 
> My view is that it would be a very bad idea to design such a tool in committee and that there would be negligible value from having the format be declared a standard prematurely. 
> 
> A better way forward would be to publish proposals for schema formats as INFORMATIONAL RFCs and see which get used.

What Phillip said.

---

From: Nico Williams <[email-masked]>
Date: Sun, 9 Feb 2014 16:57:48 -0600

There's no reason that we should have a single schema language, and
for any document it should suffice to use whichever schema language
(if any), is most appropriate.  Therefore I wouldn't suggest that the
WG have a [single] standard schema as a work item, but I'm not sure
what we stand to gain from having only Informational schema languages
-- they can't be used to specify Standards-Track protocols.  If
there's no appetite for working on one or more schema languages, so be
it.

Nico
--

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Sun, 9 Feb 2014 19:06:36 -0500

> there's no appetite for working on one or more schema languages, so be

This is no longer the case.

Internet Standards have been allowed to down-reference Informational for a
long time now. There are some additional steps but they are minor.

Since the schema itself only describes the data format, it does not even
need compliance language.

---

From: Nico Williams <[email-masked]>
Date: Mon, 10 Feb 2014 10:41:04 -0600

> need compliance language.

Then I'm happy with all JSON schema docs being Informational (and not
even WG work items).

Nico
--

---

From: Nico Williams <[email-masked]>
Date: Mon, 10 Feb 2014 11:43:42 -0600

> selector.   They practically read themselves

FYI:

.buffer.size

> //"size"

recurse(.[]).size

> //"buffers"/./"size"

recurse(.[]).buffers[].size

> //"buffers"/0/"size"

recurse(.[]).buffers[0].size

The .name syntax is a bit to type-strong and so you get spurious error
messages (safely ignored).  I think the right thing to do here is to
make .name act like 'empty' when . is not an object, or if name is an
integer, when . is not an array (but .[number] needs to continue to
produce warnings if . is not an array).  That and add ..name syntax,
then '..buffers.0.size' should do what you want without spurious error
messages.  I'll defer to Stephen either way.

BTW, I've needed // in XPath very often.  But I've never needed it
when dealing with JSON data.  The JSON data I've dealt with never had
more than one layer of structure repetition, while XML data I've dealt
with had many.

> //"buffers"/-1/.

This one's a pain as jq doesn't do negative indices, so you have -1 the length.

> "buffer"/./"key"

See above.

Nico
--

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Mon, 10 Feb 2014 13:10:20 -0500

> This one's a pain as jq doesn't do negative indices, so you have -1 the

One area I would very much like to standardize on is limits like the above.
I write my code so that the decoder can cope with messages of any size and
arbitrary depth, up to the maximum that the machine can take. But that is
obviously neither needed nor acceptable for a production machine.

Someone writing a JSON spec should specify what their limits are so that a
parser implementation can reject DoS attacks early and block future
requests from that IP.

It is also an efficiency consideration in C. If I know that no request will
ever be more than 64KB long or be nested more than 10 deep, I can
pre-allocate all the necessary structures along with the socket and they
can be cleared and reused or freed as one unit depending on the load of the
server at the time. Avoiding multiple malloc calls is a good way to make a
server fly.

---

From: John Cowan <[email-masked]>
Date: Mon, 10 Feb 2014 13:13:22 -0500

> .buffer.size

Not very general.  What happens when the key is ".foo"?

-- 
Normally I can handle panic attacks on my own;   John Cowan <[email-masked]>
but panic is, at the moment, a way of life.      http://www.ccil.org/~cowan
                --Joseph Zitt

---

From: Nico Williams <[email-masked]>
Date: Mon, 10 Feb 2014 12:29:01 -0600

> Not very general.  What happens when the key is ".foo"?

[This is an FAQ for jq.]

Then you use .[".foo"].

.name is convenient when name is identifier-like, else you must use
.[value], where value would be a string.

Nico
--