Subject: [Json] JSON Schema

From: Henry Andrews <[email-masked]>
Date: Sat, 20 Jan 2018 13:25:24 -0800

Hi folks,
  I'm one of the JSON Schema draft editors, and it's been brought to our
attention that the JSON Schema project may fit within this working group
(or a successor?  I'm a little confused as to the current status and scope
of this group).  We are definitely interested in having the project adopted
by an IETF working group and eventually reaching RFC status if possible.

  To date we have not felt like the project is quite ready for that step,
although I don't personally have a good feel for the criteria.  Since
October 2016, Austin Wright and I, with a great deal of behind-the-scenes
help from Ben Hutton and others, have published three drafts of JSON Schema
Core and Validation, four of Hyper-Schema, and two of Relative JSON
Pointer.  We are actively working on the next Core and Validation drafts
right now.

  In addition to the obvious JSON Schema community engagement, we are also
working directly with the Open API technical steering committee to resolve
the problems that led them to adopt an almost-but-not-quite-compatible form
of JSON Schema.  I feel like they are a good proxy for general adoption
problems as they have a large community of their own, and their concerns
arise from common use cases that are not quite well-served at this time
(particularly code generation).  We've also engaged a bit with folks using
JSON-LD who want to figure out how to leverage both, although that idea is
very embryonic at this time.

  We're currently working on resolving some major questions of scope for
the project overall and the Core and Validation specifications in
particular (a lot of it related to how use cases such as code generation
fit with the existing targeted use cases).  I hope to resolve these in the
next major draft before the current ones expire.  There may be a minor
bug-fix in the next week or so, separate from that effort, and one
additional scope question may be deferred to another draft beyond that.

  Once the scope is settled, I had planned to actively look into working
group status for the core and validation specifications.  Hyper-Schema
requires more work and at least a few viable implementations.  Relative
JSON Pointer could probably move to working group status as well if it is
of interest (we've also considered merging it into JSON Schema Core if
there is no interest in it as its own RFC: JSON Path is not a good fit for
our needs despite being a great idea in its own right, and we already make
extensive use of JSON Pointer).

  I believe that the ongoing broad use and implementation of JSON Schema
despite a multi-year abandonment of the draft work shows that it is a very
viable and useful technology despite the complaints around the edges (I
noticed that the complaint on this mailing list from 2016 was also about
code generation- as I mentioned, that's a current focus of ours to
resolve).  We've started to see implementations of the new drafts in use,
and interest in the latest draft picked up even more quickly than the one
before, so there seems to be real interest in taking JSON Schema over the
finish line into some sort of official standard. The current team is
committed to seeing this through. The recent change in submitting editor is
just a reflection of who had the most time; the core team members have all
remained involved since draft publication resumed.

  I'm interested in any feedback or suggestions on the path forward.

thanks,
-henry

-- 

   -

   *Henry Andrews*  |  Systems Engineer
   [email-masked]
   <https://www.cloudflare.com/>

   1 888 99 FLARE  |  www.cloudflare.com
   -

---

From: Tim Bray <[email-masked]>
Date: Sat, 20 Jan 2018 13:49:54 -0800

Could you give pointers to current drafts?  I had a horrible experience
with JSON Schema (V2 I think? Whatever's current implemented) - the
implementations were inconsistent with each other, the error messages out
of the validators were simultaneously verbose and unhelpful, and I was
looking at it trying to figure how I'd get better-quality messages, it
seemed sort of intrinsically difficult. I would want to see existence proof
of a validator that produced high-quality error messages before I could
really get behind a design.

I ended up writing my own schema language to produce a user-friendly
validator, but it's on the eccentric side and I doubt of interest to the
broader community:
https://www.tbray.org/ongoing/When/201x/2016/12/01/J2119-Validator

Having said that, I think work on a schema language for JSON might be
welcomed in the IETF, given a good draft to start with and an editor or
editors who were interested in doing things the IETF way.  It's not that
lightweight, we'd have to re-charter the working group, but far from
impossible.

On Sat, Jan 20, 2018 at 1:25 PM, Henry Andrews <[email-masked]> wrote:

> the problems that led them to adopt an almost-but-not-quite-compatible
> form of JSON Schema.  I feel like they are a good proxy for general
> adoption problems as they have a large community of their own, and their
> concerns arise from common use cases that are not quite well-served at this
> time (particularly code generation).  We've also engaged a bit with folks
> using JSON-LD who want to figure out how to leverage both, although that
> idea is very embryonic at this time.
>
> _______________________________________________
> json mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/json
>
>

-- 
- Tim Bray (If you’d like to send me a private message, see
https://keybase.io/timbray)

---

From: Henry Andrews <[email-masked]>
Date: Sat, 20 Jan 2018 14:13:04 -0800

Hi Tim,
  The current draft is draft-07 (although the actual IETF numbering is
complicated).  So draft-02 was a very long time ago :-)

You can find links to nice HTML renderings of all of the drafts, as well as
links to the official ietf.org hosted renderings, on our site at
http://json-schema.org/ (yes, we know we need to move to HTTPS, there's a
long difficult story there that is finally showing signs of resolution).
You will also find some guides to migrating to recent drafts and a bit of
explanation of the history, although we could use more/improved content on
those topics.  You will also find links to implementations, with notes
about which drafts are supported- draft-06 (April 2017) has been
implemented in several languages, and we're starting to see a few draft-07
(November 2017) implementations as well.

For discussions of improving or standardizing output (error or otherwise),
follow the "output" tag on the GitHub issues:
https://github.com/json-schema-org/json-schema-spec/issues?q=is%3Aissue+is%3Aopen+label%3Aoutput
While not an immediate focus, it is obviously on the list of concerns.  In
my view it is the responsibility of the spec to ensure that it is
*possible* to provide good output, but the fact that some implementation
fails to do so despite the possibility is not a reason to drop the spec.

Note that for the common complaint of difficult error messages with
"oneOf", we introduced "if"/"then"/"else" keywords in draft-07 (after
extensive debate as to whether they were "too imperative"- in the end the
community demand was extremely high, and the capabilities are no different
than with "oneOf", it's just easier to read and report errors with
"if"/"then"/"else" in many use cases).

thanks,
-henry

On Sat, Jan 20, 2018 at 1:49 PM, Tim Bray <[email-masked]> wrote:

> broader community: https://www.tbray.org/ongoing/When/201x/2016/12/
> 01/J2119-Validator
>
>>   In addition to the obvious JSON Schema community engagement, we are
>> also working directly with the Open API technical steering committee to
>> resolve the problems that led them to adopt an
>> almost-but-not-quite-compatible form of JSON Schema.  I feel like they
>> are a good proxy for general adoption problems as they have a large
>> community of their own, and their concerns arise from common use cases that
>> are not quite well-served at this time (particularly code generation).
>> We've also engaged a bit with folks using JSON-LD who want to figure out
>> how to leverage both, although that idea is very embryonic at this time.
>>

-- 

   -

---

From: "Paul Hoffman" <[email-masked]>
Date: Sat, 20 Jan 2018 14:59:16 -0800

On 20 Jan 2018, at 13:25, Henry Andrews wrote:

> of this group).

The WG is closed and thus has no charter. If you want an IETF WG for 
JSON Schema, it would need to be a new WG. The new WG could be chartered 
just to work on JSON Schema, and not every other JSON-y idea that comes 
by.

> complicated).  So draft-02 was a very long time ago :-)

Could you point to the actual documents you are talking about? I see 
draft-handrews-json-schema-00, which is not at -07,

--Paul Hoffman

---

From: Henry Andrews <[email-masked]>
Date: Sun, 21 Jan 2018 14:11:11 -0800

The full set of current documents is on the web page:

http://json-schema.org/specification.html

The first set of links (in the table) go to locally hosted renderings of
each document.
Beneath that, there are links to the IETF-hosted documents.

If you want to see why the numbering is so incredibly confusing, there's a
"Specification LInks" page linked under "Older Drafts"

-henry

On Sat, Jan 20, 2018 at 2:59 PM, Paul Hoffman <[email-masked]> wrote:

> The WG is closed and thus has no charter. If you want an IETF WG for JSON
> Schema, it would need to be a new WG. The new WG could be chartered just to
> work on JSON Schema, and not every other JSON-y idea that comes by.
>

-- 

   -

---

From: Rob Sayre <[email-masked]>
Date: Sun, 21 Jan 2018 14:21:50 -0800

Hi all.

There are much more efficient and unambiguous formats available if there's
a schema at hand.

thanks,
Rob

---

From: Henry Andrews <[email-masked]>
Date: Sun, 21 Jan 2018 18:08:12 -0800

I'm not sure I follow.  Are you talking about an alternative?  If so could
you provide a name or a link?  Or are you asking for the JSON Schema spec
in a different format?

On Sun, Jan 21, 2018 at 2:21 PM, Rob Sayre <[email-masked]> wrote:

>> If you want to see why the numbering is so incredibly confusing, there's
>> a "Specification LInks" page linked under "Older Drafts"
>>
>> -henry
>>
>>
>>> just to work on JSON Schema, and not every other JSON-y idea that comes by.
>>>

-- 

   -

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Sun, 21 Jan 2018 21:25:31 -0500

+1

I have been designing protocols for 25 years and I have never come across a
requirement to specify a minimum number of items in a set that is other
than 0 or 1 and never come across a reason to specify a maximum other than
1 or infinity.

When I have come across other limits, I have pretty much always found them
to be wrong. Take the requirement to have two authoritative servers for a
DNS zone (not enforced by the protocol but enforced by the social
infrastructure around it). That seemed such a good idea to me till I found
the two authoritative servers for the MIT LCS/AI running on a couple of
sparc stations plugged into the same wall socket.

The less twiddles and curlicues, the better.

Validation is not useful for a data schema. It is useful for a document
schema because it allows an intelligent editor to tell the user if they are
filling in what really amounts to a form correctly.

My other objection is to the use of JSON syntax and the limitation to JSON
encoding. I don't see a reason to limit scope to one encoding. I write my
systems using a schema language that is at least in principle capable of
targeting ASN.1 and XML as well as JSON. This approach only allows me a
subset of the capabilities of ASN.1 and XML schema which is exactly the
point.

---

From: Henry Andrews <[email-masked]>
Date: Sun, 21 Jan 2018 18:48:58 -0800

So... the point of me sending this email was to answer a request made to
the JSON Schema project that I investigate the working group.  The clear
answer to that is that this working group is closed, and a new one would
need to be chartered.  It's not clear to me that we would want to charter
such a group now, although I would be interested in anyone's input on that.

Otherwise this discussion has turned into various people raising complaints
about JSON Schema who have clearly not followed the recent developments on
the project, or just clearly want something entirely different.  For those
complaining about outdated concepts (failure to consider data definition,
which is actually a huge focus right now, or concern over a required json
encoding, which no longer exists- the spec works on a data model derived
from JSON, not on JSON itself, or the behavior of implementations that are
five drafts out of date) I don't think a closed working group mailing list
that did not have JSON Schema in its scope to begin with is the appropriate
place to deal with these complaints.  Nor does it seem to be the
appropriate place for me to educate everyone on the past few years of work.

For those of you who want something else entirely, that's great, there are
lots of good ideas out there.  I wish you well.  There are several projects
underway in the same general space, some with quite a bit of momentum of
their own.  One of them may be a better choice for an IETF working group,
I'm not in a position to say one way or the other.

For those of you who would like to take a look at the current state of the
project, outstanding topics and debates, etc., please come on over to
https://github.com/json-schema-org/json-schema-spec or join our slack
channel with this invite:

https://join.slack.com/t/json-schema/shared_invite/enQtMjk1NDcyNDI2NTAwLTcyYmYwMjdmMmUxNzZjYzIxNGU2YjdkNzdlOGZiNjIwNDI2M2Y3NmRkYjA4YmMwODMwYjgyOTFlNWZjZjAyNjg

Filing issues or asking questions on our slack channel will probably work
better than hi-jacking this mailing list, now that I understand it is no
longer active. :-)

I would be interested in pursuing the IETF process (heavyweight or not), if
there is interest from the appropriate segment of the IETF community.
However, if the reaction is primarily hostile, or just indifferent with
preferences for other projects, that is fine.  We can work through another
standards body or pursue some other path, or if our own community moves on
to other ideas we can just wind it down.  I have no interest in forcing
anyone to support JSON Schema- it stands or falls on its own.  Even without
a formal standardized specification, JSON Schema is broadly used and
survived several years of abandonment, so whatever other worthy ideas might
be out there, we have a community to support either way, and that is my
primary focus right now.

thanks,
-henry

On Sun, Jan 21, 2018 at 6:25 PM, Phillip Hallam-Baker <[email-masked]>
wrote:

> +1
>
> I have been designing protocols for 25 years and I have never come across
> a requirement to specify a minimum number of items in a set that is other
>> There are much more efficient and unambiguous formats available if
>> there's a schema at hand.
>>
>> thanks,
>> Rob
>>
>>

-- 

   -

---

From: Henry Andrews <[email-masked]>
Date: Sun, 21 Jan 2018 18:51:59 -0800

Note that this is all my personal opinion.  I'm the de-facto spokesperson
for JSON Schema mostly because I have the most time available, but I'm not
necessarily the ideal person to be running things.  So take my views with a
grain of salt.

On Sun, Jan 21, 2018 at 6:48 PM, Henry Andrews <[email-masked]> wrote:

> Otherwise this discussion has turned into various people raising
> complaints about JSON Schema who have clearly not followed the recent
> developments on the project, or just clearly want something entirely
> different.  For those complaining about outdated concepts (failure to
> consider data definition, which is actually a huge focus right now, or
> concern over a required json encoding, which no longer exists- the spec
> works on a data model derived from JSON, not on JSON itself, or the
> behavior of implementations that are five drafts out of date) I don't think
> a closed working group mailing list that did not have JSON Schema in its
> scope to begin with is the appropriate place to deal with these
> complaints.  Nor does it seem to be the appropriate place for me to educate
> everyone on the past few years of work.
>
> https://join.slack.com/t/json-schema/shared_invite/
> enQtMjk1NDcyNDI2NTAwLTcyYmYwMjdmMmUxNzZjYzIxNGU2YjdkNzdlOGZi
> NjIwNDI2M2Y3NmRkYjA4YmMwODMwYjgyOTFlNWZjZjAyNjg
>
> I would be interested in pursuing the IETF process (heavyweight or not),
> if there is interest from the appropriate segment of the IETF community.
> On Sun, Jan 21, 2018 at 6:25 PM, Phillip Hallam-Baker <
> [email-masked]> wrote:
>
>> +1
>>
>> When I have come across other limits, I have pretty much always found
>> them to be wrong. Take the requirement to have two authoritative servers
>> for a DNS zone (not enforced by the protocol but enforced by the social
>> My other objection is to the use of JSON syntax and the limitation to
>> JSON encoding. I don't see a reason to limit scope to one encoding. I write
>> my systems using a schema language that is at least in principle capable of
>>>> The first set of links (in the table) go to locally hosted renderings
>>>> of each document.
>>>> If you want to see why the numbering is so incredibly confusing,
>>>> there's a "Specification LInks" page linked under "Older Drafts"
>>>>
>>>> -henry
>>>>
>>>>

-- 

   -

---

From: "Paul Hoffman" <[email-masked]>
Date: Sun, 21 Jan 2018 19:53:10 -0800

A few procedural notes to correct some misunderstandings. (Wearing no 
hats, just lots of scars from earlier IETF work on JSON and lots of 
other data formats...)

On 21 Jan 2018, at 18:48, Henry Andrews wrote:

> need to be chartered.

If you want an IETF standard, yes.

> It's not clear to me that we would want to charter

There appears to be interest in something around schema for JSON; it is 
not clear if there is interest in starting from the specific spec at 
https://github.com/json-schema-org/json-schema-spec. I bet there is some 
interest in that, and some interest in something else. This is quite 
typical of requests for the IETF to adopt work that originated outside 
the IETF, with mixed results in any of of "adopt with the intention of 
making only minor changes", "adopt and make major changes", and "start 
from scratch while getting inspiration from the existing work".

The IETF sucks at this, as do most other SDOs. We're just more public 
about our suckage.

> There are several projects

None of us are; that's what consensus is for.

> better than hi-jacking this mailing list, now that I understand it is 

What you have done is *not* hijacking this mailing list: it's a 
completely appropriate use of it. And the list is active, as you have 
just seen. It's just not an IETF WG any more.

> I would be interested in pursuing the IETF process (heavyweight or 
> not)

It will be the former :-(

> , if

This list is a good proxy for that appropriate segment. Everyone who is 
interested in this topic should feel free to use the mailing list for 
it.

--Paul Hoffman

---

From: Henry Andrews <[email-masked]>
Date: Sun, 21 Jan 2018 20:23:43 -0800

Thanks, Paul, this is very helpful.

My inclination for now is for JSON Schema to continue on its current course
as we are.  I do not think the current set of drafts are the right base to
start from.  Again, this just my personal view, but I'd like to work within
the JSON Schema community on its own more to solve some of the most obvious
problems before throwing it in a blender with other proposals.  I had not
intended on contacting the IETF about a working group for at least two more
drafts- I only started this thread due to a request.

There is also the question of whether I would be the right point person for
bringing JSON Schema into a working group.  I'm not sure I would be, so if
someone else steps up in the meantime or wants to start working on some
schema system and use JSON Schema as it is as one input, I would be fine
with that.  Honestly, nearly every comment from anyone (here or elsewhere)
that I've heard about the working group process has, um... not really
excited me about it.

So I think I'll keep doing what I'm doing, and note that I'm happy to help
if someone else wants to lead a schema working group of some sort.  If no
one steps up, I may eventually do so.  Otherwise I need to spend some time
thinking about which paths forward would be best for the JSON Schema
community.  And if a schema working group emerges and goes in a different
direction, I can look at how to help the JSON Schema community transition
towards that.

thanks,
-henry

On Sun, Jan 21, 2018 at 7:53 PM, Paul Hoffman <[email-masked]> wrote:

> hats, just lots of scars from earlier IETF work on JSON and lots of other
> data formats...)
>
> typical of requests for the IETF to adopt work that originated outside the
> IETF, with mixed results in any of of "adopt with the intention of making
> only minor changes", "adopt and make major changes", and "start from
> scratch while getting inspiration from the existing work".
>
> What you have done is *not* hijacking this mailing list: it's a completely
> appropriate use of it. And the list is active, as you have just seen. It's
> just not an IETF WG any more.
>
> I would be interested in pursuing the IETF process (heavyweight or not)
>>
>
> interested in this topic should feel free to use the mailing list for it.
>
> --Paul Hoffman
>

-- 

   -

---

From: Henry Andrews <[email-masked]>
Date: Sun, 21 Jan 2018 20:27:07 -0800

hmm- there's a bit in that last message that could be read as me assuming
that I would lead a working group if I were interested in doing so.  I am
not making that assumption :-D

On Sun, Jan 21, 2018 at 8:23 PM, Henry Andrews <[email-masked]> wrote:

> My inclination for now is for JSON Schema to continue on its current
> course as we are.  I do not think the current set of drafts are the right
> base to start from.  Again, this just my personal view, but I'd like to
> work within the JSON Schema community on its own more to solve some of the
> most obvious problems before throwing it in a blender with other
> proposals.  I had not intended on contacting the IETF about a working group
> for at least two more drafts- I only started this thread due to a request.
>
> There is also the question of whether I would be the right point person
> for bringing JSON Schema into a working group.  I'm not sure I would be, so
> if someone else steps up in the meantime or wants to start working on some
>> completely appropriate use of it. And the list is active, as you have just
>> seen. It's just not an IETF WG any more.
>>

-- 

   -