Subject: [Json] Another problematic JSON Schema use-case

From: Tim Bray <[email-masked]>
Date: Sun, 22 May 2016 11:15:12 -0700

See https://www.tbray.org/ongoing/When/201x/2016/05/22/Json-Schema-Gripe

---

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Mon, 23 May 2016 15:41:26 +0000

>See https://www.tbray.org/ongoing/When/201x/2016/05/22/Json-Schema-Gripe

The CDDL I’m currently using for one part of draft-hildebrand-deth is:

```
update = A_update / AAAA_update / ...

A_update = {
  rtype: "A",
  v4address: tstr,
  common
}

AAAA_update = {
  rtype: "AAAA",
  v6address: tstr,
  common
}

common = (
  ? ttl: uint,
  ? comment: tstr,
  ? uid: tstr
)

```

The discriminant is the “rtype” field, and then each rtype has a set of common optional fields, and a set of required rtype-specific fields.

When you pass in this JSON: `{"rtype": "AAAA"}`

The CDDL tooling gives this error:

```
CDDL validation failure (nil for {"rtype"=>"AAAA"}):
["AAAA", [:prim, 3], nil]
```

Which is a little esoteric, granted, but when deciphered says that the “AAAA” key is required, but wasn’t there.

-- 
Joe Hildebrand

---

From: Austin William Wright <[email-masked]>
Date: Mon, 23 May 2016 18:30:15 -0700

This is a problem that I tried working on a year or two ago, but couldn't
find any good resolution to.

Let's create a JSON Schema for the described problem (slightly
modified/expanded for illustration below):

{
"type": "object",
"additionalProperties": {
"type": "object",
"properties": {"Type":{"enum":["Animal", "Vegetable", "Mineral"]}},
"required": ["Type"],
"oneOf": [
{
"properties": {
"Type":{"enum":["Animal"]},
"Thermoregulation":{"enum":["Endotherm", "Ectotherm"]}
},
"required": ["Thermoregulation"]
},
{
"properties": {
"Type":{"enum":["Vegetable"]},
"Organic":{"type":"number"}
},
"required": ["Organic"]
},
{
"properties": {
"Type":{"enum":["Mineral"]},
"Organic":{"type":"boolean"}
},
"required": ["Organic"]
}
]
}
}

Here's a valid instance for illustration:

{
"o1": {
"Type": "Animal",
"Thermoregulation": "Endotherm",
"Species": "Felis catus",
"Label": "cat"
},
"o2": {
"Type": "Mineral",
"Organic": false,
"Label": "Quartz"
}
}

This, however, is an invalid instance:

{
"o3": {
"Type": "Vegetable",
"Endotherm": true,
"Label": "Silver"
}
}

What sort of error would be appropriate here? (E.g. Is this a typo for the
"Type" property? Or is this just missing a required "Organic" property?)

When I run it through my validator, I get this error:

(1 of 1) instance.o3 is not exactly one from <http://example.org/Animal>,<
http://example.org/Vegetable>,<http://example.org/Mineral>

And within all three schemas, exactly one constraint fails. So which one do
we suggest errors for? Changing 'Vegetable' to 'Animal', and changing
'Endotherm' to 'Organic' are all equally reasonable (to a computer)
one-word fixes.

In general, JSON Schema and many other schemas are designed to ensure
things are valid. They're not well engineered to help people fix things
when they're invalid.

A similar problem is that localizing error strings. There's a few
suggestions to add custom error strings to JSON Schema, but then we have
the problem that they're not localized. If we allow multiple strings (one
per localization), then we have to update the schema every time we add a
locale, which means changing the file even though nothing about the schema
has changed.

I just created <
https://github.com/json-schema-org/json-schema-spec/issues/31> to look at
this problem as it relates to JSON Schema.

Cheers,

Austin Wright.

On Sun, May 22, 2016 at 11:15 AM, Tim Bray <[email-masked]> wrote:

> _______________________________________________
> json mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/json
>
>

---

From: Tim Bray <[email-masked]>
Date: Mon, 23 May 2016 20:54:33 -0700

Hmm, I have to excerpt one line here:

> things are valid. They're not well engineered to help people fix things

​I find this troubling.  I expect modern compilers to give me highly
actionable errors, with line numbers. Is the nature of JSON DSL’s and
programming languages so deeply different that this is not a reasonable
thing to ask a schema processor to do?

If so, then what are schemas for?  I’ve seen them as having two uses:

1. Specification aids
2. Debugging aids

For #1, I see schemas as a poor second-best to clear human-readable
description.
For #2, Schemas are appealingly declarative and presumably much more
compact than the equivalent declarative code.  But if they can’t tell me
what the problem is, why bother?  And furthermore, I’ve never thought
validators could rely on schemas alone, because schemas can’t check
semantic constraints (“is that a valid part number?”)

---

From: Austin William Wright <[email-masked]>
Date: Tue, 24 May 2016 18:07:16 -0700

I also use them as specification aids, I think JSON Hyperschema is a great
tool to make JSON APIs become RESTful (because it's self-documenting).

I also use them for validation and sanity checking, like an assert() call:
It's not something that should ever trip, but it's there in case the
impossible happens (but that would never happen, right?).

I agree with your opinion it's troubling, though. Unfortunately it's hard
to strike a balance between making something that's easy to compose and
easy to read, versus something that's expressive and gives you exactly the
right error message.

I'd love to talk how to fix this problem, though!

It seems like this problem is closely tied with automatic UI generation.
How would I design a form to enter data of the kind that you're expecting?
Three separate forms? A master drop-down that automatically changes the
remaining fields and automatically populates the "Type" one? Move the
"Type" drop-down first, and change all the other fields if it changes? What
if I want to change the type of an entity from Vegetable to Plant? It seems
if we can answer these questions, we would also tackle your validation
errors problem for free.

Austin.

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Tue, 24 May 2016 21:26:06 -0400

> if we can answer these questions, we would also tackle your validation

Funny you should say that. This past couple of weeks I have resurrected the
GUI generator I built for X-Windows before working on the Web. [The new
version targets GTK as primary and will probably do Windows Forms and HTML
in due course]

Yes, there are similarities between the problems. In both cases you have an
abstract model and a presentation of that model, a View for the UI and a
serialization for JSON. Yes, if one had the problem of producing both, you
could share a lot of code. But not so much as you might expect.

A GUI is a Model-View-Controller app and unlike the problem of
serializing/deserializing JSON, you have to do a lot of validation on the
inputs. In particular you need to be able to validate a dialog box when the
user hits 'Accept' and tell them where they went wrong. You also need to be
able to support 'cancel'.

In short, it ends up as a very much more involved problem if you are going
to do it right. XML is as complex as it is because it supports features
that allow text editors to validate input as people type. And JSON is
simpler because it does not support those features. And if you try to turn
JSON into XML you will probably end up with something almost as complex.

---

From: Andrew Newton <[email-masked]>
Date: Wed, 25 May 2016 02:52:09 -0400

On Mon, May 23, 2016 at 11:54 PM, Tim Bray <[email-masked]> wrote:
>> things are valid. They're not well engineered to help people fix things when
>> they're invalid.
>
>
> I find this troubling.  I expect modern compilers to give me highly

It's not too much to ask, but the problem may be with the JSON parser.
Many parsers ingest JSON and give back native data structures, which
is what makes JSON simple for most people. But a validator needs
context that a typical JSON parser does not provide.

> If so, then what are schemas for?  I’ve seen them as having two uses:

While not typical, being malleable enough to aid the creation of
testing harnesses is a great third use.

> For #1, I see schemas as a poor second-best to clear human-readable

In my experience, many non-English readers greatly benefit from a
formal syntax. And depending on the complexity of the problem, prose
cannot strike the right balance between precision and TLDR.

> For #2, Schemas are appealingly declarative and presumably much more compact
> than the equivalent declarative code.  But if they can’t tell me what the
> problem is, why bother?  And furthermore, I’ve never thought validators
> could rely on schemas alone, because schemas can’t check semantic
> constraints (“is that a valid part number?”)

Certainly they are no replacement for application logic, but I don't
think anybody ever made that claim.

-andy

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Wed, 25 May 2016 14:19:13 -0400

The core problem with 'JSON Schema' is that either you like strongly typed
languages or you don't.

I have some very strong opinions on the advantages of strong typing. My
college tutor was Tony Hoare. But a lot of people don't see the point just
as a lot of folk in the nuclear physics lab used to turn off compiler
warnings to 'get rid of' their errors.

If you don't like strong typing then you probably won't see the value of a
schema language because you will be using languages that allow you to write
something like this

var DOM = JSON.Parse (Textfile);

Where JSON.Parse is a completely generic function that dumps out a tree
that you can then access through dynamic binding.

Dynamic binding certainly has advantages but it has what people from my
camp consider to be the serious disadvantage that you can't do static type
checking at compile time.

It also has the disadvantage that you end up with code that looks like this:

struct Furniture {
    string: Name;
    string: Type;
    }

Which serializes as

"Furniture" : { "Name" : "Aeron", "Type" : "Chair" }

Which is fine, if you don't want to use an object oriented language where
you would write something like:

class Furniture {
    string: Name;
    }

class Chair : Furniture {
    }

Which would typically serialize as :

"Chair" : {Name: "Aeron"}

If you design a protocol with strong typing in mind it is highly unlikely
that you are going to have problems using dynamic binding as a result. But
the reverse is not the case.

Since I do security, I insist on static type checking. Perl is fine for
writing a Web site mashup. It isn't something I would want to see used on a
CA.

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 25 May 2016 20:26:38 +0200

> context that a typical JSON parser does not provide.

Can you give an example for that?

(I'm familiar with a problem like this from CBOR, where CDDL maybe
sometimes wants to make serialization choices*), which are no longer
visible at the data model level; I just haven't seen the need with JSON.)

Grüße, Carsten

*) such as float16 vs. float32 vs. float64.

---

From: Andrew Newton <[email-masked]>
Date: Thu, 26 May 2016 09:00:27 +0200

> Can you give an example for that?

Given the following:

[
  { "foo" : "bar" }
]

Here is what happens if I use the built-in Ruby parser:

irb(main):017:0> s
=> "\n[\n  { \"foo\":\"bar\"}\n]\n"
irb(main):018:0> JSON.parse s
=> [{"foo"=>"bar"}]

If a validator wants to ensure that "foo" is "baz" and not "bar", its
lost the information to tell the user to look at line 2.

By contrast, if a validator were to use Jackson as a stream parser, it
would have access to JsonLocation:
http://fasterxml.github.io/jackson-core/javadoc/2.1.0/com/fasterxml/jackson/core/JsonLocation.html

> visible at the data model level; I just haven't seen the need with JSON.)

I'm not sure we are talking about the same thing.

-andy

---

From: Tim Bray <[email-masked]>
Date: Thu, 26 May 2016 08:56:06 -0700

So, to close the loop, here's how I worked around the problem. I
auto-processed my large schema with a Ruby script to generate a small
schema for each of the "Type":"Animal/Vegetable/Mineral" variations, then
if the large schema reports any errors, I run through the objects
validating each with the appropriate micro-schema.  This way, I almost
always get a human-comprehensible error message.

Not optimal, but better than writing a fully procedural validator by hand.

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 26 May 2016 18:04:20 +0200

Tim Bray wrote:
> schema for each of the "Type":"Animal/Vegetable/Mineral" variations,
> then if the large schema reports any errors, I run through the objects

>From a grammar validation point of view, you believe the type field more
than the other fields, so if there is a conflict between them, the type
field wins and the other fields can thus be flagged as wrong.

In a definite clause grammar (DCG), cuts can be used to elicit similar
behavior.  There are proposals for cut points in parse expression
grammars (PEGs); maybe that is a good subject for further evaluation.

Grüße, Carsten

---

From: John Cowan <[email-masked]>
Date: Thu, 26 May 2016 13:37:20 -0400

Phillip Hallam-Baker scripsit:

> I have some very strong opinions on the advantages of strong typing.

Even if you accept the principle of strong typing, that is not conclusive
as to how many atomic types you have.  Most programming languages have
only one string type, or at most have a family of types varying only by
length or maximum length.  But it would be possible to declare the type
of a string using a regular expression, and Cobol approximates this with
its PICTURE types.

On the other hand, statically typed languages typically have multiple
types of numbers: C has 14, Java has 7, Pascal has 2, and Algol 68 has
a denumerably infinite number (short int, short short int, short short
short int, etc.) all statically distinct, though normally only a finite
number of distinct run-time representations.

If we stick to JSON's mutually exclusive atomic types of null, boolean,
number, string, we can devise a very simple schema language for static
typing only.  Most of the complexity comes from wanting to provide more
complex static types such as integer vs. non-integer, exact vs. inexact,
or highly constrained strings.

===========

Here's a sketch of a simple schema-by-example language:

The type 'any JSON number' is encoded by any JSON number.

The type 'any JSON boolean' is encoded by any JSON boolean.

The type 'any JSON string' is encoded by any JSON string not specially
mentioned here.

The type 'JSON array of fixed size with fixed types' is encoded by
a JSON array whose elements are the encodings of the fixed types.
This corresponds to a tuple in some statically typed languages.

The type 'JSON array of variable size with fixed element type' is encoded
by a JSON array whose first element encodes the element type and whose
second element is the string "...".  This corresponds to a list in some
statically typed languages.  It can be combined with the last rule to
allow arrays whose first few elements have different element types,
and any additional elements have the same type.

The type 'arbitrary JSON array' is encoded by the JSON string "[]".

The type 'JSON object with fixed keys' is encoded by a JSON object
whose keys are the fixed keys and whose values encode the types of their
corresponding keys.

The type 'JSON object with arbitrary keys' is encoded by the JSON string
"{}".

The type 'arbitrary JSON value' is encoded by the JSON string "*".

Open questions:

1) What about a JSON object with fixed keys that also allows arbitrary
additional keys?

2) What about null?  It can be treated as the sole inhabitant of a unique
atomic type encoded by a JSON null, or as something not mentioned in
schemas that is always allowed in place of any JSON value (like SQL null),
or as a replacement for arrays and objects only.

-- 
John Cowan          http://www.ccil.org/~cowan        [email-masked]
                if if = then then then = else else else = if;

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Thu, 26 May 2016 13:56:36 -0400

> or highly constrained strings.

I don't really consider different types of integer to be different types.
They are different representations of whole numbers, they are all
intrinsically the same type.

There is a similar error in Pascal which in the ANSI standard has the
ludicrous insistence that int [2] is a different type to int [3]. And there
is no cast operator. Small wonder people used to refer to it as being
'Wirthless'

Yes, real numbers and integers are probably things you want to be able to
treat differently in code because it is very very rare to need or want a
real64 number in a protocol.

I have no problem with limiting the intrinsic types to the set given in
JSON. What I mean by strong typing is that when I convert a JSON data
stream I convert it to a data model that has classes and structures that
are in the application data model. So I write code of the form:

Profile.Applications.Add (MailApplication);

Rather than something like

Tree.Append (Profile, "Profile.Applications", MailApplication.Tree());

Yes, if you use a fully dynamic binding language, you can write code that
looks like the first example. But what you don't get in that situation is
static type checking on your code. My code sill throw a compiler error if I
try to add a DeviceProfile in a slot typed as a list of ApplicationProfile.

Mostly agree as a starting point. But I think that you absolutely want to
be able to put binary blobs of data in a stream and distinguish them from
strings.

I have spec, running code and apps written in it.

> 1) What about a JSON object with fixed keys that also allows arbitrary

That is an interesting one because it comes down to how do you want
extensibility to work. In particular, when do you want adding things to be
backwards compatible and when do you want to cause things to halt rather
than have an application try to act on data it did not fully understand?

> or as a replacement for arrays and objects only.

This is actually a little problematic as in JSON any slot can hold null
while many languages don't allow integers or booleans to be null.

C# has nullable types for integers and booleans. But lots of languages
don't. I am very wary of writing protocol logic that would depend on such a
distinction.

---

From: John Cowan <[email-masked]>
Date: Thu, 26 May 2016 16:50:33 -0400

> 'Wirthless'

"You may call me by value, and call me 'Worth', or you may call me by
name, and call me 'Veert'."  --Wirth on how to say his name.

> Yes, real numbers and integers are probably things you want to be able
> to treat differently in code because it is very very rare to need or
> want a real64 number in a protocol.

Okay, an integer encodes the type "JSON number which is an integer",
and a non-integer encodes the type "JSON number".  Note that this might
be represented as a Java BigDecimal or equivalent.

> But I think that you absolutely want to be able to put binary blobs of
> data in a stream and distinguish them from strings.

Good idea.  If the string is "deadbeef", it encodes the type of binary
blobs represented in JSON as a string. :-)

> extensibility to work. In particular, when do you want adding things
> to be backwards compatible and when do you want to cause things to
> halt rather than have an application try to act on data it did not
> fully understand?

I think you want to be able to choose, but I'm not sure how to represent
it within the JSON-by-example format.  The obvious way is to reserve a
key like "...", but I haven't wanted to do that.

> This is actually a little problematic as in JSON any slot can hold
> null while many languages don't allow integers or booleans to be null.

True.  We can say then that null is always permitted wherever the schema
allows an array or object.

-- 
John Cowan          http://www.ccil.org/~cowan        [email-masked]
How they ever reached any conclusion at all is starkly unknowable
to the human mind.        --"Backstage Lensman", Randall Garrett

---

From: Peter Cordell <[email-masked]>
Date: Thu, 26 May 2016 22:27:30 +0100

> Here's a sketch of a simple schema-by-example language:

I think sort of thing in JCR by having a directive such as
#infer-types, e.g.

     #infer-types
     { "foo" : "bar" }

'foo' is then any string, not just "bar".

If that is too intrusive, you could have a configuration option to the 
JCR parser to select similar type inferring.

I think that would be useful as 'by-example' is perhaps an easy way to 
start creating a schema, but I feel you'd want to go beyond what it can 
offer within a matter of hours.

Pete.
-- 
---------------------------------------------------------------------
Pete Cordell
Codalogic Ltd
Read & write XML in C++, http://www.xml2cpp.com
---------------------------------------------------------------------

---

From: John Cowan <[email-masked]>
Date: Thu, 26 May 2016 17:45:22 -0400

> I think sort of thing in JCR by having a directive such as

Pretty good.

> I think that would be useful as 'by-example' is perhaps an easy way
> to start creating a schema, but I feel you'd want to go beyond what
> it can offer within a matter of hours.

Remember that I'm aiming only at static typing, not general validation.

-- 
John Cowan          http://www.ccil.org/~cowan        [email-masked]
Promises become binding when there is a meeting of the minds and consideration
is exchanged. So it was at King's Bench in common law England; so it was
under the common law in the American colonies; so it was through more than
two centuries of jurisprudence in this country; and so it is today.
       --Specht v. Netscape

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Thu, 26 May 2016 17:57:51 -0400

> name, and call me 'Veert'."  --Wirth on how to say his name.

The nickname I used was given him by someone just as well known in that
crowd :)

> be represented as a Java BigDecimal or equivalent.

Yes, there are cases where distinguishing representations is useful. This
does not include big nums for crypto in my view, I would prefer those to be
in hex or base64.

I do think it useful to distinguish integers and reals at the schema level
as the treatment is very different. In fact I do that but I have never ever
used that code. If you were doing scientific stuff, being able to represent
numbers as fractions and such like Wolfram does would be very desirable.

> blobs represented in JSON as a string. :-)

Yes, there are a small number of intrinsic types that we use in protocols.
It is slightly larger than JSON gives us but not very much larger.
Specifically:

* Boolean, Integer, String
* Binary  (Base64 (data) in string)
* DateTime (IETF time format in string)
* URI (in string)
* Label (String)
* Reference (string)
* Token (basically a reference but does not require the label to be defined)

I have parser generator tools for pretty much every IETF format (ASN.1, TLS
Schema, XML, DNS, JSON, RFC822). Those are the only ones I have ever found
a need for. (counting OIDs as a subset of URI).

> key like "...", but I haven't wanted to do that.

The way that I am doing it now is that I support both as follows:

* Fields can be added to any data structure. These are simply ignored if
the parser does not understand them.

* If the field label changes, the semantics are different and so the
decoder should abort.

My encodings always give the type of an object as a field name. So if I
have a message Hello, the C#, JSON encoding are:

class Hello {
    string Param1, Param2; }

{ "Hello" : { "Param1" : "Value1", "Param2" : "Value2" } }

If you change the protocol so that you can say who you are saying hello to,
this is backwards compatible :

{ "Hello" : { "Param1" : "Value1", "Param2" : "Value2", "To" : "Bob" } }

But if we want to add in a completely different message that replaces
Hello, we use a different tag:

{ "Hello2" : { "Param1" : "Value1", "Param2" : "Value2" } }

This approach makes it very easy to ensure that an interpreter that does
not understand Hello2 does not attempt to do the right thing and get it
wrong.

This works very well and doesn't give the problems that people are getting
wrapped around their axle when they try to use type fields:

{ "Param1" : "Value1", "Param2" : "Value2", "Type" : "Hello" } }

Why should anyone expect "Type" to be more important than the other fields?
It really doesn't make any sense. It also comes in last which means that if
we have a long message and the type field comes last we have to read the
whole thing before we try to make any sense of it.

*Object type should always be specified in a key, never a value.*

> allows an array or object.

That isn't necessarily what you want though. If you are using a schema,
there are actually four possibilities for a boolean slot 'fred':

true  { "fred" : true }
false  { "fred" : false }
null  { "fred" : null }
empty  {  }

Now you could in theory write a protocol which distinguished between the
four values but I don't think that is very useful at all. I don't think it
is practical to rely on a difference between empty and null.

You could also argue that omitting a value is equivalent to false. But that
is a case where I think you actually should be required to declare it.

---

From: Austin William Wright <[email-masked]>
Date: Fri, 27 May 2016 03:15:29 -0700

> Not optimal, but better than writing a fully procedural validator by hand.
I now recall my earlier solution, which is similar enough to this: If two
objects carry different data, they should be described by different schemas.

Things of type "Plant" should validate against a Plant schema, which
includes (with allOf) a reference to the generic schema.

So instead of the schema I listed in my first post to the thread, you would
use a schema like:

{
    "id": "http://example.com/Plant",
    "allOf": [{"$ref":"http://example.com/Thing"}],
    "properties": {
        "Type": {"enum":["Plant]}
    },
    "required": ["Thing", "Genus", "Species"]
}

This would also go over the wire with the profile="http://example.com/Plant"
media type parameter.

Notice how the allOf acts like the "subclasses" operator in a dynamic
language (like PHP or maybe Ruby; as opposed to C++ which is static or
maybe ECMAScript which is prototypical): A Plant is also a Thing, but a
Thing doesn't necessarily have the data fields found in Plant.

Please let us know if you've considered something like this, and how such a
solution would work for you.

Digging a little more advanced into JSON Schema, there's no reason an
instance can't actually be describedby multiple schemas. A server could say
"Accept: application/json;profile=Thing" and the client could send back
"Content-Type: application/json;profile=Thing;profile=Plant". Or maybe just
"Content-Type: application/json;profile=Plant" if the server understands
the JSON Schema semantics. I'll have to do some digging and see if this
would be legal...

Thanks,

Austin.