Subject: [Json] JSON Content Rules

From: Daniel P <[email-masked]>
Date: Tue, 23 Jan 2018 17:13:34 -0500

Hello everyone,

I would like to solicit feedback from members of this forum on one feature
of the JSON Content Rules specification, draft 09
<https://datatracker.ietf.org/doc/draft-newton-json-content-rules/?include_text=1>,
as I'm considering to build an implementation.

The specification states: "There are two forms of rule name assignments:
assignments of  primitive types and assignments of all other types.  Rule
name assignments to primitive type specifications [e.g. string] separate
the rule name from the type specification with the character sequence '=:',
whereas  rule name assignments for all other type specifications [e.g.
array] only require the separation using the '=' character ... This syntax
is necessary so  that JCR parsers may readily distinguish between rule name
assignments involving string and regular expressions primitive types and
member names of member specifications."

An example (I hope I have this right):

{ $bar-name : $bar-val, "foo" : $foo-val }

; member name specification
$bar-name = /^bar[0-9]$/

; primitive type specification
$foo-val =: "foo"

; non primitive type specification
$bar-val = [ integer, integer, integer ]

In what otherwise appears to me to be a fairly clean specification, I'm
having some difficulty digesting this syntax, with "=" if such and such,
and "=:" if so and so. I would be interested if anyone else on this list
has any thoughts about this.

Thanks,
Daniel Parker
https://github.com/danielaparker/jsoncons

---

From: Andrew Newton <[email-masked]>
Date: Tue, 23 Jan 2018 18:24:38 -0500

Daniel,

It's not a compromise well-loved by anyone, for sure.

With hopes that I'm explaining this correctly, the basic issue is that
without a statement termination character it is difficult for a parser
to differentiate between an assignment of a string and the assignment
of a member as they both start with the same syntax (a quoted string).

As chance would have it, an alternative is in the works.
https://github.com/arineng/jcrvalidator/issues/112

Is that a suitable fix? We're open to feedback.

-andy

On Tue, Jan 23, 2018 at 5:13 PM, Daniel P <[email-masked]> wrote:
> of the JSON Content Rules specification, draft 09, as I'm considering to
> build an implementation.
>
> name assignments to primitive type specifications [e.g. string] separate the
> rule name from the type specification with the character sequence '=:',
> having some difficulty digesting this syntax, with "=" if such and such, and
> "=:" if so and so. I would be interested if anyone else on this list has any
> thoughts about this.
>
> Thanks,
> Daniel Parker
> _______________________________________________
> json mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/json
>

---

From: Pete Cordell <[email-masked]>
Date: Wed, 24 Jan 2018 12:19:14 +0000

Hi Daniel,

As Andy said, it's been bugging us too.

The fundamental problem is (was!) that a quoted string (e.g. "foo") can 
either be a member name or a string value.  Also, a regex (e.g. /bar/) 
could either be a member name regex or a value regex.

The ambiguity could potentially be resolved with back tracking in the 
parser, but I've wanted to try to make JCR as simple to parse as 
possible in the hope that this will facilitate its adoption.  (I think 
what I've been promoting is called an LR(1) grammar.)  Hence the 
approach of adding the extra ':' or 'type' token to do the disambiguation.

The new change linked to by Andy at 
https://github.com/arineng/jcrvalidator/issues/112 essentially 
disambiguates this by adopting slightly different syntax for the various 
uses.  So we've ended up with:

    "foo" - A member name (but see caveat below)
    'bar' - A string value (new syntax, but also see below)
    /^baz\d+$/ - A regex value
    `biff\d+` - A member name regex (new syntax)

Now the various usages are unambiguous, and we don't need the colon to 
differentiate whether a quoted string is a member name or value, etc.

Now to the caveat...

We wanted to still allow JCR to be a superset of JSON syntax (to 
facilitate easy creation of JCR from example JSON).  So we wanted to allow:

     { "name" : "Fred" }

Hence, in situations where the parser has read a member name and colon, 
and knows that what follows is a value, a string value can either be 
single quoted or double quoted; 'Fred' or "Fred".

The result is that the following are all valid:

     $r1 = "name" : "Fred" ; A member rule
     $r2 = "name" : 'Fred' ; Also a member rule
     $r3 = 'Fred'          ; A string value
     $r4 = /^Fred/'        ; A regex value
     $r5 = `p[0-9]` : integer ; Member rule with regex name
     $r6 = string          ; The big win - Colon no longer needed

The following would be an error:

     $e1 = "Fred" $e2 = string

because when parsing $e1 and seeing "Fred", the parsing would interpret 
"Fred" as a member name, and therefore complain when encountering $e2 
without first seeing a colon.  (You'd have to do the following instead: 
$e1 = 'Fred' $e2 = string)

Personally I think the revision is a lot neater than what we had, and I 
hope it's not too difficult for developers to grok.  I look forward to 
your comments.

Thanks,

Pete.
-- 
---------------------------------------------------------------------
Pete Cordell
http://www.xml2cpp.com
---------------------------------------------------------------------
On 23/01/2018 22:13, Daniel P wrote:
> Hello everyone,
> 
> I would like to solicit feedback from members of this forum on one 
> feature of the JSON Content Rules specification, draft 09 
> assignments of  primitive types and assignments of all other types.  
> Rule name assignments to primitive type specifications [e.g. string] 
> separate the rule name from the type specification with the character 
> sequence '=:', whereas  rule name assignments for all other type 
> specifications [e.g. array] only require the separation using the '=' 
> character ... This syntax is necessary so  that JCR parsers may readily 
> distinguish between rule name assignments involving string and regular 
> expressions primitive types and member names of member specifications."
>

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 24 Jan 2018 14:11:11 +0100

>   `biff\d+` - A member name regex (new syntax)

Hmm.

> We wanted to still allow JCR to be a superset of JSON syntax (to facilitate easy creation of JCR from example JSON).  So we wanted to allow:
> 
>    { "name" : "Fred" }

Right.  In JSON, a string is a string…

What is the syntactical feature of JCR that makes this distinction necessary?
Maybe you can fix that feature instead?

(When we designed CDDL, we didn’t have to make a distinction between string values destined as member names and string values destined as member values — indeed strings are strings. Now the CDDL syntax is very simple and does not leave any ambiguity as to whether we are talking about a member name or a member value; I haven’t looked at JCR recently enough to answer my own question here.)

Grüße, Carsten

---

From: Pete Cordell <[email-masked]>
Date: Wed, 24 Jan 2018 13:36:07 +0000

> Right.  In JSON, a string is a string…

...disambiguated by context.

Where such disambiguating context isn't present, we give a little help 
via syntax.

> (When we designed CDDL, we didn’t have to make a distinction between string values destined as member names and string values destined as member values — indeed strings are strings. Now the CDDL sy…

Likewise, I need another look at CDDL.

Pete.

---

From: Daniel P <[email-masked]>
Date: Wed, 24 Jan 2018 09:26:30 -0500

A problem with both the original specification and the suggested revisions
is that they both involve adding inconsistent bits of syntax to resolve
ambiguities, "=" if such and such and "=": if so and so, or a double quote
if such and such and a single one if so and so. A user's mind rebels
against that.

I don't understand why it's necessary to have a distinction between name
rule specifications and value rule specifications at all. The user may
express something impossible by writing

{ $bar : $foo }

$bar = [ integer, integer, integer ]
$foo = "foo"

but who cares? it's going to get caught.

On Wed, Jan 24, 2018 at 7:19 AM, Pete Cordell <[email-masked]>
wrote:

> parser, but I've wanted to try to make JCR as simple to parse as possible
> in the hope that this will facilitate its adoption.  (I think what I've
> been promoting is called an LR(1) grammar.)  Hence the approach of adding
> the extra ':' or 'type' token to do the disambiguation.
>
> The new change linked to by Andy at https://github.com/arineng/jcr
> validator/issues/112 essentially disambiguates this by adopting slightly
> different syntax for the various uses.  So we've ended up with:
>
> and knows that what follows is a value, a string value can either be single
> quoted or double quoted; 'Fred' or "Fred".
>
> without first seeing a colon.  (You'd have to do the following instead: $e1
> = 'Fred' $e2 = string)
>
> hope it's not too difficult for developers to grok.  I look forward to your
> comments.
>
> Thanks,
>
> Pete.
> --
>> feature of the JSON Content Rules specification, draft 09 <
>> https://datatracker.ietf.org/doc/draft-newton-json-content-
>> rules/?include_text=1>, as I'm considering to build an implementation.
>>

---

From: Pete Cordell <[email-masked]>
Date: Wed, 24 Jan 2018 16:09:19 +0000

Hi Daniel,

On 24/01/2018 14:26, Daniel P wrote:
> A problem with both the original specification and the suggested 
> revisions is that they both involve adding inconsistent bits of syntax 
> to resolve ambiguities, "=" if such and such and "=": if so and so, or a 
> double quote if such and such and a single one if so and so. A user's 
> mind rebels against that.
> 

It looks like there is a misunderstanding here (possibly of minor 
consequence, but worth clearing up).

JCR works in terms of rules.  So `$bar = [ integer, integer, integer ]` 
is a named rule (called 'bar' for those less familiar with JCR).  It 
happens to be a value rule.  There are also member rules, which consist 
of a member name and a value rule.

JCR doesn't do macro substitution.  So the `$name = ...` syntax is not 
specifying a macro that can be dropped into anywhere.  So, currently, 
you can't do `{ $bar : $foo }` in JCR.

Personally, I don't see this as a problem.  I can see many cases where 
types need to be specified independent of the member names that use 
them.  But I don't think there are many practical cases where a member 
name needs to be specified independent of its type.

That said, that clarification probably doesn't change your main point.

> but who cares? it's going to get caught.

My background is C++.  One aspect of that language is that it's hard to 
parse correctly.  The consequence of that is that the tooling lacks well 
behind many other languages that are easier to parse.  I don't want that 
to be the case for JCR.  My theory is that, the easier it is to parse, 
the better the tools will be, with more consistency etc.

Also, I want to make it easy to produce accurate, helpful error reports 
that will reduce the time developers need to work out what's wrong with 
their JCR.

The structure of a rule can be pedantic, and with them parsers have a 
much better chance of saying "your error is here".  Using a macros 
approach potentially means that the errors are detected well away from 
their source, which makes them harder to give helpful error messages 
for, and harder to fix.

Normally I would push any complexities into the tool, and relieve the 
user of them as much as possible.  But I don't see JCR as a 
multi-million dollar industry.  More of a cottage industry akin to 
Bill's ABNF parser (https://tools.ietf.org/tools/bap/abnf.cgi).  So it's 
a balance.  Given the choice of better tools, and remembering to use 
single quotes; or worse tooling and not having to remember to use single 
quotes, I think developers would go for the former.

Single quoted strings also helps the human developer.  They can look at 
'foo' and know it's a value.  They don't have to look into the context 
of where it's used to work out whether it's a name or value.

I hope that help,

Pete.
http://www.xml2cpp.com

---------------- old stuff for possible context -----------------------
> 
> On Wed, Jan 24, 2018 at 7:19 AM, Pete Cordell <[email-masked] 
> <mailto:[email-masked]>> wrote:
> 
>     Hi Daniel,
> 
>     The fundamental problem is (was!) that a quoted string (e.g. "foo")
>     can either be a member name or a string value.  Also, a regex (e.g.
>     /bar/) could either be a member name regex or a value regex.
> 
>     The ambiguity could potentially be resolved with back tracking in
>     the parser, but I've wanted to try to make JCR as simple to parse as
>     possible in the hope that this will facilitate its adoption.  (I
>     think what I've been promoting is called an LR(1) grammar.)  Hence
>     the approach of adding the extra ':' or 'type' token to do the
>     disambiguation.
> 
>     <https://github.com/arineng/jcrvalidator/issues/112> essentially
>     disambiguates this by adopting slightly different syntax for the
>     various uses.  So we've ended up with:
> 
>     Now the various usages are unambiguous, and we don't need the colon
>     to differentiate whether a quoted string is a member name or value, etc.
> 
>     Now to the caveat...
> 
>     Hence, in situations where the parser has read a member name and
>     colon, and knows that what follows is a value, a string value can
>     either be single quoted or double quoted; 'Fred' or "Fred".
> 
>     because when parsing $e1 and seeing "Fred", the parsing would
>     interpret "Fred" as a member name, and therefore complain when
>     encountering $e2 without first seeing a colon.  (You'd have to do
>     the following instead: $e1 = 'Fred' $e2 = string)
> 
>     Personally I think the revision is a lot neater than what we had,
>     and I hope it's not too difficult for developers to grok.  I look
>     forward to your comments.
> 
>     Thanks,
> 
>     Pete.
>     -- 
>         I would like to solicit feedback from members of this forum on
>         one feature of the JSON Content Rules specification, draft 09
>         <https://datatracker.ietf.org/doc/draft-newton-json-content-rules/?include_text=1
>         <https://datatracker.ietf.org/doc/draft-newton-json-content-rules/?include_text=1>>,
>         The specification states: "There are two forms of rule name
>         assignments: assignments of  primitive types and assignments of
>         all other types.  Rule name assignments to primitive type
>         specifications [e.g. string] separate the rule name from the
>         type specification with the character sequence '=:', whereas
>           rule name assignments for all other type specifications [e.g.
>         array] only require the separation using the '=' character ...
>         This syntax is necessary so  that JCR parsers may readily
>         distinguish between rule name assignments involving string and
>         regular expressions primitive types and member names of member
>         specifications."
> 
>         In what otherwise appears to me to be a fairly clean
>         specification, I'm having some difficulty digesting this syntax,
>         with "=" if such and such, and "=:" if so and so. I would be
>         interested if anyone else on this list has any thoughts about this.
> 
>         Thanks,
>         Daniel Parker
>         <https://github.com/danielaparker/jsoncons>
> 
> 
> 
>         [email-masked] <mailto:[email-masked]>
>         <https://www.ietf.org/mailman/listinfo/json>
> 
>

---

From: Daniel P <[email-masked]>
Date: Wed, 24 Jan 2018 12:51:03 -0500

On Wed, Jan 24, 2018 at 11:09 AM, Pete Cordell <[email-masked]>
 wrote:

> JCR works in terms of rules ... JCR doesn't do macro substitution.

Of course, but the issue is whether the grammar needs to distinguish
between name rule specifications and
value rule specifications. I don't think that it does, and I don't think
that that provides any meaningful benefit to
the user, compared to the loss in simplicity of the syntax. That's just my
opinion, of course, but I would be
interested to see what others on the list think.

> My background is C++.

Mine too, but if you want a C++ inspired solution, you might want to study
the C++ rules for template method disambiguation :-)

Daniel

---

From: Pete Cordell <[email-masked]>
Date: Thu, 25 Jan 2018 09:15:49 +0000

On 24/01/2018 17:51, Daniel P wrote:
> On Wed, Jan 24, 2018 at 11:09 AM, Pete Cordell <[email-masked] 
> the user, compared to the loss in simplicity of the syntax. That's just 
> my opinion, of course, but I would be

I too am interested in the opinions of the group.  Maybe I'm being over 
cautious?

Pete.
http://www.xml2cpp.com