Subject: [apps-discuss] Update to JSON Predicates

From: James M Snell <[email-masked]>
Date: Thu, 18 Oct 2012 14:56:34 -0700

Based on some initial direct feedback I received on the -00 draft, I have
submitted an updated draft..

  http://www.ietf.org/id/draft-snell-json-test-01.txt

Several key modifications were made..

 1. Specific mention is made re: the use of the JSON-Patch "test" operation
as a Predicate with the addition of an optional "ignore_case" member
consistent with the other text comparison predicates.

 2. Addition of an "in" predicate operation that tests whether the value is
part of a given list of values.. for instance:

    {"op": "in", "path": "/a/b/c", "value": [1,2,3,4,5]}

    Returns true if the value of "/a/b/c" is in the array [1,2,3,4,5]

 3. Additional types for the "type" predicate operation including: date,
time, date-time, lang, lang-range, iri and absolute-iri. These are intended
to test if a JSON string conforms to a specific set of common RFC defined
data types. The types are drawn from RFC's 3339, 3987, 4646, and 4647.
These types are common to many JSON data formats.

    {"op": "type", "path": "/a/b/c", "value": "date-time"}

    Returns true if the value of "/a/b/c" is an RFC3339 date-time like
"2012-12-12T12:12:12Z"

    {"op": "type", "path": "/a/b/c", "value": "absolute-iri"}

    Returns true if the value of "/a/b/c" is an Absolute IRI reference like
"http://example.org/foo"

    {"op": "type", "path": "/a/b/c", "value": "lang"}

    Returns true if the value of "/a/b/c" is a Language Tag like "en-US"

4. Several clarifications and editorial fixes were made.

As always, comments are welcome. Unless there are significant changes made
to JSON Patch or significant issues found with this specification, I do not
anticipate making any further technical changes.

- James

---

From: "Manger, James H" <[email-masked]>
Date: Fri, 19 Oct 2012 18:10:50 +1100

James,

This predicate syntax has significant problems [draft-snell-json-test-01].

1.       It doesn’t play nicely with json-patch. It stomps all over json-patch’s namespace for “op” names, defining another 14 values. It would be better to use one name, say, “test2”.

2.       The separate patch op for each predicate is not the right model. The syntax for the predicates should be embedded in a wrapper that provides: the value to test; and any actions to do if the test passes. The wrapper can be a json-patch operation.

3.       “and”/”or”/”not” have an “apply” array of ops. Can these ops only be other tests, or can they modify the target?

4.       A json-patch says ops MUST have a path; while this doc says “and”/”or”/”not” MAY have a path.

5.       The example in the introduction is wrong. The paths for the “type” & “contains” ops should be “/c” (not “/a/b/c”).

6.       Way too verbose.

7.       Less and more are not sufficient, you also need <= and >=.

8.       “contains”, “ends”, “starts” are subsets of “matches”. Might be cleaner to only have the latter.

9.       “in” can be written using “or”. Perhaps “in” is common enough to have a dedicated, less verbose syntax.

10.   “not” should apply to 1 predicate, not an array of them.

11.   “or” is implied for “not”’s array of ops; while  “and” is implied for the top-level array of ops. Inconsistent.

12.   There are multiple ways to do “and”: explicitly with “and”; just concatenating all the ops

How about:

A <predicate> evaluates to true or false when applied to a target JSON value (which may not exist).
A <predicate> is an object with any number of <predicate-op> elements. The <predicate> is true if, and only if, each <predicate-op> is true ({} is true).
The <predicate-op>s are:
* "and":[<predicate-op>…] — true if none of the <predicate-op>s in the array are false; false otherwise. "and":[] is true.
* "or":[<predicate-op>…] — true if any of the <predicate-op>s in its value are true; false otherwise. "or":[] is false.
* "not":<predicate> — true if <predicate> evaluates to false; false otherwise
* "present":<boolean> — true if the value is true and the target is present, or the value is false and the target is absent; false otherwise
* "=":<any> — true if the value and target are equal
* "type":<any> — true if the value and target have the same type; false otherwise. Suggested values to use for the 6 JSON types are null, true, 0, "", [], and {} for null, boolean, number, string, array, and object respectively.
* "match":<regex> — true if the target is a string, and it matches the give regular expression; false otherwise. The expression “(?i)” can be included in the regex for case-insensitive matching.
* "<":<number> — true if the target is a number, and is less than the given value; false otherwise
* ">", "<=", ">=" — similar to "<"
* <pointer>:<predicate> — evaluate the predicate against the part of the target selected by the pointer. A <pointer> can only be the empty string or a string starting with "/", which allows all possible pointers to be distinguished from other <predicate-op> names.
* <unrecognized>:<any> — any unrecognized element in a <predicate> is treated as though it evaluates to false

"test2" is a new operation defined for json-patch. It MUST have a "?" member whose value is a <predicate> that evaluates the target selected by the "path" member. It MAY have a "ops" member whose value is an array of operations. Those operations are applied only if the predicate evaluates to true. It is an error if the predicate evaluates to false and "ops" is not present, ie processing the patch fails. Nothing happens in the other two cases (predicate is false and ops present; predicate true and ops absent).

The examples from draft-snell-json-test-01 re-written in this syntax.

§1. Introduction [match implements “contains”, and implies “type” since it only works against strings]

  [{"op":"test2", "path":"/a/b/c", "?":{"match":".*ABC.*"} },
   {"op":"replace", "path":"/a/b/c", "value":123}]

§2.2.1. contains

  [{"op":"test2", "path":"/a/b", "?":{"match":".* is a .*"} }]

  [{"op":"test2", "path":"/a/b", "?":{"match":"(?i).* Is A .*"} }]

§2.2.2 defined

  [{"op":"test2", "path":"/a/b", "?":{"present":true} }]

  [{"op":"test2", "path":"/a/c", "?":{"present":true} }]

§2.2.3. ends

  [{"op":"test2", "path":"/a/b", "?":{"match":".* test"} }]

  [{"op":"test2", "path":"/a/b", "?":{"match":"(?i).* TEST"} }]

§2.2.4. in

  [{"op":"test2", "path":"/a/b", "?":
     {"or":[{"=":1}, {"=":"foo"}, {"=":10}, {"=":{"z":"y"}}]} }]

§2.2.5. less

  [{"op":"test2", "path":"/a/b", "?":{"<":15} }]

§2.2.6. matches [an example where case sensitivity matters would be better]

  [{"op":"test2", "path":"/a/b", "?":{"match":"[\\w\\s]*"} }]

  [{"op":"test2", "path":"/a/b", "?":{"match":"(?i)[\\w\\s]*"} }]

§2.2.7. more

  [{"op":"test2", "path":"/a/b", "?":{">":5} }]

§2.2.8. starts

  [{"op":"test2", "path":"/a/b", "?":{"match":"This .*"} }]

  [{"op":"test2", "path":"/a/b", "?":{"match":"(?i)This .*"} }]

§2.2.9. test

  [{"op":"test2", "path":"/a/b", "?":{"=":"this is a test"} }]

§2.2.10. type [use "present":false for "undefined"; "match":<regex> for date, date-time, time, lang, lang-range, iri, absolute-iri]

  [{"op":"test2", "path":"/a/b", "?":{"type":""} }]

§2.2.11 undefined

  [{"op":"test2", "path":"/a/c", "?":{"present":false} }]

  [{"op":"test2", "path":"/a/b", "?":{"present":false} }]

§2.3. Second-Order Predicates

  [{"op":"test2", "path":"/a/b", "?":{"/c":{"present":true}} }]

§2.3.1. and (don’t need to use “and” unless applying the same <predicate-op> to the same value two or more times, which should be rare) [checking presence is redundant if checking type]

  [{"op":"test2", "path":"", "?":{
      "/a/b":{"present":true},
      "/a/c/d":{"<":15} } }]

  [{"op":"test2", "path":"", "?":{
      "/a/c":{"present":true},
      "/a/c":{"type":""} } }]

§2.3.2. not

  [{"op":"test2", "path":"", "?":{"not":{"or":[
      {"/a/b/e":{"present":true}},
      {"/a/c/d":{"<":5}} } }]

  [{"op":"test2", "path":"", "?":{"not":{"or":[
      {"/a/c":{"present":false}},
      {"/a/b":{"match":"f.*"}} ]}} }]

§2.3.3. or [examples are wrong as they use “test” without a “value”]

  [{"op":"test2", "path":"", "?":{"or":[
      {"/a/b":{"present":true}},
      {"/a/c/d":{"<":5}} ]} }]

  [{"op":"test2", "path":"", "?":{"or":[
      {"/a/e":{"present":true}},
      {"/a/f":{"present":true}} ]} }]

§2.3.4. Nesting Second Order Predicates  [type test is redundant here]

  [{"op":"test2", "path":"/a/b", "?":
    {"or":[
      {"/c":{"not":{"or":[
        {"present":false}, {"match":"f.*"}
      ]}}},
      {"/d":{"not":{"or":[
        {"present":true}, {"type":0}
      ]}}}
    ]}
  ]

--
James Manger

From: [email-masked] [mailto:[email-masked]] On Behalf Of James M Snell
Sent: Friday, 19 October 2012 8:57 AM
To: IETF Apps Discuss
Subject: [apps-discuss] Update to JSON Predicates

Based on some initial direct feedback I received on the -00 draft, I have submitted an updated draft..

 1. Specific mention is made re: the use of the JSON-Patch "test" operation as a Predicate with the addition of an optional "ignore_case" member consistent with the other text comparison predicates.

 2. Addition of an "in" predicate operation that tests whether the value is part of a given list of values.. for instance:

 3. Additional types for the "type" predicate operation including: date, time, date-time, lang, lang-range, iri and absolute-iri. These are intended to test if a JSON string conforms to a specific set of common RFC defined data types. The types are drawn from RFC's 3339, 3987, 4646, and 4647. These types are common to many JSON data formats.

    Returns true if the value of "/a/b/c" is an RFC3339 date-time like "2012-12-12T12:12:12Z"

    Returns true if the value of "/a/b/c" is an Absolute IRI reference like "http://example.org/foo"

As always, comments are welcome. Unless there are significant changes made to JSON Patch or significant issues found with this specification, I do not anticipate making any further technical changes.

- James