Subject: Re: [Jsonpath] Expression language -- index expressions and filter expressions

From: Daniel P <[email-masked]>
Date: Wed, 10 Mar 2021 10:24:00 -0500

On Tue, Mar 9, 2021 at 10:44 PM <[email-masked]> wrote:
>
> From: Greg Dennis <[email-masked]>

> Daniel, could you explain the two grammars, please? I'm not sure I follow what you're
> suggesting.  I see only a single overall grammar, where expressions are a subset of
> that grammar and only valid under certain circumstances.
>
> Surely, the expression grammar will be defined within the same spec (it doesn't make
> sense to write an entirely new spec for an expression grammar that will only be used
> in JSON Path), but I can't see expression grammar just being used anywhere within a
> path.  It only makes sense where an expression is valid.  Additionally, I would expect
> that the path grammar should be valid _within_ an expression, e.g. `@.name`.
>
Sure. To make it clear, I'm in favour of a single integrated grammar
describing JSONPath selectors and filter expressions, much as other
query languages like XPATH and JMESPath have.

But historically, many JSONPath "grammars" (implied grammars), defined

[?(<expr>)]

and treated <expr> within parentheses as a single token, to be
extracted and handed off to a separate "script engine" to evaluate.
Javascript was a popular script engine. Javascript has its own
grammar, it doesn't know anything about JSONPath. The syntax in the
Javascript engine would not  be exactly the same as in the path
grammar, for example, completely general JSONPath expressions would
not be supported in Javascript comparator expressions. Unquoted names
could have different conventions. It is Javascript, afterall. This
works, because while JSONPath depends on Javascript, Javascript has no
dependencies on Jsonpath. Of course, not all JSONPath implementations
used Javascript, some used different "script engines", and some took
the other approach of a single integrated grammar.

Let's consider another fully specified language for the moment, JMESPath.

In JMESPath, filter expressions are defined in the grammar as

list-filter-expr      = "[?" expression "]"
comparator-expression = expression comparator expression
comparator            = "<" / "<=" / "==" / ">=" / ">" / "!="

The operands "expression" can be any kind of expression defined in the
grammar, whether identifier-expressions, index-expressions,
comparator-expressions, JSON literal, and so on.

Now let's ask the question, instead of integrating filter expressions
into the main grammar, could JMESPath have defined

list-filter-expr  = "[?" "(" <expr> ")" "]"

and treated <expr> within parentheses as a single token, to be
extracted and handed off to a separate "script engine" to evaluate?

I don't see how. That script engine wouldn't be like Javascript, which
has its own grammar. Rather, it would be intimately dependent on
productions in the main grammar, which in turn would be intimately
dependent on productions in it.

Now, let's come back to JSONPath. Consider

[?(<expr>)]

Now, say we want to specify our own script engine for <expr>.

We could make it very much like Javascript, or different but still
independent of the main grammar. That would work.

But suppose we want to support things like

comparator-expression = expression comparator expression

where, analogous to JMESPath, any kind of JSONPath expression is
allowed as an operand, including selectors with filters. Now we're
back to a situation where productions in the main grammar and
productions in the filter expression grammar are intimately dependent
on each other.

So what could we do?

We could do some hand waving and leave it up to implementers to figure
it out. We could say "expression can be any JSONPath expression"
without further elaboration, effectively treating them as a single
token, and let implementers worry about how to extract them.

Or, we could place restrictions on "expression".  For example, we
could restrict "expression" to simple paths like "$.foo.bar", without
allowing filters, and have a separate grammar to describe these, that
doesn't have a dependency on the filter expression.

These are options. Some people may be happy with them. And that's fine.

My preferences, though, are (1) for the JSONPath specification to be
of the same quality as the JMESPath specification, with an unambiguous
grammar, and (2) to allow JSONPath expressions in comparator and
arithmetic expressions to be general and not constrained. And for
those reasons, I'm in favor of one integrated JSONPath grammar, and
not a separately defined "script engine".

Daniel

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 10 Mar 2021 16:28:53 +0100

> extracted and handed off to a separate "script engine" to evaluate.

Why would we want to do that?

Grüße, Carsten

---

From: Daniel P <[email-masked]>
Date: Wed, 10 Mar 2021 10:46:12 -0500

> Why would we want to do that?

Don't ask me, I don't know. That question should be asked of people
that want to do that,
if you can find any. One person has written to me privately that they wanted to
do that.

In my original post, attempting to make a modest contribution to a
discussion about
filter expressions, I simply "remarked"

"Another issue is whether expression evaluation should be a separate
component, or integrated into the JSONPath grammar proper. My view is
the latter. A number of implementations take that approach."

which, as far as I can tell, is a true statement. I didn't expect that to be
a controversial statement.

Daniel

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 10 Mar 2021 16:59:31 +0100

> a controversial statement.

Finally got it.  It seems you just stretched our imagination a bit :-)

I think we already have established consensus for not making the expression sublanguage a pluggable component(*); instead the plan is to define a single standard one that is part of the standard and build the grammar around that.

Grüße, Carsten

(*) Which to me would mean that you can parse a JSONpath query and hand over the expressions to an implementation that believes ++++++++[>++++[>++>+++>+++>+<<<<-]>+>+>->>+[<]<-]>>.>---.+++++++..+++.>>.<-.<.+++.------.--------.>>+.>++. is a reasonable expression.

---

From: Stefan Gössner <[email-masked]>
Date: Wed, 10 Mar 2021 17:01:49 +0100

.... in #54 I wrote:

 > The invocation of 'the underlying scripting engine' wasn't meant a 
serious normative aspect, but rather a quick and dirty solution for 
JavaScript and PHP implementations at that time.

so we should not care about that solution ...

---

From: Greg Dennis <[email-masked]>
Date: Wed, 10 Mar 2021 18:30:41 +0000 (UTC)

I was looking for that...

  On Thu, 11 Mar 2021 at 5:02 am, Stefan Gössner<[email-masked]> wrote:   -- 
Jsonpath mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/jsonpath