Subject: [Jsonpath] Agenda for Interim meeting, May 11 2021

From: Tim Bray <[email-masked]>
Date: Sun, 9 May 2021 11:28:46 -0700

At our meeting Tuesday we need to do some agenda-building.  Here are issues
that your co-chairs think appropriate for the agenda at this time;
consensus is not obvious and there are good arguments posted on the
issues.

It would be helpful to familiarize yourself with the editor's draft
https://ietf-wg-jsonpath.github.io/draft-ietf-jsonpath-base/draft-ietf-jsonpath-base.html
and
perhaps its differences with the current WG draft:
https://tools.ietf.org/rfcdiff?url1=https://tools.ietf.org/id/draft-ietf-jsonpath-base.txt&url2=https://ietf-wg-jsonpath.github.io/draft-ietf-jsonpath-base/draft-ietf-jsonpath-base.txt

We've proposed three issues but we think there's a good chance of getting
through all of them in two hours, so don't be shy about suggesting more.

Suggested issues for discussion:

Terminology:
- Issue #84:
https://github.com/ietf-wg-jsonpath/draft-ietf-jsonpath-base/issues/84

Relative paths support:
- Issue #59:
https://github.com/ietf-wg-jsonpath/draft-ietf-jsonpath-base/issues/59

Regular expressions in filters?
Issue #70:
https://github.com/ietf-wg-jsonpath/draft-ietf-jsonpath-base/issues/70

---

From: Glyn Normington <[email-masked]>
Date: Mon, 10 May 2021 16:17:59 +0100

Do we need to discuss why progress on the spec seems to have stalled and
what can be done about it? (Since I haven't worked in an IETF WG before,
perhaps the "glacial" pace is normal, in which case this discussion point
may be unnecessary.)

Perhaps we could also knock off the following issue, which was discussed at
the last IETF meeting:

Duplicates in selector output:
- Issue #23:
https://github.com/ietf-wg-jsonpath/draft-ietf-jsonpath-base/issues/23

I have the impression that the consensus is converging on removing
duplicate paths, but not duplicate values.

On Sun, 9 May 2021 at 19:29, Tim Bray <[email-masked]> wrote:

> At our meeting Tuesday we need to do some agenda-building.  Here are
> issues that your co-chairs think appropriate for the agenda at this time;
> https://ietf-wg-jsonpath.github.io/draft-ietf-jsonpath-base/draft-ietf-jsonpath-base.html and
> --
> Jsonpath mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/jsonpath
>

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 10 May 2021 17:22:16 +0200

On 2021-05-10, at 17:17, Glyn Normington <[email-masked]> wrote:
> 
> Do we need to discuss why progress on the spec seems to have stalled and what can be done about it? 

If we can focus on the latter, absolutely.
(I’m not interested in meting out blame.
Or actually, if there are processes to blame, let’s change them.
Or perceptions!)

My perception of the main problem is that we have tons of active issues and aren’t active in closing them or moving the remaining essence to more focused issues.

Grüße, Carsten

---

From: Stefan Gössner <[email-masked]>
Date: Mon, 10 May 2021 19:07:02 +0200

* Terminology converges to a really good collection. I am confident to 
get that straight very soon.
* `duplicates removal` ... Glyn's "removing duplicate paths, but not 
duplicate values" seems to bring it to a good point.
* Selectors list proposed by me might be discussed for completeness:
     * Root selector `'$'`
     * Current item selector `'@'`
     * Nested descendants selector `'..'`.
     * Dot selector `'.<name>'`, used with object member names exclusively.
     * Dot wild card selector `'.*'`.
     * Index selector `'[<index>]'`, where `<index>` is either an 
(possibly negative) array index or an object member name.
     * Index wild card selector `'[*]'`.
     * Array slice selector `'[<start>:<end>:<step>]'`, where `<start>`, 
`<end>`, `<step>` are integer literals.
     * Union selector `'[<index1>,<index2>,...,<indexN>]'`, holding a 
comma delimited list of indices (indexes ?).
     * Filter selector `[?(<expr>)]`

* The biggest construction site at current seems to be filter selectors 
`[?(<expr>)]`. Here I started/continued issue [Filter Expressions 
#64](https://github.com/ietf-wg-jsonpath/draft-ietf-jsonpath-base/issues/64) 
with a following PR in mind. Not to forget the regular expressions 
challenge with it.
* I would like to know opinions regarding index selectors `[(<expr>)]` 
... to add or not to add.

I cannot see progress really stalled. But since I am new to this IETF WG 
work, I would like to know a bit more about expectations here.
--
sg

---

From: Glyn Normington <[email-masked]>
Date: Mon, 10 May 2021 18:45:32 +0100

Stefan wrote:

> I cannot see progress really stalled. But since I am new to this IETF WG work, I would like to know a bit more about expectations here.

It may be that my expectations are unrealistic or out of line and
please note that I certainly didn't intend to blame any individuals
when I described the rate of progress as "glacial".

I guess I am happy with a certain amount of discussion and work in
issues, but then I would expect to see (bold/brave) PRs as a means of
breaking deadlocks, testing consensus, and making substantive, if
incremental, progress. My hope is that we could chip away with small
PRs and gradually make the remaining gaps easier to fill.

When I stepped back from authoring and editing to focus on the
compliance test suite and reference implementation, I guess I hoped
others would step in on the authoring front, but it's not really clear
who actually sees it as their role and has sufficient time to author
PRs. Carsten has done an excellent job in the past, but less recently.
I get the impression that a number of others involved would prefer to
thrash things out almost completely in issues before venturing into
PRs.

I think discussing our expectations a little tomorrow will help us all
pull together more effectively.

---

From: Greg Dennis <[email-masked]>
Date: Mon, 10 May 2021 20:02:36 +0000 (UTC)

> it's not really clear who actually sees it as their role 
I'd be happy to work up some PRs, but I experience two roadblocks:
1. Discussions I've had, whether over email or in slack, have indicated to me that the WG retains authorship and editorship credit.  I'm not part of the WG, and (it may seem petty, but it's important to me), I'd like to be credited with the contributions I make.2. I can't build the repo because none of the tools are conducive to a Windows environment, and I can't seem to resolve dependencies within the WSL extension.
Greg

---

From: Tim Bray <[email-masked]>
Date: Mon, 10 May 2021 13:19:09 -0700

[co-chair hat on]

On Mon, May 10, 2021 at 1:02 PM Greg Dennis <[email-masked]> wrote:

>
> 1. Discussions I've had, whether over email or in slack, have indicated to
> me that the WG retains authorship and editorship credit.  I'm not part of
> the WG, and (it may seem petty, but it's important to me), I'd like to be
> credited with the contributions I make.
>

Yeah, sorry, that's just how the IETF is.  Some RFCs do contain
acknowledgements to non-editors who've made significant contributions, but
usually not.  Hmm, you do leave a permanent record of your contributions on
some combination of the email archive and the github repo traffic.

> 2. I can't build the repo because none of the tools are conducive to a
> Windows environment, and I can't seem to resolve dependencies within the
> WSL extension.
>

This is very annoying. A lot of people are using WSL these days.

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 10 May 2021 22:34:19 +0200

On 2021-05-10, at 22:02, Greg Dennis <[email-masked]> wrote:
> 
> 2. I can't build the repo because none of the tools are conducive to a Windows environment, and I can't seem to resolve dependencies within the WSL extension.

We should collect some information about how to run things in WSL.

For now, you may simply want to use the docker image from https://github.com/martinthomson/i-d-template

The somewhat unusual tool xmlstarlet is actually included in the docker image at [email-masked]:ietf-wg-jsonpath/draft-ietf-jsonpath-jsonpath.git, but the Dockerfile there is inexplicably stuck at some ancient versions

RUN gem install kramdown-rfc2629 -v 1.3.17
RUN pip3 install xml2rfc==3.3.0

If you delete the version pinnings, you probably get a good built environment.

Grüße, Carsten

---

From: Glyn Normington <[email-masked]>
Date: Mon, 10 May 2021 21:45:06 +0100

Greg wrote:

> I'm not part of the WG

I understand there is no concept of becoming a member of an IETF WG, so
Greg is as much a member as anyone. Not sure that helps the need for
attribution though.

The Chairs will need to think carefully about who to credit because, for
example, at least one of the contributors to my early draft hasn't been
involved since whereas Greg and others have done a tonne of work in issues.

---

From: Greg Dennis <[email-masked]>
Date: Tue, 11 May 2021 00:09:58 +0000 (UTC)

Some RFCs do contain acknowledgements to non-editors who've made significant contributions, but usually not.  Hmm, you do leave a permanent record of your contributions on some combination of the email archive and the github repo traffic.
 
Recording a person's ideas and arguments in GitHub issues is completely different than formally acknowledging them for authoring portions of a specification.  I don't expect someone to look through the GH repos and email archives to determine who actually wrote the words that appear in a specification.  They're going to look at the authors.
Attributing portions of a specification that I WROTE to someone else on the basis that they are part of the IETF and I'm not is UNACCEPTABLE.  I'm appalled that a group with as much influence as the IETF would even consider not including an obvious co-author of a spec because they're not in some exclusive club.
I understand that there is a threshold of contribution, but supposing I were to write the section on, for example, filter expressions, which is a major component of JSON Path, I should receive credit for that authorship.  I would not, however, expect such credit for adding a clarifying paragraph somewhere.  But to summarily dismiss the idea that someone else could be credited for authorship, regardless of the level of contribution, is abominable.
    On Tuesday, May 11, 2021, 08:45:18 AM GMT+12, Glyn Normington <[email-masked]> wrote:  
 
> I'm not part of the WG
I understand there is no concept of becoming a member of an IETF WG, so Greg is as much a member as anyone. Not sure that helps the need for attribution though.
The Chairs will need to think carefully about who to credit because, for example, at least one of the contributors to my early draft hasn't been involved since whereas Greg and others have done a tonne of work in issues.

---

From: Tim Bray <[email-masked]>
Date: Mon, 10 May 2021 17:30:28 -0700

Here’s the kind of thing I’m talking about:
https://datatracker.ietf.org/doc/html/rfc3986#section-9 (I know because I'm
in there; I wrote the first draft of Section 6.)

If you’re not willing to contribute to this effort without a guarantee of
recognition that meets your needs that's a pity, but you're within your
rights.

On Mon, May 10, 2021 at 5:10 PM Greg Dennis <[email-masked]> wrote:

> *Some RFCs do contain acknowledgements to non-editors who've made
> significant contributions, but usually not.  Hmm, you do leave a permanent
> record of your contributions on some combination of the email archive and
> the github repo traffic.*
>
>
> Recording a person's ideas and arguments in GitHub issues is completely
> different than formally acknowledging them for authoring portions of a
> specification.  I don't expect someone to look through the GH repos and
> email archives to determine who actually wrote the words that appear in a
> specification.  They're going to look at the authors.
>
> Attributing portions of a specification that I WROTE to someone else on
> the basis that they are part of the IETF and I'm not is UNACCEPTABLE.  I'm
> appalled that a group with as much influence as the IETF would even
> consider not including an obvious co-author of a spec because they're not
> in some exclusive club.
>
> I understand that there is a threshold of contribution, but supposing I
> were to write the section on, for example, filter expressions, which is a
> major component of JSON Path, I should receive credit for that authorship.
> I would not, however, expect such credit for adding a clarifying paragraph
> somewhere.  But to summarily dismiss the idea that someone else could be
> credited for authorship, regardless of the level of contribution, is
> abominable.
>

---

From: Greg Dennis <[email-masked]>
Date: Tue, 11 May 2021 00:42:00 +0000 (UTC)

Okay, that's a good example, but I think that's more appropriate for people like me who have contributed be discussing the ideas (e.g. in GitHub issues).
I want to explore this a bit more.
Why does IETF reserve authorship to its members?
Is there a written policy that describes the requirements for authorship or acknowledgement?
I'm not threatening to leave.  This project is important to me as an implementor.  I just want to be sure that the people are credited appropriately for their contributions.

  On Tue, 11 May 2021 at 12:30 pm, Tim Bray<[email-masked]> wrote:   -- 
Jsonpath mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/jsonpath

---

From: Glyn Normington <[email-masked]>
Date: Tue, 11 May 2021 03:23:11 +0100

>Why does IETF reserve authorship to its members?

I haven't seen any statement reserving authorship to IETF members. Did
I miss something?

> Is there a written policy that describes the requirements for authorship or acknowledgement?

I found the expired I-D
https://datatracker.ietf.org/doc/html/draft-rfc-editor-author-lists-00
gave some helpful background.

The RFC Style Guide (https://www.rfc-editor.org/styleguide/) points at
this statement on authorship:
https://www.rfc-editor.org/pipermail/rfc-interest/2015-May/008869.html

On Tue, 11 May 2021 at 01:42, Greg Dennis <[email-masked]> wrote:
> On Tue, 11 May 2021 at 12:30 pm, Tim Bray
> <[email-masked]> wrote:
> --
> Jsonpath mailing list
> [email-masked]

---

From: Tim Bray <[email-masked]>
Date: Mon, 10 May 2021 19:40:35 -0700

Um, there’s such a thing as an “IETF member”. There are two recognized
roles, chair and editor, outside that everyone is equal to everyone else.
You are a member of the WG by virtue of showing up.  In some WGs, some of
the people are there on behalf of their employer, but employers have no
standing whatsoever in the IETF.

On Mon, May 10, 2021 at 5:42 PM Greg Dennis <[email-masked]> wrote:

> Okay, that's a good example, but I think that's more appropriate for
> people like me who have contributed be discussing the ideas (e.g. in GitHub
> issues).
>
> I'm not threatening to leave.  This project is important to me as an
> implementor.  I just want to be sure that the people are credited
> appropriately for their contributions.
>

---

From: Julian Reschke <[email-masked]>
Date: Tue, 11 May 2021 08:51:35 +0200

Am 10.05.2021 um 22:02 schrieb Greg Dennis:
> ...
> ...

FWIW - if you can't or don't want to use WSL - Cygwin works just fine.
You will need base tools, git, xml2rfc and kramdown-rfc2629 in it, but
that should be it. (Works fine over here)

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Tue, 11 May 2021 09:13:05 +0200

> I haven't seen any statement reserving authorship to IETF members. Did

The important point being: there is no formal membership. By providing
feedback on the mailing list or through Github (respecting
<https://www.ietf.org/about/note-well/>) you participate in the Working
Group, and thus consider yourself an "IETF member" if you want.

> https://www.rfc-editor.org/pipermail/rfc-interest/2015-May/008869.html

And yes, substantive contributions by people not listed as authors
usually are acknowledged in an appendix. How to identify these people
can sometimes be tricky.

Best regards, Julian

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 11 May 2021 09:23:31 +0200

> usually are acknowledged in an appendix. How to identify these people

Yes.  Since RFC 7322, we also have a contributors section, which already has its first entry (yours truly); we could just list the names or include text about the topic of the contribution.
(Smaller contributions can be acknowledged in the acknowledgements section.)
So we already have started with this and should simply continue on this path.

> The Chairs will need to think carefully about who to credit 

Managing this information, just like managing the rest of this document, is done by the authors (“editors").
The chairs have no role, except maybe in giving some guidance (and selecting the author (“editor”) set).
It’s all about the subjective sense of the authors.
Traditionally, the WG has not interfered in those sections, either, so I’d ask the authors (“editors”) to simply charge ahead, at a time and a pace of their choosing.

Grüße, Carsten

---

From: Greg Dennis <[email-masked]>
Date: Tue, 11 May 2021 07:25:30 +0000 (UTC)

I think the topic of credit has been answered sufficiently.  I will continue to monitor and raise any further issues if I see anything.
Thank you.Greg

Grüße, Carsten

---

From: Greg Dennis <[email-masked]>
Date: Tue, 11 May 2021 07:35:57 +0000 (UTC)

My problem is specifically with installing those tools.  I can't seem to get kramdown installed properly.

gem says it's installed, but then the make says it's not.
This is where I got and gave up because I couldn't get help.  I'm not familiar with these tools, and it seems you need some prerequisite level of knowledge.
I've been following the installation guide.

| 
| 
| 
|  |  |

 |

 |
| 
|  | 
martinthomson/i-d-template

A template for IETF internet draft git repositories - martinthomson/i-d-template
 |

 |

 |

If you can help, I'd greatly appreciate it.
    On Tuesday, May 11, 2021, 06:51:48 PM GMT+12, Julian Reschke <[email-masked]> wrote:  

Best regards, Julian

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 11 May 2021 09:37:26 +0200

> gem says it's installed, but then the make says it's not.

You may have to fix up the PATH.

Grüße, Carsten

---

From: Greg Dennis <[email-masked]>
Date: Tue, 11 May 2021 08:07:39 +0000 (UTC)

I thought about that, but where is kramdown (as a ruby gem) installed so I can add it to the path?
    On Tuesday, May 11, 2021, 07:37:35 PM GMT+12, Carsten Bormann <[email-masked]> wrote:

---

From: Julian Reschke <[email-masked]>
Date: Tue, 11 May 2021 10:08:57 +0200

Am 11.05.2021 um 09:37 schrieb Carsten Bormann:
> On 11. May 2021, at 09:35, Greg Dennis
> <gregsdennis=[email-masked]
> <mailto:gregsdennis=[email-masked]>> wrote:
>>

Ah, this reminds me of
<https://jreschke.blogspot.com/2019/01/installing-xml2rfc-and-kramdown-rfc2629.html>.

Best regards, Julian

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 11 May 2021 10:12:18 +0200

$ gem environment | grep EXE
  - RUBY EXECUTABLE: /Users/cabo/opt/ruby/bin/ruby
  - GIT EXECUTABLE: /Users/cabo/bin/git
  - EXECUTABLE DIRECTORY: /Volumes/nar/Users/cabo-rescue/lib/ruby/gems/3.0.0/bin

The first two files are in your PATH already, I hope.

(Check with
  ruby -v
)

The third (EXECUTABLE DIRECTORY) may be needed to be added to the path.

(Or use Julian’s approach, install with -n ~/bin or a similar directory already in the path, but then you may need to remember that for every upgrade.)

Grüße, Carsten

> On 2021-05-11, at 10:07, Greg Dennis <[email-masked]> wrote:
>

---

From: Greg Dennis <[email-masked]>
Date: Tue, 11 May 2021 08:29:34 +0000 (UTC)

That did it!  I can build!
I had to sudo-install ruby because I didn't have write access to /bin, but it works now.
Thank you.
    On Tuesday, May 11, 2021, 08:09:24 PM GMT+12, Julian Reschke <[email-masked]> wrote:  

Ah, this reminds me of

Best regards, Julian

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 11 May 2021 10:58:21 +0200

> I had to sudo-install ruby because I didn't have write access to /bin, but it works now.

Yes, but you are now stuck having to sudo for every update.

The specific directory /bin that Julian proposes makes sense in an environment without authorization (such as Cygwin, which is what Julian uses), but not in Linux (or, I presume, WSL).
The directory Martin Thomson proposes, ~/.local/bin, appears to be a newer (systemd) rendition of ~/bin [1], which is where I install most user-specific things (macOS is not systemd-infected).  Choices, choices…  Makes it hard to provide good installation documentation.

Grüße, Carsten

[1] https://unix.stackexchange.com/questions/316765/which-distributions-have-home-local-bin-in-path

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 12 May 2021 15:03:31 +0200

On 2021-05-09, at 20:28, Tim Bray <[email-masked]> wrote:
> Issue #70: https://github.com/ietf-wg-jsonpath/draft-ietf-jsonpath-base/issues/70

We had a good discussion about issue #70 yesterday in the interim.
I promised to supply a PR.

It turned out that the work that is needed is to a large part reusable in other contexts.

So instead of writing the PR, I submitted a separate Internet-draft first.
This should supply the answer to the question what flavor of regexps we want to use:

https://www.ietf.org/archive/id/draft-bormann-jsonpath-iregexp-00.html

If we like this approach (which certainly needs a bit more work, as we want to make sure we hit the right level of complexity), I can supply the PR that anchors this specification in the jsonpath specification.

Grüße, Carsten

---

From: Glyn Normington <[email-masked]>
Date: Wed, 12 May 2021 16:50:55 +0100

> It turned out that the work that is needed is to a large part reusable in other contexts.

Indeed! In fact, there would be some benefit in keeping this spec,
together with a compliance test suite and reference implementation
separate so it could be reused in other contexts and would avoid
bloating the JSONPath spec/CTS/RI.

> https://www.ietf.org/archive/id/draft-bormann-jsonpath-iregexp-00.html

This looks good to me, at least as a starting point.

> If we like this approach (which certainly needs a bit more work, as we want to make sure we hit the right level of complexity), I can supply the PR that anchors this specification in the jsonpath s…

Not sure what you mean by "anchors". Would you refer to this spec from
the JSONPath spec (my preferred option - see above) or copy it in
place?

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 12 May 2021 18:03:23 +0200

On 2021-05-12, at 17:50, Glyn Normington <[email-masked]> wrote:
> the JSONPath spec (my preferred option - see above)

Yes.  But you need glue (what is the syntax for including the iregexp in the query?).

> or copy it in place?

Copies are bad…
So I would propose to keep them separate documents.
We just should get iregexp started in the jsonpath WG so we can make sure it fulfills our requirements, but at some point we should ask DISPATCH whether it should stay that way.

Grüße, Carsten

---

From: Martin J. Dürst <[email-masked]>
Date: Thu, 13 May 2021 18:50:27 +0900

Hello Carsten,

> https://www.ietf.org/archive/id/draft-bormann-jsonpath-iregexp-00.html

I haven't looked at the details, and I'm guessing that you are also 
aware of it, but I guess Section 4 should be entitled
"4. Formal definition of I-Regexp Syntax" rather than just
"4. Formal definition of I-Regexp". At the moment, the semantics are 
completely implicit. That could be fixed with a clear pointer to the XSD 
semantics, such as saying "The semantics of all the constructs defined 
in this ABNF are the same as those in [XSD]." or some such.

Regards,   Martin.

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 13 May 2021 13:44:45 +0200

Guten Tag Martin,

On 2021-05-13, at 11:50, Martin J. Dürst <[email-masked]> wrote:
> 
> At the moment, the semantics are completely implicit.

Thank you for the reminder.

Now in

https://github.com/cabo/iregexp/commit/ed6aa65

Grüße, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Sat, 15 May 2021 14:07:24 +0200

In iregexp-00, I kept the shorthand character classes that W3C XSD2 regexes provide.

https://www.w3.org/TR/xmlschema11-2/#cces-mce
has a list of the “multi-character” shorthands, which I abridge (removing the upper case variants, which are just complements) here:

.	[^\n\r]
\s	[#x20\t\n\r]
\i	initial name characters,  NameStartChar [XML]
\c	name characters, NameChar [XML]
\d	\p{Nd}
\w	[#x0000-#x10FFFF]-[\p{P}\p{Z}\p{C}] (all characters except the set of "punctuation", "separator" and "other" characters)

\i and \c are inventions of W3C XSD2; they will need to be translated to other flavors if we leave them in.

\w and \d are Unicode-enabled, so for instance \d includes (*)

0123456789٠١٢٣٤٥٦٧٨٩۰۱۲۳۴۵۶۷۸۹߀߁߂߃߄߅߆߇߈߉०१२३४५६७८९০১২৩৪৫৬৭৮৯੦੧੨੩੪੫੬੭੮੯૦૧૨૩૪૫૬૭૮૯୦୧୨୩୪୫୬୭୮୯௦௧௨௩௪௫௬௭௮௯౦౧౨౩౪౫౬౭౮౯೦೧೨೩೪೫೬೭೮೯൦൧൨൩൪൫൬൭

Similarly, \w includes letters from all scripts, math symbols, etc.

(For comparison, in ES2020 \d is [0-9] and \w is approximately [A-Za-z0-9_], with some weirdness added, and, in even weirder contrast, \s is a wider, Unicode-enabled set (***).)

For each of these (‘.’, '\'[sSiIcCdDwW]), should we:

(1) leave it out
(2) redefine it to what is considered /normal/ (so it is no longer XSD2-compatible)
(3) leave it in and provide any necessary translations for ECMAScript, RE2, …?

Do we have a corpus of regexp-using JSON Path expressions we could use for guidance?

Grüße, Carsten

(*)
   	NameStartChar	   ::=	":" | [A-Z] | "_" | [a-z] | [#xC0-#xD6] | [#xD8-#xF6] | [#xF8-#x2FF] | [#x370-#x37D] | [#x37F-#x1FFF] | [#x200C-#x200D] | [#x2070-#x218F] | [#x2C00-#x2FEF] | [#x3001-#xD7FF] | [#xF900-#xFDCF] | [#xFDF0-#xFFFD] | [#x10000-#xEFFFF]
	NameChar	   ::=	NameStartChar | "-" | "." | [0-9] | #xB7 | [#x0300-#x036F] | [#x203F-#x2040]

(**) In case you wonder: "", "ARABIC-INDIC", “EXTENDED ARABIC-INDIC", "NKO", "DEVANAGARI", "BENGALI", "GURMUKHI", "GUJARATI", "ORIYA", "TAMIL", "TELUGU", "KANNADA", “MALAYALAM”

(***) <TAB> <VT> <FF> <SP> <NBSP> <ZWNBSP> <USP>, <LF> <CR> <LS> <PS>
Where USP is ES2020’s name for \p{Zs}, i.e.
0020;SPACE;Zs;0;WS;;;;;N;;;;;
00A0;NO-BREAK SPACE;Zs;0;CS;<noBreak> 0020;;;;N;NON-BREAKING SPACE;;;;
1680;OGHAM SPACE MARK;Zs;0;WS;;;;;N;;;;;
2000;EN QUAD;Zs;0;WS;2002;;;;N;;;;;
2001;EM QUAD;Zs;0;WS;2003;;;;N;;;;;
2002;EN SPACE;Zs;0;WS;<compat> 0020;;;;N;;;;;
2003;EM SPACE;Zs;0;WS;<compat> 0020;;;;N;;;;;
2004;THREE-PER-EM SPACE;Zs;0;WS;<compat> 0020;;;;N;;;;;
2005;FOUR-PER-EM SPACE;Zs;0;WS;<compat> 0020;;;;N;;;;;
2006;SIX-PER-EM SPACE;Zs;0;WS;<compat> 0020;;;;N;;;;;
2007;FIGURE SPACE;Zs;0;WS;<noBreak> 0020;;;;N;;;;;
2008;PUNCTUATION SPACE;Zs;0;WS;<compat> 0020;;;;N;;;;;
2009;THIN SPACE;Zs;0;WS;<compat> 0020;;;;N;;;;;
200A;HAIR SPACE;Zs;0;WS;<compat> 0020;;;;N;;;;;
202F;NARROW NO-BREAK SPACE;Zs;0;CS;<noBreak> 0020;;;;N;;;;;
205F;MEDIUM MATHEMATICAL SPACE;Zs;0;WS;<compat> 0020;;;;N;;;;;
3000;IDEOGRAPHIC SPACE;Zs;0;WS;<wide> 0020;;;;N;;;;;

---

From: Glyn Normington <[email-masked]>
Date: Sat, 15 May 2021 20:09:28 +0100

> \w and \d are Unicode-enabled, so for instance \d includes (*)

These are hard decisions, especially if we have to form a consensus. I
wonder if regexp should be an extension point, or even a collection of
extension points, which would then allow multiple varieties of regexp
to be registered and used?

> Do we have a corpus of regexp-using JSON Path expressions we could use for guidance?

Probably not. I don't even think we have a corpus of JSONPath
expressions other than those used in the comparison project, which are
hardly representative of real-world usage.

---

From: Carsten Bormann <[email-masked]>
Date: Sat, 15 May 2021 22:32:56 +0200

> to be registered and used?

To me, that would be “giving up”.

I agree that having well-defined extension points is good.
(E.g., in CDDL we can just add control operators for different flavors of REs.)
But that doesn’t mean that we can’t have a basic flavour.

Even if we can’t (or, really, decide that we shouldn’t!) agree on the meaning of /\d+/, it would still be good for interoperability to have /[0-9]+/ and /\p{Nd}+/ — i.e., option (1) above is a valid fallback position.

> hardly representative of real-world usage.

Hmm.  So it seems we need to procure some corpora from the esteemed members of this WG…

Hey, JSONPathers!  Have any real-world examples that you could share?
In documentation, in actual traces, etc.
Even collections of tests might help (these aren’t going to be as “real-world”, as they will be corner cases, but they will probably still show corner cases that people care about).
What are the open-source projects that use JSONPath?

Grüße, Carsten

---

From: Glyn Normington <[email-masked]>
Date: Tue, 18 May 2021 12:08:14 +0100

Hi Carsten

You wrote:
> So I would propose to keep them [JSONPath and iregexp] separate documents.

Do you envisage there being a compliance test suite and/or a reference
implementation for iregexp? I doubt the JSONPath compliance test suite
could fully test compliance of regexps without becoming rather
bloated.

Regards,
Glyn

---

From: Michael Kay <[email-masked]>
Date: Tue, 18 May 2021 12:49:11 +0100

You might like to look at how the Perl regex tests have been integrated into the XQuery test suite:

See test fn-matches-50 at 

https://github.com/qt4cg/qt4tests/blob/master/fn/matches.xml

which reads data from

https://github.com/qt4cg/qt4tests/blob/master/fn/matches/perl-tests.xml

Michael Kay
Saxonica

> On 18 May 2021, at 12:08, Glyn Normington <[email-masked]> wrote:
> 
> Hi Carsten
> 
> You wrote:

---

From: Glyn Normington <[email-masked]>
Date: Tue, 18 May 2021 16:11:12 +0100

> https://github.com/qt4cg/qt4tests/blob/master/fn/matches/perl-tests.xml

Yes, that would be ideal. iregexp tests could be pulled into the
JSONPath tests and generate corresponding filter tests involving
regexps.

---

From: Greg Dennis <[email-masked]>
Date: Tue, 18 May 2021 15:45:47 +0000 (UTC)

Additionally, here's the JSON Schema `pattern` tests.  JSON Schema specifies "SHOULD" compliance with RFC 262.
https://www.github.com/json-schema-org/JSON-Schema-Test-Suite/tree/master/tests%2Fdraft2020-12%2Foptional%2Fecmascript-regex.json
These tests are optional due to the "SHOULD", but it's still a good example of test layout.
Greg

  On Tue, 18 May 2021 at 11:49 pm, Michael Kay<[email-masked]> wrote:   You might like to look at how the Perl regex tests have been integrated into the XQuery test suite:

which reads data from

Michael Kay
Saxonica

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 18 May 2021 18:27:28 +0200

Hi Michael,

thank you for this great input!
Do you recollect what the different classes of match results mean?

$ grep -o 'result="."' perl-tests.xml  | sort | uniq -c
 122 result="c"
 174 result="n"
 775 result="p"
 522 result="y"

It seems to me the driver in matches.xml only considers y and n, while almost half of the tests are p.

Grüße, Carsten

> On 2021-05-18, at 13:49, Michael Kay <[email-masked]> wrote:
>

---

From: Michael Kay <[email-masked]>
Date: Tue, 18 May 2021 18:13:35 +0100

p and c are clearly invalid regular expressions, where fn-matches-50 is expected to throw an error, but I don't recall the distinction between the two. Perhaps the distinction is whether it is valid in Perl?

Michael Kay
Saxonica

> On 18 May 2021, at 17:27, Carsten Bormann <[email-masked]> wrote:
> 
> Hi Michael,
>