Subject: [apps-discuss] json-pointer in fragment identifiers

From: Mark Nottingham <[email-masked]>
Date: Wed, 27 Jun 2012 13:35:26 +1000

I think this has been brought up before, but couldn't find where I'd seen it.

In our current draft, it says:

>             <t>A JSON Pointer can be represented in a URI fragment identifier.
>             The JSON pointer MUST be <xref target="RFC3629">UTF-8</xref>
>             encoded as octets; octets not in the URI "unreserved" set SHOULD
>             be percent-encoded, per <xref target="RFC3986"/>, section 2.5.</t>

This is far more restrictive than RFC3986, which defines a fragment identifer as:

>    fragment      = *( pchar / "/" / "?" )

>    pchar         = unreserved / pct-encoded / sub-delims / ":" / "@"

as becomes evident once you look at the examples I just generated, e.g.:

>  #%2Ffoo%5B0%5D     "bar"

I think we need to make this advisory (no need to re-require what 3986 specifies), as well as more permissive. 

Proposal:

"""
A JSON Pointer can be represented in a URI fragment identifier by encoding it using UTF-8 [ref] into octets, percent-encoding those characters not allowed by the fragment rule in RFC3986.
"""

Thoughts?

--
Mark Nottingham   http://www.mnot.net/

---

From: "Manger, James H" <[email-masked]>
Date: Wed, 27 Jun 2012 14:35:12 +1000

Changeset 70 <http://trac.tools.ietf.org/wg/appsawg/trac/changeset/70/json-pointer/draft-ietf-appsawg-json-pointer.xml> seems to have some errors:

1. The leading "/" is present for the 1st 3 JSON pointers, but missing from the next 5. For instance "c%d" should be "/c%d"

2. "/foo[0]" should be "/foo/0". The former was a suggestion (from Julien?), but I don't think it was adopted.

3. A JSON string holding a pointer to 5 in { "i\\j":5 } should be "/i^\\j" (not "i\\j")

4. "k\"l" should be "/k^\"l"

5. Most of the fragment id examples are wrong in the same ways as points 1, 2, 3 & 4 above

6. #g%7C should be #g%7Ch

After the JSON document, I would be tempted to list the raw JSON pointers (eg /i^\j) then list the JSON pointers as JSON strings (eg "/i^\\j") then the pointers as fragments (eg #%2Fi%5E%5Cj).

P.S. the proposal below looks good:

> A JSON Pointer can be represented in a URI fragment identifier by
> encoding it using UTF-8 [ref] into octets, percent-encoding those
> characters not allowed by the fragment rule in RFC3986.
> """
> 
> Thoughts?

--
James Manger

---

From: "Manger, James H" <[email-masked]>
Date: Wed, 27 Jun 2012 14:44:24 +1000

"/" does not need to be escaped in a fragment so unescape %2F in all the fragment examples.

#/foo
#/foo/0
#/c%25d
#/e%5E%5Ef
#/g%7Ch
#/i%5E%5Cj
#/k%22l

--
James Manger

> 6. #g%7C should be #g%7Ch

Opps. Should be #/g%7Ch

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 27 Jun 2012 14:47:42 +1000

I didn't want to spend too much time on them until we had agreement on the change I suggested.

Since I've got at least one +1 now, I'll incorporate that and update the examples.

Thanks!

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 27 Jun 2012 15:02:17 +1000

See <http://trac.tools.ietf.org/wg/appsawg/trac/changeset/77>; I *think* it's correct now.

The choice of the caret as an escape character is a bit unfortunate in the URI fragment encoding, because it's not in sub-delims (or the other chars allowed in fragment).

A few comments below.

> 2. "/foo[0]" should be "/foo/0". The former was a suggestion (from Julien?), but I don't think it was adopted.

Ah, old habits die hard.

> 3. A JSON string holding a pointer to 5 in { "i\\j":5 } should be "/i^\\j" (not "i\\j")

We don't caret escape backslash...

> 4. "k\"l" should be "/k^\"l"

Same as above.

> After the JSON document, I would be tempted to list the raw JSON pointers (eg /i^\j) then list the JSON pointers as JSON strings (eg "/i^\\j") then the pointers as fragments (eg #%2Fi%5E%5Cj).

Yeah. Let's let things settle a bit.

Thanks again,

---

From: "Manger, James H" <[email-masked]>
Date: Wed, 27 Jun 2012 15:20:45 +1000

> *think* it's correct now.

Getting close.

Line 157 should be
  "/"   0 
instead of
  ""    0

The "m/n" element is unnecessary as we have the "a/b" element so lines 145, 164, and 191 can go.

Line 185 should be
  #/a%5E/b    1
instead of
  #/a/b       1

--
James Manger

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 27 Jun 2012 15:28:10 +1000

Done in 78. Thanks for the extra pair of eyes.

Cheers,

---

From: Julian Reschke <[email-masked]>
Date: Wed, 27 Jun 2012 09:33:58 +0200

On 2012-06-27 05:35, Mark Nottingham wrote:
> ...

I believe we should discuss 
<http://trac.tools.ietf.org/wg/appsawg/trac/ticket/10> first (which is 
about whether to do fragment identifiers at all).

On the other hand, I agree that the escaping should just follow from the 
character repertoire and RFC 3986. Note that JSON pointers in path 
segments might be much more interesting than in fragment identifiers...

Best regards, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 27 Jun 2012 17:38:19 +1000

On 27/06/2012, at 5:33 PM, Julian Reschke wrote:

> I believe we should discuss <http://trac.tools.ietf.org/wg/appsawg/trac/ticket/10> first (which is about whether to do fragment identifiers at all).

Was that intended for json-pointer? If so, I'll change the details (right now it says it's about patch).

> On the other hand, I agree that the escaping should just follow from the character repertoire and RFC 3986.

I'll take that as a +1, thank you.

> Note that JSON pointers in path segments might be much more interesting than in fragment identifiers...

Indeed.

---

From: "Martin J. Dürst" <[email-masked]>
Date: Wed, 27 Jun 2012 16:50:35 +0900

Hello Mark,

On 2012/06/27 12:35, Mark Nottingham wrote:
>>              The JSON pointer MUST be<xref target="RFC3629">UTF-8</xref>
>>              be percent-encoded, per<xref target="RFC3986"/>, section 2.5.</t>
>

RFC 3986 does not nail down UTF-8 for fragment identifiers, and your new 
text makes that advisory, so we could end up with fragment identifiers 
based on Shift_JIS, Latin-1, and other legacy stuff, which we definitely 
don't want. So:

- Use of UTF-8 has to be mandatory (for interoperability),
   and we have to mandate it in this spec. That's where your proposal
   falls short.

- %-encoding for bytes/octets not allowed in fragment identifiers is
   mandatory, but that's a given in RFC 3986 (and somewhat differently
   in RFC 3987, because in IRIs, you can use Unicode characters).

- There is no need for %-escaping sub-delims or ":" or "@", so we
   should not require that.

Regards,   Martin.

---

From: Julian Reschke <[email-masked]>
Date: Wed, 27 Jun 2012 09:55:59 +0200

> Was that intended for json-pointer? If so, I'll change the details (right now it says it's about patch).

Yes and no :-). json-patch contains sections that suggest that fragment 
identifiers are defined, and uses the syntax defined in json-pointer.

Best regards, Julian