Subject: [Json] JSON-text for sequences

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Mon, 25 Aug 2014 16:20:05 +0000

Starting a new thread as Paul requested.

"JSON-text" in the ABNF is wrong, in my opinion.  A sequence can be 
syntactically valid with invalid (e.g. truncated) JSON-text in the 
sequence.  I suggest:

potential-JSON-text = <valid JSON-text as given by RFC7159, otherwise 
ignored>

Or something to that effect.

On 8/23/14, 3:08 PM, "Paul Hoffman" <[email-masked]> wrote:

>As much as it is fun to keep debating old topics, it would help the WG if 
>folks could focus on the current work. We would like to finish up, and 
>the following WG LC is a way for us to determine how close we are  to 
>that.
>
>--Paul Hoffman
>
>On Aug 22, 2014, at 10:34 AM, Paul Hoffman <[email-masked]> wrote:
>
>> Greetings again. The first WG Last Call ended up with no consensus 
>>about whether the document was useful as it stood. Nico has made a few 
>>rounds of changes, and so I'd like to ask folks to read it again, fresh.
>> 
>> Please respond on the list whether the technical proposal in the new 
>>document could be useful in meeting the use cases and scenarios 
>>described in the document. Note that this is *not* a request for 
>>comments about whether or not you liked the old proposal better; it is a 
>>request about the current document.
>> 
>> The current document:
>> 
>> URL:            
>>http://www.ietf.org/internet-drafts/draft-ietf-json-text-sequence-06.txt
>> Status:         
>>https://datatracker.ietf.org/doc/draft-ietf-json-text-sequence/
>> Htmlized:       
>>http://tools.ietf.org/html/draft-ietf-json-text-sequence-06
>> Diff:           
>>http://www.ietf.org/rfcdiff?url2=draft-ietf-json-text-sequence-06
>
>_______________________________________________
>json mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/json
>

-- 
Joe Hildebrand

---

From: Paul Hoffman <[email-masked]>
Date: Mon, 25 Aug 2014 09:44:43 -0700

<no hat>

> Or something to that effect.

What is the purpose of defining that? The protocol is meant for JSON-texts, but also *happens* to work for potential-JSON-texts. Why complicate the definition?

--Paul Hoffman

---

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Mon, 25 Aug 2014 17:04:45 +0000

On 8/25/14, 4:44 PM, "Paul Hoffman" <[email-masked]> wrote:

>What is the purpose of defining that? The protocol is meant for 
>JSON-texts, but also *happens* to work for potential-JSON-texts. Why 
>complicate the definition?

Because ABNF doesn't define partial failures in parsing, as far as I know. 
With the ABNF as-is, technically I should stop parsing the entire file 
when I get one corrupt entry.

-- 
Joe Hildebrand

---

From: Paul Hoffman <[email-masked]>
Date: Mon, 25 Aug 2014 11:35:42 -0700

> when I get one corrupt entry.

OK, I get it. That's not the way that I think of using ABNF, but I trust you as a developer on that. So, the draft should in fact specify what type of thing is being surrounded by the sequence delimiters.

--Paul Hoffman

---

From: Tim Bray <[email-masked]>
Date: Mon, 25 Aug 2014 11:45:49 -0700

Well, Nico’s nice “not necessarily fatal” section is another way of
achieving the same goal without trying to push nondeterministic situations
into abnf.

On Mon, Aug 25, 2014 at 11:35 AM, Paul Hoffman <[email-masked]>
wrote:

> OK, I get it. That's not the way that I think of using ABNF, but I trust
> you as a developer on that. So, the draft should in fact specify what type
> of thing is being surrounded by the sequence delimiters.
>
> --Paul Hoffman

-- 
- Tim Bray (If you’d like to send me a private message, see
https://keybase.io/timbray)

---

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Mon, 25 Aug 2014 21:22:10 +0000

On 8/25/14, 6:45 PM, "Tim Bray" <[email-masked]> wrote:

>achieving the same goal without trying to push nondeterministic 
>situations into abnf.

Can we put a couple of extra words in the comment at least?  Example:

JSON-text = <given by RFC7159, but see section 2.1>

-- 
Joe Hildebrand

---

From: Nico Williams <[email-masked]>
Date: Tue, 26 Aug 2014 00:58:37 -0500

> achieving the same goal without trying to push nondeterministic situations

+1.  ABNF is nice, but it's not ASN.1, and I doubt anyone will be
generating code from it for this.

Nico
--

---

From: Nico Williams <[email-masked]>
Date: Tue, 26 Aug 2014 00:59:00 -0500

> JSON-text = <given by RFC7159, but see section 2.1>

Sure!

---

From: Julian Reschke <[email-masked]>
Date: Tue, 26 Aug 2014 08:14:06 +0200

> JSON-text = <given by RFC7159, but see section 2.1>

That's the ABNF equivalent of hand-waving.

If a parser is supposed not to require correct JSON, than the ABNF 
should be in sync with that (essentially enumerating the code points 
allowed here, and not saying anything more).

Best regards, Julian

---

From: Nico Williams <[email-masked]>
Date: Tue, 26 Aug 2014 01:41:27 -0500

On Tue, Aug 26, 2014 at 1:14 AM, Julian Reschke <[email-masked]> wrote:
> If a parser is supposed not to require correct JSON, than the ABNF should be
> in sync with that (essentially enumerating the code points allowed here, and
> not saying anything more).

Yes, it is handwaving.  It'd be trivial to write the ABNF so any octet
is anllowed
other than RS, in between RSes, and then we's say "attempt to parse
the entry as a JSON-text".  Would that be better?

We do this sort of thing in ASN.1-land all the time: OCTET STRING with
a comment saying what's meant to be in the OCTET STRING.

Nico
--

---

From: Julian Reschke <[email-masked]>
Date: Tue, 26 Aug 2014 08:54:34 +0200

> the entry as a JSON-text".  Would that be better?

Yes. That would make it very lcear how the format is parsed.

> a comment saying what's meant to be in the OCTET STRING.

I'd recommend to have a precise ABNF description of the character ranges 
allowed, and to have a comment such as "all characters except RS" (if 
that's what it is...).

Best regards, Julian

---

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Tue, 26 Aug 2014 16:17:01 +0000

>allowed, and to have a comment such as "all characters except RS" (if 

I think that's right, since you don't want the parser to choke even if 
some mangled UTF8 comes in the JSON piece.

So:

JSON-sequence = *(1*RS potential-JSON) <or whatever we decide>
     RS = %x1E; See RFC20
potential-JSON = *not-RS
not-RS = %x01-1D / %x1F-FF

This makes it clear that you're parsing bytes at this level, NOT any 
unicode encoding.  We probably have *always* needed some text about 
encoding in the draft, so I suggest adding a section:

2.x Encoding

The format, when encoded into bytes for transmission or storage, MUST be 
stored as UTF-8, however consumers of the format MUST process 
potential-JSON items with invalid encoding in the same way they process 
other invalid JSON (e.g. ignore the error and move to the next RS).

(or something.  that's just a starting point)

And then modify section 2.1 to also explicitly cover all invalid JSON, not 
just incomplete JSON.

-- 
Joe Hildebrand