Subject: [Json] I-D Action: draft-ietf-json-text-sequence-02.txt

From: [email-masked]
Date: Fri, 09 May 2014 10:31:59 -0700

A New Internet-Draft is available from the on-line Internet-Drafts directories.
 This draft is a work item of the JavaScript Object Notation Working Group of the IETF.

        Title           : JavaScript Object Notation (JSON) Text Sequences
        Author          : Nicolas Williams
	Filename        : draft-ietf-json-text-sequence-02.txt
	Pages           : 10
	Date            : 2014-05-09

Abstract:
   This document describes the JSON text sequence format and associated
   media type.

The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-json-text-sequence/

There's also a htmlized version available at:
http://tools.ietf.org/html/draft-ietf-json-text-sequence-02

A diff from the previous version is available at:
http://www.ietf.org/rfcdiff?url2=draft-ietf-json-text-sequence-02

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

---

From: Paul Hoffman <[email-masked]>
Date: Wed, 14 May 2014 11:32:35 -0700

Greetings again. Wearing my chair hat, I encourage people in the group (especially you quiet types) to review this WG draft and let the WG know what you think. The idea of JSON text sequences has many applications, and folks have invented things like this in the past for their own internal purposes. We would like to make sure that this document covers as many use cases as possible.

--Paul Hoffman

---

From: Stefan Drees <[email-masked]>
Date: Sun, 18 May 2014 10:56:56 +0200

On 2014-05-14 20:32 +01:00, Paul Hoffman wrote:
> Greetings again. Wearing my chair hat, I encourage people in the
> group(especially you quiet types) to review this WG draft and let the WG know
> what you think. The idea of JSON text sequences has many applications,
> and folks have invented things like this in the past for their own
> internal purposes. We would like to make sure that this document covers
> as many use cases as possible.
>
> --Paul Hoffman

I like to understand grammars also bottom up - which might qualify as a 
lexer's perspective - overlapping rules like the one for ws1 and ws2 in 
the suggested grammar do not really make things clearer for me. 
Additionally the reader will benefit from the expressed clear 
assumption, how these ws(''|1|2) relate to each other.

So, I suggest to change (as found in draft-02):

# --- 8< ----
2.  JSON Text Sequence Format

    The ABNF [RFC5234] for the JSON text sequence format is as follows:

      JSON-sequence = *ws1 *(JSON-text *ws2 %x0A *ws1)
      ws1 = %x20 / %x09 / %x0A / %x0D
      ws2 = %x20 / %x09 / %x0D
      JSON-text = <given by RFC7159>

                      Figure 1: JSON text sequence ABNF

# --- 8< ----

into either:

# --- 8< ----
2.  JSON Text Sequence Format

    The ABNF [RFC5234] for the JSON text sequence format is as follows:

      JSON-sequence = *(ws-no-nl
                        / nl) *(JSON-text *ws-no-nl nl *(ws-no-nl / nl))
      nl = %x0A
      ws-no-nl = ( %x20 / %x09 / %x0D )
      JSON-text = <given by RFC7159>

                      Figure 1: JSON text sequence ABNF

    When combining the grammars, the rule for ws in the JSON-text is
    therefore equivalent to *( ws-without-newline / newline )

# --- 8< ----

or less cryptic but wrapped some more:

# --- 8< ----
2.  JSON Text Sequence Format

    The ABNF [RFC5234] for the JSON text sequence format is as follows:

      JSON-sequence = *(
         ws-without-newline /
         newline ) *( JSON-text *ws-without-newline newline *(
             ws-without-newline / newline ) )
      newline = %x0A
      ws-without-newline = ( %x20 / %x09 / %x0D )
      JSON-text = <given by RFC7159>

                      Figure 1: JSON text sequence ABNF

    When combining the grammars, the production for ws in the JSON-text
    is therefore equivalent to *( ws-without-newline / newline )

# --- 8< ----

  ... but I am *not* passionate about this.

It is just that the mix of optional sets, where one is a proper subset 
of the other to me smells very implicit, where being explicit (choosing 
a direct decomopsition like "X/x and x" to me reads well and is not that 
much longer.

What do the others think?

PS: If the parentheses in the simple rules do not match our current 
styleguide, just remove them, it is the decomposition I am interested in.

All the best,
Stefan

> _______________________________________________
> json mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/json
>

---

From: Stefan Drees <[email-masked]>
Date: Sun, 18 May 2014 11:48:09 +0200

On 2014-05-18 10:56 +01:00, Stefan Drees sent during streaming read 
> So, I suggest to change (as found in draft-02):

*preferred* (cf. below)

>   ... but I am *not* passionate about this.

Given the short length, focused style of the document, I think these 
changes do form a good contribution towards clarity and readability.

> styleguide, just remove them, it is the decomposition I am interested in.

and of course adapt accordingly figure 2 located in section "3.  Use for 
Logfiles, or How to Resynchronize Following Truncated entries"

from the current:

# --- 8< ----
     boundary = endchar *ws2 %0xA *ws1 startchar
     endchar = ( "}" / "]" / %x22 / "e" / "l" / DIGIT )
     startchar =  ( "{" / "[" / %x22 / "t" / "f" / "n" / "-" / DIGIT )
# --- 8< ----

to the (short version):
# --- 8< ----
     boundary = endchar *ws-no-nl nl *(ws-no-nl / nl) startchar
     endchar = ( "}" / "]" / %x22 / "e" / "l" / DIGIT )
     startchar =  ( "{" / "[" / %x22 / "t" / "f" / "n" / "-" / DIGIT )
# --- 8< ----

or *preferred* (more explicit version):
# --- 8< ----
     boundary = endchar *ws-without-newline newline *(
                    ws-without-newline / newline) startchar
     endchar = ( "}" / "]" / %x22 / "e" / "l" / DIGIT )
     startchar =  ( "{" / "[" / %x22 / "t" / "f" / "n" / "-" / DIGIT )
# --- 8< ----

Note: Aliasing %0xA with a readable term lke newline to me shines as a 
service to the reader, emphasizing the special role this token has in 
the recommendation which is so nicely stated in:

# --- 8< ----
    Requirements:

    o  JSON text sequence encoders MUST emit a newline after any JSON
       text.
# --- 8< ----

So the longer versions presented above are the ones, I would prefer.
(those with "newline" instead of "nl").

Sorry for streaming feedback mode,
Stefan.

---

From: Nico Williams <[email-masked]>
Date: Tue, 20 May 2014 20:17:12 -0500

On Sun, May 18, 2014 at 3:56 AM, Stefan Drees <[email-masked]> wrote:
> lexer's perspective - overlapping rules like the one for ws1 and ws2 in the
> suggested grammar do not really make things clearer for me. Additionally the
> reader will benefit from the expressed clear assumption, how these
> ws(''|1|2) relate to each other.

I'm open to rewriting it but... there are many ways to do it and I
don't really want to bikeshed the grammar -- it's enough to make sure
it's correct.

> [...]

My counter:

  JSON-sequence = *ws-nl *(JSON-text *ws-no-nl %x0A *ws-nl)
  ws-no-nl = %x20 / %x09 / %x0D
  ws-nl = ws-no-nl / %x0A
  JSON-text = <given by RFC7159>

or, since I agree with you about giving %xXX names:

  JSON-sequence = *ws-LF *(JSON-text *ws-no-LF %x0A *ws-LF)
  ascii-SP = %x20
  ascii-HT = %x09
  ascii-CR = %x0D
  ascii-LF = %x0A
  ws-no-LF = ascii-SP / ascii-HT / ascii-CR
  ws-LF = ws-no-LF /  ascii-LF ; same as ws in RFC7159
  JSON-text = <given by RFC7159>

Or we could use pre-defined rules from RFC5234:

  ; SP HTAB CR LF are defined in RFC5234
  JSON-sequence = *ws-LF *(JSON-text *ws-no-LF %x0A *ws-LF)
  ws-no-LF = SP HTAB CR
  ws-LF = ws-no-LF /  LF ; same as ws in RFC7159
  JSON-text = <given by RFC7159>

But maybe we should use ws from RFC7159, or maybe we shouldn't given
single characters names but instead name them in comments like RFC7159
does:

  JSON-sequence = *ws-LF *(JSON-text *ws-no-LF %x0A *ws-LF)
  ws-no-nl = SP HTAB CR
  ws-LF = ws ; ws is defined in RFC7159
  JSON-text = <given by RFC7159>

There's a lot of variations.  I see you refer to a styleguide -- is there one?

> What do the others think?

It's just a grammar :)

Nico
--

---

From: Nico Williams <[email-masked]>
Date: Tue, 20 May 2014 20:28:18 -0500

>   ws-no-LF = SP HTAB CR

Missing alternators...  Anyways, my final offer:

  JSON-sequence = *ws-lf *(JSON-text *ws-no-lf LF *ws-no-lf)
  ws-no-lf = SP / HTAB / CR
  ws = <given by RFC7159>
  JSON-text = <given by RFC7159>

and

 boundary = endchar *ws-no-lf LF *ws-lf startchar
 endchar = ( "}" / "]" / DQUOTE / "e" / "l" / DIGIT )
 startchar =  ( "{" / "[" / DQUOTE / "t" / "f" / "n" / "-" / DIGIT )

I was already using DIGIT from RFC5234, so using DQUOTE, SP, HTAB, CR,
and LF seems right, including the upper case (though it doesn't
matter).  I'm surprised RFC7159 doesn't use them (it uses DIGIT, oddly
enough, but not the others).

Nico
--

---

From: Nico Williams <[email-masked]>
Date: Tue, 20 May 2014 22:36:03 -0500

James Manger points out that I can't help the greedy parser problem
because the JSON-text rule uses the ws rule.  Well, yeah.  Therefore
the ABNF can be a bit simpler.

---

From: Stefan Drees <[email-masked]>
Date: Wed, 21 May 2014 07:32:58 +0200

> the ABNF can be a bit simpler.

I seem to have not received that mail of James Manger. But maybe some 
feedback on top of that "bridged" comment:

We define a profile here, extending JSON-text (seen as a core).
So how someone implemented her JSON-text parser is not our business.
How we suggest parsing our decorating functionality is our business.
Maybe the single nicely coined "natural language rule" would suffice.

If we use ABNF, I suggested how to translate the simple recipe of this 
profile "just add at least one newline character between JSON-texts and 
you're done" into a concise ABNF plus relating (as a service) to the 
overlapping construct in the referenced JSON-text grammar.
why not have the same term stand out in the grammar, that we use in the 
nicely coined "natural language rule"?

All the best,
Stefan.

---

From: Nico Williams <[email-masked]>
Date: Wed, 21 May 2014 14:19:11 -0500

I'll give two ABNFs:

  JSON-sequence = *(JSON-text LF)
  LF = <given by RFC5234>
  JSON-text = <given by RFC7159>

and,

  JSON-sequence = *ws *(value text-sep *ws)
  text-sep = *(SP / HTAB / CR) LF ; these are from RFC5234
  ws = <given by RFC7159>
  value = <given by RFC7159>

<hand-waving>
The latter being the alternative to the first for dealing with ABNF
greediness.  Greediness is not specified in RFC5234 NAICT.  Nor do we
speak of which rules are "exported"/"imported" by "modules".
</hand-waving>

I think this is good enough.  I doubt anyone will ever use ABNF
tooling to produce a JSON text sequence parser.  Two alternative
formulations + prose should be sufficient, if not excessive.

For the resync ABNF I'll give this:

  boundary = endchar text-sep *ws startchar
  text-sep = *(SP / HTAB / CR) LF ; these are from RFC5234
  endchar = ( "}" / "]" / DQUOTE / "e" / "l" / DIGIT )
  startchar =  ( "{" / "[" / DQUOTE / "t" / "f" / "n" / "-" / DIGIT )

Nico
--

---

From: "Manger, James" <[email-masked]>
Date: Thu, 22 May 2014 15:21:37 +1000

>  JSON-sequence = *(JSON-text LF)

No, because that means a payload of, say, 2 newlines is invalid. It should be a valid sequence of 0 JSON items. It is too easy to get stray newlines to reject them.

>  value = <given by RFC7159>

This still does not deal with greedy parsers as <value> includes leading and trailing optional whitespace for objects and arrays (but not for primitives).

Acceptable versions:

  JSON-sequence = *(JSON-text LF) ws
or
  JSON-sequence = ws *(JSON-text ws LF ws)

with
  ws = <given by RFC7159> ; optional whitespace
  JSON-text = <given by RFC7159>
  LF = <given by RFC5234> ; linefeed %x0A

--
James Manger

---

From: Nico Williams <[email-masked]>
Date: Thu, 22 May 2014 12:22:57 -0500

> No, because that means a payload of, say, 2 newlines is invalid. It should be a valid sequence of 0 JSON items. It is too easy to get stray newlines to reject them.

Good catch.

> This still does not deal with greedy parsers as <value> includes leading and trailing optional whitespace for objects and arrays (but not for primitives).

Ah, yes, because begin-* and end-* have ws.  Oh well, then I give up
on the greedy matching issue.

>   LF = <given by RFC5234> ; linefeed %x0A

I'll take the second.  Thanks!

Nico
--