Subject: [apps-discuss] ABNF changes in draft-ietf-appsawg-http-forwarded-06

From: Julian Reschke <[email-masked]>
Date: Mon, 23 Jul 2012 20:15:32 +0200

Hi there,

I just became aware of some ABNF related changes in the latest draft; 
see 
<http://tools.ietf.org/html/draft-ietf-appsawg-http-forwarded-06#section-4>:

        Forwarded   = 1#forwarded-element

        forwarded-element =
            [ forwarded-pair ] *[ ";" forwarded-pair ]

        forwarded-pair = token "=" value
        value          = token / quoted-string

        node     = nodename [ ":" node-port ]
        nodename = IPv4address / "[" IPv6address "]" /
                    "unknown" / obfnode

        IPv4address = <Defined in [RFC3986], Section 3.2.2>
        IPv6address = <Defined in [RFC3986], Section 3.2.2>
        obfnode = "_" 1*( ALPHA / DIGIT / "." / "_" / "-")

        node-port     = port / obfport
        port          = 1*5DIGIT
        obfport       = "_" 1*(ALPHA / DIGIT / "." / "_" / "-")

        token = <Defined in
            [I-D.ietf-httpbis-p1-messaging], Section 3.2.4>
        quoted-string = <Defined in
            [I-D.ietf-httpbis-p1-messaging], Section 3.2.4>

        DIGIT = <Defined in [RFC5234], Section 3.4>
        ALPHA = <Defined in [RFC5234], Section B.1>

1) The change to forwarded-element makes leading and trailing semicolons 
invalid. I believe that's a bad idea, and also contrary to prior WG 
discussions. Disallowing them in the ABNF will have zero effect on 
implementations (we know that from other header fields that use similar 
syntax), so pretending they won't happen isn't useful. It just creates a 
situation where some recipients accept them and some do not.

2) Moving the "node" ABNF here is very misleading; these productions are 
used in *values*, after potential unescaping of quoted-string; so at a 
completely different processing level.

3) Nit: the prose productions for token and quoted-string are invalid as 
ABNF doesn't allow CRLF here. That probably will resolve itself once the 
reference is replaced with a shorter string.

Best regards, Julian

---

From: Barry Leiba <[email-masked]>
Date: Mon, 23 Jul 2012 14:45:05 -0400

> situation where some recipients accept them and some do not.

This was done unintentionally: I asked a question in my AD review, and
Andreas responded with the ABNF change.  I'm sure he'll be happy to
change that one back.

> completely different processing level.

I don't understand what you're complaining about here.  Is it just the
ordering of the ABNF productions?  If so, suggest specifically where
"node" should go in the order.  If it's that you don't want the ABNF
all in one place, we're going to have a row about that, because I
strongly think that it's easier on implementors to have it together.

> reference is replaced with a shorter string.

Yes; it might help to put in a note to the RFC Editor about this.
I'll do that.,

Barry

---

From: Julian Reschke <[email-masked]>
Date: Mon, 23 Jul 2012 21:28:33 +0200

> Andreas responded with the ABNF change.  I'm sure he'll be happy to

Ok.

> strongly think that it's easier on implementors to have it together.

I'd like to be where it was.

Having the productions in a single place suggests that they are somehow 
related to each other, which they are not at all.

> ...

Best regards, Julian

---

From: Barry Leiba <[email-masked]>
Date: Mon, 23 Jul 2012 15:37:42 -0400

> related to each other, which they are not at all.

Nonsense.  Having them in a single place makes them easier to find and
easier to refer to.

I'm happy to defer to you on the content of the productions.  But for
the arrangement, I think it's a bad idea to have small bits of ABNF in
different places in the document.

Does anyone besides Julian and me have a comment on this point?

Barry

---

From: Julian Reschke <[email-masked]>
Date: Mon, 23 Jul 2012 21:47:27 +0200

> Nonsense.  Having them in a single place makes them easier to find and

I have to disagree. ABNF productions should be placed where they are 
used (or referenced from there). If these turn out to be "many" places, 
one should consider an appendix with all productions, an index, or both.

Again: if somebody writes a parser that deals with both "forwarded-pair" 
and "node" in the same piece of code, then that parser is VERY likely 
broken.

This header field uses a generic name/value syntax, and then puts 
additional constrains on certain values (after unescaping). Just because 
both use ABNF doesn't mean that one thing has anything to do with the 
other thing.

> Does anyone besides Julian and me have a comment on this point?

The same issue will come up very soon in HTTPbis; if you disagree how 
we're doing things over there it would be very good to find out about 
that before we're after LC.

Best regards, Julian

---

From: Barry Leiba <[email-masked]>
Date: Mon, 23 Jul 2012 16:20:06 -0400

> I have to disagree. ABNF productions should be placed where they are used
> (or referenced from there). If these turn out to be "many" places, one
> should consider an appendix with all productions, an index, or both.

Ack; we disagree.  Neither unusual, nor disastrous.

> Again: if somebody writes a parser that deals with both "forwarded-pair" and
> "node" in the same piece of code, then that parser is VERY likely broken.

Oh, I think the component you're referring to isn't the parser (the
same parser absolutely will be used for both in many implementations),
but the lexical processor.  But that doesn't really matter; you're
right that if someone tries to treat a "node" as the same part of
speech as a "forwarded-pair", we have problems.

> The same issue will come up very soon in HTTPbis; if you disagree how we're
> doing things over there it would be very good to find out about that before
> we're after LC.

How about this as a compromise; I would be very happy with this alternative:

The ABNF stays together in the same section, but it is separated into
groupings (or subsections, that would be OK too) to make it clear that
the grouping that contains "node" is lexically different to the other
grouping(s).

What do you think?  The productions could be grouped as they were
before, along with forward references to where they're discussed.

Barry

---

From: Alexey Melnikov <[email-masked]>
Date: Mon, 23 Jul 2012 21:24:16 +0100

Hi,

On 23 Jul 2012, at 20:28, Julian Reschke <[email-masked]> wrote:

> Having the productions in a single place suggests that they are somehow related to each other, which they are not at all.
> 

My 2p: having ABNF in one place is Ok. Besides it means that all productions are defined before being referenced.

---

From: Ned Freed <[email-masked]>
Date: Mon, 23 Jul 2012 13:24:53 -0700 (PDT)

> Nonsense.  Having them in a single place makes them easier to find and

Indeed it does. But it also means that the text referencing the ABNF may
now be disconnected from it.

> Does anyone besides Julian and me have a comment on this point?

It's a tradeoff and there is no one right answer. Of the three options:

(1) ABNF where the text defines/refers to the construct.
(2) Collected ABNF.
(3) Both (1) and (2).

The only one I think has no place is (3), and that's only because our
relatively weak tools have no means of doing the collecting for us, leading
to silly states when revisions are made.

Generally speaking I prefer (1) when there are only a few bits of ABNF in
the document. (2) is strongly preferred when you're defining, say, an entire
language. Anything in between is a judgement call.

Another factor that needs to be considered is whether or not the text talks
about or elaborates on specific ABNF details. In that case the ABNF needs
to be adjacent. Otherwise I tend to perfer collecting.

Getting back to the document at hand, one problem I see is that the recent
change has the text and ABNF at odds with each other: There's now a section
that purports to describe a header field but it now includes ABNF that isn't
directly references into that definition at all. That's confusing, but it can
easily be solved by moving the collected ABNF to the end in a collected ABNF
section.

However, that really doesn't address the main issue here, which is that this
document is using something I call a syntax overlap. Specifically, there's a
general ABNF for the field, then there is additional ABNF that kinda sorta
replaces one ABNF production (value) with another (node). Except it doesn't
really do that, because that blurs the distinction between a token and a quoted
string. It's ABNF restricting what can appear inside a value, and AFAIK we have
no representation for that sort of thing.

The original MIME RFC was full of this sort of stuff, and for exactly the same
reason: We were trying to restrict the syntax of media type parameter values
And people complained about it because it was - big duh here - inconsistent and
therefore quite confusing.

When I took over editorship from Nathaniel I spent a bunch of time trying to
getting rid of the inconsistencies without adding unnecessary complexity, and
failing miserably. I then took a much more radical step: I deleted every last
bit of the overlapping ABNF. And if memory serves, exactly one person
complained, and they went silent when I asked them how they would have fixed
the inconsistency issue.

My takeaway from that is overlapping ABNF should be used sparingly if at all,
but when it is unavoidable keeping it separate and unlinked is actuallly
preferablea. As a result my fairly strong preference in this case is to go with
the approach in the previous draft version. But this has everything to do with
the overlap, and nothing to do with how ABNF is best presented when there is no
overlap. If there was no overlap I'd prefer to have all the ABNF in one place.

				Ned

---

From: Julian Reschke <[email-masked]>
Date: Tue, 24 Jul 2012 09:54:36 +0200

> same parser absolutely will be used for both in many implementations),

What I'm trying to say is that the natural way to process this header is 
a two-step process (starting with the field value extracted from the 
HTTP header block):

1) Parse into a list of forwarded-elements, containing lists of 
forwarded-pairs, consisting of parameter name and value (unescaping the 
quoted-string value when needed).

2) Parse the values inside the forwarded-pair parameter values according 
to the syntactical constraints of the given parameter.

The ABNF needed for 1) belongs into Section 4. The ABNF fragments for 2) 
belong into Section 5 (or later).

There's absolutely no point to present ABNF productions that haven't 
anything to do with 1) in Section 4.

> before, along with forward references to where they're discussed.

I think it's marginally better, but still much much worse than what we 
had before.

> no representation for that sort of thing.

That's exactly what's going on here, and it would be cool if we could 
come up with a recommendation how to deal with cases like these.

The ABNF in the draft we're discussing defines *multiple* grammars; one 
for parsing the field value in general, and then multiple mini-grammars 
for specific parameter values. These have nothing to do with each other, 
except for potentially re-using common ABNF constructs. We shouldn't 
give the impression that this is not the case.

Best regards, Julian