Subject: Re: [spfbis] #2: RFC 4408 Section B.3 - Errata #2250

From: S Moonesamy <[email-masked]>
Date: Mon, 20 Feb 2012 12:16:11 -0800

At 12:57 07-02-2012, spfbis issue tracker wrote:
>#2: RFC 4408 Section B.3 - Errata #2250
>
>  Section B.3 says:
>
>     $ORIGIN _spf.example.com.  mary.mobile-users                   A
>     127.0.0.2 fred.mobile-users                   A 127.0.0.2
>     15.15.168.192.joel.remote-users     A 127.0.0.2
>     16.15.168.192.joel.remote-users     A 127.0.0.2
>
>
>  It should say:
>
>     $ORIGIN _spf.example.com.
>     mary.mobile-users                   A 127.0.0.2
>     fred.mobile-users                   A 127.0.0.2
>     15.15.168.192.joel.remote-users     A 127.0.0.2
>     16.15.168.192.joel.remote-users     A 127.0.0.2
>
>
>  Notes:
>
>  The DNS zone file example has incorrect line breaks.

[snip]
>Ticket URL: <http://trac.tools.ietf.org/wg/spfbis/trac/ticket/2>

I'll classify this issue as "hold for document update".  When the 
4408bis draft is opened, the working group can look into the above.

Regards,
S. Moonesamy
SPFBIS WG co-chair

---

From: Scott Kitterman <[email-masked]>
Date: Mon, 20 Feb 2012 22:03:33 -0500

On Monday, February 20, 2012 12:16:11 PM S Moonesamy wrote:
...
> 4408bis draft is opened, the working group can look into the above.
...

I'm confused again.  Isn't this what we are working on now?

Scott K

---

From: S Moonesamy <[email-masked]>
Date: Mon, 20 Feb 2012 20:52:13 -0800

>I'm confused again.  Isn't this what we are working on now?

Andrew and I asked for participants to review RFC 4408 without 
getting into a discussion about the issues.  The working group now 
has a list of open issues. The discussion on issue #9, #2, #3 and #8 
started today.  As Issue #2, #3 and #8 are editorial, they don't need 
to be addressed immediately.  The group could wait until there is a 
working group Internet-Draft to discuss about editorial 
questions.  We are not ignoring these issues; they are listed in the 
tracker and we will go through them in a few months.

Issue #9 was opened for discussion as we don't know what to do about 
it.  It's better to tackle issues such as that one now as it may be 
useful for the conclusion document.  As the discussions evolves, we 
will have a clearer view of the direction the working group would 
like to take, what needs to be fixed, how to fix, etc.

I understand that the way the work is being done is unusual.  The 
group started without any idea about how to conclude the experiments 
or what should go in the conclusion document.  Over the next few 
weeks, or more, the discussions should shape up that document.

At 19:01 20-02-2012, Scott Kitterman wrote:
>I did already mention these errata in http://www.ietf.org/mail-
>archive/web/spfbis/current/msg00274.html - but they don't seem to 
>have made it
>into the tracker.

I went through http://www.rfc-editor.org/errata_search.php?rfc=4408 
and opened tickets for each item.  If the comments from 
http://www.openspf.net/RFC_4408/Errata which require discussion are 
posted to the mailing list, I don't have to figure out the Errata 
status or worry about process.

>Do you want one mail per item?

No, they can be posted to this mailing list as one message and I'll 
sort the items for tracking.

Regards,
S. Moonesamy

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 21 Feb 2012 09:05:16 -0500

On Monday, February 20, 2012 08:52:13 PM S Moonesamy wrote:
...
> sort the items for tracking.

Errata approved by the SPF council and/or by the RFC authors

    Add %v macro to ABNF grammar
    Undefined "uric" set
    Recommend an SMTP reply code for optional PermError rejections
    Received-SPF examples have incorrect syntax
    unknown-modifier is too greedy
    Empty domain spec on exp modifier
    PermError on invalid domains after macro expansion
    Miscellaneous confirmed typos

Add %v macro to ABNF grammar

Suggested wording:

The ABNF grammar in section 8.1 and appendix A are missing the letter "v". The 
macro-letter production should look like this:

      macro-letter     = "s" / "l" / "o" / "d" / "i" / "p" / "h" /
                         "c" / "r" / "t" / "v"

Rationale:

The %v macro is listed in the table of macros in section 8.1, and used in the 
examples.
Undefined uric set

Suggested wording:

Replace "uric" by "unreserved" in section 8.1.

Rationale:

uric was defined in RFC 2396, but later removed in RFC 3986. Because RFC 4408 
has a normative reference to RFC 3986 (AKA STD 66) uric has to be replaced by 
a modern set of characters affected by the URL escaping for uppercased macros.

Reported by Julian and confirmed by Wayne.
Recommend an SMTP reply code for optional PermError rejections

Suggested wording:

 If the message is rejected during the SMTP transaction for
 this reason, the software SHOULD use an SMTP reply code of
 550 and, if supported, the 5.5.2 DSN code.

(to be added at the end of 2.5.7 in the style of the wording at the end of 
2.5.6)

Rationale:

The PermError section in Wayne's first SPF I-D stated:

 A PermError result means that the domain's published records
 couldn't be correctly interpreted.  Checking software SHOULD
 reject the message.  If rejecting during SMTP transaction
 time, it SHOULD use an SMTP reply code of 550 and, if supported,
 the 5.5.2 DSN code.

This was removed later when the first SHOULD was identified as receiver policy.

However the second SHOULD recommends an appropriate reply code for receivers 
wishing to reject PermError. The issue was resolved in an appeal to the SPF 
Council proposing to use a similar wording as for FAIL, but the missing status 
code didn't make it again into the final draft.
Received-SPF examples use incorrect syntax

Suggested wording:

The examples in section 7 are incorrectly quoted, and should look like this:

      Received-SPF: Pass (mybox.example.org: domain of
       [email-masked] designates 192.0.2.1 as permitted sender)
          receiver=mybox.example.org; client-ip=192.0.2.1;
          envelope-from="[email-masked]"; helo=foo.example.com;

      Received-SPF: Fail (mybox.example.org: domain of
                        [email-masked] does not designate
                        192.0.2.1 as permitted sender)
                        identity=mailfrom; client-ip=192.0.2.1;
                        envelope-from="[email-masked]";

Rationale:

In RFC4408, key-value-pair is defined as:

      key-value-pair   = key [CFWS] "=" ( dot-atom / quoted-string )

However, the examples look like this:

      Received-SPF: Pass (mybox.example.org: domain of
       [email-masked] designates 192.0.2.1 as permitted sender)
          receiver=mybox.example.org; client-ip=192.0.2.1;
          envelope-from=<[email-masked]>; helo=foo.example.com;

Notice that envelope-from in both cases is NOT a dot-atom or quoted-string.

None of '<','>','@' are legal chars for dot-atom.
unknown-modifier clause is too greedy in ABNF

In section 4.6.1/2, unknown-modifier should be changed to:

      unknown-modifier = name "=" macro-string
                         ; where name is not any known modifier

Rationale:

The unknown-modifier clause is too greedy, causing any syntax errors in 
redirect or explanation to simply become unknown-modifier. For instance, "exp=-
all" would match unknown-modifier. This was not the intent.

For a similar case see RFC 2822, where the field names of any optional-field 
MUST NOT be identical to any field name specified elsewhere in this standard.
Empty domain-spec on exp modifier

Problem statement:

Section 6.2/4 says:

    If domain-spec is empty, or there are any DNS processing errors (any RCODE 
other than 0), or if no records are returned, or if more than one record is 
returned, or if there are syntax errors in the explanation string, then 
proceed as if no exp modifier was given.

However, 6.2/1 gives the grammar for exp as:

      explanation = "exp" "=" domain-spec
      domain-spec      = macro-string domain-end
      domain-end       = ( "." toplabel [ "." ] ) / macro-expand

Thus, domain-spec can never be empty. Syntax errors require a permerror 
result, which conflicts with the instruction to "proceed as if no exp modifier 
was given."

Suggested fix and rationale:

Section 6.2/4 should be changed to say:

    If there are any DNS processing errors (any RCODE other than 0), or if no 
records are returned, or if more than one record is returned, or if there are 
syntax errors in the explanation string, then proceed as if no exp modifier was 
given.

The exp and redirect modifiers are special since they are included in the spec. 
They must be fully parsed, with any resulting Permerror, even when exp= is not 
otherwise supported. Clear syntax errors are supposed to be identified a.s.a.p. 
and reported as PermError, and supporting explanations in syntax isn't 
optional.

The prose obviously wants that an empty <target-name> and similar issues after 
the macro expansion of <domain-spec> are ignored. It's more important to 
reject the FAIL than to deal with obscure explanation issues.
PermError on invalid domains after macro expansion

Problem statement:

The Permerror definition ends with:

    Be aware that if the domain owner uses macros (Section 8), it is possible 
that this result is due to the checked identities having an unexpected format.

A <target-name> after macro expansion of <domain-spec> can be invalid, i.e. a 
string not suited for DNS queries like foo..example (adjacent dots), a 
<target-name> with overlong labels, or similar issues not permitted in the DNS 
syntax.

Suggested fix (variant 1):

The last sentence in the Permerror definition is misleading. A syntactically 
malformed <target-name> can be also handled as no match. The specification 
should say:

    Be aware that if the domain owner uses macros (Section 8), it is possible 
that this result is due to the checked identities having an unexpected format.

    Please note that an unexpected <target-name> can be also handled as no 
match, ideally implementations document how they handle such issues. The 
outcome for an unexpected <domain-spec> without macros might differ from the 
outcome for an unexpected <target-name> after macro expansion.

Suggested fix (Variant 2):

The last sentence in the Permerror definition is misleading. A syntactically 
malformed <target-name> can be also handled as no match. The specification 
should say:

    Be aware that it is also possible that this result is generated by certain 
SPF clients due to the input arguments having an unexpected format; see 
Section 4.8.

At the end of Section 4.8 add:

    Note: Historically, this document has made no provisions for how to handle 
<domain-spec>s, or macro-expansions thereof, that are syntactically invalid 
per [RFC1035], such as names with empty labels (e.g., "foo..example.com") or 
overlong labels (more than 63 characters). Some implementations choose to 
treat as a no-match mechanisms, and ignore modifiers, with such names, whereas 
others throw a "PermError" exception. The outcome for an unexpected <domain-
spec> without macros might even differ from that for an unexpected <target-
name> after macro expansion.

Rationale:

Reporting a TempError in such cases is no option, the syntax problem won't go 
away for a given sender policy, HELO identity, envelope sender address, and 
sending IP combination with a try again later TempError approach. If anything 
else is as expected the sender policy might need to be fixed manually, 
supporting PermError.

If the DNS syntax problem is caused by random net abuse, or intentionally by 
an attacker, a no match approach, eventually reaching a FAIL for -all or 
whatever the sender policy offers, is better. In common practice this problem 
is handled as no match OR PermError, like similar problems explicitly 
addressed in the specification.
Miscellaneous typos

    Missing indentation for the second paragraph of the [US-ASCII] reference 
reported by Wayne.
    Missing line break on page 46 before mary.mobile-users reported by Wayne.
    Typo: "Danish" instead of "Danisch" in the references confirmed by Wayne 
and Frank.
    Missing words in the last sentence of the third paragraph of section 6.1: 
"The <ip> and <sender> arguments remain the same as in the current evaluation 
of check_host()." (reported by Julian)

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 21 Feb 2012 09:18:15 -0500

> weeks, or more, the discussions should shape up that document.

Not understanding how to write a technical document that wraps up an 
"experiment" that was started for ill defined (and IMO non-technical) reasons 
is unrelated to what we understand about what needs to go into 4408bis, so I'm 
still confused.

I fear that this process of discussing issues without a draft to document the 
consensus in is going to result in repetitive work.  First we discuss an issue 
and reach some rough consensus and then later when we have to put words on 
paper in a draft we'll inevitably end up rehashing the issue.

Every other IETF working group I've been involved in (which is admittedly not 
a large number) has started with a draft and then documented the results of 
their discussions in the draft as they went along (including groups that used 
an issue tracker).  This doesn't eliminate the risk you'll get to the end and 
have to revisit issues, but the current process this group seems to be 
following guarantees it.

Scott K

---

From: Dave CROCKER <[email-masked]>
Date: Tue, 21 Feb 2012 06:45:12 -0800

> paper in a draft we'll inevitably end up rehashing the issue.

You are likely correct. However I'll suggest that in this case it might be a 
Good Thing.

Normally, revision efforts start with the group's having good context about the 
document being revised.  The group typically has a significant number of folk 
who participated in the original writing effort and it typically has a 
significant number of people who have been "using" the document.

My guess is that SPFbis does not enjoy quite as much of such shared context as 
is usual.  Consequently, it might actually be helpful to take a "spiral" 
approach to the work; that is, visiting issues a couple of times, during 
different phases of the work.

That said, there certainly is a danger of re-hashing settled issues -- a serious 
concern, since of course it rarely happens in other IETF efforts...

There are two basic mechanisms that mitigate against inappropriate re-hashing 
(and yes, there are cases of 'appropriate' rehashing.)

The first is making sure the the original process of resolving a point has a 
clear, consensus-based outcome.

The second is wg management that enforces those decisions, absent explicit 
justification to re-open issues.

I suspect this wg's management team is  entirely up to both tasks...

d/

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 21 Feb 2012 10:29:30 -0500

On Tuesday, February 21, 2012 06:45:12 AM Dave CROCKER wrote:
> Normally, revision efforts start with the group's having good context about
> the  document being revised.  The group typically has a significant number
> of folk who participated in the original writing effort and it typically
> has a significant number of people who have been "using" the document.
> 
> My guess is that SPFbis does not enjoy quite as much of such shared context
> as  is usual.  Consequently, it might actually be helpful to take a
> "spiral" approach to the work; that is, visiting issues a couple of times,
> during different phases of the work.

There are quite a number of people here who have significant experience with 
the document and implementing SPF.  I don't see any need to design WG process 
around the assumption this is not the case.

There are a few significant issues that need discussion, but this isn't a case 
of something that's not working as designed that needs major overhaul.  I feel 
like we're making this harder than it is.  It would have been nice (IMO) for 
there to have been some discussion about how this is going to work.

I keep having questions and I'm pretty sure the chairs know how they intend to 
proceed, but I don't think it's ever been completely explained.

Scott K

---

From: Andrew Sullivan <[email-masked]>
Date: Tue, 21 Feb 2012 11:18:35 -0500

Hi,

On Tue, Feb 21, 2012 at 10:29:30AM -0500, Scott Kitterman wrote:
> 
> There are quite a number of people here who have significant
> experience with the document and implementing SPF.  I don't see any
> need to design WG process around the assumption this is not the
> case.

There are also people who do not have such experience, and there are
people who have strong opinions about the topic.  Finally, and maybe
most importantly, there are several people who have extremely
wide-ranging opinions about matters, and have a history of collapsing
different issues and treating them as though they're all the same
thing.  

We want to avoid all of that.  You are correct in your impression that
this will probably result in some issues being discussed more than
once.  The idea is precisely that it's up to the chairs to interrupt,
if a discussion goes back over well-tilled ground, and ask what is
different this time.

 > There are a few significant issues that need discussion, but this
> isn't a case of something that's not working as designed that needs
> major overhaul.

Also, in any case, we're not chartered to do a major overhaul.  But
this is the classic bikeshed situation: everyone has a strong opinion
about the details, and will argue about each of them, weaving in other
details of the bikeshed's construction and colour scheme without
regard for various distinctions.  That's exactly what our methodology
is intended to avoid.  Once the more controversial questions have been
addressed, we can integrate them into a candidate draft.  It seems to
me that there was an IETF contribution that someone thoughtfully
provided that could be a good start for that, but before we commit to
anything it seemed to me that it would be wise to clear away some of
the controversy.

> like we're making this harder than it is.  It would have been nice
> (IMO) for there to have been some discussion about how this is going
> to work.

When SM and I proposed taking this approach to the AD, the question of
whether to discuss the methodology first was broached.  I was, and
remain, opposed to that; all it would do is turn the management of the
WG's process itself into another bikeshed problem.  It wouldn't be the
first time, but I think the WG will be more productive the way we're
going.

I hope that explains things to your satisfaction.

Best,

A

-- 
Andrew Sullivan
[email-masked]

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 21 Feb 2012 11:24:11 -0500

On Tuesday, February 21, 2012 11:18:35 AM Andrew Sullivan wrote:
> ... It seems to
> provided that could be a good start for that ...

Yes.  That was me.

https://datatracker.ietf.org/doc/draft-kitterman-4408bis/

> I hope that explains things to your satisfaction.

It doesn't, but at least I know it's on purpose now.  I'll strive for 
patience.  No promises.

Scott K