Subject: [spfbis] 4408bis update

From: Scott Kitterman <[email-masked]>
Date: Thu, 16 Aug 2012 02:50:46 -0400

I just uploaded 04 and it's too late for me to wait up for the notification.

This version has a lot of changes, mostly based on Murray's excellent and 
detailed review and some of the discussions that follow up from it.  I think 
it addresses all the on list comments.  If I missed something, please remind 
me and we'll put it in the hopper for 05.

It's enough changes that my summary is "go read the diff."

I'll try and look at the issue tracker tomorrow and see where we are.

Scott K

---

From: Alessandro Vesely <[email-masked]>
Date: Thu, 16 Aug 2012 14:52:28 +0200

> It's enough changes that my summary is "go read the diff."

Yes, many changes.  At last.

Here's some comments:

1. *Introduction*

 This document defines a protocol by which ADMDs can authorize hosts
 to use their domain names in the RFC5321.MailFrom or RFC5321.HELO/
 .EHLO identities.  Compliant ADMDs publish Sender Policy Framework
 (SPF) records in DNS [...]

Two notes about that snippet.  First the positive one:  I like the
phrase "Compliant ADMDs".  I suggest to stick to this concept as it is
much clearer than "compliant domain" (see below).

For the negative note:  Is it possible to roll back that 5598-based
replacement of identity names, _please_?  RFC5321.HELO/.EHLO is
excessively long and lowers the readability of the document.  I don't
object on that string appearing in Section 1.3.4.  After precisely
defining what HELO means, it is not necessary to repeat the whole
definition any more.

                                                      Furthermore,
 if a claimed identity fails verification, local policy can take
 stronger action against such email, such as rejecting it.

Kudos for Section 9.3!  Now that we have it, that last sentence in the
Introduction can be reworded so as to anticipate it.  For example:

 Furthermore, receivers can devise local policies (Section 9.3)
 in order to facilitate processing of messages that get specific
 results; for example, they can whitelist domains that pass the
 verification, or reject mail that fails it.

2.3. *Publishing Authorization*

 An SPF-compliant domain MUST publish a valid SPF record as described
 in Section 3.  This record authorizes the use of the domain name in
 the RFC5321.HELO/.EHLO and RFC5321.MailFrom identities by the MTAs it
 specifies.

It is possible to address compliance by ADMD, rather than by domain:

 An SPF-compliant ADMD MUST publish valid SPF records as described
 in Section 3.  These records authorize the use of the relevant
 domain names in HELO and MAILFROM identities by the MTAs it
 specifies.

That way, the spec can modulate the requirements about which domains
MUST/SHOULD have an SPF record.  Until the term "domain" is used
interchangeably for a DNS label or an ADMD, wording such requirements
is difficult.

 SPF results can be used to make both positive (source is authorized)
 and negative (source is not authorized) determinations.  If domain
 owners choose to publish SPF records and want to support receivers
 making negative authorization determinations, then they MUST publish
 records that end in "-all", or redirect to other records that do,
 otherwise, no definitive determination of authorization can be made.

IMHO, constructs like "If you choose X then you MUST Y" are
meaningless.  I'll never be able to tell whether you are compliant,
unless I have a method to determine what you chose.  And since you can
always change your mind, the normative sense of the verb is completely
lost.

For the merit, we'll need to discuss this point.  Of course, we are
defining in what cases an ADMD can be said to be SPF compliant.  I'd
put that there should no reason a domain wouldn't comply, except for
ignorance of mistake.  Indeed, there is no advantage in standardizing
a protocol that cannot be adhered to.

For example, consider a mail admin who is unable to devise an SPF
policy, for the time being.  While she takes steps for gathering the
necessary data, she puts some RRs like "v=spf1 ?all" just to optimize
DNS caches.  How would we rate her, as far as compliance is concerned?

2.5.4. *Fail*

 A "fail" result is an explicit statement that the client is not
 authorized to use the domain in the given identity.  Disposition of
 SPF fail messages is a matter of local policy.  See Section 9.3 for
 considerations on developing local policy.

For the sake of clarity, cannot we say explicitly that the receiver
"MAY reject the message, according to local policy"?  It is what that
paragraph is actually saying, except that we force the reader to
interpret the term "disposition".  I realize that being explicit may
emphasize a weakness of SPF, which I hope we'll address later on.

As a further point on that snippet, that "given identity" sounds
obscure.  The result doesn't formally depend on which identity is
being checked.

3. *SPF Records*

The paragraph "Domain owners wishing to be SPF compliant [...]" is
redundant after Section 2.3.  I'd strike it.

In the paragraph next to it, s/in Section 3/above/.

  Since TXT records have multiple uses, beware of other TXT records
  published there for other purposes.  They might cause problems with
  size limits (see Section 3.4) and care MUST be taken to ensure only
  SPF records are used for SPF processing.

I'd s/ and care MUST be taken/.  Verifiers MUST take care/.

3.3. *Multiple Strings in a Single DNS record*

 As defined in [RFC1035] sections 3.3.14 and 3.3, a single text DNS
 record (either TXT or SPF RR types) can be composed of more than one
 string.

s/[RFC1035] sections 3.3.14 and 3.3/Section 5.1 of [RFC1035]/

Remove the parenthesized phrase.

3.4. *Record Size*

Why would TCP be less reliable than UDP?

Instead, I'd mention that, in the presence of multiple TXT records,
two UDP queries done through a redirection to _spf.example.com still
cost less than setting up a TCP connection.

4.6.4. *DNS Lookup Limits*

 SPF implementations MUST limit the number of mechanisms and modifiers
 ("terms") that DNS lookups to at most 10 during SPF evaluation.

s/that DNS lookups/that cause any DNS query/ if it would make for
better English.

5.1. *"all"*

                         Any "redirect" modifier (Section 6.1) has
 MUST be ignored when there is an "all" mechanism in the record.

s/has //

5.5. *"ptr" (deprecated)*

I like the numbered algorithm.  It's more readable.

Does the grammar allow "ptr:example.com/24"?  Step 4 doesn't seem to
allow fuzzy IP comparisons based on given CIDR-length.

Another question:  If there was a {temp/perm}error while doing an A RR
lookup, and then the mechanism failed to match, MAY a {temp/perm}error
be returned instead of non-match?

s/yearrs/years/

7. *Recording The Result*

 It is RECOMMENDED that SMTP receivers record the result of SPF
 processing in the message header.  There are two methods for doing
 this: the Received-SPF Header Field defined here and the more generic
 Authentication-Results Header Field defined in [RFC5451].  Because
 these fields are generally used within a receiving ADMD, it is a
 local policy choice which to include.  In general, the more broadly
 applicable Authentication-Results Header Field ought to be used, but
 it SHOULD provide the same information as if a Received-SPF Header
 Field were used.

Is there a reason why "Header Field" is capitalized that way?

I don't think we can be normative-by-cmparison on what information is
actually put on the field, because we don't know which tags would have
been filled in the Received-SPF field if the latter were used.  For an
alternative wording:

 In general, the more broadly applicable Authentication-Results header
 field ought to be used, as long as it can convey the same
 information that the verifier would have written in a Received-SPF
 header field if the latter had been used.

The next paragraph should be split in two:

 If an SMTP receiver chooses to do so, it SHOULD use one of these
 header fields for each identity that was checked.  This information
 is intended for the recipient.  (Information intended for the sender
 is described in Section 6.2, Explanation.)

See above for the "If ... SHOULD".  I'd propose:

 An SMTP receiver SHOULD use one and only one of these header fields.
 In case both of them are used, they MUST be consistent with one
 another.

 This information is intended for the recipient.  (Information
 intended for the sender is described in Section 6.2, Explanation.)

Note that A-R can convey results for multiple identities within a
single header field, so it doesn't make much sense to require to use
two of them.

7.2. *SPF Results in the Authentication-Results Header Field*

We need to discuss this further:

Why "Authserv-id SHOULD be the name of the receiving host performing
the check"?  What if the check is performed by a different host, or
the host has multiple names?  Being consistent with the corresponding
Received: field seems to be a good idea, but then RFC 5451 allows
different strategies.

Why overload the "reason" tag?  RFC 5451 is extensible and this spec
may add any missing tags.

The example misses the explanation.

8.1. *Macro Definitions*

 Implementations SHOULD be aware that if no directive processed during
 the evaluation of check_host() contains an "s", "l", "o", or "h"
 macro, then the results of the evaluation can be cached on the basis
 of <domain> and <ip> alone for as long as the shortest Time To Live
 (TTL) of all the DNS records involved.

"SHOULD be aware" doesn't mean anything to me.  I'd make it non-normative.

9.1.3. *Bounces*

                                              Zone file generation for
 significant numbers of hosts can be consolidated using the redirect
 modifier and scripted for initial deployment.  Administrators might
 alternatively consider publishing an SPF for *.domain (wildcard
 domains) in that case if the involved hostnames do not have already
 any other DNS record.

I'd strike the last sentence, or at least recall that wildcard records
are discouraged and provide a link to Section 3.5.

Further advice is needed for scripting the zone.  For example, mention
that any host that has an IP address is a good candidate for an SPF
record; those that do send mail really deserve one; and so forth.
Should we provide a Perl script in an appendix?

9.2.4. *MTA Relays*

 For mail senders, this means that published SPF records must
 authorize any MTAs that actually send across the Internet.  Usually,
 these are just the border MTAs as internal MTAs simply forward mail
 to these MTAs for delivery.

I believe the last word should be "relaying".

*Last and least*

I counted 13 occurrences of "SPF client(s)".  Since an SPF client is
likely an SMTP the server, that term may be confusing.  How about
using "SPF verifier(s)" instead?

---

From: S Moonesamy <[email-masked]>
Date: Thu, 16 Aug 2012 12:07:02 -0700

>I just uploaded 04 and it's too late for me to wait up for the notification.

Thanks for moving the draft forward.

Andrew and I are waiting for feedback from the Responsible AD about 
an issue.  Meanwhile, could the working group please review 
draft-ietf-spfbis-4408bis-04 ( 
http://tools.ietf.org/html/draft-ietf-spfbis-4408bis-04 ) and comment?

Thanks,
S. Moonesamy

---

From: Hector Santos <[email-masked]>
Date: Thu, 16 Aug 2012 16:02:47 -0400

S Moonesamy wrote:
> Andrew and I are waiting for feedback from the Responsible AD about an 
> issue.  Meanwhile, could the working group please review 

There are massive changes that needs time to review. With a quick 
perusal, I don't particular agree with and/or needs a lot of work, in 
particular the new section 9.0.

It would of been nice if the editor would of posted for the WG list to 
first review the substantial changes he intended to make or add 
because a good bit of it seems highly subjective, not consistent with 
our 9 years of practice and leans towards to specific deployment 
concepts. Now we are struck in a repeated OPT-OUT "Rough Consensus" 
situation where unexpected changes is done first and now a very high 
bar to reverse it without controversies developing. IMO, an usual 
source for contention and waste of time.

Looking for specific items I was focused with, I can see security 
aspects and concerns I posted were also ignored. But I hope to read 
the diff and make sense of it all more thoroughly over the next week 
or days.

Thanks

-- 
HLS

---

From: Andrew Sullivan <[email-masked]>
Date: Thu, 16 Aug 2012 16:28:08 -0400

On Thu, Aug 16, 2012 at 04:02:47PM -0400, Hector Santos wrote:
> 
> It would of been nice if the editor would of posted for the WG list
> to first review the substantial changes he intended to make or add

Why?  This is an Internet Draft.  This is the format we have for
reviewing and commenting.  Sending it to the list in advance is not
something we want to encourage.

If there are specific things that one sent that one thinks were not
reflected, then it is entirely correct to raise that issue.  I note
that suggested replacement text is more likely to reflect the change
one wants (particularly if it also attracts support).

Best,

A (as co-chair)

-- 
Andrew Sullivan
[email-masked]

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 16 Aug 2012 16:42:26 -0400

On Thursday, August 16, 2012 04:02:47 pm Hector Santos wrote:
...
> concepts. 

It's called a draft for a reason.  It's there for review.  I decline to engage 
in the silliness of pre-reviewing everything before I post drafts for review.

Nothing gains consensus on the basis of my having written it.

Almost nothing that's new in this draft was written by me without it being 
stimulated by comments from this list.

Please leave the name calling aside and let us know your specific issues.  I 
added explicit language about PTR support being something recipients MUST do 
based on your concerns.  I may not have agreed with or felt like the balance 
of the group was for some of your comments, but I did not ignore them.

Scott K

---

From: Hector Santos <[email-masked]>
Date: Thu, 16 Aug 2012 17:47:23 -0400

> something we want to encourage.

Because of past experience, and I stress only an isolated experience, 
not my general experience.  From quick reading so far, section 9.0 was 
written without any consensus and now it (and changes) will be much 
harder to overcome, especially if it originates from me, 
unfortunately. Look, too more unprofessional ignorance is going on and 
if thats ok by the group, so be it.  But this is not a HOBBY for me. 
The output of this group directly impacts in my product and how my 
customers will use SPF and I will have my input or the new document 
will not get adopted by me, which means changes not implemented, 
largely ignored. Status quo, picking and choosing what goods vs bad 
and quite frankly, I am tired of this trend for the RFC documents.  I 
had to bring up a more general issue, but its part of what you see 
here.  Sorry.

Section 9.0 is linked to other existing sections which required a much 
wider engineering review than what was done and I strongly feel there 
is actual empirical information that is not reflected. No problem if 
it wasn't so subjective, was more generic protocol material and more 
reflective of actual practice out there. i.e. some of the stuff we 
should of extracted during the experiment report, unfortunately, we 
focused on other things.  The issues were/are already on the table 
(documented) and I don't think they are un-addressable.

-- 
HLS

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 16 Aug 2012 18:02:19 -0400

> (documented) and I don't think they are un-addressable.

Section 9 is not new.  It's in RFC 4408 and every draft that follows.  There 
are three major changes for -04 in section 9:

9.1.1. DNS Resource Considerations: Rewritten/expanded.  This is Alessandro 
Vesely's text that he provided me offline and I posted here almost a month ago 
for review.

9.1.3. Bounces:  New subsection provided by Franck Martin that was previously 
published on this list.  What's in the draft is almost exactly what he posted.

9.3. Receivers:  This is new.  I wrote the first two subsections in response to 
comments by Murray Kucherawy.  He provided the text (on this list) for the 
last subsection.

So most of the change in the section you are complaining about was previously 
posted here.  Keep in mind that all of section 9 is non-normative.  If you've 
got actual suggestions to make, please make them.  Vague assertions that it 
could be better don't help me get this done.

Scott K

---

From: "John Levine" <[email-masked]>
Date: 17 Aug 2012 03:46:03 -0000

>It's enough changes that my summary is "go read the diff."

Man, what a lot of work, thanks.

Most of these are nits:

Abstract: Why "author's address"?  Usually that means the address in
the From: header.  Suggest using one of the terms in 1.3.3

2.5.6: suggest adding "software" to the end of the new sentence.  As
it stands one could read it as referring to problems in the DNS
records that the receiver publishes.

3: suggest ref to 4.6.4 as well as 9.1.1

3.3: remove parenthetical ref to SPF RR type

3.4: there's nothing wrong in what this says, but it's a generic
problem with all DNS usage, nothing specific to SPF

3.5: in examples X.COM is a real domain (it's Paypal).  Suggest
using X.EXAMPLE or EXAMPLE.COM

Also in 3.5, there are real uses of MX wildcards, typically
used to treat mail to [email-masked] as [email-masked]
so people can have less obviously tagged addresses.  I'd suggest
saying to be sure that the SPF records match the domains actually
in use in outgoing mail and leave it at that.

4.1: suggest removing last sentence.  It's a conceptual function,
it might not be coded as a function at all.

5: last pp, throws the exception? I hope you mean that evaluation
stops and the topmost check_host() returns temperror.  Also in the
table in 5.2

5.5: yearrs -> years

6.1: modifier intended -> modifier is intended

end of 8.1: what is an implementation supposed to do if a macro
expansion creates a bogus domain due to input data, e.g., using %l in
a domain name since local parts can contain spaces, punctuation, and
other random ASCII crud?

10.3, second bullet: I think TCP spoofing was fixed long enough ago
that this item can go

---

From: Hector Santos <[email-masked]>
Date: Fri, 17 Aug 2012 05:14:27 -0400

Scott Kitterman wrote:
> 
> 9.3. Receivers:  This is new.  

I will try to be more specific with specific section #s next time. 
Massive changes across the doc nonetheless and no longer sure its 
worth the stress to go over the relatively long new set of concerns 
(or complaints if you wish to label it that way), I believe they are. 
  The more important one changes (or lack there of) related to Issue 29.

> could be better don't help me get this done.

hmmm, I did and you made it quite apparent it would be ignored as the 
doc shows. My input were specific to certain things, not doc nits. 
For example, like issue 29, it went a different route and doesn't 
solve the security loophole.

But you made it clear it wasn't a security issue and I strongly 
disagree and nonchalantly blew it aside. Various inputs regarding 
REJECT vs MARK, the perhaps with erroneous (IMO) beliefs of how MARK 
prevails over REJECT in deployments made it clear to me it is a 
security loophole and now a concern with watering down the publication 
of -ALL policies and now presenting two different paths for RFC2119 
compliant REJECT language and a loophole with non-RFC2119 MARK-ON-FAIL 
semantics.   What was apparent was the MARK "group" was HERE, in this 
FORA, prevailing and I felt that was an engineering error (for the SPF 
protocol) if it was not before defined.

REJECT is clear with RFC2119 language and the essence of SPF -ALL 
policies. As with most things we do to not force things per se or to 
provide protocol features and alternatives, the deployment alternative 
in the protocol MUST provide at the very least the same level or 
equivalent security level, otherwise it presents a loophole or entry 
point for confusion, design complexity, problems and a minimization of 
problem solving (i.e. purpose is minimized).

I provided text that suggest deployments MUST|SHOULD|RECOMMENDED (i 
prefer MUST) or essentially MUST be highly aware to separate the 
MARK-ON-FAIL stream from normal accepted mail streams - thats the 
security loophole.   I provided practical product design input how it 
can affect backends and MUAs - offline vs online,  POP3 vs IMAP, etc.

In short, a deployment using MARK-ON-FAIL, even if its (erroneously) 
prevailing in this group, they must be made aware to separate the 
stream (negatively classify it, isolate, etc) and that may be site 
specific and/or user connectivity specific.  If he reads mail online 
(HTML, IMAP, etc) and that may be part time, full time), the backend 
can better handle the MARK-ON-FAIL status message.  If he reads mail 
offline, via POP3, the backend MUST be aware again to separate the 
MARK-ON-FAIL messages form the normal new mail stream.  In general, 
for any mail reading portal access, the backend offers MUST deal with 
MARK-ON-FAIL to offer the same level or equivalent of user protection 
that a SMTP level REJECT-ON-FAIL offers users.  We are talking about 
user level vs system level filtering at this point and IMV, SPF caters 
to the latter but can work with the former IFF the security offering 
is the same.

I provided some started text - ignored.  I could care less how its 
stated, but it shouldn't be ignored and thrown side as irrelevant input.

However, I will note when Issue 29 was finally added to the tracker 
with a reference message that was not what I expected. I felt maybe 
that it was the shorter version but it was definitely not as complete 
as discussed in various messages:

   http://www.ietf.org/mail-archive/web/spfbis/current/msg01408.html
   http://www.ietf.org/mail-archive/web/spfbis/current/msg01486.html

So I can see if it was missed because of that.  Yet, maybe the issue 
text was sufficient, I waited but eventually saw it was closed and 
when noted, you said it wasn't an security issue.

I disagree.  The spec should via clear what design options need to be 
considered when alternatives are offered, especially when one is very 
simple, clear (REJECT) with RFC2119 language and the other (MARK) is 
left open ended with no RFC2119 language.

There are other change I think need to be reviewed, but this issue was 
long undue and the more critical one IMO from other things. The leap 
to section 9 from section 2.5.4 should be more clear as the 
alternative deployment option implementators need to consider in their 
code.

-- 
HLS

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 17 Aug 2012 09:56:06 -0400

On Friday, August 17, 2012 05:14:27 AM Hector Santos wrote:
> > So most of the change in the section you are complaining about was
> > previously posted here.  Keep in mind that all of section 9 is
> > non-normative.  If you've got actual suggestions to make, please make
> > them.  Vague assertions that it could be better don't help me get this
> > done.
> 

SPF was designed for SMTP time rejection.  I know that.  I was there when it 
happened.  I've distributed software for half a decade that by default rejects 
on fail during the SMTP session.  Whoever that "MARK 'group'" (sic) is, I'm 
not in it and I'm the one that wrote the words you're apparently unhappy with, 
so please lay off the conspiracy theories and give me something I can work 
with.

> can affect backends and MUAs - offline vs online,  POP3 vs IMAP, etc.

I attempted to address this in -04.  There was a LOT of email to go through 
when I was preparing it and it is quite likely I missed something.  Please 
give me a pointer to the text in the archive or resend it.  I can assure you I 
didn't intentionally ignore any useful input.

> to the latter but can work with the former IFF the security offering

I can see this, but at the same time if a receiver it is marking and 
delivering mail it is because they want that mail to be available in some 
manner to the end user, so it is inherently not the same.  Security is never 
binary.  It is always a trade between risk and benefit.  Marking and delivering 
has the risk that something 'bad' will be delivered.  It also has the 
potential benefit that something otherwise desirable will be delivered.  Any 
language we include (and I could see possibly adding more information about 
this - although we need to avoid providing an MUA design tutorial) needs to 
recognize this trade off and provide useful guidance for receivers and 
implementers.

> stated, but it shouldn't be ignored and thrown side as irrelevant input.

As I said, I didn't intentionally ignore anything from you.  If you have 
something to add, please provide it or a pointer to it and it won't be just 
ignored.

> when noted, you said it wasn't an security issue.

You are entitled to your own set of opinions, but not your own set of facts.  
See http://tools.ietf.org/wg/spfbis/trac/ticket/29.  The issue is not closed.

In fact, the last thing I wrote in the tracker was:

> New text in the -04 draft should address this issue.
> 
> Waiting for WG review.

So not only is it not closed, it is explicitly waiting for people to comment 
on it so we can improve it.

> left open ended with no RFC2119 language.

I think the text in -04 does exactly this.  Here's what it says on this exact 
issue:

> 9.3.2.  Policy For SPF Fail
> 
>    SPF fail results can be used to reject messages during the SMTP
>    transaction based on either RFC5321.MailFrom or RFC5321.HELO/.EHLO
>    identity results.  This reduces resource requirements for various
>    content filtering methods and conserves bandwidth since rejection can
>    be done before the SMTP content is transferred.  It also gives
>    immediate feedback to the sender who might then be able to resolve
>    the issue.  Due to some of the issues described above in this
>    Section 9 section, SPF based rejection does present some risk of
>    rejecting legitimate email when rejecting based on RFC5321.MailFrom
>    results.
>    
>    SPF fail results can also be used as one input into a larger set of
>    evaluations which might, based on the overall evaluation result in
>    the email being marked negatively in some way (this might be via
>    delivery to a special spam folder, modifying subject lines, or other
>    locally determined means).  Developing the details of such an
>    approach have to be based on local conditions and requirements.
>    Using SPF results in this way does not have the advantages of
>    resource conservation and immediate feedback to the sender associated
>    with SMTP rejection, but could produce fewer undesireable rejections
>    in a well designed system.
>    
>    Either general approach can be used as they both leave a clear
>    disposition of emails.  They are either delivered in some manner or
>    the sender is notified of the failure.  Other dispositions such as
>    "dropping" or deleting email after acceptance are inappropriate
>    because they leave uncertainty and reduce the overall reliabilility
>    and utility of email across the Internet.

I attempted to provide a balanced view of the advantages and disadvantages of 
either approach.  If you think it needs improvement, please provide text.  At 
this point "I think it should X" is not very helpful.  I believe I understand 
what you think it should do and have attempted to accommodate that.  If you 
think it should be different, I need exact text as I'm already doing the best I 
can without it.

> alternative deployment option implementators need to consider in their

Please provide text.

Scott K

---

From: Hector Santos <[email-masked]>
Date: Fri, 17 Aug 2012 13:30:47 -0400

> See http://tools.ietf.org/wg/spfbis/trac/ticket/29.  The issue is not closed.

Changing perceptions is not something you need to do.  Issue 29 has 
not been addressed because it was indicated as CLOSED.  If it was 
reopen as requested, then you have new facts.

I don't wish to waste time going over this with discussions of SM, 
with direct email to you questioning why it was closed and your 
eventual posting that it was not a security problem, rude post I may 
say, then yes, we have a new set of convenient facts you wish to 
portray. No problem.

>> alternative deployment option implementators need to consider in their

See above links.

-- 
HLS

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 17 Aug 2012 14:13:39 -0400

On Friday, August 17, 2012 01:30:47 PM Hector Santos wrote:
> > You are entitled to your own set of opinions, but not your own set of
> > facts. See http://tools.ietf.org/wg/spfbis/trac/ticket/29.  The issue is
> > not closed.

http://trac.tools.ietf.org/wg/spfbis/trac/ticket/29 gives the change history.  
I get you think it was closed, but the history clearly shows it isn't and 
never was.

> say, then yes, we have a new set of convenient facts you wish to

Meh.  It's not the main point, so whatever.

> >> alternative deployment option implementators need to consider in their

I see there is text in the second link there that's relevant to this issue 
that I did, indeed, miss:

> NEW:
>    A "Fail" result is an explicit domain policy violation statement that
>    the client is not authorized to use the domain in the given identity.
>    
>    When the result is a "Fail", the checking software MUST offer local
>    receiver SPF deployments the option to choose between marking and
>    classifying
> 
>    (MARK-ON-FAIL) the the message having failed the domain SPF policy or
>    rejecting the mail outright (REJECT-ON-FAIL).
> 
> When the REJECT-ON-FAIL option is used, the SMTP server MUST reject the mail
> during the SMTP transaction and it SHOULD issue a permanent rejection reply
> code of 550 and for servers supporting extended reply code [RFC3463], use a
> policy based rejection description subject sub-code of 5.7.1. The following> 
> are some examples of REJECT-ON-FAIL server responses:
>        550 SPF MAIL FROM check failed:
>        550-5.7.1 SPF MAIL FROM check failed:
>        550-5.7.1 The domain example.com explains:
>        550 5.7.1 Please see http://www.example.com/mailpolicy.html
> 
> When the MARK-ON-FAIL option is used, it server SHOULD report this failure
> by adding a Receiver-SPF header to the received message. See Section 7.
> 
> The REJECT-ON-FAIL should be regarded a System Level local policy for global
> filtering for all users, where the MARK-ON-FAIL should be regarded as a
> User Level Policy framework where good vs bad mail is separated for the
> user.

The first question is what belongs in Section 2 (normative protocol) versus 
Section 9 (non-normative advice).  I don't see benefit in making MUST mark or 
reject part of the protocol as what you're calling REJECT-ON-FAIL and MARK-ON-
FAIL are really the only two options, so I think it's not necessary for it to 
be normative.  MUST do one of the only two choices isn't needed.

I think that your suggestion to add a SHOULD add a header field if marking is a 
good one.  It's accurate and a good companion to the how to go about rejecting 
discussion that's already there.  I've added words to that effect (but also 
including the option of authentication-results since that is now discussed in 
the draft) for the next update.  Here's what I added:

>   If the checking software chooses not to reject the mail during the
>   SMTP transaction, then is SHOULD add a Received-SPF or
>   Authentication-Results header field (see <xref
>   target="results-headers"/>) to communicate this result to downstream
>   message processors.  While this is true for all SPF results, it is
>   of particular importance for "fail" results since the message is 
>   explicitly not authorized by the domain owner.

I tried to make it parallel to what's there about rejection.

I think the last bit about system level policy versus user level policy is an 
interesting idea.  Where do you think it would fit in 9.3?  

I do not like the idea of introducing the all caps terms MARK-ON/REJECT-ON, so 
I haven't used them in my update.  I think they come across as shouting and 
are distracting.  If anyone else thinks I should, please speak up.

Scott K

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 17 Aug 2012 13:48:03 -0700

> defining what HELO means, it is not necessary to repeat the whole

I'd settle for something that makes it clear that "HELO Identity"
elsewhere means RFC5321.HELO or RFC5321.EHLO (whichever is available),
but I disagree that it's a cumbersome notation.  That could just be
that I'm used to it.

>  verification, or reject mail that fails it.

I don't know that it's necessary to add that up front.  Referring to
Section 9.3 at this point is sufficient.

>  domain names in HELO and MAILFROM identities by the MTAs it

As above.

> interchangeably for a DNS label or an ADMD, wording such requirements

The goal of introducing the term "ADMD" is specifically to avoid using
the terms interchangeably.

> always change your mind, the normative sense of the verb is completely

I'm not so sure.  I read that as saying "the only way to coerce
rejection from a receiver willing to do so is to use -all".

> emphasize a weakness of SPF, which I hope we'll address later on.

It would be equally correct to say "MAY keep the message, according to
local policy".  I think the language that's there is correct and
concise, and we should leave it.

> obscure.  The result doesn't formally depend on which identity is

How can it not?

But you could say "the identity under evaluation" or something like that.

> redundant after Section 2.3.  I'd strike it.

+1.

> In the paragraph next to it, s/in Section 3/above/.

+1.

> I'd s/ and care MUST be taken/.  Verifiers MUST take care/.

Meh.  Take it or leave it.

> s/[RFC1035] sections 3.3.14 and 3.3/Section 5.1 of [RFC1035]/

Nope.  Section 5.1 talks about the zone file format.  SPF receivers
don't get zone files, they get raw DNS data.  The Section 3 references
are correct.

> Remove the parenthesized phrase.

+1.

> Why would TCP be less reliable than UDP?

It's not uncommon for operators, out of ignorance, to interfere with
DNS over TCP because the need for that fallback is historically rare.
There were hints of this family of problems in the experiment
document.

> s/that DNS lookups/that cause any DNS query/ if it would make for

+1.

>  MUST be ignored when there is an "all" mechanism in the record.

+1.

> allow fuzzy IP comparisons based on given CIDR-length.

I don't believe the grammar does allow that.

> be returned instead of non-match?

Section 4.4 should make it clear that it's only talking about the
outermost TXT query in the second paragraph.

Section 4.6.2 doesn't actually say what to do in a parent evaluation
when a child mechanism is being evaluated and the child throws an
exception; does this exception propagate upward and terminate the
entire evaluation, or does processing continue?

> Is there a reason why "Header Field" is capitalized that way?

It should be lowercased.

>  header field if the latter had been used.

s/as long as it can convey/in such a way that it conveys/

> The next paragraph should be split in two:

+1.

>  In case both of them are used, they MUST be consistent with one

"one" is enough; "one and only one" is too wordy.  That's fine otherwise.

> single header field, so it doesn't make much sense to require to use

+1.

> Received: field seems to be a good idea, but then RFC 5451 allows

I agree; RFC5451 says use of the hostname or ADMD domain name is
common, but not necessary.  It also doesn't say anything at all about
coordinating with Received:, but that seems a sensible thing to do.

> may add any missing tags.

What would you do with the client IP address as its own dedicated part
of an A-R header field?  This was a matter of substantial debate
during RFC5451's development.  If all you want to do is log it or
include it in Received: fields for information, doing it in CFWS is
fine.  It's useful to register tags of this nature only if something
will extract that value and attempt to do something more with it.  In
the case of DKIM, for example, one might extract the signing domain,
compare it to the From: domain, and decide to do something special
with message presentation there.  But providing the IP address,
something the user never sees, doesn't seem to be a useful thing to
do.  The argument back then is that one might want to evaluate SPF
again at the MUA, but there was no support for that and plenty against
it.  That's why we didn't do that in the first place.

> "SHOULD be aware" doesn't mean anything to me.  I'd make it non-normative.

+1.

> are discouraged and provide a link to Section 3.5.

Indifferent on that suggestion.

> Should we provide a Perl script in an appendix?

I don't want to stray too far into operational advice.  We're simply
describing an algorithm here.  The further we go into operational
stuff, the more complex this all gets.  Including anything more than
pseudocode needs to be very carefully considered.

> I believe the last word should be "relaying".

+1.

> using "SPF verifier(s)" instead?

+1.

-MSK

---

From: Franck Martin <[email-masked]>
Date: Fri, 17 Aug 2012 21:07:42 +0000

>pseudocode needs to be very carefully considered.

I like the text there, this draws the attention on bounces and similar
forgotten folklore.

Yes the wildcard thing is a potential issue if you do FCDNS. In rDNS it is
fine.

So instead mentioning wildcard, may be we could simply suggests the
following simple record for each sending hostname.
Hostname TXT "v=SPF1 a ~all"

And for a non sending hostname
Hostname TXT "v=SPF1 redirect=domain"

As Murray says this go into operations, but that would cover things simply
for the seasoned admin.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 17 Aug 2012 14:46:24 -0700

>>    rejecting the mail outright (REJECT-ON-FAIL).

First, I agree with Scott that we don't need to introduce these as
all-caps terms.

Second, I think the current text for "Fail" explains perfectly well
that the option of what to do with the message is left to the
receiver, as it should be.  That's what "disposition" is.  If we
really want more discussion up in Section 2.5.x, we can make reference
down to Section 9.3.2 which discusses the point in more detail.  But
putting more text up here clutters what is now a simple definition.

>> are some examples of REJECT-ON-FAIL server responses:

I'm not liking the idea of identifying specific SMTP reply codes or
enhanced status codes that should be used for SPF rejections.  RFC3463
and RFC5321 already define those.  We should leave that alone.  If we
want to give examples, that's fine, but I don't think we should be
normative about it.

>> by adding a Receiver-SPF header to the received message. See Section 7.

This is covered elsewhere already.

>> User Level Policy framework where good vs bad mail is separated for the

I don't even know what the basis for this guidance is, much less that
Section 2 is the right place for it.

+1.

> discussion that's already there.

It's also a current best practice.  And we even have a choice of which to use.

I like the current text on that topic.

> interesting idea.  Where do you think it would fit in 9.3?

I think at best we could discuss the notion of having the two "levels"
or application points, but we need to stop short of offering advice of
how it should be done unless we have substantial evidence that doing
so is useful.  Do such (non-anecdotal) data exist?

This is Section 9 at best, if not an appendix.  But overall I think it
makes for more confusion for implementers to bring up this topic at
all.

> are distracting.  If anyone else thinks I should, please speak up.

I agree, don't do it.

-MSK

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 17 Aug 2012 22:04:01 -0400

On Friday, August 17, 2012 02:13:39 PM Scott Kitterman wrote:
> > The REJECT-ON-FAIL should be regarded a System Level local policy for
> > global filtering for all users, where the MARK-ON-FAIL should be regarded
> > as a User Level Policy framework where good vs bad mail is separated for
> > the user.
...
> I think the last bit about system level policy versus user level policy is
> an  interesting idea.  Where do you think it would fit in 9.3?

I thought about this some more and I don't think it's helpful after all.

Mark/Reject can be done differently on a per-user basis.  It sometimes produces 
odd results for multi-recipient mail, but that's manageable.

There are other SMTP time tests that can be done, including, if one is so 
inclined content filtering and virus checks.

The distinction is really a different set of design approaches.

It is possible and reasonable to approach design of a "spam defense" system 
two different ways:

1.  A system with a series of discrete steps where each step produces it's own 
definitive result.  A message that fails any single step along the way is 
rejected or marked 'bad'.  It might be Local IP black list check -> DNSBL 
check -> SPF HELO check (and there might be a whitelist that then skips some 
or all the rest of the checks for whiltelisted hosts) -> rDNS check -> SPF 
Mail From check -> anti-virus check -> anti-spam content filtering -> accepted.

2.  A system where information about various attributes of the message (e.g. 
the results of all the above checks and more) are collected and evaluated in 
some weighted evaluation scheme to produce a decision about what to do with 
the message.

The first one is easier to conceptualize, design, and implement because the 
problem can be trivially broken down into a series of small, largely 
independent efforts.  The second requires more information to be collect and 
stored and development/testing of a complex/robust algorithm for integrating 
and balancing the various results.  Neither one of these is inherently better 
than the other.

When I'm doing systems work, I prefer the first kind of approach when I can get 
a satisfactory result.  It's easier to design and to verify it's working as 
expected.  There is no guarantee that the second 'cook it into a stew' 
approach will produce a better result.

Given that, I don't think this kind of design philosophy has a place in 
4408bis.  I think we should (as I've done) try to limit ourselves to factual 
advantages and disadvantages of reject and mark and leave it at that.

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 17 Aug 2012 22:51:26 -0400

> the From: header.  Suggest using one of the terms in 1.3.3

Done.

> records that the receiver publishes.

Done.

> 3: suggest ref to 4.6.4 as well as 9.1.1

Done

> 3.3: remove parenthetical ref to SPF RR type

Done.

> problem with all DNS usage, nothing specific to SPF

This is true, but it's all limits you have to keep in mind when developing an 
SPF record.

> using X.EXAMPLE or EXAMPLE.COM

3.5 specifically states it's extending an example in RFC 1034 (which uses 
X.COM), so I think it should stay as is to match the rest of the example.  
Since it's in other longstanding RFCs, I think there's no harm in having it 
here.

> in use in outgoing mail and leave it at that.

I took a whack at this.  Please check the next revision and see if you think 
it's adequate.

> it might not be coded as a function at all.

Done.  Given the pre-amble we have at the start of Section 4 now, I think 
that's appropriate.

> stops and the topmost check_host() returns temperror.  Also in the

Fixed those and one other.

> 5.5: yearrs -> years

Fixed.

> 6.1: modifier intended -> modifier is intended

Fixed.

Does today's message from Arthur Thisell answer this:

> As far as whether the result is a valid domain name, that is spelled out a
> few paragraphs above where it says "When the result of macro expansion is
> used in a domain name query, ...".  It doesn't matter if the result isn't a
> valid host name, the DNS allows for 8-bit data in labels, except \0.  Some
> of the DNS RFCs even give examples of such, and you can enter these labels
> in bind-format zone files.

As long as it meets RFC 1034/5 requirements (I don't remember which and I'm 
too tired to look) then isn't it a matter of issuing a query and seeing what 
you get back?  If it doesn't meet the relevant requirements to do than, then 
it's got to be a permerror.  I'd welcome improved text.

> 10.3, second bullet: I think TCP spoofing was fixed long enough ago

I don't put it past anyone to mess stuff up, so I'd prefer not to remove it.  
Instead I added "In a modern, correctly configured system the risk of this is 
negligible." to the end.  Does that work for you?

Scott K

---

From: "John Levine" <[email-masked]>
Date: 18 Aug 2012 02:55:24 -0000

>I thought about this some more and I don't think it's helpful after all.

Agreed.  This is way, way out of scope of this document.  Our job is
to tell people how to determine SPF's opinion of a message.  It is not
to to speculate about how one might build a spam filtering system on
top of it.

Beyond being out of scope, it's also beyond our area of expertise.  I
think that if you showed some of the proffered advice to people who
run large mail systems, the best you'd get would be a polite chuckle.

R's,
John

---

From: "John Levine" <[email-masked]>
Date: 18 Aug 2012 03:06:52 -0000

>Since it's in other longstanding RFCs, I think there's no harm in having it 

Doesn't matter to me, but the IESG has been known to obsess on this.

>I took a whack at this.  Please check the next revision and see if you think 

OK, will do so.

>Does today's message from Arthur Thisell answer this:

Not really.  

> then isn't it a matter of issuing a query and seeing what 

The problem is that it's ill specified.  If you give a resolver
library an invalid domain name, there's no guarantee that it will
politely reject it.  It might crash or do other random badness.  This
isn't a permerror, since the SPF record might well work if used on a
message with a less nasty bounce address.  Waving my hands, I'd
suggest a soft fail if a macro expansion creates an invalid domain
name.

>negligible." to the end.  Does that work for you?

You'd have to find a TCP stack more than 15 years old for TCP sequence
guessing to be a problem.  Really, it's dead.  I've been trying to
stamp out discussions of TCP spoofing because incompetent system
managers still point to them and insist that they're not sending spam,
it was spoofed.  Not.

R's,
John

---

From: Hector Santos <[email-masked]>
Date: Fri, 17 Aug 2012 23:43:36 -0400

Note: I was not proposing the terms REJECT/MARK-ON-FAIL, only using it 
for discussion.

I only pointed out how it was being discussed by deployment 
discussions with MARK-ON-FAIL because once the mail is accepted, any 
filtering will be a USER LEVEL POLICY concept and that is where it 
becomes more complex and I am glad to see, the view is now realized. 
I wanted to emphasize is it not the prevailing idea for SPF as it was 
made out to be. REJECT is a very much a big part of the SPF protocol 
and domains publishing strong exclusive -ALL policies do have a high 
expectation for SPOOF protection where no other INPUT is required, 
additional input undefined or otherwise. -ALL means its "This Way" and 
not "That Way!"  There is no ambiguity. There is no reputation concept 
here to muddy the water with.  Of course, there could be, but that is 
a RECEIVER thing, not a SPF PUBLISHER thing. -ALL means a specific 
thing, well all the mechanisms do, and -ALL (always) had a specific 
RFC2119 strong compliance language. MARK, quite frankly, I personally 
believe is only being mentioned as the prevailing method by a few that 
never quite believed in SPF since the git-go. Thats not a bad rap, its 
the truth and their publicly expressed viewed for a long time.  Many 
people in the SMTP world don't believe in strong deterministic SMTP 
level rejection concepts. Many do believe of leaving it all up to the 
USER.  Well, SPF was not one of those "iffy" protocols, we needed 
something straight forward and the strong hard, even softfail 
policies, and perhaps that is what bothered many. DKIM tried to 
changed that and couldn't.

Anyway, the whole point is that REJECT-ON-FAIL vs MARK-ON-FAIL are the 
protocol options for the receiver and MARK-ON-FAIL MUST NOT circumvent 
the security REJECT-ON-FAIL provides. The only recommendation or 
additional insights for MARK-ON-FAIL is to make sure accepted mail 
streams and user mail access, etc are separated otherwise you have a 
protocol security loophole.

How this is stated, I leave to you or others that wish to describe 
this basic concept here for REJECT vs MARK.  The MUA method is very 
important here.

Scott Kitterman wrote:
> _______________________________________________
> spfbis mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/spfbis
> 
> 

-- 
HLS

---

From: Scott Kitterman <[email-masked]>
Date: Sat, 18 Aug 2012 00:36:52 -0400

> but I disagree that it's a cumbersome notation.  That could just be

I'm inclined to put it back, but then that could be just because I'm not used 
to it.  I'll wait for more feedback.

> the terms interchangeably.

If I used the wrong one in some place, please cite specifics, so I can look at 
it and fix it.

> rejection from a receiver willing to do so is to use -all".

I think it's reasonable to say if you want X, Y is how you get it.  I think 
that's what this does.  It takes the place of a generic "use -all" 
recommendation in 4408 that isn't particularly defensible IMO.

> concise, and we should leave it.

I think the text is precisely correct at the moment.  It's a matter of local 
policy.  Period.  No more.  As part of the protocol I don't think we can do 
more.  If more is needed, I believe it has to be in Section 9 where we provide 
some advice on what to consider when developing local policy.

> But you could say "the identity under evaluation" or something like that.

The SPF result would be the same either way (mfrom or helo), but the 
implications of it might be different.  I don't see the benefit in the change, 
but if people think it would be better, I'm fine with it.

> > redundant after Section 2.3.  I'd strike it.

The first sentence is redundant.  I've removed it.  The second one is not.  
I've left it.

> > In the paragraph next to it, s/in Section 3/above/.

Changed to "above (Section 3)".

> Meh.  Take it or leave it.

Me too.  IIRC this is unchanged from 4408, so I'd rather leave it unless 
there's a real problem.

> > Remove the parenthesized phrase.

Fixed

Done.

Done.

> I don't believe the grammar does allow that.

I don't think it should.

> entire evaluation, or does processing continue?

It does propagate upward.  It has to for consistency.

In -04 it says:

>    If it matches, processing ends and the qualifier value is returned as
>    the result of that record.  If it does not match, processing
>    continues with the next mechanism.  If it throws an exception,
>    mechanism processing ends and the exception value is returned.

I think "mechanism processing ends and the exception value is returned" is 
clear, particularly if you look at it in combination with what's in Section 
5.2 on includes:

>    3.  The recursive evaluation returns either match, not match, or an
>        error.  If it matches, then the appropriate result for the
>        include: mechanism is used (e.g. include or +include gives a
>        "pass" result and -include gives "fail).

Does either of those need change to make it clear the error isn't somehow 
discarded?

> > Is there a reason why "Header Field" is capitalized that way?

Fixed in all but a few cases where it was part of a title.

Done.

> > The next paragraph should be split in two:

This one:

      If an SMTP receiver chooses to do so, it SHOULD use one of these header
      fields for each identity that was checked.  This information is intended
      for the recipient.  (Information intended for the sender is described in
      <xref target="mod-exp"/>, Explanation.)

If so, I don't see it?

> > single header field, so it doesn't make much sense to require to use

I think I cleaned all this up a bit and it should ~OK now.

> coordinating with Received:, but that seems a sensible thing to do.

I took that out.

> it.  That's why we didn't do that in the first place.

I don't think this is the place to extend RFC 5451 and I also don't think 
providing this information in "reason" is overloading at all.  The information 
I suggest be provided is, in fact, the reason for the result.

> > "SHOULD be aware" doesn't mean anything to me.  I'd make it non-normative.

I changed it to Note: If ....

> Indifferent on that suggestion.

I changed 3.5, so I think it's better to leave it in.

> pseudocode needs to be very carefully considered.

Implementation specific DNS setup advice is, I think, far beyond what we should 
provide.  I know a lot of people are used to thinking in terms of BIND zone 
files, but there's more than one way to do it.  I was using script in a very 
generic sense.  In many cases these days the tricky part would be the SQL for 
the back end database.

Done.

> > using "SPF verifier(s)" instead?

I replaced 12.

Thanks for all the comments.

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Sat, 18 Aug 2012 00:40:05 -0400

> Hostname TXT "v=SPF1 a ~all"

For a sending hostname there's no reason not use -all.  If we want to 
recommend something that should be it.

> Hostname TXT "v=SPF1 redirect=domain"

Why would this not be just TXT "v=SPF1 -all"

These are two of the safest SPF records to publish.  For them to be wrong you 
either have to get IP addresses of hostnames wrong or have hosts sending mail 
and not know it.  If you want to make a recommendation, don't pussyfoot around 
here.

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Sat, 18 Aug 2012 00:42:29 -0400

On Friday, August 17, 2012 02:46:24 PM Murray S. Kucherawy wrote:
...
...

We discussed this before.  This was in RFC 4408 (modulo one case that was 
accidentally missed and was an erratum), so including this isn't novel.  I 
view this as part of consistency.  I miss what is the case for removing it.  

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Sat, 18 Aug 2012 01:09:44 -0400

> Doesn't matter to me, but the IESG has been known to obsess on this.

My usual response to such things is "I'll be glad to fix it after you go fix the 
reference I copied it from".  I'll leave it as is I guess unless one there's a 
upswell to change it.

> >I took a whack at this.  Please check the next revision and see if you
> >think it's adequate.
> 
> OK, will do so.
> 
> >you get back?  If it doesn't meet the relevant requirements to do than,
> >then it's got to be a permerror.  I'd welcome improved text.
> 

So I think we already have this problem without EAI.  If I read  the 
intersection of RFC 5322 correctly, there are non-EAI characters allowed in 
local-part that aren't allowed in domain names, they can be quoted, I would 
guess.  So leaving EAI aside for a moment, how do we specify it?  I think 
either quote and then look up or raise permerror as this isn't something 
that's going to change?

> >I don't put it past anyone to mess stuff up, so I'd prefer not to remove
> >it. Instead I added "In a modern, correctly configured system the risk of
> >this is negligible." to the end.  Does that work for you?
> 

OK.  How about:

      The client IP address, <ip>, is assumed to be correct.  In a
      modern, correctly configured system the risk of this not being true is
      nil

How about that?   I put that in for the moment.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 17 Aug 2012 23:09:11 -0700

> Does either of those need change to make it clear the error isn't somehow

I think if you were to say "all mechanism processing ends", it's more
clear.  Right now I think one could read it as "this mechanism
processing ends, but continue".  Tacking "all" before it is more
explicit.

> I suggest be provided is, in fact, the reason for the result.

Yep, that's fine.

-MSK

---

From: S Moonesamy <[email-masked]>
Date: Fri, 17 Aug 2012 23:05:53 -0700

>Since it's in other longstanding RFCs, I think there's no harm in having it

Please see https://www.ietf.org/iesg/statement/examples.html  I don't 
think that it is worth arguing about this during an IESG Evaluation.

Regards,
S. Moonesamy

---

From: Hector Santos <[email-masked]>
Date: Sat, 18 Aug 2012 03:37:56 -0400

> view this as part of consistency.  I miss what is the case for removing it.  

Thats a matter of style. I prefer not having multiple documents open. 
  Being specific is terrific, making it easy for readers.

-- 
HLS

---

From: Hector Santos <[email-masked]>
Date: Sat, 18 Aug 2012 03:42:38 -0400

> to it.  I'll wait for more feedback.

This is among one of the review comment items I will be posting soon.

+1.  I don't quite like the massive changes to use RFC5321 prefixing 
from what are otherwise generic SMTP terms well understood for 
decades.  It also places a tight requirement for a specific RFC # when 
its possible that the # can change in the further. Plus, in good 
acceptable technical writing style, a first time reference is all that 
is needed. I don't think we were in trouble with using well understand 
SMTP terms.  It also makes it very redundant, harder to read, to speak 
out, too technical. Functionality is all that is conveyed.

-- 
HLS

---

From: Franck Martin <[email-masked]>
Date: Sat, 18 Aug 2012 09:18:02 +0000

On 8/17/12 9:40 PM, "Scott Kitterman" <[email-masked]> wrote:

>> >>                                               Zone file generation
>>for
Agreed.

I just did not want to look like the usual dictator I am :P

---

From: Alessandro Vesely <[email-masked]>
Date: Sat, 18 Aug 2012 16:31:25 +0200

On Sat 18/Aug/2012 06:36:52 +0200 Scott Kitterman wrote:
> I'm inclined to put it back, but then that could be just because
> I'm not used to it.  I'll wait for more feedback.

<More feedback to be added here, please>

>> Section 9.3 at this point is sufficient.

I just tried to make the reference look smoother.  Any change of
meaning was unintended.

> If I used the wrong one in some place, please cite specifics, so I
> can look at it and fix it.

There are lots of them.  Most occurrences of "domain" in sections 2.*
and 3 actually mean ADMD, especially "domain owner" or "domain
holder".  In Section 3.5 the term "zone file" might be better.

I'd suggest to dedicate an I-D version entirely to such changes, so as
to ease reading the diff.

> that's what this does.

I agree with Murray's reading of that snippet.  IMHO, the sentence he
quotes is much better.  It uses no 2119-language, though.  Compare it
with the worsen rewriting:

   The RECOMMENDED way to coerce rejection from a receiver willing to
   do so is to use -all.

> It takes the place of a generic "use -all" recommendation in 4408
> that isn't particularly defensible IMO.

I agree, otherwise it should be the default.  In addition, since -all
can only be used by ADMDs that are able to publish a well crafted set
of records, we'd have to explain why we prefer that the other ones
publish no record at all rather than a coarse ~all.

>> local policy".

While correct that's redundant, since accepting and delivering
messages is the normal activity.

> more.

It'd be odd to have no normative statements about rejection, given
that that is one of the relevant points of the protocol.

> implications of it might be different.

Yes.

> I don't see the benefit in the change, but if people think it would
> be better, I'm fine with it.

See my next message, in another branch of this thread.

> Me too.  IIRC this is unchanged from 4408, so I'd rather leave it unless 

No real problem, both concerns are addressed separately, later in the
spec.

>> don't get zone files, they get raw DNS data.

Uh, I thought it was advice to the publishers.

>> The Section 3 references are correct.

In that case, may I suggest they be worded as "[RFC1035] Section
3.3.14 and [RFC1035] Section 3.3"?  That's to work around a limit of
rfcmarkup.

>> DNS over TCP because the need for that fallback is historically rare.

Fine.  Perhaps "[...] that cause DNS over TCP to work intermittently"?
That use of "reliable" in a TCP versus UDP discussion sounds odd.

>> There were hints of this family of problems in the experiment

Worth a citation?

>> I don't believe the grammar does allow that.

Right, it doesn't.  It is "ptr" [ ":" domain-spec ], with no
cidr-length.  Sorry for doubting.

> Does either of those need change to make it clear the error isn't somehow 

Perhaps, a note could be appended to the paragraph after the
algorithm, so as to read, say:

   If a DNS error occurs while doing an A RR lookup, then that domain
   name is skipped and the search continues, but the error is noted
   so as to eventually return an error code if no match is found.

>>> The next paragraph should be split in two:

The idea was two paragraphs instead of one, as exemplified below.

>> "one" is enough; "one and only one" is too wordy.  That's fine otherwise.

I'd like better the short version too, but the meaning is different:

"SHOULD use one"  was already said at the beginning of the section.
"SHOULD use only one"  prevents using both (why?  See below.)
"SHOULD use one and only one"  is admittedly too wordy.

> I think I cleaned all this up a bit and it should ~OK now.

The point here is the phrase "for each identity that was checked" in
the second paragraph of Section 7 (before the split).  It could be
moved to the first sentence of the previous paragraph ("the result of
SPF processing for each identity that was checked")  --if we decide we
won't define a combined result.

>> it.  That's why we didn't do that in the first place.

I read part of those threads in mail-vet-discuss.  SPF wasn't
different at the time, and the same argument seems to be valid here
today.  Besides "client-ip", A-R only misses "problem", and
"mechanism":  authserv-id doesn't match "receiver" exactly, but bears
the same semantics.  The "identity" can be implicit as in:

   Authentication-Results: receiver.example;
     spf=pass smtp.mailfrom=sender.example;
     spf=none smtp.helo=mail.sender.example

The explanation is a comment in Received-SPF and can stay that way.

> I don't think this is the place to extend RFC 5451 and I also don't
> think providing this information in "reason" is overloading at all.
> The information I suggest be provided is, in fact, the reason for
> the result.

It worsens a field that is already difficult to parse, especially if
you also add problem and mechanism.  Users who currently set those
tags in Received-SPF and have software for parsing it downstream will
never want to convert it to a new format for whatever reason --and
that's why I wouldn't insist on "SHOULD use only one" above.

After all, that information is internal to the given ADMD, and the
missing info seems to be only relevant for debugging.  We have better
means to do that, nowadays.  If passing those values in the headers is
what users really want, so be it.  But setting a value to a name=value
string to be interpreted on its own is clearly a kludge that we
shouldn't even suggest, let alone putting a SHOULD for doing so.

---

From: Alessandro Vesely <[email-masked]>
Date: Sat, 18 Aug 2012 16:31:35 +0200

On Fri 17/Aug/2012 23:46:24 +0200 Murray S. Kucherawy wrote:
> 
> I think the current text for "Fail" explains perfectly well that
> the option of what to do with the message is left to the receiver,
> as it should be.  That's what "disposition" is.

I beg to disagree.  The current wording resembles a generic statement
that receivers can do as they like:

   Disposition of SPF fail messages is a matter of local policy.

Does that mean local policy doesn't have a say in case the result is
"none"?  In other 2.5.x sections we may want to say something like
Section 2.5.5 does:

   Receiving servers SHOULD NOT reject mail based solely on this
   SPF result.

Not "In this case disposition is not a matter of local policy."

BTW, there is still a lowercase "should interpret" in the first
paragraph of Section 2.5.

> If we really want more discussion up in Section 2.5.x, we can make
> reference down to Section 9.3.2 which discusses the point in more
> detail.

A reference to Section 9.3 is there already.

> want to give examples, that's fine, but I don't think we should be

+1!  In addition, the current spec doesn't mention that a server may
want to decline a whole session as a result of a failed HELO check.
SHOULD a server reject each and every transaction instead?

Although this was in RFC 4408 and not novel, the relevant SMTP text
was moved from Section 3.7 of RFC 2821 to its own Section 3.6.2 of RFC
5321, where the "policy reasons" are further characterized by using
the term "SPF".  Repeating normative parts introduces ambiguity as to
whether the two protocols overlap.

> This is covered elsewhere already.

Correct, just in Section 7's opening words.

>> are distracting.  If anyone else thinks I should, please speak up.

+1.  However, a note somewhere in Section 1.3 might say:

   We use the verb "reject" in predicates involving email messages or
   sessions as a shortcut for rejecting the corresponding SMTP
   [RFC5321] commands;  that is, replying a 5xx code to them.  The
   phrase "temporarily reject" indicates 4xx codes instead.

---

From: "John Levine" <[email-masked]>
Date: 18 Aug 2012 15:15:09 -0000

>to it.  I'll wait for more feedback.

It reads better if you put it back.  Anyone who can't understand that
the HELO identity can come from either HELO or EHLO is unlikely to be
able to understand the rest of the document, either.

R's,
John

---

From: Dave Crocker <[email-masked]>
Date: Sat, 18 Aug 2012 11:29:35 -0400

> able to understand the rest of the document, either.

There is a pattern of confusion about transport commands vs. message
field names.  the spec.name convention disambiguates.  Use of the name
alone makes sense when the context has been set.  Using the name alone
without setting that context (by an initial use of the qualified name)
invites confusion even when the name is unique.

As for HELO vs. EHLO, they really are two different 'commands'.  Hence,
saying HELO is a precise reference that excludes EHLO, even though
that's not what's meant.  Would that we had the practice of saying SMTP
Greeting, or some other generic term.

I'll suggest RFC5321.[E]HELO as a compromise.

Really demanding sticklers might be more comfortable with RFC5321.HELO/EHLO

d/

-- 
 Dave Crocker
 Brandenburg InternetWorking
 bbiw.net

---

From: "John R Levine" <[email-masked]>
Date: 18 Aug 2012 11:36:57 -0400

> Really demanding sticklers might be more comfortable with RFC5321.HELO/EHLO

This grouch thinks it would be nice to say something like "The HELO 
identity is the RFC5321.HELO or RFC5321.EHLO field from the beginning of 
the SMTP session" and thereafter say something less stilted.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
"I dropped the toothpaste", said Tom, crestfallenly.

---

From: Scott Kitterman <[email-masked]>
Date: Sat, 18 Aug 2012 12:25:21 -0400

> think that it is worth arguing about this during an IESG Evaluation.

The statement speaks of harm.  Since this exact information is already in RFC 
1035, there is no harm.  Was this message sent as chair or participant?

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Sat, 18 Aug 2012 12:26:45 -0400

> the SMTP session" and thereafter say something less stilted.

This one too.

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Sat, 18 Aug 2012 12:51:26 -0400

...
> I just did not want to look like the usual dictator I am :P

I've included something similar to this for the next draft.  Please have a 
look when I've published it.

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Sat, 18 Aug 2012 13:02:39 -0400

> to ease reading the diff.

Or:

I uploaded the XML with -04 so if someone wants to send me a patch, that'd be 
lovely.

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Sat, 18 Aug 2012 13:10:04 -0400

> messages is the normal activity.

Your ham/spam ratio must be much higher than mine.  I think in most cases 
'bad' mail significantly outnumbers good mail and figuring out how not to 
deliver those message is the predominate activity.

Given that the next paragraph starts with "If the checking software chooses to 
reject the mail ..." I don't think adding what you proposed says anything that 
isn't already said.  That and no matter what we say in 4408bis, receivers can 
reject whatever they want anyway.  

With my free software developer hat on, I've had feature requests to enable 
rejecting mail from all domains that don't publish SPF records (I said no, 
there's a limit to the degree of foot shooting I'm willing to write code for).  
I don't think anyone is confused that the possibility of rejection doesn't 
exist.

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Sat, 18 Aug 2012 13:40 -0400

>    Disposition of SPF fail messages is a matter of local policy.

It does and it's correct.  Receivers can do as they like.  They don't get to 
decide how to determine if an SPF result is "fail", that is standardized, but 
once they have that result they get to decide what to do with it.

> Not "In this case disposition is not a matter of local policy."

If there's no SPF result there can be no SPF related policy, so I don't see a 
need for a change.

Fixed.

> SHOULD a server reject each and every transaction instead?

I think that's up to local policy/an implementation detail.  HELO/EHLO can 
change during a session, so it's not axiomatic that one should.

> whether the two protocols overlap.

Perhaps.  I agree it's reasonably obvious what one should pick for the SMTP 
reply code.  For enhanced status codes, it's much less clear.  Reviewing 
http://www.iana.org/assignments/smtp-enhanced-status-codes/smtp-enhanced-
status-codes.xml does not lead me to one obviously correct solution.  I think 
providing additional specificity is useful.

> >> I do not like the idea of introducing the all caps terms
> >> MARK-ON/REJECT-ON, so I haven't used them in my update.  I think they
> >> come across as shouting and are distracting.  If anyone else thinks I
> >> should, please speak up.> 
> > I agree, don't do it.
> 

Since we have the SMTP reply and enhanced status codes, I think this is 
already clear.  If removing them would make this unclear, I think that's 
another reason not to remove them.

Scott K

---

From: S Moonesamy <[email-masked]>
Date: Sat, 18 Aug 2012 11:23:04 -0700

>1035, there is no harm.  Was this message sent as chair or participant?

It was not as WG co-chair as I usually explicitly mention that.

As WG co-chair, I would like to remind the working group of the IESG 
statement at https://www.ietf.org/iesg/statement/examples.html

Regards,
S. Moonesamy
SPFBIS WG co-chair

---

From: "John Levine" <[email-masked]>
Date: 18 Aug 2012 20:06:25 -0000

>reference I copied it from".  I'll leave it as is I guess unless one there's a 

Like I said, it doesn't matter to me, but when the IESG puts a discuss on
this for X.COM, we told you so.

R's,
John

---

From: Hector Santos <[email-masked]>
Date: Sat, 18 Aug 2012 17:54:22 -0400

My review:

o Abstract.

I prefer the previous version with the reverse-path. SPF protects the 
REVERSE PATH, not the AUTHOR'S ADDRESS. The abstract can provide the 
wrong idea on what SPF attempts to protect.

o Introduction

In the introduction, it introduces a new vague term with a vague 
definition.

    .... "reverse pointer" which is the purported hostname
    of the SMTP client as far as the DNS is concerned [RFC1035].

SPF attempts to offer technology to help protect against the forging 
of the EHLO/HELO and MAIL FROM domain fields.  No mas.  I suggest that 
we keep with long time terms we know these two fields to be.

   MAIL FROM -> Reverse Path
   EHLO/HELO -> host, primary host name, host domain name, machine 
host name,
                address literals, etc.

In practice, I have used Primary Host Name (PHN) or generally Client 
Domain Name (CDN) in process modeling and that is not far off from how 
SMTP described it:

     fully-qualified domain name of the SMTP client

o usage of RFC5321 prefixes

I don't quite like the massive changes to use RFC5321 prefixing from 
what are otherwise generic SMTP terms well understood for decades.  It 
also places a tight requirement for a specific RFC # when its possible 
that the # can change in the further. Plus, in good acceptable 
technical writing style, a first time reference is all that is needed.
I don't think we were in trouble with using well understand SMTP 
terms.  It also makes it very redundant, harder to read, to speak out, 
too technical. Functionality is all that is conveyed.

o 1.3.5.  Deprecated

    There are [RFC4408] features that are marked "deprecated".  In the
    context of this document, deprecated means that senders SHOULD NOT
    publish SPF records that make use of the feature in question.

Here you made a specific definition of what "deprecated" means making 
that much harder to use the term in general form across the board.  I 
suggest to remove the section and just be specific where it is 
necessary to describe when PUBLISHING is not recommended.  This allows
us to use the generic and open ended form of the term deprecation when 
and where applicable.   As you have it now, its a puzzle!  You locked 
it in for the publishing usage, but its quite possible that there 
could be a receiver feature that we may deprecate, i.e. suggest ideas 
such as deprecating Receiver-SPF if we can get the Auth-Res correctly 
matched for SPF.  It is not a far fetch idea for Receiver designs to 
offer to customers especially for those designs that are have 
integrated concepts with other protocols using Auth-Res.

o 2.1.  The HELO RFC5321.HELO/.EHLO Identity

I don't like how this is written - too strong and needs more 
information.  Practical deployments and experience suggest a continue 
high rate of false and bad client domain names (EHLO/HELO) and thus 
will continue to be a high yield of wasted lookup overhead.

I prefer this to be a MAY with more details gathered from what we 
learned over the 9 years and also probably debate if there is a trend 
towards correction which was the main reason I pushed Meng Wong to 
re-add HELO checking - to promote RFC5321 (RFC2821 back then) strong 
SMTP compliance and correctness especially for the primary client 
machine name.

What I am saying is that an automatic Check_Host() function can not be 
always be applied without other syntax and local domain checking 
considerations. Implementators need to make this very flexible 
otherwise there is a high overhead. For the sake of example, you don't
need to do an EHLO/HELO SPF check for fields such as:

    - loopback domains such as "localhost"
    - square bracketed ip literals
    - strings without a dot (could be a netbios name)

The field should syntactically correct before the SPF check is done 
and at this expected point, I believe it should remain as a MAY, not 
RECOMMENDED with local domain spoof protection being the #1 benefit 
you can get here.

o 2.2.  The RFC5321.MailFrom Identity

I don't like this section beginning with a NULL discussion without 
describing this field.  The prev revision paragraph removed was 
sufficient - suggest to add back then lead to NULL use cases.

I never quite noticed this before, but the sentence with the usage of 
"postmaster"

    When the reverse-path is null, this document defines the
    RFC5321.MailFrom identity to be the mailbox composed of the local-part
    "postmaster" ....

Two things:

- Confusing. RFC5321.MailFrom is NULL. Do you mean the RFC5322.From 
with have the "postmaster" for the local part?   So would this be a 
correction?

     When the reverse-path is null, this document expects the
     RFC5322.From identity to be the mailbox composed of the local-part
     "postmaster" ....

- There may not be any control of the SMTP receiver what local part 
address is used for the bounce since the bounce could be done outside 
the completion and SMTP reception of mail with a gateway or mail 
importer.  It is still very common to see mailer-daemon and there is 
RFC2142 for decades old mailbox names for what are acceptable 
local-parts for NULL addresses and other functions.

Overall, this is implying that an SPF implementation SHOULD or MUST 
use "postmaster" for any bounce/dsn message it creates and in 
additional, it implies if you get a bounce (NULL reverse path), then 
expect the 5322.From to have a local-part "postmaster".

I suggest that this sentence is vague and should be removed to avoid 
any false logic checks that may not exist.  The bounce can be from 
"mailer-daemon" or other mailbox names.  What makes it a true SMTP 
defined bounce is the NULL reverse path - not the mailbox name.  The 
different kinds of mailbox name generally drives different mail bots 
or translations.

o 2.3.  Publishing Authorization

    An SPF-compliant domain MUST publish a valid SPF record as described
    in Section 3.  This record authorizes the use of the domain name in
    the RFC5321.HELO/.EHLO and RFC5321.MailFrom identities
    by the MTAs it specifies.

The last sentence implies ONE record (via use of "This record" ...) is 
  sufficient to cover both fields when in practice it usually does not.

    When changing SPF records, care must be taken to ensure that there is
    a transition period so that the old policy remains valid until all
    legitimate email has can reasonably expect to have been checked.  This
    can be as much as 30 days.

Where does this 30 days come from?

IME, that could be 4, 5, 7 days, 2 weeks or even more than 30 days. It 
must be some particular setup/experience, but certainly not universal. 
As far as adding some  "SMTP" related factor, it would be 4-5 days for 
the recommended exhaustive transport retries where it is very much 
applicable to SPF. In other words, if a message does not get sent 
within 4-5 SMTP "example guideline" days, it usually gets bounced. 
That is the guideline we used for other things too like DKIM revoked 
records as well.  Whatever the limit, it should be pretty close to the 
SMTP practiced of 4-5 days.

- Address the security issues first.

Per issue 29 and section 2.5.4 and the links I provided in a previous 
post for additional info, overall, IMO, we need to define what 
"marking" the SPF hard failed message means and how this deployment 
alternative can expose a security level difference compared the 
RFC2119 compliance option of REJECT.

We need to note that implementators SHOULD offer both options to 
deployments (operators, admins, etc).  I think we should state what is 
the MINIMUM requirement for SPF compliance, and one can argue since 
REJECT is the only RFC2119 level language, it should be the minimum 
offering.  The alternative is MARKING and I believe, we only need to 
describe the responsibility regarding mail separation at this point.

I personally believe even adding a section or paragraph regarding 
System Level vs User Level Filtering is warranted as a nice way to 
introduce and/or reference the SMTP level REJECT vs mail acceptance 
MARKing and the recommended mail stream separation design/deployment 
guideline.

o TBD - other items to comment on

-- 
HLS

---

From: S Moonesamy <[email-masked]>
Date: Sat, 18 Aug 2012 16:00:06 -0700

>this for X.COM, we told you so.

I am using the above comment from John Levine for context.

A DISCUSS from the IESG is not an unsolvable problem.  It means that 
an Area Director has identified an issue and he/she would like to 
talk about it with the working group.  Let's assume that there is a 
DISCUSS about X.COM.  The DISCUSS is for the working group and it is 
up to the working group to agree on a text change or an explanation 
about why it decided not to make any change.

Regards,
S. Moonesamy
SPFBIS WG co-chair

---

From: Scott Kitterman <[email-masked]>
Date: Sat, 18 Aug 2012 21:00:45 -0400

On Saturday, August 18, 2012 11:23:04 AM S Moonesamy wrote:
> >The statement speaks of harm.  Since this exact information is already in
> >RFC 1035, there is no harm.  Was this message sent as chair or
> >participant?

I've reviewed it.  The operative bit is:

> The IETF should minimize any potential impact on the entity having been
> assigned such a codepoint from unwanted traffic or other concerns. To help
> ensure this the IESG will expect the author of any Internet Draft that
> defines a new specification to use values assigned for examples whenever
> possible. The IESG also recommends new protocol specifications creating new
> codepoint, address or names spaces to consider if some part of the space
> should be reserved for documentation and example usage.

I don't think it insists on a change between 4408 and 4408bis.  We are not 
defining a new protocol in 4408bis.  It was defined in 4408.  Additionally, I 
don't think the current use in an example will have any affect or if it does, 
it's de minimus in comparison to that same example being in RFC 1035.  The 
statement does not suggest document writers should go back and rewrite 
existing examples.  I am assuming if the IESG wanted people to do that, they 
would have said so.

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Sat, 18 Aug 2012 21:59:51 -0400

> wrong idea on what SPF attempts to protect.

I've already removed author's address for the next revision based on a 
previous comment, so please check -05 when it's published.

>     of the SMTP client as far as the DNS is concerned [RFC1035].

I went back and looked and in RFC 1035, Section 3.5 the actual term used is 
"primary domain names", which I don't think is particularly helpful either.

Looking over it again, I think that the ", plus ... " clause isn't necessary 
and have deleted it.

> too technical. Functionality is all that is conveyed.

Based on quite a number of comments, including this one, I'm switching back to 
using "HELO" or "MAIL FROM" in the document with the 5321.XXX forms being 
included in the definitions for each for technical clarity.  I'm reworking this 
a bit, so please review the next draft.

> integrated concepts with other protocols using Auth-Res.

So far we didn't have to deprecate anything that didn't fit this definition and 
I don't think we will.  If we do, that would be the time to revisit it.  This 
is here because there were a number of questions about what does deprecated 
mean.  I think it's useful to answer it once.

> RECOMMENDED with local domain spoof protection being the #1 benefit

This is RECOMMENDED in RFC 4408, so MAY would be a change, not RECOMMENDED.

I added "SPF checks can only be performed when the "HELO" string is a valid 
fully qualified domain." to the end of the "Note ..." paragraph to clarify when 
SPF checks have to be skipped.

> sufficient - suggest to add back then lead to NULL use cases.

Nothing was removed.  It was moved up into 1.3.3, the definition.  I agree 
though that diving right into the bounce case was a bit rough.  I reversed the 
two paragraphs in the section and I think it flows much better.

> different kinds of mailbox name generally drives different mail bots

Here's the full sentence (updated for my terminology changes I mentioned 
above):

> When the reverse-path is null, this document defines the "MAIL FROM"
> identity to be the mailbox composed of the local-part "postmaster" and the
> "HELO" identity (which might or might not have been checked separately
> before).

All this says is that if he reverse-path is null, the construct to use for an 
SPF "MAIL FROM" check is "postmaster@" + "HELO identity".  This is unchanged 
from RFC 4408 and I think is fine.  I think you may be reading too much into 
it.

>   sufficient to cover both fields when in practice it usually does not.

For a single domain it is.  This is an SPF-compliant domain, not an SPF-
compliant ADMD.  Yes, every domain needs it's own record, but it needs only 
one and that one can be used either for "MAIL FROM" or "HELO".

> Where does this 30 days come from?

I got a comment that the RFC 4408 language was too vague and we should provide 
specificity.  It used to say:

>    legitimate E-Mail has been checked.

The problem is that there's really no way to do that, so I tried to improve 
it.

> SMTP practiced of 4-5 days.

The problem is that there is no universal.  The only spec I recall seeing a 
specific recommendation on this is the original Microsoft Caller ID for Email 
specification.  It suggested 30 days.  I asked a few people with experience 
(which I don't have) in supporting domains with large consumer mail flows and 
the feedback I got was that 30 days would cover most, but not all cases.  I 
had been thinking a week should be enough, similar to your suggestion, but if 
we're going to try and bound the problem, I think we need to do it keeping in 
mind something that will broadly cover many situations.

> RFC2119 compliance option of REJECT.

I went back over your previous posts and I think that what we have in section 
9 does this.  Please review -05 and provide updated text based on that if you 
don't feel it does.

> MARKing and the recommended mail stream separation design/deployment

I don't see how to put this in a protocol document.  As I mentioned in a 
previous mail, system/user level designs can both do reject or 'marking' so it 
seems orthogonal to this discussion.

Scott K

---

From: Hector Santos <[email-masked]>
Date: Sat, 18 Aug 2012 23:46:37 -0400

> from RFC 4408 and I think is fine.  I think you may be reading too much into 

Yes I was.  Perhaps as you stated here would clear this because I 
didn't catch this before.  I can see how it can help to fill in data 
for macros, i.e. %{l}.  Perhaps necessary to add something for the 8.2 
Expansion examples for the special case:

    8.2.  Expansion Examples

    ..

    macro             expansion
    -------  -------------------
    ...
    %{l}              strong-bad
    %{l}              postmaster  (when reverse path is NULL, see 
section 2.2)
    ...

> mind something that will broadly cover many situations.

There are two delays:

   - SMTP Transport Delays
   - SMTP Acceptance/Mail Pickup Delays (Transport is complete)

Transport delays are well defined in practice. SMTP long recommended 
4-5 days.  But once received, Local Mail Storage policies come into 
play.  Despite what offline MUAs attempt to take control of, the 
backend is in full control. For our system, see attach image showing 
an example mail conference policies.  I intentionally set some typical 
past values for email you may see in the customer base. But these 
days, its archived forever anyway, unlimited or huge disk storage 
allocations for users allows for more server side storage which make 
the offline MUA work better too, especially for Roaming Users.

Unless I am missing something, I don't see how SPF applies to accepted 
mail payload pickup delays going beyond the 4-5 transport UNLESS we 
are time shifting the SPF checking application to the MUA.  In this 
case, yes, I can see the offlist time shift need to have a MUA related 
delay.

For DKIM, I had written a 2006 I-D related to this time shifting problem.

       http://tools.ietf.org/html/draft-santos-dkim-rcvd-00

allowing for a migration and partial DKIM receiver support to add the 
DKIM dns record to the headers so that delayed checkers can validate 
the message in the future just in case the DKIM record was revoked 
before the MUA picked up the mail.

I guess, for SPF, this would allow a Partial SPF Support receiver to 
just lookup, save the SPF policy record in the RFC3522 payload to 
allow a more complete SPF checker to do the work.  This could be the MUA.

-- 
HLS

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sat, 18 Aug 2012 20:53:27 -0700

I believe we are compelled by the IESG statement to change it.  And
even without that, I would push to change it anyway.

As the IESG Note on RFC4408 points out, it was published as-is for all
the reasons that applied at the time.  It never underwent official
IETF review of any kind, and the climate surrounding this work
probably means that even if this IESG statement had come out by then
(note that it came out some years later), it would've sidestepped any
such checking.

And as I would guess Andrew will happily attest, few people hold up
RFC103[45] as high examples of quality writing.  And our (ahem)
standards are a lot higher since then.

We are going to have bigger battles to fight when this document gets
to IESG evaluation.  Let's not burn much energy on this one.

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sat, 18 Aug 2012 21:34:46 -0700

> 9 does this.  Please review -05 and provide updated text based on that if you

+1.  There's no need to embellish what we're saying about handling of
"Fail" any more than what's there.

(Note a typo in the changed text in 2.5.4: "is" should be "it".)

> seems orthogonal to this discussion.

"REJECT" isn't an RFC2119 word last I checked, so I don't even
understand this claim.  There is no normative requirement to reject or
even mark a message, and there shouldn't be.  The operator gets to
decide what to do with a "fail" result and in the aggregate there are
good reasons to do either.  This isn't a new notion.

System vs. user level filtering and stream separation constitute
applicability statements in my view, and are not part of the base
protocol.

Finally, please, let's not get into the mess of minimum compliance statements.

-MSK

---

From: Scott Kitterman <[email-masked]>
Date: Sun, 19 Aug 2012 00:37:37 -0400

> (note that it came out some years later), it would've sidestepped any

That's not consistent with the record.  If you go back and look at 
http://datatracker.ietf.org/doc/rfc4408/history/ you'll find that the IESG did 
not just rubber stamp draft-schlitt-spf-classic.  Some significant changes were 
made between the initial submission and the final approval.  While your 
statement that "It never underwent official IETF review of any kind" is 
literally true it was reviewed on the namedroppers, ietf-smtp, and ietf-822 
lists in addition to the IESG review.

> standards are a lot higher since then.

It's not about them being well written, but that since the example is already 
there, there's no harm in reusing it.  

> to IESG evaluation.  Let's not burn much energy on this one.

Perhaps.  Is the IESG generally requiring examples to be rewritten in $bis 
documents?

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Sun, 19 Aug 2012 00:39:51 -0400

On Saturday, August 18, 2012 09:34:46 PM Murray S. Kucherawy wrote:
> > I went back over your previous posts and I think that what we have in
> > section 9 does this.  Please review -05 and provide updated text based on
> > that if you don't feel it does.
> 

Fixed.  Thanks.

> > previous mail, system/user level designs can both do reject or 'marking'
> > so it seems orthogonal to this discussion.
>

---

From: Hector Santos <[email-masked]>
Date: Sun, 19 Aug 2012 01:21:04 -0400

Scott,

(first, in 2.5.4. I don't agree with it, but it should reference 
9.3.2, not 9.3)

I disagree with the weakening of the SPF "Hard Fail" protocol crafted 
into section 2.5.4.  The  previous text was MORE clear.  The only 
issue is defining Marking.

No Receiver can claim SPF compliance when it comes to section 2.5.4 
and dealing with hard fails if it does not honor the publisher policy 
with either a dynamic SMTP level permanent 55x reject or a undefined 
"marking" of the messages.

Short of being non-SPF compliant, there are no other choices here, Its 
that simple.  This is not about reputation modeling, DKIM or DMARC, 
reporting or whatever, this is a straight forward nine year old 
deterministic protocol which for some here was very hard to swallow, 
but nonetheless, very much highly successful in the market place with 
a very clear understanding and fast entry point unlike anything else 
the past 30 years.

The issue here is defining what the alternative MARKING means and in 
my opinion section 9.3.2 is too vague on the subject.  Its not clear.

Section 2.5.4 is where the PROTOCOL options are introduced for 
rejection and marking and that is where it should be clear with the 
functionality is expected.  We don't need more mumbo jumbo, redundant, 
obvious "its local policy" claims, that only adds more confusion.  of 
course, its local policy what a system will do with SPF. But as far as 
the protocol is concern,  its very clear:

o Implementations (coders) MUST offer two options:

    - SMTP level permanent rejection
    - SMTP level acception with SPF hardfail marking

o Deployment MAY choose rejection or marking.

I am stating which is all that is being proposed added to 2.5.4 is 
that the MARKING functionality is an Security Loophole IFF you don't 
talk about stream separation.

More review into:

9.3.1.  Policy For SPF Pass

We don't need a SPF if the client is already part of a white list. 
Who does what is described in this section?   Is this is a SPF 
specific WHITE LIST?

We keep getting lost with a GOOD MODELING concept when SPF is about 
detecting SPOOFS for pete sake.

I can see advance parsing of the policy to look at the final yield, 
-ALL, ~ALL, ?ALL, etc and try to create some magic source formula, 
neural network, fuzzy or what have you for a PASS:

     > 1.0 for a PASS ending with a -ALL
     > 0.5 for a PASS ending with a ~ALL
     = 0.0 for a PASS ending with a ?ALL

Or some stuff like this, but a pass is not going to be usually against 
a white list. in fact, I suggest, SPF will be SKIPPED - like on our 
system if the sender is already white listed.

-- 
HLS

---

From: Scott Kitterman <[email-masked]>
Date: Sun, 19 Aug 2012 01:49:50 -0400

Fixed.

> talk about stream separation.

As far as I understand this comment, I think what you are saying is that 
instead of having 2.5.4 say this is a matter of local policy, see 9.3.2 (which 
then goes on to say something like you have to reject or mark) you want 2.5.4 
to say you have to reject or mark with further discussion about marking in 
9.3.2?

Which section would you propose have the discussion about the advantages and 
disadvantages of each approach?

If you think 9.3.2 is insufficient, please provide text based on what's there 
now.  I know you've provided text before, but it's difficult for me to relate 
and adapt it to where we are now since I don't understand your objections to 
the current text.

> Who does what is described in this section?   Is this is a SPF

It's a name based white list.  It's risky to whitelist domains without some 
kind of authorization check first.

> detecting SPOOFS for pete sake.

It's been about both since the beginning.

> system if the sender is already white listed.

If the SMTP client is already whitelisted, then you're probably right.  The 
idea is name based whitelists (and it's not a new idea).  For those you need 
to check that you're not applying a whitelist based on a spoofed name.

Scott K

---

From: SM <[email-masked]>
Date: Sun, 19 Aug 2012 01:10:19 -0700

>Perhaps.  Is the IESG generally requiring examples to be rewritten in $bis

It has been pointed out during the IESG Evaluation of a specification 
which was advanced on the Standards Track.  There were two DISCUSSes.

Regards,
-sm

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sun, 19 Aug 2012 10:07:48 -0700

> lists in addition to the IESG review.

Was this particular point discussed in those venues?

> Perhaps.  Is the IESG generally requiring examples to be rewritten in $bis

I believe so, yes.

Regardless what's the harm in correcting the examples to be more
acceptable to common document conventions?

-MSK

---

From: Scott Kitterman <[email-masked]>
Date: Sun, 19 Aug 2012 13:27:57 -0400

On Sunday, August 19, 2012 10:07:48 AM Murray S. Kucherawy wrote:
> > http://datatracker.ietf.org/doc/rfc4408/history/ you'll find that the IESG
> > did not just rubber stamp draft-schlitt-spf-classic.  Some significant
> > changes were made between the initial submission and the final approval. 
> > While your statement that "It never underwent official IETF review of any
> > kind" is literally true it was reviewed on the namedroppers, ietf-smtp,
> > and ietf-822 lists in addition to the IESG review.
> 

No.  The point is that it was reviewed.

> acceptable to common document conventions?

I think it's very odd for the text to say "For example, the example given in 
[RFC1034], Section 4.3.3, could be extended with the following:" and then not 
extend it, but completely rewrite it.  It totally loses the point of taking a 
standard example and showing how it changes.  So it's potentially confusing.

I'm against making things more confusing for the sake of ...  Well I'm not 
sure what it would be for the sake of.  There is no way this creates any harm, 
which is what the IESG note was about.  I guess it must be for the sake of 
applying a tangentially related rule to the text in advance of IESG review to 
avoid potential negative feedback.  That's what I've come up with as a reason 
and I don't think it makes sense.

I'm reasonably certain I'm in the rough part of the consensus, so if someone 
sends me a patch I'll apply it (I uploaded the XML with -05) because I'm about 
98% sure that whatever text I wrote would need revision due to excessive 
snarkyness anyway.

Scott K

---

From: Dave Crocker <[email-masked]>
Date: Sun, 19 Aug 2012 13:55:45 -0400

> acceptable to common document conventions?

I, too, am curious about this.

Certainly there are times when it's important to preserve the details of
an example, even when those details are problematic.  The linkage to its
original occurrence is required, then.

I don't see how this is one of those time.

d/

---

From: Barry Leiba <[email-masked]>
Date: Sun, 19 Aug 2012 20:01:30 -0400

>
> II think it's very odd for the text to say "For example, the example given
> in

You could certainly say , "Consider the example in RFC 1034, Section 4.3.3.
 Based on that, we can do the following:"

Trust me: it will be lots easier to use currently acceptable examples, and
this is not an issue that's worth making a big stink and falling on your
sword about.  Really, it's not.

Barry

---

From: Scott Kitterman <[email-masked]>
Date: Sun, 19 Aug 2012 20:07:26 -0400

> sword about.  Really, it's not.

I get that that's true as a practical matter, which is why I said I'd change 
it.  That doesn't change my opinion about it's ridiculousness.

Thanks for the suggestion.

Scott K

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sun, 19 Aug 2012 21:32:56 -0700

> and adapt it to where we are now since I don't understand your objections to

+1.  There's a complaint that we haven't defined "marking", but we
also don't use that term anywhere in the document, only on this
mailing list.  Adding a definition for it doesn't make any sense.

The current 2.5.4 text lays out the two most obvious actions an
operator can take on SPF "fail" evaluations.  I can't imagine what's
missing.

We are all aware that some people believe that those who don't treat
"-all" as a mandatory rejection are nuts.  That opinion is not
unanimous, nor is it even a consensus opinion, and hasn't been since
RFC4408 and probably before.  It's not likely to get added now.
Therefore, I don't believe this line of debate is a wise use of our
remaining energy.

-MSK

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 20 Aug 2012 14:01:39 +0200

> I uploaded the XML with -04 so if someone wants to send me a patch, that'd be 

I did that for -05.  I've left many "domain owner" as that is shorter
than "ADMD that owns/controls the domain".

I put a couple of very some minor changes, but I resisted relevant
changes.  In particular, in:

I would put this:

  ADMDs MAY publish SPF records that explicitly authorize no hosts for
  domain names that are neither used in the domain part of email
  addresses nor expected to originate mail.

That "MAY" is currently "can".  I'd also move that paragraph from the
third to the second place, so as to have normative stuff at the
beginning of the section.

Please note that the "MUST" in the first paragraph bears a slightly
different meaning after the change.  An ADMD could get away with some
compliant domains only, according to the previous wording.  Now, it
has to define a record for each relevant host.  Should we relax that
"MUST" to a "SHOULD"?  Or maybe keep it at "MUST" for mfrom and relax
to "SHOULD" for helo identities only?
--- draft-ietf-spfbis-4408bis-05.xml	2012-08-20 12:25:11.000000000 +0200
+++ draft-ietf-spfbis-4408bis-05-ale.xml	2012-08-20 13:24:38.000000000 +0200
@@ -68,9 +68,9 @@
         particular, existing protocols place no restriction on what a sending
         host can use as the "MAIL FROM" of a message or the domain given on
         the SMTP HELO/EHLO commands.  This document describes version 1 of
-        the Sender Policy Framework (SPF) protocol, whereby a domain can
+        the Sender Policy Framework (SPF) protocol, whereby an ADMD can
         explicitly authorize the hosts that are allowed to use its domain
-        name, and a receiving host can check such authorization.
+        names, and a receiving host can check such authorization.
       </t>
       <t>This document obsoletes RFC4408.
       </t>
@@ -85,7 +85,7 @@
         each of the various identifiers specified by <xref target="RFC5321"/>
         and <xref target="RFC5322"/>.  Although this feature is desirable in
         some circumstances, it is a major obstacle to reducing Unsolicited Bulk
-        email (UBE, aka spam).  Furthermore, many domain owning ADMDs
+        Email (UBE, aka spam).  Furthermore, many domain owning ADMDs
         (ADministrative Management Domains, see <xref target="RFC5598"/>) are
         understandably concerned about the ease with which other entities can
         make use of their domain names, often with malicious intent.
@@ -94,7 +94,7 @@
         This document defines a protocol by which ADMDs can authorize hosts to
         use their domain names in the "MAIL FROM" or "HELO"
         identities.  Compliant ADMDs publish Sender Policy Framework (SPF)
-        records in DNS specifying which hosts are permitted to use their names,
+        records in the DNS specifying which hosts are permitted to use their names,
         and compliant mail receivers use the published SPF records to test the
         authorization of sending Mail Transfer Agents (MTAs) using a given
         "HELO" or "MAIL FROM" identity during a mail
@@ -227,8 +227,8 @@
           Note that requirements for the domain presented in the EHLO or HELO
           command are not always clear to the sending party, and SPF verifiers
           MUST be prepared for the "HELO" identity to be malformed
-          or an IP address literal.  SPF checks can only be performed when the
-          "HELO" string is a valid fully qualified domain.
+          or an IP address literal.  This SPF check can only be performed when
+          the "HELO" string is a valid fully qualified domain.
         </t>
       </section>
       <section title="The &quot;MAIL FROM&quot; Identity" anchor="mfrom-ident">
@@ -251,10 +251,10 @@
       </section>
       <section title="Publishing Authorization">
         <t>
-          An SPF-compliant domain MUST publish a valid SPF record as
-          described in <xref target="records"/>.  This record authorizes the
-          use of the domain name in the "HELO" and "MAIL FROM"
-          identities by the MTAs it specifies.
+          An SPF-compliant ADMD MUST publish valid SPF records as
+          described in <xref target="records"/>.  These records authorize the
+          use of the relevant domain names in the "HELO" and "MAIL FROM"
+          identities by the MTAs specified therein.
         </t>
         <t>
           SPF results can be used to make both positive (source is authorized)
@@ -267,8 +267,9 @@
           determinations are discussed in <xref target="implications"/>.
         </t>
         <t>
-          Domain holders can publish SPF records that explicitly authorize no
-          hosts if mail is not expected to originate using that domain.
+          ADMDs can publish SPF records that explicitly authorize no
+          hosts for domain names that are neither used in the domain part of
+          email addresses nor expected to originate mail.
         </t>
         <t>
           When changing SPF records, care must be taken to ensure that there
@@ -449,7 +450,7 @@
         <section title="Softfail" anchor="op-result-softfail">
           <t>
             A "softfail" result ought to be treated as somewhere between "fail"
-            and "neutral"/"none".  The domain believes the host is not
+            and "neutral"/"none".  The domain owner believes the host is not
             authorized but is not willing to make a strong policy statement.
             Receiving software SHOULD NOT reject the message based solely on
             this result, but MAY subject the message to closer scrutiny than
@@ -512,7 +513,7 @@
         "a:colo.example.com/28" (the + is implied), and "-all".
       </t>
         <t>
-          The SPF records are placed in the DNS tree at the host name it
+          Each SPF record is placed in the DNS tree at the host name it
           pertains to, not a subdomain under it, such as is done with SRV
           records <xref target="RFC2782"/>.
         </t>
@@ -533,9 +534,9 @@
           ensure only SPF records are used for SPF processing.
         </t>
         <t>
-          Domains publishing records SHOULD try to keep the number of
+          ADMDs publishing SPF records SHOULD try to keep the number of
           "include" mechanisms and chained "redirect" modifiers to a minimum.
-          Domains SHOULD also try to minimize the amount of other DNS
+          ADMDs SHOULD also try to minimize the amount of other DNS
           information needed to evaluate a record.  <xref target="eval-limits"/>
           and <xref target="sending-resources"/> provide some suggestions on how
           to achieve this.
@@ -603,7 +604,7 @@
         <section title="Wildcard Records">
           <t>
             Use of wildcard records for publishing is NOT RECOMMENDED. Care
-            must be taken if wildcard records are used. If a domain publishes
+            must be taken if wildcard records are used. If a zone publishes
             wildcard MX records, it might want to publish wildcard
             declarations, subject to the same requirements and problems. In
             particular, the declaration MUST be repeated for any host that
@@ -628,9 +629,9 @@
             </artwork>
           </figure>
           <t>
-            SPF records MUST be listed twice for every name within the domain:
+            SPF records MUST be listed twice for every name within the zone:
             once for the name, and once with a wildcard to cover the tree under
-            the name in order to cover all domains in use in outgoing mail.
+            the name, in order to cover all domains in use in outgoing mail.
           </t>
         </section>
     </section>
@@ -2072,7 +2073,7 @@
             normal to have the HELO identity set to hostname instead of domain.
             Zone file generation for significant numbers of hosts can be
             consolidated using the redirect modifier and scripted for initial
-            deployment.  Specific deployment advice is given above in Section
+            deployment.  Specific deployment advice is given above in
             <xref target="sending-admin"/>.
           </t>
         </section>

---

From: Hector Santos <[email-masked]>
Date: Mon, 20 Aug 2012 08:22:38 -0400

> mailing list.  Adding a definition for it doesn't make any sense.

Not so.  It was defined in 2.5.4 in RFC4408. You moved it to 9.3.2 in 
a vague way. I can suggeste you strengthen RFC2119 reading with SMTP 
level rejection, so maybe I should no longer be "complaining" - no, 
both are needed.

I hope to provide example text later today.

-- 
HLS

---

From: Hector Santos <[email-masked]>
Date: Mon, 20 Aug 2012 08:24:41 -0400

Hector Santos wrote:

>> Murray stated:
> Not so.  It was defined in 2.5.4 in RFC4408. You moved it to 9.3.2 in a 
> vague way.

Sorry, it was mentioned, not defined.

> I can suggest you strengthen RFC2119 reading with SMTP level 
> rejection, so maybe I should no longer be "complaining" - no, both are 
> needed.
> 

-- 
HLS

---

From: S Moonesamy <[email-masked]>
Date: Mon, 20 Aug 2012 08:41:19 -0700

Hello,

[Bcc to [email-masked]]

This is the text from RFC 4408:

   'A "Fail" result is an explicit statement that the client is not
    authorized to use the domain in the given identity.  The checking
    software can choose to mark the mail based on this or to reject the
    mail outright.

    If the checking software chooses to reject the mail during the SMTP
    transaction, then it SHOULD use an SMTP reply code of 550 (see
    [RFC2821]) and, if supported, the 5.7.1 Delivery Status Notification
    (DSN) code (see [RFC3464]), in addition to an appropriate reply text.'

My reading of the above is that there isn't any RFC 2119 [1] 
requirement or RFC 2119 recommendation to do SMTP level 
rejection.  Am I reading the text incorrect?

Regards,
S. Moonesamy
SPFBIS WG co-chair

1. http://www.rfc-editor.org/rfc/rfc2119.txt

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 20 Aug 2012 18:45:19 +0200

> If there's no SPF result there can be no SPF related policy, so I don't see a 

Well, "none" /is/ an SPF result, isn't it?  The "SHOULD NOT" I quoted
is from Section 2.5.5 (softfail).  The question is why is the latter
normative while the advice for "fail" is not.

> change during a session, so it's not axiomatic that one should.

Thus, a receiver may reject whichever command it likes.  That's fine
for me.  However, from the second paragraph onward, the text of
Section 2.5.4 is written assuming that the failed identity is MAIL
FROM.  (I partially retract the "+1" I gave on
http://www.ietf.org/mail-archive/web/spfbis/current/msg01844.html )

Technically, "during the SMTP transaction" means that the transaction
has to be started, which is after the last HELO/EHLO command.  Can we
say "before accepting the SMTP transaction" instead?

> providing additional specificity is useful.

Yes, it is useful, but putting the codes in the example is clear
enough.  That "SHOULD" is spurious.  Besides, the description in the
page you cite, although not SPF-specific, is quite clear:

   X.7.1   Delivery not authorized, message refused
      451, 454, 502, 503, 533, 550, 551
         The sender is not authorized to send to the destination.

Thus, we could say:

   The receiving server MAY reject messages using codes as specified
   by SMTP ([RFC5321]) and, if supported, Enhanced Mail System Status
   Codes ([RFC3463]).

That way, we'd use normative language for the point pertaining to the
SPF protocol, while sparing it for what pertains to other standards.

---

From: Hector Santos <[email-masked]>
Date: Mon, 20 Aug 2012 13:06:41 -0400

S Moonesamy wrote:
> My reading of the above is that there isn't any RFC 2119 [1] requirement 
> or RFC 2119 recommendation to do SMTP level rejection.  Am I reading the 
> text incorrect?

We can really lost with rhetorical judo RFC2119 semantics games.  I 
prefer to design protocol software from an practical engineering 
standpoint, but lets go ahead and play it out.

What it the difference in the following three sentences for the 
SOFTWARE ENGINEER or his programmer writing the "checking software?"

   The checking software can choose to mark the mail based on this or
   to reject the mail outright.

   The checking software CAN choose to mark the mail based on this or
   to reject the mail outright.

   The checking software MAY choose to mark the mail based on this or
   to reject the mail outright.

In my view, the first two are the same but a RFC2119 purist may argue 
the first one is not standard compliance material "throw book in your 
face" enforcement stuff.  We can get into the CAN vs MAY semantics 
game and suggest the CAN is the mechanical form of what is possible in 
the protocol, where as MAY may imply there is another lay of human 
level considerations and options, authorization permission based 
concept such as ON/OFF.

So the CAN or can suggest the software SHOULD have in place the 
software capability to reject or accept+mark the SMTP transaction, and 
the MAY or may would suggest that none of it is enforceable per 
RFC2119 "Book in face" compliance.

It could be presented as follows by the software engineer, programmer, 
GUI designer implementing this stuff:

    [X] Enable honoring HARD FAIL SPF results
       (o) Reject HARD FAIL results
       (_) Mark HARD FAIL results

Note: [_] is a checkbox (on/off switch), (_) is a radio checkbox 
offering mutual exclusive optioms.

So again, if we really want to go this deep into what it all means, 
then maybe we should:

If we have a group that believe SPF receivers DO NOT REQUIRED to honor 
HARD FAIL publications, then I believe the sentence needs to be:

   The checking software MAY choose to mark the mail based on this or
   to reject the mail outright.

I don't agree with it personally, but I can't argue one's 
justification for it. IMV, one should not call itself supportive of 
SPF if it doesn't want to honor the HARD FAIL, but again, if we want 
to get picky, how it was written in 4408 with lower case can, to me, 
as an software engineer suggest the compliant protocol software MUST 
have the option to do both.

At the end of the day, what are we offering the protocol 
implementation, is one, two or three options?  What additional doubt 
are we adding here?

   The checking software MAY choose to mark the mail based on this or
   to reject the mail outright, but it NOT REQUIRED to honor the hard fail
   and therefore the DOMAINS should not expect it to be either rejected or
   marked.

Anyway, I plan later to provide my text version that basically 
clarifies what the receiver responsibility here either with REJECT or 
MARK. Despite if the receiver does either, SPFBIS should be very clear 
what is the expected protocol for each.  My only real original point 
is that a reject is a very powerful actiom and of acception + mark is 
done, stream separation SHOULD be in place.

With the 04/05 change done, I am suggestion just make it clear in 
2.5.4 where the options are mentioned, RFC2119 defined for one 
(reject), left open-ended for the marking option.

-- 
HLS

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 20 Aug 2012 10:29:51 -0700

On Mon, Aug 20, 2012 at 8:41 AM, S Moonesamy <[email-masked]> wrote:
> My reading of the above is that there isn't any RFC 2119 [1] requirement or
> RFC 2119 recommendation to do SMTP level rejection.  Am I reading the text
> incorrect?

That's also how I read it.

See also http://www.ietf.org/mail-archive/web/spfbis/current/msg01340.html.

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 20 Aug 2012 10:44:17 -0700

> normative while the advice for "fail" is not.

Interesting point.

I think what's there is fine, actually.  That's based on the fact that
DKIM (Section 6.1 of RFC6376) does what we're doing here; things that
fail SHOULD NOT get negative treatment to a message because of the
known failure modes.  In effect, this applies not only to "softfail"
but also to what we're calling the "mark" option of "fail".

> http://www.ietf.org/mail-archive/web/spfbis/current/msg01844.html )

So change "during the SMTP transaction" to "via the SMTP session"?  Or
"dialog" instead of "session"?  Then it doesn't matter whether you're
inside a transaction or not (meaning it applies to HELO/EHLO or MAIL
or RCPT equally).

> say "before accepting the SMTP transaction" instead?

Not quite.  Sessions begin with HELO/EHLO; transactions begin with MAIL.

-MSK

---

From: Dave Crocker <[email-masked]>
Date: Mon, 20 Aug 2012 10:51:09 -0700

> but also to what we're calling the "mark" option of "fail".

The asymmetry in handling failure vs. success, for these types of
mechanisms, should come naturally from the questions:

     What do we know when there is success?

     What do we know when there is failure.

Because failure can be due to many factors -- both malign and benign --
we know rather little when it happens.

In contrast, we know quite a lot, when there is success.

d/

---

From: S Moonesamy <[email-masked]>
Date: Mon, 20 Aug 2012 11:12:06 -0700

>standpoint, but lets go ahead and play it out.

There is nothing in a RFC which forces anyone to follow it.  If 
anyone decides to follow a RFC it would help me if the person says 
so.  If the person wants to cherry-pick what to follow, it is not for 
me to tell him or her to what to follow.

This mailing list is used by the IETF SPFBIS working group to discuss 
about the IETF work on RFC 4408.  I am ok if anyone does not like the 
IETF, the way it works or the way this working group works.  I follow 
IETF process and practices as that is what I am tasked to do.

>SOFTWARE ENGINEER or his programmer writing the "checking software?"

That is a question for the above-mentioned persons to determine.

>In my view, the first two are the same but a RFC2119 purist may 
>argue the first one is not standard compliance material "throw book 
>in your face" enforcement stuff.  We can get into the CAN vs MAY 
>semantics game and suggest the CAN is the mechanical form of what is 
>possible in the protocol, where as MAY may imply there is another 
>lay of human level considerations and options, authorization 
>permission based concept such as ON/OFF.

I do not do standard compliance on this mailing list.  I am ok with 
RFC 2119 purist labels or any other label which people want to use as 
long as it does not give me cause to have to read a process BCP to 
the working group.  I look at RFC 2119 key words in terms of 
interoperability.  I do not see any interoperability problem here.

>software capability to reject or accept+mark the SMTP transaction, 
>and the MAY or may would suggest that none of it is enforceable per 

The above mentions compliance.  I cannot comment about compliance.

>If we have a group that believe SPF receivers DO NOT REQUIRED to 
>honor HARD FAIL publications, then I believe the sentence needs to be:
>

I do not see any explanation in terms of interoperability for the 
suggested text.  I see a proposed change to RFC 4408.  I will 
interpret it in IETF terms.

Regards,
S. Moonesamy
SPFBIS WG co-chair

---

From: Hector Santos <[email-masked]>
Date: Mon, 20 Aug 2012 14:26:15 -0400

> normative while the advice for "fail" is not.

That depends on your view on lower case can vs upper case CAN in the 
sentence:

Never mind this services no justice to the visually impaired or audio 
world without one reasons one's pitch, but semantically, there is only 
one practical meaning.

Second, in rfc 4408, section 2.3, 2nd paragraph:

    If domain owners choose to publish SPF records, it is RECOMMENDED
    that they end in "-all", or redirect to other records that do, so
    that a definitive determination of authorization can be made.

This has a direct implication of what a domain expects a receiver to 
be doing with a -ALL, and for the protocol to work.

Now I see rev 05, has:

To me, this reinforces properly protocol client/server design to make 
sure 2.5.4 is very clear about domains MUST rfc2119-level requirement 
to get a definitive negative result.

That means REJECT must be clear (which is already is) and the 
alternative "mark the mail" MUST also be very clear and to result in a 
negative classification.  That means "Spam box" and/or stream 
separation for the user level filter.

I really do not like how SPFBIS is shaping into a deja-vu "DKIM" spec 
full of engineering doubt and too many cross section semantics 
confusing the reader, leaving items falling the  RFC2119 and protocol 
cracks.

Folks, SPF is not NEW! its 9 years OLD!  Its well established and now 
is not the time to question's domains logic and reasoning for using 
-ALL.   If the domain wants DOUBT, then it easily use ~ALL or ?ALL and 
it will get it 100%.  But -ALL has a special meaning and also has been 
that way.

What is surpising to me, and I'm not convince how much truth is there 
is that many implementations actually just do accept/mark rather than 
the reject for a -ALL result. We don't even know if that is mostly a 
result of SPF2.0 (SENDERID) support where the payloads is already 
accepted.

As far as I am concern, a reject is the only RFC2119 item, while a 
accept/mark is open-ended, IMV it needs to be corrected to say the 
stream has to be separated or negatively classified from the POV of 
the end-user.

-- 
HLS

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 20 Aug 2012 11:37:03 -0700

On Mon, Aug 20, 2012 at 11:12 AM, S Moonesamy <[email-masked]> wrote:
>> What it the difference in the following three sentences for the SOFTWARE
>> ENGINEER or his programmer writing the "checking software?"
>

>>   to reject the mail outright.

Those three things are synonymous to me.

Furthermore, "CAN" isn't an RFC2119 word, so capitalizing it just
looks strange to me.

> I do not do standard compliance on this mailing list.  I am ok with RFC 2119
> purist labels or any other label which people want to use as long as it does
> not give me cause to have to read a process BCP to the working group.  I
> look at RFC 2119 key words in terms of interoperability.  I do not see any
> interoperability problem here.

+1.

> I do not see any explanation in terms of interoperability for the suggested
> text.  I see a proposed change to RFC 4408.  I will interpret it in IETF
> terms.

Moreover, the proposed change introduces no clarification or new
ideas, so I don't see the point.  It already says that.

-MSK

---

From: Hector Santos <[email-masked]>
Date: Mon, 20 Aug 2012 14:57:25 -0400

> Furthermore, "CAN" isn't an RFC2119 word, so capitalizing it just

One may argue or suggest, and have, that "CAN" and "can" are in the 
same boat "may" vs "MAY" have also been argued, as well as SHOULD vs 
should.  So want it implied, others like to see specifics.  To me? The 
protocol engineering speaks for itself.

>> interoperability problem here.

Then we have a BUG conflict between section 2.3 with a very strong 
RFC2119 language regarding DOMAIN expectation for a -ALL to translate 
to a negative classification where in 2.5.4, based on what is begin 
stated we have three options:

(o) DO NOT SUPPORT HARD FAIL RESULTS
(_) REJECT HARD FAIL
(_) ACCEPT and MARK HARD FAIL

(By the way, what is the default?)

The point is, depending on how one reads the new 2.3, 2.5.4 with 
9.3.2, we raised doubt, and lowered the effectiveness of the SPF -ALL 
policy.

> ideas, so I don't see the point.  It already says that.

See above.

-- 
HLS

---

From: Hector Santos <[email-masked]>
Date: Mon, 20 Aug 2012 15:12:24 -0400

> In contrast, we know quite a lot, when there is success.

Dave, you should take into account many system engineers, especially 
those with experience in the unit ops world (like chemical, process, 
industrial engineers), have deep training in optimization, fault 
detection, dissemination of detectable errors, faults and spoofs as 
can be provided with strong domain policies like SPF before you can 
get a steady state stream of "good" material.

Section 2.3 in RFC4408 provide well understood strong RFC2119 language 
regarding the strong exclusive -ALL policy and the high payoff 
expectation it provides for query dissemination optimization concepts.

It works so well, its scary to many.

Take into account that receivers do see many spoofs attempted on their 
own local mail on their own system. I suspect as a well to make it 
look like its a local client.  SPF provide the highest benefit for 
local domain or locally hosted domain protection at the receiver.  The 
overhead begins when SPF receivers begins to attempt to provide this 
protection to remote domains.

--
HLS

---

From: Andrew Sullivan <[email-masked]>
Date: Mon, 20 Aug 2012 16:05:46 -0400

Dear colleagues

Procedural hat.

On Mon, Aug 20, 2012 at 02:57:25PM -0400, Hector Santos wrote:

> same boat "may" vs "MAY" have also been argued

No, they're not. 

MAY and other 2119 friends are reasonably well-defined and widely
accepted within the community.  CAN is a novel use.  Given our
charter, we are certainly not going to be innovators in the area of
standards level words.

We shall stick to 2119, please.

Thanks,

A

-- 
Andrew Sullivan
[email-masked]

---

From: Andrew Sullivan <[email-masked]>
Date: Mon, 20 Aug 2012 16:13:28 -0400

By the way, "STTN" expands to "Stop This Thread Now."  Sorry for
failing to expand on first use.

-- 
Andrew Sullivan
[email-masked]

---

From: Hector Santos <[email-masked]>
Date: Mon, 20 Aug 2012 16:29:59 -0400

> We shall stick to 2119, please.

So not sticking to 2119 is just the same as advocating a protocol 
design?   RFC 4408 has:

Which by your definition, the PROTOCOL has the compliance level or 
lack there of option to NOT mark NOR reject. We can argue this many 
ways and thats by sticking with 2119 sir.  It means the technical 
specification is to design a protocol with a mutual exclusive option:

   (o) do not honor hard fail
   (_) honor using SMTP REJECT (RFC 2119 compliance)
   (_) honor using SMTP ACCEPT+RECEIVE-SPF

We are presuming a "Mark" means adding RECEIVER-SPF header.

If this is the case, then we have a RFC2119 conflict in 2.3 made even 
more pronounced in the rev 04/05 version which does have a 2119 MUST 
expectation for -ALL translating to a negative dissemination of the 
transaction.

At some point, we need to end the rhetoric judo and just work with 
common since. Relying squaring on RFC2119 is fine, if you are 
consistent across the board and everyone is one board on what means what.

-- 
HLS

---

From: SM <[email-masked]>
Date: Mon, 20 Aug 2012 13:16:20 -0700

Hi Hector,
At 11:57 20-08-2012, Hector Santos wrote:
>should.  So want it implied, others like to see specifics.  To me? 
>The protocol engineering speaks for itself.

There is a thread about the boat at 
http://www.ietf.org/mail-archive/web/ietf/current/msg73338.html  This 
topic has been a recurring in the IETF since a long time.

>RFC2119 language regarding DOMAIN expectation for a -ALL to 
>translate to a negative classification where in 2.5.4, based on what 
>is begin stated we have three options:

The problem with an error in a specification is that it generally 
translates into a software bug.

Regards,
-sm

---

From: Scott Kitterman <[email-masked]>
Date: Mon, 20 Aug 2012 16:46:25 -0400

On Monday, August 20, 2012 06:45:19 PM Alessandro Vesely wrote:
> > It does and it's correct.  Receivers can do as they like.  They don't get
> > to decide how to determine if an SPF result is "fail", that is
> > standardized, but once they have that result they get to decide what to
> > do with it.
> > 
> > If there's no SPF result there can be no SPF related policy, so I don't
> > see a need for a change.
> 
> > status-codes.xml does not lead me to one obviously correct solution.  I
> > think providing additional specificity is useful.
> 

The problem with this is that the rationale is about the recipient and not the 
sender.  Why would it be appropriate to pick x.7.1 over x.7.0:

X.7.0   Other or undefined security status

While the overall designation includes the word policy:

X.7.XXX Security or Policy Status

None of the X.7.? codes mention it at all (unlike 550).  I don't think there 
is an obvious best choice and so I think that for consistency we recommend one 
that should be used.

Scott K

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 20 Aug 2012 14:15:20 -0700

On Mon, Aug 20, 2012 at 1:16 PM, SM <[email-masked]> wrote:
>> Then we have a BUG conflict between section 2.3 with a very strong RFC2119
>> language regarding DOMAIN expectation for a -ALL to translate to a negative
>> classification where in 2.5.4, based on what is begin stated we have three
>> options:
>
> The problem with an error in a specification is that it generally translates
> into a software bug.

The current Section 2.3 text says, in essence: If you want to give
receivers the ability to make a definite negative conclusion ("fail")
about an incoming message, you have to use "-all".  It says nothing
whatsoever about what a receiver does upon reaching that conclusion.

Section 2.5.4 then goes into detail about what choices a receiver has
when a "fail" conclusion is reached.

I don't see a conflict or "bug" here.  They're complementary.

-MSK

---

From: Hector Santos <[email-masked]>
Date: Mon, 20 Aug 2012 17:21:04 -0400

> topic has been a recurring in the IETF since a long time.

Well, I tend to come from the school of thought gracefully cited here:

    http://www.ietf.org/mail-archive/web/ietf/current/msg73338.html

> translates into a software bug.

And in my engineering opinion, the new revs will greatly increase

   - increase the complexity of designs,
   - interoperability issues,
   - lower the SPF -ALL payoff
   - increase doubt in the minds of SPF domains

We need to separate the documentation people from the software 
engineers and technical programmers! That use to be separate 
disciplines in the SE process, you know? <grin> Today, everyone wants 
to be everything - and that causes issues, well, management problems 
and a status quo for the IETF process.

-- 
HLS

---

From: Scott Kitterman <[email-masked]>
Date: Mon, 20 Aug 2012 17:22:55 -0400

On Monday, August 20, 2012 02:15:20 PM Murray S. Kucherawy wrote:
> >> RFC2119

Agreed.  In any case, 4408 was very careful not to prescribe what receivers 
should do with "fail" mail.  Changing that in 4408bis would be a 
substantiative change that's out of scope of the charter even if there was 
consensus for it.

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Mon, 20 Aug 2012 17:29:03 -0400

> and a status quo for the IETF process.

The new revisions present exactly the same technical requirements that RFC 
4408 presented.  There is no change.  There is some change in how the words 
are arranged and some added background about why certain choices might be 
made, but the design choices that are available to the implementer are exactly 
the same.  Nothing has changed.

Also, while you're throwing your accusations about documentation people versus 
real software engineers and technical programmers around, please don't forget 
that the guy that's editing the document is also an implementer.  So far the 
only change I'm going to make in the current implementations I support is to 
format the Authentication-Results header fields the way that is suggested in 
-05.  None of the changes you are complaining about affect implementation.

Scott K

---

From: Hector Santos <[email-masked]>
Date: Mon, 20 Aug 2012 18:02:16 -0400

> the same.  Nothing has changed.

The effacy for SPF -ALL publication has been greatly diminished with 
the changed in the revs.  Today, unlike yesterday, we will now be put 
into a change position for our documentation SPF references to say:

     NOTE: Publishing -ALL no longer guarantees SPF receivers will 
honor your
     HARD FAIL -ALL policy.

While between you, I and that post over that knows that was always the 
case - for anything, the revision now certainly raises doubt, and in 
addition unlike over the years where casual discussions or support had 
no dispute on what -ALL meant and does, you and murray have now added 
a new level of debate anytime it now even comes up by throwing the 
RFC4408BIS book at them.

4408 2.5.4 said the checking software can mark or reject. It does not 
say it does not have to do either or it can be silent although there 
was no point in saying that a receiver would do anything at all.  The 
rev have made that clear now, the new 2.5.4 does now ever mention 
marking the mail any more and moved it to a discussion of doubt.

> that the guy that's editing the document is also an implementer.

Exactly, I know that.  But the point still remains, sometimes these 
things need to be separated. Of course, thats the beauty of the RFC 
process - the merging of disciplines, and keeping aware of the mixed 
discipline readers is part of the process.

> None of the changes you are complaining about affect implementation.

I believe it does. At the rudimentary level its harder to read, more 
vague and odds are now higher, it inadvertently lowered the "mark the 
message" aspect and strengthen the RFC2119 reject behavior.

-- 
HLS

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 20 Aug 2012 16:15:29 -0700

> the same.  Nothing has changed.

+1.  Moreover, the chairs declared this a closed topic back in April.
Could we please spend our energy on topics that are open?  I'd like to
get this work finished before my daughter hits high school.

-MSK

---

From: Hector Santos <[email-masked]>
Date: Mon, 20 Aug 2012 21:34:14 -0400

Murray S. Kucherawy wrote:
> Could we please spend our energy on topics that are open?  

What topic is that? Issue 29? Is this ISSUE 29 closed?  Scott 
indicated it was still open.

> I'd like to get this work finished before my daughter hits high school.

You are the one that is creating a mess for SPF. Not me buddy.  The 
controversies have begins with you, saying dumb stuff like above.  I 
am not the one ignoring input from participants and then finding you 
were wrong in the end.

There is a still a problem with the rev text.  The rev changes, which 
I presumed is your work, needs serious work with added doubt added to 
the protocol.  Learn to accept the criticism and concerns and work 
with it rather than ignoring it and brushing it off.

-- 
HLS

---

From: S Moonesamy <[email-masked]>
Date: Mon, 20 Aug 2012 19:07:36 -0700

Hi Hector,
>indicated it was still open.

Issue #29 is at http://trac.tools.ietf.org/wg/spfbis/trac/ticket/29

This is part of the current text in Section 2.5.4 of 
draft-ietf-spfbis-4408bis-05:

   "If the checking software chooses to reject the mail during the SMTP
    transaction, then it SHOULD use an SMTP reply code of 550

If the mail receiver decides to reject the message it should use the 
designated SMTP reply code.

The following text is from RFC 4408:

   "If the checking software chooses to reject the mail during the SMTP
    transaction, then it SHOULD use an SMTP reply code of 550"

There hasn't been any change to the sentence except for an update to 
the references and the change from "Delivery Status Notification" to 
"enhanced status code".

Is the following text an issue:

   "Disposition of SPF fail messages is a matter of local policy.  See
    Section 9.3 for considerations on developing local policy."

Regards,
S. Moonesamy
SPF WG co-chair

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 21 Aug 2012 09:40:43 +0200

>    Section 9.3 for considerations on developing local policy."

IMHO it should be:

    Local policy MAY dispose to reject SPF fail messages.  See [...]

That specifies the interoperability among the receiver, the domain
owner, and any unauthorized senders abusing its domain name.

No change of meaning is implied, as there is no magic in "MAY", except
to have a somewhat uniform method of specifying standards throughout
the whole RFC series.  I suggest we follow that rule by adding such
clarifying language.

There may be good reasons to specify SPF in its own way, using the
text you quoted.  I'd imagine the IETF can accommodate that too.

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 21 Aug 2012 09:40:51 +0200

> X.7.0   Other or undefined security status

In a previous message you said "receivers are reluctant to get into
detail about why the accept or reject messages for fear that senders
will use the information to game the system."
http://www.ietf.org/mail-archive/web/spfbis/current/msg01693.html

Since 5.7.1 is given in the example, the added meaning of a SHOULD is
that there are serious issues that the receiver should be aware of
before using a different code.  Correct?

In that case, would it be overly picky to remove the SHOULD for 550
and just use it for X.7.1?  E.g.

   The receiving server MAY reject messages using codes as specified
   by SMTP ([RFC5321]).  If Enhanced Mail System Status Codes are
   supported, it SHOULD use 5.7.1 ([RFC3463]).

---

From: Andrew Sullivan <[email-masked]>
Date: Tue, 21 Aug 2012 10:29:57 -0400

No hat.

On Mon, Aug 20, 2012 at 04:29:59PM -0400, Hector Santos wrote:
> design?

What I intended to say was simply, "No new all-caps terms."

> lack there of option to NOT mark NOR reject.

No.  There is no "compliance level" to the protocol.  A protocol
specifies the necessary and sufficient conditions of behaviour for
something to be called an implementation of it.  If an implementation
does not follow those conditions, then it is not conforming to the
protocol specification; but the IETF does not do "compliance", and
there are no protocol cops.  

> We are presuming a "Mark" means adding RECEIVER-SPF header.

While these may be mutually exclusive options, it is entirely possible
for the text to leave open to the receiver which of the options to
use.  This would still provide a complete specification, with
behaviour that covers all the possibilities while yet leaving local
policy to the local operator.  As nearly as I can tell from your
message downthread, you actually agree that RFC 4408 leaves the
options open to the receiver.  You appear to be arguing that we should
tighten this portion of the specification.  But given the chairs'
recent ruling on a procedural question, and the results of the appeal
to the responsible AD, I argue that the WG could not tighten this
portion anyway.  Therefore, what is correct for the text to do is to
lay out more clearly perhaps than it was in 4408 what the options
are.  You seem to think it does that.

If I have misunderstood your argument, perhaps you can state it less
colourfully so that I may understand it properly.  If I have correctly
stated your argument, then it is time to let this thread die.

Best regards,

A

-- 
Andrew Sullivan
[email-masked]

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 21 Aug 2012 10:45:18 -0400

>   supported, it SHOULD use 5.7.1 ([RFC3463]).

I think it's quite awkward to specify one and not the other. I think we should keep them both.

Scott K

---

From: S Moonesamy <[email-masked]>
Date: Tue, 21 Aug 2012 10:23:48 -0700

>    supported, it SHOULD use 5.7.1 ([RFC3463]).

The above question is one which I might ask.  One of the problem, 
which is generally referred to as a rathole, is that the working 
group would have to be picky about all the RFC 2119 key words which 
are already in RFC 4408.  That's going to end up as a lot of 
work.  The WG Chairs would have to see whether there is an adequate 
number of WG participants to review the changes.  I am aware that 
some individuals who committed to review the WG work have not done 
that.  My opinion is that all this would end up as a WG Chair 
problem.  It is unlikely that the Responsible AD would try to 
convince me to the contrary as he might have a guess about I would do then. :-)

Regards,
S. Moonesamy
SPFBIS WG co-chair

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 21 Aug 2012 11:08:46 -0700

On Tue, Aug 21, 2012 at 10:23 AM, S Moonesamy <[email-masked]> wrote:
> The above question is one which I might ask.  One of the problem, which is
> generally referred to as a rathole, is that the working group would have to
> be picky about all the RFC 2119 key words which are already in RFC 4408.
> That's going to end up as a lot of work.  The WG Chairs would have to see
> whether there is an adequate number of WG participants to review the
> changes.  I am aware that some individuals who committed to review the WG
> work have not done that.  My opinion is that all this would end up as a WG
> Chair problem.  It is unlikely that the Responsible AD would try to convince
> me to the contrary as he might have a guess about I would do then. :-)

In this particular case, the question is around interoperability
between the SMTP client and server.  The question we need to ask is:
If someone doesn't do this, what's the impact?

I think we can be certain that using 5yz codes is the right thing to
do.  Is there any harm in picking the wrong 5yz code?  What do we need
to say that RFC5321 doesn't, that's specific to SPF?

Same question for enhanced status codes: 5.y.z is obviously correct,
but beyond that, why is there a need to be more precise?

Perhaps more generally: What problem are we trying to solve?  Is it
simply a question of trying to avoid sending implementers off to read
those other documents?

-MSK

---

From: S Moonesamy <[email-masked]>
Date: Tue, 21 Aug 2012 15:16:59 -0700

>If someone doesn't do this, what's the impact?

Yes.  I am in favor of the "think about it".  I might ask which 
implementation encounters an irrecoverable error because of an 
interoperability issue in RFC 4408.

>but beyond that, why is there a need to be more precise?

This is more of a matter of style.  Some specifications defer to the 
ones referenced normatively for that level of detail.  Some 
specifications attempt to be self-contained by providing all the 
details.  Let's look at this another way.  If RFC 5321 changes the 
5yz code to 6yz, what is the effect?

By the way, the answer to the question might end up as a rewrite of 
the specification.

Regards,
S. Moonesamy

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 21 Aug 2012 18:39:42 -0400

On Tuesday, August 21, 2012 11:08:46 AM Murray S. Kucherawy wrote:
> > Chair problem.  It is unlikely that the Responsible AD would try to

Particularly the problem I'm trying to solve is having change for unimportant 
reasons.  While some have argued the codes don't need to be in 4408bis, I 
don't think anyone has suggested it's a problem.  Given that these codes are 
(with one exception that was a mistake) in 4408, I think there needs to some 
positive benefit associated with removing them.

I do think there's value in uniformity on this.  Not so much for hard core 
interoperability between computers but to aid in human understanding of what 
the various systems are doing.  If different implementations use different 
codes, then there is some potential for confusion.

Finally, I think it's better in this case to simply give the preferred answer 
in 4408bis than to send implementers off to read two references, particularly 
since one of which does not give a clear answer about how to proceed.

If we're going to remove things, I think there needs to be more to it than "I 
feel like that's not the best way to have done it."

Scott K

---

From: Hector Santos <[email-masked]>
Date: Wed, 22 Aug 2012 03:23:13 -0400

Andrew,

This has already taken more time that I anticipated, but wasn't 
surprise. I believe BIS has been watered down with this increase 
obvious emphasis on local policies (DUH!).   The goal is to provide 
insight for designers to provide operators flexibility (i.e. local 
policies).

I was simply looking for a definition or clear description of "Mark 
the mail" as this has direct implication on backend storage methods 
based on MUA method of access. Security is provided with a hard SMTP 
reject. Marking "should not" be a way to circumvent this security 
provided and domain intent and expectation for using a hard -ALL 
failure policy.

Anyway, the following is my last input on this and let scott and 
others decide from here.  Yes, in theory we can PULL all the parts 
from the revisions.  I just don't like how its written which in my 
view does change the SPF practice and view of the protocol to new 
level of awareness of more, not less doubt.

My proposal for 2.5.4 which hopefully is on par in codifying 9 years 
worth of experience plus what I have read from others "views" in this 
less in regards to a true or false notion for the prevailing nature of 
"marking" the message in lieu of rejecting it outright.

2.5.4.  Fail

    Per section 2.3, a "fail" result is an explicit domain policy
    violation statement that the client is not authorized to use the
    domain in the given identity.  The -ALL policy is the domains
    intent to obtain a negative classification for the transaction
    with the client not authorized to use the domain in the given
    MAIL FROM or EHLO/HELO identity.

    For a negative classification, the checking software SHOULD choose
    one of two options:

      -  reject the transaction at the SMTP level, or
      -  accept, add a Receiver-SPF header and it SHOULD separate the
         message stream from the user's normal access to mail.

    If the checking software chooses to reject the transaction during the
    SMTP transaction, then it SHOULD use an SMTP reply code of 550 (see
    [RFC5321]) and, if supported, the 5.7.1 enhanced status code (see
    [RFC3463]), in addition to an appropriate reply text.  The
    check_host() function will return either a default explanation string
    or one from the domain that published the SPF records (see Section
    6.2).  If the information does not originate with the checking
    software, it should be made clear that the text is provided by the
    sender's domain.  For example:

        550 SPF MAIL FROM check failed
        550-5.7.1 SPF MAIL FROM check failed:
        550-5.7.1 The domain example.com explains:
        550 5.7.1 Please see http://www.example.com/mailpolicy.html

    If the checking software chooses to accept the message, it MUST
    trace|mark|report this failure by adding a Receiver-SPF header to the
    received message. See Section 7.  In addition, it SHOULD separate the
    mail stream from the user as this can open a security issue for the
    user.

    Despite the out of scope nature of local mail policies, rejecting the
    failed SPF transaction is regarded a System Level local policy for
    global filtering for all users, where accepting and marking the
    message is regarded as a User Level Policy framework where good
    versus bad mail policies are implemented by separating mail on behalf
    of the user before the MUA access the mail.

-- 
HLS

---

From: Andrew Sullivan <[email-masked]>
Date: Wed, 22 Aug 2012 11:36:56 -0400

Dear Hector,

Your message of 20 August, quoted in its entirety below, included _ad
hominem_ attacks against another participant.  _Ad hominem_ is a
fallacious form of argument wherein the person arguing attacks the
person holding an opposing position, rather than attacking the
position itself.  This is not acceptable, and we believe you have been
warned before.  My co-chair SM has contacted you off-list, under the
procedures in BCP 94, but you have refused to take what we regard as
rectifying action.

Accordingly, please understand this message as a formal warning under
the procedures of BCP 94 that, if we observe continued behaviour of
the sort you have exhibited, we shall suspend your posting privileges
to the spfbis mailing list for 30 days.  Behaviour of the sort
includes anything that we believe to be needlessly personalized, and
especially includes _ad hominem_ forms of argument.  We will also
treat returning to points that have been closed without raising new
arguments as attempts to disrput the functioning of the Working Group.
We will act without further warning in such an event.

If we take that action, and, after restoration of your privileges, we
observe you to return to disrupting the work of the group, we shall
undertake action under BCP 83.

Sincerely,

Andrew (for the chairs)

On Mon, Aug 20, 2012 at 09:34:14PM -0400, Hector Santos wrote:
> There is a still a problem with the rev text.  The rev changes,
> which I presumed is your work, needs serious work with added doubt
> added to the protocol.  Learn to accept the criticism and concerns
> and work with it rather than ignoring it and brushing it off.
> 

-- 
Andrew Sullivan
[email-masked]

---

From: Hector Santos <[email-masked]>
Date: Wed, 22 Aug 2012 12:13:01 -0400

I will not apologize for this incident as the person in question has 
continued to offend me with indirect ad hominem attacks with attempts 
to shut off my input, despite making blatant statement of facts that 
are questionable and in many cases incorrect.

I will minimize my input to serve my SPF interest for our product line 
and will avoid Murray to the fullest extent but I will issue a formal 
complaint on my own if his indirect unprofessional offenses continue 
towards me to limit and shut down my input does not stop.

Do as you wish.

Sincerely,

Hector Santos, CEO/CTO
Santronics Software, Inc.
http://www.santronics.com

-- 
HLS

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 22 Aug 2012 20:01:38 +0200

> inside a transaction or not (meaning it applies to HELO/EHLO or MAIL

Something like that...  I guess "on the fly" would require a detailed
definition.  There are some 8 occurrences of that phrase.

> Not quite.  Sessions begin with HELO/EHLO; transactions begin with MAIL.

So what?  If rejection comes before a transaction starts, then there
is no transaction.  Rejection was still before the transaction end.
Like turning off the alarm before it rings...  no?

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 22 Aug 2012 11:10:42 -0700

> Like turning off the alarm before it rings...  no?

I'm challenging your definition of "transaction" in the context of
SMTP, not with respect to SPF.  In both cases there's no completed
transaction (which is all SPF cares about), but you can't say "during
a transaction" if the only thing that's happened so far is HELO.

-MSK

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 22 Aug 2012 20:29:11 +0200

On Tue 21/Aug/2012 19:23:48 +0200 S Moonesamy wrote:
> which is generally referred to as a rathole, is that the working group
> would have to be picky about all the RFC 2119 key words which are
> already in RFC 4408.  That's going to end up as a lot of work.

In general, I prefer clarity over formal purity.  But this particular
knot isn't overly clear either.  Discussing the semantics seems to be
one of those thing, like defining spam, that can go on for ages
without reaching any useful conclusion.  Formalism, in this case,
seems to offer a neat hold, allowing us to handle the question of how
to gear policies into protocols without generating too much heat.  As
we have a restricted charter and the protocol is well established
already, it should be easy...

> The WG Chairs would have to see whether there is an adequate number
> of WG participants to review the changes.  I am aware that some
> individuals who committed to review the WG work have not done that.
> My opinion is that all this would end up as a WG Chair problem.  It
> is unlikely that the Responsible AD would try to convince me to the
> contrary as he might have a guess about I would do then. :-)

I hope the WG can stay away from drastic measures.

---

From: Barry Leiba <[email-masked]>
Date: Wed, 22 Aug 2012 19:53:49 -0400

Anyone who's been following IESG reviews knows that Pete has been very
picky about "bad" 2119 language.  I have been less picky than he, but I've
also been pushing back on it when I've seen it as detrimental to proper
protocol documentation.

I can't speak for Pete, but I will say for myself that I expect 4408bis to
be an exception, in that I see the goal as minimizing changes in the
process of moving the spec to Standards Track.  I therefore expect that my
review will be especially lenient about existing 2119 language, preferring
to leave it unless it looks not just wrong, but specifically harmful.

On the other hand, I expect to treat changes with due scepticism, with the
idea that the change was made to fix a real problem, and it had better be
right.

Barry, AD

On Wednesday, August 22, 2012, Alessandro Vesely wrote:

> [email-masked] <javascript:;>

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 23 Aug 2012 00:17:13 -0400

> sword about.  Really, it's not.

Done.  For added political correctness points I also changed the IP address to 
one from the designated block for documentation while I was at it.

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 23 Aug 2012 00:23:27 -0400

On Monday, August 20, 2012 02:01:39 PM Alessandro Vesely wrote:
> > I uploaded the XML with -04 so if someone wants to send me a patch, that'd
> > be lovely.
> 

Patch applied.  For 2.3, I changed "domain MUST publish" to "domain MUST have" 
so that it makes it about the DNS domain and not an ADMD, which is consisten 
with what we have now.

Thanks,

Scott K

---

From: Alessandro Vesely <[email-masked]>
Date: Thu, 23 Aug 2012 09:50:59 +0200

On Wed 22/Aug/2012 20:10:42 +0200 Murray S. Kucherawy wrote:
> SMTP, not with respect to SPF.

We are both following the same definition, the one given by RFC 5321.
 I'm not sure I'm following you...

> In both cases there's no completed transaction (which is all SPF
> cares about), but you can't say "during a transaction" if the only
> thing that's happened so far is HELO.

That's why I proposed to s/during/before/

---

From: Alessandro Vesely <[email-masked]>
Date: Thu, 23 Aug 2012 10:51:43 +0200

>> but beyond that, why is there a need to be more precise?

The difference between 5yz an 5.y.z is that while SMTP mandates a
single specific code, RFC 3463 allows a choice.  No existing codes
unambiguously identify SPF.  We seem to agree on that being a feature,
not a defect.  Hence, the above question makes more sense in this
case:  What sort of confusion can be generated if one chooses a
different enhanced status code?

> positive benefit associated with removing them.

Being error-prone is a problem by itself.

In addition, making the wording more clear-cut can provoke revision of
existing policies and configuration of existing software.

Finally, it may ease future development (see below.)

> codes, then there is some potential for confusion.

See question above.

> since one of which does not give a clear answer about how to proceed.

The example provides a useful shortcut to those who don't read the
references, without confusing those who do.

> feel like that's not the best way to have done it."

Sure.  It identifies "insertion points" for further development.

Some further development is needed.  SPF originally emphasized the
negative results, assuming that senders would have changed their
forwarding methods.  Some still uphold and/or deploy such silver
bullet view.  However, it is not fully compatible with SMTP, and RFC
5598, Section 5.1 documents that changing the MailFrom implies a
number of problems that a mail admin may not be willing to tackle.

Current SPF deployments emphasize positive results.  The protocol
evolved, albeit not on the paper.  However, domain reputation is not
an exact science yet, especially for smaller ADMDs having a somewhat
limited view of the global scenario.  For that reason, and because
they have a better knowledge of their users, small ADMDs tend to
deploy the silver-bullet aspect of SPF more than others.  Hence, we
need to recover that aspect, and adapt it to the current trend.  To
precisely locate where on the paper that lies is a prerequisite for
such work.

---

From: Alessandro Vesely <[email-masked]>
Date: Thu, 23 Aug 2012 11:56:03 +0200

On Thu 23/Aug/2012 06:23:27 +0200 Scott Kitterman wrote:
> so that it makes it about the DNS domain and not an ADMD, which is consistent 
> with what we have now.

Thank you.  However, that way there is still a "domain" vs "ADMD"
ambiguity, especially because of the plural ("records").

Are there more opinions on this?  Let me recall that we RECOMMEND to
test the helo identity, but formalize about neither publishing it nor
interpreting its result if different from that of the other identity.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 23 Aug 2012 06:54:07 -0700

On Thu, Aug 23, 2012 at 1:51 AM, Alessandro Vesely <[email-masked]> wrote:
> unambiguously identify SPF.

Why is that something that's become a necessity?

> We seem to agree on that being a feature,

This feels like change for the sake of change.  How about: What sort
of benefit is generated by adding this additional detail?

As I've said, I'm fine with examples using specific SMTP and enhanced
status codes, but normative language there steps on the documents that
define them, and I'd rather we omit it.  It doesn't seem necessary.

-MSK

---

From: Dave Crocker <[email-masked]>
Date: Thu, 23 Aug 2012 07:04:40 -0700

> of benefit is generated by adding this additional detail?

As asked, that's a theoretical question.  Benefits can be appealing as
ideas but turn out to satisfy no actual market need.

So in addition to asking about the benefit, I'll ask for the basis for
believing that the market of SPF users (publishers and/or queriers) have
demonstrated a desire or need for this change.

> define them, and I'd rather we omit it.  It doesn't seem necessary.

+1

d/

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 23 Aug 2012 10:42:58 -0400

>precisely locate where on the paper that lies is a prerequisite for

Perhaps I'm dense, but I don't see how any of that answers the question about then benefit of removing the status codes.  And no, I don't think ambiguity is a feature.

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 23 Aug 2012 10:57:28 -0400

>define them, and I'd rather we omit it.  It doesn't seem necessary.

Having this language is not a change, except for one case where it was inadvertently omitted it's all there in 4408.  I think removing what's there is what's change for change sake, not adding the missing information for one result to make the document internally consistent (it's one of the errata for 4408).

Scott K

---

From: Alessandro Vesely <[email-masked]>
Date: Thu, 23 Aug 2012 17:56:45 +0200

On Thu 23/Aug/2012 16:42:58 +0200 Scott Kitterman wrote:
>> [...]
> Perhaps I'm dense,

More probably, my English is as if it was written by a foreigner ;-)

> I don't see how any of that answers the question about then benefit
> of removing the status codes.

It might allow playing a different mantra, where the stress is more on
what the receiver does and less on how it does it.  According to the
current text, reject and discard have the same conformance.

> And no, I don't think ambiguity is a feature.

Well you said the omission of an SPF-specific enhanced status code was
not out of inattention.  I inferred that if it's not a bug it must be
a feature...

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 23 Aug 2012 11:14:36 -0700

> Having this language is not a change, except for one case where it was inadvertently omitted it's all there in 4408.  I think removing what's there is what's change for change sake, not adding the…

My point is not change for the sake of change, but that it was a
mistake to include them in the first place.  Since cleaning up errors
is within charter, I think it's a valid thing to consider.

What special behaviour are we hoping to provoke in MTAs by providing
normative guidance for use of specific codes when a rejection occurs
due to SPF failures?  Unless there is some, then there's no new data
to encode, and thus providing normative guidance of specific codes is
not valuable.

-MSK

---

From: Alessandro Vesely <[email-masked]>
Date: Thu, 23 Aug 2012 20:38:41 +0200

> to encode, and thus providing normative guidance of specific codes is

+1 for removing 2119-language for codes

---

From: S Moonesamy <[email-masked]>
Date: Thu, 23 Aug 2012 12:45:06 -0700

Hello,

I have read the messages on this thread.  I will flag the following 
text in Section 2.5.4 as an issue for AD review:

   "If the checking software chooses to reject the mail during the SMTP
    transaction, then it SHOULD use an SMTP reply code of 550 (see
    [RFC5321]) and, if supported, the 5.7.1 enhanced status code (see
    [RFC3463]), in addition to an appropriate reply text."

Barry Leiba, as Area Director, provided some very useful information 
which may be relevant to the discussion about the above text ( 
http://www.ietf.org/mail-archive/web/spfbis/current/msg02240.html ).

I would be picky about the "checking software". :-)

Regards,
S. Moonesamy

---

From: Hector Santos <[email-masked]>
Date: Thu, 23 Aug 2012 16:27:57 -0400

S Moonesamy wrote:
> Hello,
> 
> I have read the messages on this thread.  I will flag the following text 
> in Section 2.5.4 as an issue for AD review:
> 

Yes, you have argued for 550 when you saw software using 551, like 
with quite a number of GLS (GreyListing Servers) and there are client 
software that might be triggering logic off the 551 since the 550 is 
very generic, hence why it (551) was used to cover non-extended reply 
code server support.

My view.

I don't see a problem with having specific examples. Perhaps if 
anything, they are just poor examples for reply text and also the 
exact paragraph can be improved too.

IMO, if we want to remove the codes and link other docs, then we need 
to change the paragraph to state specifically the subject type of 
reply codes. Something along the line:

    If the checking software chooses to reject the mail during the SMTP
    transaction, then it SHOULD use policy-based reply codes with the
    appropriate policy-based reply reason text. See policy examples in
    RFC5321, RFC3463.

But to me, even for the best programmer in the world, he will be 
tickled pink if he had specific examples to work with. Otherwise, 
while he might have open windows on his desktop for RFC5321 and 
perhaps RFC3463 or even perhaps already printed out if his software is 
going to support extended reply codes,  in the end, he might look else 
where for real illustrative transaction examples in the market place 
for the sort of the text that is being used.  Spare him the need to 
search for servers that have working operational examples of rejects. 
  No harm to anyone except maybe to a RFC/IETF purist.

We need to remember, our "customers" of these docs are the real 
readers, NOT people like us (in this WG). They are certainly not IETF 
Technocrats who are privy to every known RFC document on earth and can 
put it all together.  This is one reason I believed and also believed 
many also believed, RFC1123 was the like the "Holy Bible" for internet 
hosting!  While it filled in many gaps missing at the time, it did one 
great thing - a single doc to work with with a sweet summary for a 
requirements table.

-- 
HLS

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 23 Aug 2012 18:54:06 -0700

On Thu, Aug 23, 2012 at 12:45 PM, S Moonesamy <[email-masked]> wrote:
> I have read the messages on this thread.  I will flag the following text in
> Section 2.5.4 as an issue for AD review:
>
> Barry Leiba, as Area Director, provided some very useful information which
> may be relevant to the discussion about the above text (

I'd be happy to argue with the ADs in support of dumping the normative
text and leaving useful examples.

-MSK

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 24 Aug 2012 10:21:13 +0200

> I would be picky about the "checking software". :-)

If that means to s/checking software/server/, I agree.

And there is a lowercase "should" further down, right before the
example, that should be capitalized.

---

From: Scott Kitterman <[email-masked]>
Date: Sun, 26 Aug 2012 01:57:47 -0400

> example, that should be capitalized.

Fixed.  Found some others too.

Scott K