Subject: [spfbis] Review of draft-ietf-spfbis-4408

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 13 Jul 2012 01:20:28 -0700

Attached is a top-down review of the RFC4408bis document.  Apologies that
it took so long, but it's a big document.

The majority of the things I've tagged in there are editorial in nature.
Some refer to things that are open in the tracker as well, including my
suggestion for making "check_host()" not look so much like an API
definition.  Note that it's only a suggestion; I'm happy to discuss other
ways to resolve that issue.

tl;dr version: It's well on its way to being ready, but we need to spend
some more time polishing it as well as resolving the open issues.  None of
it creates a need for face time in Vancouver.

Comments welcome.

-MSK

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 13 Jul 2012 17:54:57 +0200

On Fri 13/Jul/2012 10:20:28 +0200 Murray S. Kucherawy wrote:
> Attached is a top-down review of the RFC4408bis document.  Apologies
> that it took so long, but it's a big document.

That's great of you, thanks.

> Comments welcome.

I comment only some of Murray's suggestions, for the time being.

@ 445
>    It is RECOMMENDED that SPF clients not only check the "MAIL FROM"
>    identity, but also separately check the "HELO" identity by applying
>    the check_host() function (Section 4) to the "HELO" identity as the
>    <sender>.
> 
> [MSK: We need to explain why this is RECOMMENDED.  We might later,
> but I haven't gotten that far yet and I might forget to come back
> and delete this.  :-)]

Good point.  I don't see how we can explain that "why" without also
telling how the result can affect the server's behavior.  See the text
at line 653 of Murray's review, quoted below.

@ 480
>    If domain owners choose to publish SPF records, it is RECOMMENDED
>    that they end in "-all", or redirect to other records that do, so
>    that a definitive determination of authorization can be made.
> 
> [MSK: RECOMMENDED in RFC2119 terms is effectively a SHOULD.  I
> don't think we should do this, as that's strictly a local policy
> matter and not one of interoperability.]

Although local policy is intertwined with interoperability in various
ways, I agree 2119-language is not adequate to express policy issues.
For example, publishers ShOuLd NoT use +all.  How do we convey such stuff?

@ 599
>    Generating non-delivery notifications to forged identities that have
>    failed the authorization check is generally abusive and against the
>    explicit wishes of the identity owner.
> 
> [MSK: I think we should say more here.  For example, how exactly is
> it abusive?  The recipient has done exactly what it was asked to
> do, yet it's abusing someone?  Rather than saying it's abusive, I
> suggest we say it introduces backscatter, and then define that
> term.]

E.g.:

 Backscatter:  An unwitting abuse of domain names caused by using
               unverified addresses as targets for NDNs.

@ 653
> [MSK: We should probably give a sentence somewhere above this about
> what's meant by "mark the mail".  Maybe even a small section about
> possible "fail" actions that underscores the fact that rejection
> vs. marking is a choice and neither is specifically required would
> be a good idea.]

A new Section 2?  It had better be a "non normative" section, since we
want to avoid 2119-language for this topic.  Some parts of Section 9
might go there as well.

@ 711
> [MSK: Do we really need to spell out SMTP reply codes and enhanced
> status codes?  Isn't it enough to say those codes are defined in
> RFC5321 and RFC3463, and leave it at that?]

+1, those constructs are indirect ways to insinuate behavioral
suggestions.  They should be stated in a more direct manner, perhaps
in the new Section 2 aired above.

@ 761
>                             Loosely, the record partitions all hosts
>    into permitted and not-permitted sets (though some hosts might fall
>    into neither category).
> 
> [MSK: That sounds confusing.  How is that possible?]

I attempted a better wording, but my English doesn't support me.
That quoted sentence was transformed into the less loose but too lengthy:
                                     For each identity, an SPF
 evaluation starting at the relevant domain name associates an
 authorization result to each possible sender, based on the
 authentication token provided by lower level protocols, the IP
 address.
 http://www.ietf.org/mail-archive/web/spfbis/current/msg01775.html

Can someone compose something that's precise and readable at the same
time?

@ 777
>    Domain owners wishing to be SPF compliant must publish SPF records
> 
> [MSK: s/must/MUST/]

I'd suggest MAY.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 13 Jul 2012 13:36:19 -0700

> I attempted a better wording, but my English doesn't support me.

Actually, I withdraw the comment.  It was late and I read "either" rather
than "neither", i.e., a client could somehow be both permitted and
not-permitted simultaneously, which is of course absurd.  It's fine as
"neither".

-MSK

---

From: Alessandro Vesely <[email-masked]>
Date: Sat, 14 Jul 2012 20:28:16 +0200

@ 589
> [MSK: I have an open ticket in the tracker about check_host(), so
> I'll make my first comment about it here though there will be many
> more later.  The very string "check_host()" looks like a function
> definition, and as I said in the tracker item, the IETF prefers to
> avoid the business of API definitions.  It's possible that I might
> be able to search-and-replace it into something more palatable.

The term "function" as well as the use of parentheses was borrowed
from mathematics and logic.  It does not, by itself, imply any API
definition.  However, the use of parentheses to indicate that it is a
function is typical of programming.  It can be safely removed, IMHO.

@ 972
> [MSK: I'm reading this on the assumption that "check_host()" is
> replaced by "CheckHost".  Let's see if that works.]

Switching to camel-case is not particularly meaningful, as far as
telling logical functions from API is concerned.  The downside of a
name change is that it will never propagate to all SPF-related
documents.  That way we'll end up with two lexically different terms
to indicate exactly the same thing.

@ 984
> 4.1.  Arguments
> 
>    The check_host() function takes these arguments:
> 
> [MSK: "This section describes the procedure for evaluating an SMTP
> client's authorization to use a domain name as an identity in the
> SMTP session based on SPF policy retrieved from the DNS.  The
> procedure is referred to as the "CheckHost function".  It uses
> these inputs:"]

If the I-D specifies each and every step, I'd call it the "check_host
algorithm".  Using the term function seems to stress that some details
are left to the implementation, while the logical relationship between
input and output is essential.

s/4.1.  Arguments/4.1.  Input/?

@ 999
> [MSK: "(The names in angle brackets are symbolic names for these
> inputs, and are discussed using these symbols elsewhere in this
> document.)"]

@ 1006
>    Actual implementations of the check_host() function might need
>    additional arguments.
> 
> [MSK: "Specific implementations of CheckHost may, of course,
> consider other parameters as inputs in addition to these."]

I'm quite unhappy with either wording.  The I-D specifies what input
results in what output.  I propose:

 The names in angle brackets are symbolic names for the inputs, and
 correspond to values discussed using these symbols elsewhere in this
 document.  No resemblance is implied with the names, order, and
 number of parameters of actual implementations of this function.

---

From: Scott Kitterman <[email-masked]>
Date: Mon, 16 Jul 2012 01:07:03 -0400

> it creates a need for face time in Vancouver.

I'm only commenting on the things that I think need discussion.

> 1.  Introduction
> 
>    The current email infrastructure has the property that any host
>    injecting mail into the mail system can identify itself as any domain
>    name it wants.  Hosts can do this at a variety of levels: in
>    particular, the session, the envelope, and the mail headers.
> 
> [MSK: Does "the session" refer to HELO/EHLO, rDNS, or something else?]

Good question.  I don't actually recall.  I went back and looked and that 
paragraph first appears in the draft done for MARID.  The pre-MARID drafts 
don't have anything like it at all and it wasn't touched after.  My guess 
would be the SMTP session and the identity in question is HELO/EHLO.

This paragraph seems to beg for editorial improvements, so I'd welcome text if 
someone is up to it.

>    Although this feature is desirable in some circumstances, it is a
>    major obstacle to reducing Unsolicited Bulk email (UBE, aka spam).
>    Furthermore, many domain name holders are understandably concerned
>    about the ease with which other entities can make use of their domain
>    names, often with malicious intent.
> 
> [MSK: Would ADMD from RFC5598 be helpful in place of "domain name holders"
> above or "domain owners" below?]> 
>    This document defines a protocol by which domain owners can authorize
>    hosts to use their domain name in the "MAIL FROM" or "HELO" identity.

Sort of.  I think it would have to be something like "ADMD owning the domain", 
since there are plenty of non-domain owning ADMDs.  I'm not sure how much that 
helps.

> [MSK: We need to explain why this is RECOMMENDED.  We might later, but I
> haven't gotten that far yet and I might forget to come back and delete
> this.  :-)]

It doesn't.  It's RECOMMENDED as a general practice both in the sense of 
"being a useful thing to do" and for overall consistency of results.  I don't 
know that the latter is enough of an interoperability point to keep the 2119 
language.  The SPF design,  up to the point where local policy starts, is 
meant to be pretty deterministic and get consistent results.

> [MSK: RECOMMENDED in RFC2119 terms is effectively a SHOULD.  I don't think
> we should do this, as that's strictly a local policy matter and not one of
> interoperability.]

Agreed.  This needs to be reworded into something along the lines of "If the 
domain owning ADMDs purpose in publishing the SPF record includes allowing 
other parties to make a definitive determination of which hosts are not 
authorized by the ADMD to send mail from the domain, then they MUST publish a 
record that ends in "-all" or redirect to other records that do."

Maybe that belongs in section with a non-2119 word for MUST.

>    When changing SPF records, care must be taken to ensure that there is
>    a transition period so that the old policy remains valid until all
>    legitimate email has been checked.
> 
> [MSK: There's no way to ever know this.  Who's to say that a message hasn't
> sat queued for a month and finally goes through, or something erroneously
> begins relaying and thus re-evaluating a ton of old mail?  We saw that all
> the time at Cloudmark, where someone would report an entire chunk of
> his/her inbox as "spam" all at once.  Re-evaluations like that could have
> undesirable effects.  Thus, I think that last bit isn't actionable.  DKIM
> talked about overlapping selectors as a transition policy, but I'm not sure
> it applies here.  Don't know what to suggest either.]

Microsoft's original Caller ID for Email specification suggested 30 days, but 
they also envisioned checking in MUAs, which SPF never has (people do it, but 
it often doesn't end well).  The intent of this was to warn people that there 
needed to be a transition, but we didn't try to define the time requirement 
because, as you say, there's no way to know.  This should probably move to 
section 9.

> 2.5.  Interpreting the Result
> 
>    This section describes how software that performs the authorization
>    should interpret the results of the check_host() function.  The
>    authorization check SHOULD be performed during the processing of the
>    SMTP transaction that sends the mail.  This allows errors to be
>    returned directly to the sending MTA by way of SMTP replies.
> 
> [MSK: I have an open ticket in the tracker about check_host(), so I'll make
> my first comment about it here though there will be many more later.  The
> very string "check_host()" looks like a function definition, and as I said
> in the tracker item, the IETF prefers to avoid the business of API
> definitions.  It's possible that I might be able to search-and-replace it
> into something more palatable.  This is just a placeholder comment to note
> that I still think it's an issue and I'm figuring out how to suggest we
> deal with it.]> 

It looks like later you started replacing check_host() with CheckHost.  If 
it's just a string replacement like that, I don't object.  I don't like the 
idea of creating diff for things like this, but as long as there's support in 
the group for it, no problem.

>    Performing the authorization after the SMTP transaction has finished
>    can cause problems, such as the following: (1) It might be difficult
>    to accurately extract the required information from potentially
>    deceptive headers; (2) legitimate email might fail because the
>    sender's policy had since changed.
> 
> [MSK: How is (1) above specific to post-transaction processing?  Seems to me
> it's a problem at the end of DATA as well.]> 

I guess this is a bit implementation dependent.  In Postfix, the MTA I'm most 
familiar with, all of the information from the SMTP session is still available 
to me through it's policy interface.  Meng Wong was also a Postfix user, IIRC, 
so this wording may be too implementation specific and need to be generalized.  
I'd appreciate help on text.

> [MSK: I think we should say more here.  For example, how exactly is it
> abusive?  The recipient has done exactly what it was asked to do, yet it's
> abusing someone?  Rather than saying it's abusive, I suggest we say it
> introduces backscatter, and then define that term.]

Makes sense.  If someone can suggest a definition or a reference for a 
definition (even better), I would appreciate it.

> 2.5.1.  None
> 
>    A result of "none" means that no records were published by the domain
>    or that no checkable sender domain could be determined from the given
>    identity.  The checking software cannot ascertain whether or not the
>    client host is authorized.
> 
> [MSK: s/means that/means/, s/or that/or/]
> 
> [MSK: What constitutes "checkable"?]

It means a domain could not be extracted to be used in check_host() 
(CheckHost).  The two possible conditions are "couldn't figure out what domain 
to query for a TXT record" and "queried for a record, got no DNS errror, and 
did not get an SPF record back (could have gotten other TXT records."  It 
would be good if someone could suggest a clearer way to put this.

> 2.5.4.  Fail
> 
>    A "fail" result is an explicit statement that the client is not
>    authorized to use the domain in the given identity.  The checking
>    software can choose to mark the mail based on this or to reject the
>    mail outright.
> 
> [MSK: We should probably give a sentence somewhere above this about what's
> meant by "mark the mail".  Maybe even a small section about possible "fail"
> actions that underscores the fact that rejection vs. marking is a choice
> and neither is specifically required would be a good idea.]> 

I think change the second sentence to something like "Disposition of SPF fail 
messages is a matter of local policy.  See [new part of section 9] for 
considerations on developing local policy." Then we'd add what you're 
suggesting to section 9.  That makes it very clear that the mark/reject 
decision is up to the receiver and not a normative part of the protocol.

>    example, the recipient's Mail User Agent (MUA) could highlight the
>    "softfail" status, or the receiving MTA could give the sender a
>    message using greylisting, [RFC6647], with a note the first time the
>    message is received, but accept it on a later attempt based on
>    receiver policy.
> 
> [MSK: Are there any implementations that do this?]

Yes.  Tumgreyspf can do this.  It's a blended SPF/Grey listing application.

> 2.5.6.  TempError
> 
>    A "temperror" result means that the SPF client encountered a
>    transient error while performing the check.  Checking software can
>    choose to accept or temporarily reject the message.  If the message
>    is rejected during the SMTP transaction for this reason, the software
>    SHOULD use an SMTP reply code of 451 and, if supported, the 4.4.3
>    enhanced status code.
> 
> [MSK: Do we really need to spell out SMTP reply codes and enhanced status
> codes?  Isn't it enough to say those codes are defined in RFC5321 and
> RFC3463, and leave it at that?]

I think we should.  There was an erratum about one of them being missing and 
it would be odd to resolve that erratum by removing them all.  Additionally, 
I'm not sure it's 100% obvious.  When we were reviewing that erratum, we 
discovered RFC 4408 got one of them wrong.  I think there is an 
interoperability point here about consistency that it's worth not removing 
text for.

> 2.5.7.  PermError
> 
>    A "permerror" result means that the domain's published records could
>    not be correctly interpreted.  This signals an error condition that
>    requires manual intervention to be resolved, as opposed to the
>    temperror result.  If the message is rejected during the SMTP
>    transaction for this reason, the software SHOULD use an SMTP reply
>    code of 550 and, if supported, the 5.5.2 enhanced status code.  Be
>    aware that if the domain owner uses macros (Section 8), it is
>    possible that this result is due to the checked identities having an
>    unexpected format.
> 
> [MSK: s/means that/means/]
> 
> [MSK: Same point as above about reply codes, etc.]
> 
> [MSK: If not covered elsewhere, we need to indicate which of these is an
> appropriate return when a DNS error occurs.  It may be the case that we
> leave this as an implementation detail, but I'd prefer we say that.]
> 
Is this not already addressed under temperror?

> 3.  SPF Records
> 
>    An SPF record is a DNS Resource Record (RR) that declares which hosts
> 
> [MSK: This needs to be adjusted, because there's an SPF RRTYPE, but also a
> TXT RRTYPE, and as of this draft we prefer only the latter.  Suggest adding
> "TXT (type 16)" after "DNS" here.]> 

Agreed.

>    are, and are not, authorized to use a domain name for the "HELO" and
>    "MAIL FROM" identities.  Loosely, the record partitions all hosts

Pass = permitted and Fail = not permitted.  The none, neutral, softfail, 
permerror, and temperror results are variants of "I don't know".  I don't know 
are the ones that are in neither category.  I'd appreciate someone providing 
recommended text to make this clearer.

> [MSK: As I found in the experiment survey, there are some domains that have
> as many as 37 records at the domain level.  We might want to include any
> other permanent references we can find that establish other TXT records
> published at the domain level that might interfere with these guidelines.]

If someone want to look into gathering a list, I don't object, but I'm not 
sure of the value.  Many/most uses are not standardized, so whatever we do 
will be guaranteed to be substantially incomplete.  I'm not sure what benefit a 
partial list would do.

> 4.  The check_host() Function
> 
> [MSK: I'm reading this on the assumption that "check_host()" is replaced by
> "CheckHost".  Let's see if that works.]

FWIW, I see what you're going after here and I think it 'works', but I don't 
think we should change it.  It's a distinction without a difference and the 
existence of references to both check_host() and CheckHost is documentation 
would be confusing.  I'm not worried about references withing 4408bis, but in 
other documents.

> 4.3.  Initial Processing
> 
>    If the <domain> is malformed (label longer than 63 characters, zero-
>    length label not at the end, etc.) or is not a fully qualified domain
>    name, or if the DNS lookup returns "domain does not exist" (RCODE 3),
>    check_host() immediately returns the result "none".
> 
> [MSK: I suggest <domain> be defined in terms of whatever a legal domain name
> is in [RFC5321], as amended by EAI, and further require that it be
> fully-qualified.  In terms of I18N, we should just do what DKIM did to
> handle this case.]

What did "DKIM do"?
 
>    When evaluating the "mx" and "ptr" mechanisms, or the %{p} macro,
>    there MUST be a limit of no more than 10 MX or PTR RRs looked up and
>    checked.  If more than 10 "mx" or "ptr" records are returned for this
>    further lookup, a permerror MUST be returned.  This limit is per
>    mechanism or macro in the record and in addition to the lookup limits
>    above.
> 
> [MSK: So this is where the concern about 110 queries comes from.  The
> minutes from the Paris meeting say we would ask the DNS Directorate for
> guidance here as to whether we should be concerned about it.  I don't think
> that's happened yet.  Someone should put that question together and send it
> over to them ASAP.]

I believe that the actual number of concern is 111 which you achieve by 
including the original record lookup.

>    Note that records SHOULD always use either a "redirect" modifier or
>    an "all" mechanism to explicitly terminate processing.
> 
> [MSK: So there's a default, but people SHOULD NOT rely on it?  That seems
> awkward.]

SPF records are read by humans in addition to computers and they are often 
confused about this feature of the protocol.  There is an implicit result, 
but, as the "Zen of Python" says, "Explicit is better than implicit."  The 
default result is fine for the silicon based computers that read SPF records, 
but the carbon based ones do better if one specifies it.
    
>    For several mechanisms, the <domain-spec> is optional.  If it is not
>    provided, the <domain> is used as the <target-name>.
> 
> [MSK: How are <domain-spec> and <domain> different?  Do we need them both?]

Domain-spec is a defined ABNF term.  Domain isn't.  It's one of the input 
arguments to check_host()/CheckHost defined in section 4.1.  In the case of an 
SPF record like:

example.com In TXT "v=spf1 a:mail.example.com -all"

the domain is example.com, an input value applicable to the entire evaluation 
(see redirect= for a limited exception to this) while domain-spec is a 
property associated with a particular mechanism or modifer.

I think they are distinct enough that we ought not try and combine them.

>    If no CIDR-length is given in the directive, then <ip> and the IP
>    address are compared for equality.  (Here, CIDR is Classless Inter-
>    Domain Routing.)
> 
> [MSK: Provide a reference to the definition of CIDR, and make sure
> "CIDR-length" is defined there or use the term that document uses.]> 
>    If a CIDR-length is specified, then only the specified number of
>    high-order bits of <ip> and the IP address are compared for equality.

If someone knows/would look up the reference, I'd appreciate it.

> [MSK: Do we need to mention network byte order?  Probably not, just asking.]

No that I know of.

>    
>    Mechanisms after "all" will never be tested.  Any "redirect" modifier
>    (Section 6.1) has no effect when there is an "all" mechanism.
> 
> [MSK: Unless it appears before "all", right?]

I'm not sure how well it's defined, but the spec at least hints in more than 
one place (and this is one of them) that modifiers are processed after 
mechanisms.  I believe that's the original intent and we ought to clarify 
things to explicitly say so.

So,  answer to your question, I don't think thats right, but I think it's not 
at all clear.

>    In hindsight, the name "include" was poorly chosen.  Only the
>    evaluated result of the referenced SPF record is used, rather than
>    acting as if the referenced SPF record was literally included in the
>    first.  For example, evaluating a "-all" directive in the referenced
>    record does not terminate the overall processing and does not
>    necessarily result in an overall "fail".  (Better names for this
>    mechanism would have been "if-pass", "on-pass", etc.)
> 
> [MSK: Interesting.  Should we make a new name for it, synonymous with
> "include", to facilitate introduction of a new name in a future version?]

I don't see how we can do that without running into transition problems.  You 
can't publish both the new/old name as the new name would cause an error on 
existing implementations, so it couldn't be deployed as part of SPF v 1.   
This is one of the changes (like combining a/ip4/ip6) that would be good to 
consider in a future update that is free to worry less about backward 
compatibliilty.

>    The "include" mechanism is intended for crossing administrative
>    boundaries.  Although it is possible to use includes to consolidate
>    multiple domains that share the same set of designated hosts, domains
>    are encouraged to use redirects where possible, and to minimize the
>    number of includes within a single administrative domain.  For
>    example, if example.com and example.org were managed by the same
>    entity, and if the permitted set of hosts for both domains was
>    "mx:example.com", it would be possible for example.org to specify
>    "include:example.com", but it would be preferable to specify
>    "redirect=example.com" or even "mx:example.com".
> 
> [MSK: I get that it's preferable, but I don't see why.  What difference does
> it make?]

It's a question of how the records expand and what results you get.

example.com IN TXT "v=spf1 include: another.example.net ?all"
example.com IN A 192.0.2.1
another.example.net IN TXT "v=spf1 a -all"
another.example.net IN A 192.0.2.200

This will match for mail from 192.0.2.200 since the domain used in the 
evaluation of another.example.net's SPF record is another.example.net.  The 
result for mail from any other IP address is Neutral because include with just 
match/notmatch and the original SPF record's "all" mechanism is what controls 
the result if nothing matches.

With include you are specifying you want another domain's authorized hosts to 
be authorized for your domain, but you still want to control the sender policy 
from this one (what result to give for things that don't match).

Similarly, consider:

example.com IN TXT "v=spf1 redirect=another.example.net"
example.com IN A 192.0.2.1
another.example.net IN TXT "v=spf1 a -all"
another.example.net IN A 192.0.2.200

In this case, mail from 192.0.2.1 matches the record since the check_host 
expansion is done using the orignal domain name, not the domain-spec of the 
redirect modifier.  The result is fail is the mail doesn't match.

Redirect is much more like a common code element to be shared among records in 
a single ADMD.  You could control both authorized hosts and policy for an 
arbitrary number of domains from a single record.

> 
> 5.5.  "ptr" (deprecated)
> 
> [MSK: Wouldn't it be easier to drop this section?]

It's in use.  We can't drop it, only discourage it harder.
    
>    sending-domain_names := ptr_lookup(sending-host_IP);
>    if more than 10 sending-domain_names are found, use at most 10.
>    for each name in (sending-domain_names) {
>    
>      IP_addresses := a_lookup(name);
>      if the sending-domain_IP is one of the IP_addresses {
>      
>        validated-sending-domain_names += name;
>      
>      }
>    
>    }
>    
>    Check all validated domain names to see if they end in the
>    <target-name> domain.  If any do, this mechanism matches.  If no
> 
> [MSK: So if xebay.com is validated, and <target-name> is ebay.com, the
> mechanism matches?  Surely not, but it illustrates that the definition of
> this is weak.]> 

That's meant to be limited to subdomain matches.  We should improve the 
language.

> 5.6.  "ip4" and "ip6"
> 
>    These mechanisms test whether <ip> is contained within a given IP
>    network.
>    
>    IP4              = "ip4"      ":" ip4-network   [ ip4-cidr-length ]
>    IP6              = "ip6"      ":" ip6-network   [ ip6-cidr-length ]
>    
>    ip4-cidr-length  = "/" 1*DIGIT
>    ip6-cidr-length  = "/" 1*DIGIT
>    dual-cidr-length = [ ip4-cidr-length ] [ "/" ip6-cidr-length ]
> 
> [MSK: Can't this be simplified to just a "cidr-length", and the above three
> dropped?  Really the only difference is that ip4-cidr-length can't be more
> than 32 and ip6-cidr-length can't be more than 128.  So either say that in
> an ABNF comment for each (and you could make the former be 1*2DIGIT and the
> latter 1*3DIGIT), or simplify.]> 

Would that still allow one to use a:example.com/30/64?  That's legit now for 
dual homed IPv4/6 hosts.

>    ip4-network      = qnum "." qnum "." qnum "." qnum
>    qnum             = DIGIT                 ; 0-9
>    
>                       / %x31-39 DIGIT       ; 10-99
>                       / "1" 2DIGIT          ; 100-199
>                       / "2" %x30-34 DIGIT   ; 200-249
>                       / "25" %x30-35        ; 250-255
>             
>             ; as per conventional dotted quad notation.  e.g., 192.0.2.0
> 
> [MSK: Is there really not a reference for this syntax?]

There probably is, but we didn't find it in 2005.  If someone can dig it up, 
I'll add it.
 
>    The domain-spec is expanded as per Section 8.  The resulting domain
>    name is used for a DNS A RR lookup.  If any A record is returned,
>    this mechanism matches.  The lookup type is A even when the
>    connection type is IPv6.
> 
> [MSK: What about AAAA?]

This is one of the few places where it says A and it should say A/AAAA.  I 
think fixing it is an obvious bug fix.

> 7.  The Received-SPF Header Field
> 
> [MSK: This should be updated as agreed (I think) to say implementations
> SHOULD generate A-R and MUST NOT generate this, while they SHOULD be able
> to parse this.]> 

We argued about this.  I agree that Section 7 should cover both Received SPF 
and Authentication Results.  I think it should also described how to create an 
Authentication Results header that has all the information found in a Received 
SPF header.  

I don't believe that there is an Internet interoperability point in preferring 
one over the other.  These are generated and consumed within an ADMD and so 
within that ADMD, they should use whichever works for them.  On that basis, I 
think it's reasonable to encourage adoption of Authentication Results by 
software developers so that ADMD administrators can choose.

>    It is RECOMMENDED that SMTP receivers record the result of SPF
>    processing in the message header.  If an SMTP receiver chooses to do
>    so, it SHOULD use the "Received-SPF" header field defined here for
>    each identity that was checked.  This information is intended for the
>    recipient.  (Information intended for the sender is described in
>    Section 6.2, Explanation.)
>    
>    The Received-SPF header field is a trace field (see [RFC5322] Section
>    3.6.7) and SHOULD be prepended to the existing header, above the
>    Received: field that is generated by the SMTP receiver.  It MUST
>    appear above all other Received-SPF fields in the message.  The
>    header field has the following format:
>    
>    header-field     = "Received-SPF:" [CFWS] result FWS [comment FWS]
>    
>                       [ key-value-list ] CRLF
>    
>    result           = "pass" / "fail" / "softfail" / "neutral" /
>    
>                       "none" / "temperror" / "permerror"
>    
>    key-value-list   = key-value-pair *( ";" [CFWS] key-value-pair )
>    
>                       [";"]
>    
>    key-value-pair   = key [CFWS] "=" ( dot-atom / quoted-string )
>    
>    key              = "client-ip" / "envelope-from" / "helo" /
>    
>                       "problem" / "receiver" / "identity" /
>                       
>                        mechanism / name
>    
>    identity         = "mailfrom"   ; for the "MAIL FROM" identity
>    
>                       / "helo"     ; for the "HELO" identity
>                       / name       ; other identities
> 
> [MSK: Should "identity" be quoted in the "key" list?]
> 
> [MSK: There are two paths to "key" generating "name": one direct, one
> through "identity"]> 

Is this a problem?

>    "MAIL FROM" identity was checked, "envelope-from".
>    
>    client-ip      the IP address of the SMTP client
>    
>    envelope-from  the envelope sender mailbox
> 
> [MSK: Suggest "the RFC5321.MailFrom address" or simply "the return-path"]

There's a section at the beginning where we cover all the different names for 
mail from.  We ought to use this terminology from there on.

>    macro-expand     = ( "%{" macro-letter transformers *delimiter "}" )
>    
>                       / "%%" / "%_" / "%-"
> 
> [MSK: The "*delimiter" allows this: %{s.-+,/_=.+,..+.}  What would that
> mean?]> 

Good question.  It's almost 1AM, so I'm not even going to try and figure that 
one out right now.  What does it mean?

>       p = the validated domain name of <ip> (deprecated)
> 
> [MSK: This makes me think we need to say somewhere what it means for this to
> be deprecated.  When verifiers see it, for example, what should they do?]> 

Deprecated is defined early in the draft.

>    By default, strings are split on "." (dots).  Note that no special
>    treatment is given to leading, trailing, or consecutive delimiters,
> 
> [MSK: In input strings, right?]

Yes.

> 
>    Macros MAY specify delimiter characters that are used instead of ".".
> 
> [MSK: Isn't this redundant to the ABNF?]

Yes, but so is a lot of the text.

>    The 'r' transformer indicates a reversal operation: if the client IP
>    address were 192.0.2.1, the macro %{i} would expand to "192.0.2.1"
>    and the macro %{ir} would expand to "1.2.0.192".
>    
>    The DIGIT transformer indicates the number of right-hand parts to
>    use, after optional reversal.  If a DIGIT is specified, the value
>    MUST be nonzero.  If no DIGITs are specified, or if the value
>    specifies more parts than are available, all the available parts are
>    used.  If the DIGIT was 5, and only 3 parts were available, the macro
>    interpreter would pretend the DIGIT was 3.  Implementations MUST
>    support at least a value of 128, as that is the maximum number of
>    labels in a domain name.
>    
>    The "s" macro expands to the <sender> argument.  It is an email
>    address with a localpart, an "@" character, and a domain.  The "l"
>    macro expands to just the localpart.  The "o" macro expands to just
>    the domain part.  Note that these values remain the same during
>    recursive and chained evaluations due to "include" and/or "redirect".
>    Note also that if the original <sender> had no localpart, the
>    localpart was set to "postmaster" in initial processing (see
>    Section 4.3).
> 
> [MSK: s/localpart/local-part/, x4]
> 
>    For IPv4 addresses, both the "i" and "c" macros expand to the
> 
> Kitterman                Expires January 6, 2013               [Page 33]
> 
> Internet-Draft        Sender Policy Framework (SPF)            July 2012
> 
>    standard dotted-quad format.
>    
>    For IPv6 addresses, the "i" macro expands to a dot-format address; it
>    is intended for use in %{ir}.  The "c" macro MAY expand to any of the
>    hexadecimal colon-format addresses specified in [RFC4291], Section
>    2.2.  It is intended for humans to read.
>    
>    The "p" macro expands to the validated domain name of <ip>.  The
>    procedure for finding the validated domain name is defined in
>    Section 5.5.  If the <domain> is present in the list of validated
>    domains, it SHOULD be used.  Otherwise, if a subdomain of the
>    <domain> is present, it SHOULD be used.  Otherwise, any name from the
>    list MAY be used.  If there are no validated domain names or if a DNS
>    error occurs, the string "unknown" is used.  This macro is deprecated
>    and SHOULD NOT be used.
> 
> [MSK: There's probably not a better algorithm, but it's uncomfortable having
> the "any name" part, because two evaluations of the same name could yield
> different expansions.]> 

Multiple PTR names and how that can get confused are part of why this was 
considered unreliable and why I think deprecation is a good idea.

>    The "t" macro expands to the decimal representation of the
>    approximate number of seconds since the Epoch (Midnight, January 1,
>    1970, UTC).  This is the same value as is returned by the POSIX
>    time() function in most standards-compliant libraries.
> 
> [MSK:  When?  At time of evaluation?  At time of message receipt?  Some
> other time?]> 

At evaluation.  It has to be.

>    When the result of macro expansion is used in a domain name query, if
>    the expanded domain name exceeds 253 characters (the maximum length
>    of a domain name), the left side is truncated to fit, by removing
>    successive domain labels until the total length does not exceed 253
>    characters.
> 
> [MSK: …and their following dots, right?]

Yes.

> 9.3.  Mail Services
> 
>    MSPs (Mail Service Providers - [RFC5598] Section 2.3) that offer mail
>    services to third-party domains, such as sending of bulk mail, might
>    want to adjust their setup in light of the authorization check

> [MSK: I'm getting wary of the use of "domain" now, and in retrospect I
> should've been looking at this throughout the document.  DNSDIR has been
> known to get cranky about the use of that word when the specific context
> isn't clear.  Did you mean ADMD here, or DNS domain name, for example?]> 

I can make this one clearer.  If there are others, let's address them.

> 
>    at least 20 seconds.  If such a limit is exceeded, the result of
>    authorization SHOULD be "temperror".
> 
> [MSK: Where are we these days on normative language in Security
> Considerations?  I thought we currently didn't like it.]

So did I, but I got pushback when I tried to move it up into section 4.  I 
think it does fit rather better there.

>    (Section 6.2), were generated by the sender policy published by the
>    domain holders themselves.  As long as messages are only returned
>    with non-delivery notification to domains publishing the explanation
>    strings from their own DNS SPF records, the only affected parties are
>    the original publishers of the domain's SPF records.
> 
> [MSK: Reference for NDNs?]

If someone could provide this, it would be appreciated.

Thanks for the detailed review.  I've just left in the things that I'd like 
help with or that I think need some further discussion.

Scott K

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sun, 15 Jul 2012 23:31:57 -0700

> This paragraph seems to beg for editorial improvements, so I'd welcome

The current email infrastructure has the property that any host injecting
mail into the system can use any DNS domain name it wants in each of the
various identifiers specified by [RFC5321] and [RFC5322], plus the "reverse
pointer" which is the purported hostname of the SMTP client as far as the
DNS is concerned [RFC1035].

> since there are plenty of non-domain owning ADMDs.  I'm not sure how much

The opposite is true though: Every domain is owned by an ADMD.  Your
suggestion seems okay to me.

> meant to be pretty deterministic and get consistent results.

I think saying exactly that is probably sufficient.

> record that ends in "-all" or redirect to other records that do."

That's probably fine, especially if there's a "however" type reference to a
section that discusses caveats in use of SPF, such as its known failure
modes (forwarders, for example).

> because, as you say, there's no way to know.  This should probably move to

Sure, let's try that.

> the group for it, no problem.

The chatter in Paris suggested simply adding a paragraph before the
definition of check_host() that says emphatically that this is not an API
definition, but rather the mechanism is used to illustrate the algorithm; a
compliant SPF implementation MUST do something semantically equivalent to
what we describe for check_host().  Pete said this was probably
acceptable.  It's probably less work than all the s// I did here as well.
We should probably try that approach, in retrospect.

> I'd appreciate help on text.

Just change "after the SMTP transaction has finished" to "other than using
the return-path and client address at the time of the MAIL command during
the SMTP session", and I'd be happy.

> definition (even better), I would appreciate it.

RFC3834 comes closest, I think.  Section 2, especially.

> would be good if someone could suggest a clearer way to put this.

A result of "none" means either (a) no syntactically valid DNS domain name
was extracted from the SMTP session that could be used as the one to be
authorized, or (b) no TXT records were retrieved from the DNS that appeared
to be intended for use by SPF verifiers.

> what's

That sounds much better.  I'll wait to see what you develop there.

> Yes.  Tumgreyspf can do this.  It's a blended SPF/Grey listing application.

Hmm.  It sounds complicated.  Although it's possible and legal, does it add
anything here to call it out as an example?  I'm pushing for simplicity
here.

> interoperability point here about consistency that it's worth not removing

What do others think?  I always prefer to refer to other protocols (or
specific parts of them) rather than copying details from them.

> recommended text to make this clearer.

You don't need to.  I originally read "neither" as "either".  It's fine the
way it is.

> will be guaranteed to be substantially incomplete.  I'm not sure what

The point of this would be to encourage implementers to harden their code
against receiving TXT records that have nothing to do with SPF.  As an
opposite example of sorts, ADSP, DKIM, VBR, and various other protocols
have all had to be hardened against the notion of asking for one of their
own TXT records and getting back an SPF record because of the use of DNS
wildcards since SPF pre-dates them.  DMARC will have to do this too.

We could reference those as examples of other protocols that use DNS TXT
records, and due to wildcarding, an SPF implementation needs to anticipate
getting data it doesn't expect.  In practice this is far less likely than
the opposite, but who knows what the future will yield.

> would be confusing.  I'm not worried about references withing 4408bis, but

I commented on this above already.  This was just one example of dealing
with the concern.  There might be something much simpler we can do.

> > fully-qualified.  In terms of I18N, we should just do what DKIM did to

I'm referring to Section 3.5 of RFC6376, the definition of "d=".

> but the carbon based ones do better if one specifies it.

We should say at the end "Although the latter is the default (specifically
"?all"), it aids debugging efforts if it is explicitly included."

(At least, I think it's "?all", but whatever it is should go there.)

> I think they are distinct enough that we ought not try and combine them.

OK, then I think the right thing would be to say that they are
syntactically identical, but used in different parts of the evaluation
process.  In ABNF, something like this would do:

domain = <whatever the definition we want to use is>

domain-spec = domain
  ; syntactically identical to "domain", but used differently in the
evaluation

Dave, do you have any other insight here?

> If someone knows/would look up the reference, I'd appreciate it.

RFC4632 is the current reference, but alas it contains no ABNF.

> things to explicitly say so.

I think that needs to be stated explicitly someplace, maybe an "overall
flow of the evaluation process" paragraph or something before it gets too
detailed.

> consider in a future update that is free to worry less about backward

I hadn't thought of a/ip4/ip6.  This makes me think a short appendix about
"notes for future revisions to SPF" or such might be a good idea.

> arbitrary number of domains from a single record.

Your example is a lot more illustrative than the paragraph I cited.
Perhaps a briefer summary of the difference would be helpful.  The concept
that speaks to me as a coder is something like "redirect never returns".
For example, in C, if you do this:

status = foo();
return status;

You have the choice to use the return value from foo() as it does, or to
insert code before the "return" to do your own thing and eventually return
your own result (which "include" does), while:

return foo();

This executes foo() and its return value is the return value of the caller
with no opportunity to do something different (which "redirect" does).

> It's in use.  We can't drop it, only discourage it harder.

I would remove it to an appendix or something if we don't intend for it to
be part of the live protocol.

But this is another example of the question, which I think is still being
discussed, about whether it's appropriate to remove or at least deprecate
things.  The same survey that started the other argument finds 93 domains
using "ptr" out of over 150,000 reporting.  If it's okay do deprecate ptr,
then it's okay to deprecate macros.  And if it's deprecated and moving from
Experimental to Proposed Standard, I think this is a serious consideration.

> That's meant to be limited to subdomain matches.  We should improve the

Something about a "right-to-left label-for-label subset match" might do.

> Would that still allow one to use a:example.com/30/64?  That's legit now

My survey found zero dual-cidr-lengths in use.

> There probably is, but we didn't find it in 2005.  If someone can dig it

I actually can't find one either.  Pete or Dave, perhaps?

> software developers so that ADMD administrators can choose.

Sounds good to me.

> > [MSK: There are two paths to "key" generating "name": one direct, one

I thought it was considered an ambiguous grammar in that case, and thus
"improper".

You should run this through a BNF checker too just to see if any bugs are
found.  There's one on tools.ietf.org someplace.

> mail from.  We ought to use this terminology from there on.

Yeah, using one throughout would be ideal.

> one out right now.  What does it mean?

My point is that the grammar allows it, and I'm not certain we want that.
What's the point of allowing lots of delimiters?  How would one apply more
than one?

> Deprecated is defined early in the draft.

So it does; withdrawn.

> > [MSK: In input strings, right?]

So if the "delimiter" part of the ABNF is "+.", what does a parser do?
Break on +, then within those break on .?  Break on either (like strtok()
does)?

> Yes, but so is a lot of the text.

I guess I'd like to keep that to a minimum.

> considered unreliable and why I think deprecation is a good idea.

Let's add a sentence in there to this effect then.

> At evaluation.  It has to be.

Let's add that clause.

> I can make this one clearer.  If there are others, let's address them.

That was the first one that got my attention.  When you post -04 I'll do a
quick pass looking for that word and see if I can spot any that need
massaging.

> think it does fit rather better there.

Pete or Barry?

> If someone could provide this, it would be appreciated.

 RFC3464, I believe.

-MSK

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 16 Jul 2012 11:31:29 +0200

On Mon 16/Jul/2012 08:31:57 +0200 Murray S. Kucherawy wrote:
> On Sun, Jul 15, 2012 at 10:07 PM, Scott Kitterman <[email-masked]> wrote:
> 
>>> it's a problem at the end of DATA as well.]
> Just change "after the SMTP transaction has finished" to "other than
> using the return-path and client address at the time of the MAIL
> command during the SMTP session", and I'd be happy.

+1, except that "transaction" seems to be a better term than
"session".  A client can issue multiple MAIL commands in the same session.

---

From: Scott Kitterman <[email-masked]>
Date: Mon, 16 Jul 2012 10:30:11 -0400

> On Sun, Jul 15, 2012 at 10:07 PM, Scott Kitterman <[email-masked]>wrote:
...
> We should probably try that approach, in retrospect.

OK.  I'll take a shot at that in the next revision.  Thanks.

> anything here to call it out as an example?  I'm pushing for simplicity

It's an example.  Similarly one might use different spam thresholds in post-
SMTP content filtering.  I'm open for suggestions.  This could get a "see 
section 9 for examples" here and then something new there if people would 
prefer (seems overkill to me though).

OT: Tumgreyspf isn't particularly complex.  It is essentially as complex as 
both SPF and greylisting and due to taking advantage of existing SPF 
libraries, the amount of SPF related code is pretty small.

> > > status
> > it would be odd to resolve that erratum by removing them all.
> > 
> >  Additionally,
> > 

This always seemed to me like adding specificity, not mere copying.

> the opposite, but who knows what the future will yield.

OK.  If it needs to be more explicit, I think that's fine.  If someone would 
provide text, it would be helpful.

> > > [MSK: I'm reading this on the assumption that "check_host()" is replaced
> > 
> > by
> > 
> > > [MSK: I suggest <domain> be defined in terms of whatever a legal domain
> > 
> > name
> > 

It says:

      Internationalized domain names MUST be encoded as A-labels, as
      described in Section 2.3 of [RFC5890].

I think that's fine.  In a related point, I think we can dispose of the local-
part macro/EAI issue similarly.

> (At least, I think it's "?all", but whatever it is should go there.)

It is.  Agreed.

> Dave, do you have any other insight here?

The current ABNF says:

   domain-spec      = macro-string domain-end

So you're proposing changing this to:

   domain-spec      = macro-string domain-end / domain
                                    ; syntactically identical to "domain", but
                                    ; used differently in the evaluation

This doesn't read quite right to me.  I think it's actually the other way 
around:

   domain-spec      = macro-string domain-end / domain
   domain                 = domain-spec / "input value"
                                    ; either provided as an input for the
                                    ; evaluation process or a domain-spec used
                                    ; as an input for recursive evaluation

As I've said before though ABNF is not my thing, so that might be totally 
wrong.

> RFC4632 is the current reference, but alas it contains no ABNF.

So we should add the reference, but we need to keep our own ABNF since there 
isn't one to refer to, correct?

> "notes for future revisions to SPF" or such might be a good idea.

I think it would distract from the purpose of the document and 4408bis is more 
than long enough already.  The particular example I gave you is well known in 
the SPF community.  If we need an IETF document for "stuff we didn't do in 
SPFbis" (and I'm not sure we do), I'd suggest a post-WG individual submission 
from the party or parties that have enough energy left to do it.  It doesn't 
need IETF consensus, so it could be informational.

> > evaluation of another.example.net's SPF record is another.example.net.
> > 
> >  The

I think some of this can go in section 9.  I'll look at making it more clear.

> Experimental to Proposed Standard, I think this is a serious consideration.

In my view, things that are deprecated are still part of the protocol, for 
now.  Implementers still need to implement them to fully implement the 
protocol.  I think moving deprecated things into their own section would make 
the document harder to operate and more confusing.

The fact that you found so few is the result of a lot of advocacy in the SPF 
community against it's use.  

> My survey found zero dual-cidr-lengths in use.

Zero explicit dual-cidr-lengths that is.  For any dual homed host there is an 
implicit /32/128.  My dual homed SPF records use ip4/ip6 mechanisms to reduce 
DNS lookups, but for people with dual homed systems that prefer the "A" 
mechanism, I think this will increasing come up.  

Without IPv6 deployment, dual-cidr-length won't get used.  Given the current 
level of IPv6 deployment, I think it's premature for us to remove IPv6 related 
features as unused because we just don't know yet what will get used or not.

The original SPF design included careful theoretical consideration of IPv6 
because we knew it was coming (eventually), but the operational experience 
with the IPv6 specific parts of the protocol is still limited.

> > There probably is, but we didn't find it in 2005.  If someone can dig it
> > up, I'll add it.
> 

OK.  Standing by.

> > > [MSK: Should "identity" be quoted in the "key" list?]

I think it should not.

> found.  There's one on tools.ietf.org someplace.

I did already.  The previous round of ABNF discussion we had we engendered by 
just such a trip to Bill's ABNF checker.
 
> What's the point of allowing lots of delimiters?  How would one apply more

One could publish an SPF record that would match multiple sequential 
delimiters.  That would be odd, and not very useful, but I don't think it's a 
problem.  I don't think there's an interoperability benefit to modifying the 
ABNF to prohibit it.

> Break on +, then within those break on .?  Break on either (like strtok()

OK.  Now I'm not sure about my previous comment.  Let's hold this until the 
overall macro discussion is resolved.

> I guess I'd like to keep that to a minimum.

I'm going to leave it for now as I think it's useful.  Let's revisit after we 
get done beating up the concept of macros.

Thanks again for the additional comments.  Once again, I've just left in the 
things that I think might need further discussion.

---

From: S Moonesamy <[email-masked]>
Date: Mon, 16 Jul 2012 08:58:38 -0700

>need IETF consensus, so it could be informational.

As a guideline, an individual submission with an intended status of 
Informational usually requires IETF consensus.

Regards,
-sm

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 16 Jul 2012 19:24:28 +0200

On Mon 16/Jul/2012 16:30:11 +0200 Scott Kitterman wrote:
>>>> [MSK: I suggest <domain> be defined in terms of whatever a
>>>> legal domain name is in [RFC5321], as amended by EAI, and
>>>> further require that it be fully-qualified.  In terms of
>>>> I18N, we should just do what DKIM did to handle this case.]
>>> 
>>> What did "DKIM do"?
>> 

DKIM had good reasons to do so:  RFCs 6530-6533 were not published at
the time, and having d= tags as A-labels could avoid considerations
about UTF-8 completely.  Neither of those are true for 4408bis.

MAIL command can legally have UTF-8 addresses.  SPF implementations
may request that parameters be converted by the caller, and prohibit
use of UTF-8 in SPF records.  But they need to sanitize the arguments
they pass to (and receive from) the DNS software anyway, so they could
do IDNA calls as well.

Disallowing UTF-8 in local explanations, including local-parts, sounds
arbitrarily against current trends.  Using UTF-8 for both records and
parameters may make for smoother specs, implementations, and
deployment --especially in the long term.

I don't think it will be any easier to switch to UTF-8 in some future
SPF version, possibly the opposite.  Is that correct?

Could we try and see how it works, or at least state the pros and cons
we'd expect?

---

From: Scott Kitterman <[email-masked]>
Date: Mon, 16 Jul 2012 14:27:03 -0400

On Monday, July 16, 2012 07:24:28 PM Alessandro Vesely wrote:
> > I think that's fine.  In a related point, I think we can dispose of the
> > local- part macro/EAI issue similarly.
> 

Current implementations don't deal with UTF-8.  To the extent they might, they 
are buggy.  I think that converting to an A-label is the best we can do 
without breaking backward compatibility, which, as I read the charter we 
aren't allowed to do for non-essential reasons.  Given the level of EAI 
deployment, I don't think it's such a reason.

I think full EAI compatibility needs to go into the stack of things to do in a 
future, non-backward compatible update.  If EAI ever gets enough traction, 
that may be enough to drive it.  I've been giving a little bit of thought to 
how to start experimentation with non-backwards compatible updates which I'll 
post about on spf-discuss (it's OT here) once my thoughts are a bit more fully 
formed.

Scott K

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 18 Jul 2012 13:33:03 +0200

On Mon 16/Jul/2012 20:27:03 +0200 Scott Kitterman wrote:
> Current implementations don't deal with UTF-8.  To the extent they
> might, they are buggy.

True.  But then we can have funky local-parts even without UTF-8.

> I think that converting to an A-label is the best we can do without
> breaking backward compatibility, which, as I read the charter we 
> aren't allowed to do for non-essential reasons.

We don't break backward compatibility.  To publishers and senders, we
can non-normatively suggest to use A-labels, which is just common
sense as of today, unless they know that all their receivers are
SMTPUTF8-compliant.  Receivers who are not SMTPUTF8-compliant
normatively MUST return permerror when they see non-ASCII or invalid
chars either in the smtp.mailfrom or in the SPF record.  That's
consistent with EAI, and is not different from what we'd do anyway. In
addition, it prompts developers to make sure they check the strings
they get.  Relying on callers in order to skip parameters validation
is not a good practice for library functions...

> Given the level of EAI deployment, I don't think it's such a reason.

Now that Andrew is questioning our sampling, I note I never saw any
IDN domain using SPF, let alone UTF-8 local-parts.  However, EAI has
been experimental for quite a while.  Didn't they ever send mail to
one another?  According to http://eggert.org/meter/spf there is some
56.9% SPF domains in South Korea.  Do they use IDN?

By s/US-ASCII/UTF-8/ we'd just put our 2 cents for that slow, global
change.  We can continue undisturbed in our ASCII world, yet allow
those who need extra characters to use them.

jm2c