Subject: [spfbis] I-D Action: draft-ietf-spfbis-experiment-03.txt

From: [email-masked]
Date: Mon, 16 Apr 2012 21:11:38 -0700

A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the SPF Update Working Group of the IETF.

	Title           : Resolution of The SPF/Sender-ID Experiment
	Author(s)       : Murray S. Kucherawy
	Filename        : draft-ietf-spfbis-experiment-03.txt
	Pages           : 10
	Date            : 2012-04-16

   In 2006 the IETF published a suite of protocol documents comprising
   SPF and Sender-ID, two proposed email authentication protocols.
   Because of interoperability concerns created by simultaneous use of
   the two protocols by a receiver, and some concerns with Sender-ID and
   compatibility with existing standards, the IESG required them to have
   Experimental status and invited the community to observe their
   deployments for a period of time, hoping convergence would be
   possible later.

   After six years, sufficient experience and evidence have been
   collected that the experiment thus created can be considered
   concluded, and a single protocol can be advanced.  This memo presents
   those findings.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-spfbis-experiment-03.txt

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

This Internet-Draft can be retrieved at:
ftp://ftp.ietf.org/internet-drafts/draft-ietf-spfbis-experiment-03.txt

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 17 Apr 2012 04:36:33 +0000

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of [email-masked]
> Sent: Monday, April 16, 2012 9:12 PM
> To: [email-masked]
> Cc: [email-masked]
> Subject: [spfbis] I-D Action: draft-ietf-spfbis-experiment-03.txt
> 
> 
> A New Internet-Draft is available from the on-line Internet-Drafts
> directories. This draft is a work item of the SPF Update Working Group
> of the IETF.
> 
> [...]

Includes the following changes:

- remove old Section 2

- add text to remind readers that this document doesn't do a thorough technical comparison of the two protocols; the community at large has already done that, and I don't believe we either need to or could be expected to track down consensus on that point beyond the evidence in hand

- expand "MUA" on first use

- mention DNS provisioning tools as a concern that may have hindered type 99 adoption

- some other tweaks John Levine suggested

- update acknowledgements

I'm told there are a couple of complete reviews outstanding, so there will be an -04 before we go about requesting WGLC.  In -04 I will also update the numbers from the two surveys I'm running, namely the DNS survey and the SUBMITTER survey, but I don't expect that the updated results will change anything.

-MSK

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 17 Apr 2012 19:38:16 +0200

On Tue 17/Apr/2012 18:03:23 +0200 Murray S. Kucherawy wrote:
> 
> there will be an -04 before we go about requesting WGLC.

I'd like to run an extra test, but I'm out of the office right now, and it
will presumably take some more time anyway to complete.

That's the test we talked about in Paris.  As inconclusive as we concluded it
would be, it may still bring some more knowledge of the existing implementations.

Let me recall the idea:  For the MXes corresponding to the surveyed domains,
start an SMTP session and initiate transactions that contain the command, say,

   MAIL FROM:<[email-masked]>

Some servers are setup to plainly fail that command.  If not, the test could
check whether a recipient like postmaster@the-domain-being-tested succeeds.
Finally, if a rejection is substantiated, the test could check whether using
an SPF-authorized helo identity has any effect.

I think we can consider any treatment after an accepted RCPT TO an indirect
effect of the SPF check, even a reject after data --that the test will never
reach.  Thus, the outcome will indicate how many servers directly reject on fail.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 17 Apr 2012 20:14:32 +0000

> From: [email-masked] [mailto:[email-masked]] On Behalf Of Alessandro Vesely
> Sent: Tuesday, April 17, 2012 10:38 AM
> To: [email-masked]
> Subject: Re: [spfbis] I-D Action: draft-ietf-spfbis-experiment-03.txt
> 
> I'd like to run an extra test, but I'm out of the office right now, and
> it will presumably take some more time anyway to complete.
> 
> That's the test we talked about in Paris.  As inconclusive as we
> concluded it would be, it may still bring some more knowledge of the
> existing implementations.
> 
> Let me recall the idea:  For the MXes corresponding to the surveyed
> domains, start an SMTP session and initiate transactions that contain
> the command, say,
> 
> Some servers are setup to plainly fail that command.  If not, the test
> could check whether a recipient like postmaster@the-domain-being-tested
> succeeds.
> Finally, if a rejection is substantiated, the test could check whether
> using an SPF-authorized helo identity has any effect.
> 
> I think we can consider any treatment after an accepted RCPT TO an
> indirect effect of the SPF check, even a reject after data --that the
> test will never reach.  Thus, the outcome will indicate how many
> servers directly reject on fail.

My concern with this approach is that it's not accurate.  A 5yz reply to MAIL FROM isn't guaranteed to be caused by SPF because there isn't a specific SMTP reply code or enhanced status code that guarantees the rejection is caused by SPF.  For all you know, a DNSBL could be the cause of the error.

The same goes for a DATA failure and SIDF, where those could be caused by DKIM failures, ADSP failures, or any content analysis error you can imagine.

By contrast, the survey data currently represented in the experiment document don't have the "maybe" sort of quality to them.

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 17 Apr 2012 20:43:53 +0000

> 	Filename        : draft-ietf-spfbis-experiment-03.txt

To avoid ratholing on the whole "which is better" question, the document currently avoids any substantial comparison of the methods of the two protocols.  The exception is Section 3 of this version which spends the first two paragraphs comparing the costs of evaluation, independent of their semantics.  That text supports what is currently said in Section 4.1, bullet 6, which speculates as to one of the likely reasons of near-zero adoption of Sender ID.

On review, as currently written, this bit looks out-of-place along with the rest of the document.  That is, it doesn't matter (to this document) what the two protocols do or how they do it; it matters that the overwhelming majority appears to have decided which one is "standard".

So should we remove it, or work to accommodate it?  I think I'm inclined to remove it, in line with Andrew's previous Saint-Exupery reference.

-MSK

---

From: "Rolf E. Sonneveld" <[email-masked]>
Date: Wed, 18 Apr 2012 00:18:32 +0200

> So should we remove it, or work to accommodate it?  I think I'm inclined to remove it, in line with Andrew's previous Saint-Exupery reference.

Summarizing the differences between SPF and Sender-ID is OK. Leave in 
section 3, but rephrase it: s/ substantially// for both occurrences in 
the second paragraph of 3.

Another aspect / difference between SPF and Sender-ID is that the 
post-DATA processing prevents the SMTP server from per-recipient SMTP 
responses. Most of the time _in the context of SPF/Sender-ID_ this will 
not be a problem, but suppose a recipient has a whitelist entry for 
example.com, where the SPF policy of example.com would instruct the 
server to reject the message, the user policy would be overruled by the 
system-level decision.

I agree with you that bullet 6 of section 4.1 does not belong in the 
list of 'From the Evidence', as it merely poses a possible reason for 
the low adoption rate of Sender-ID, so I'd suggest to leave it out, or 
add it to section 3 with a text like:

        It is possible that the additional cost of having to accept and
        process the entire message rather than just a portion of its
        envelope is (part of) the reason of the low adoption rate of Sender-ID.

/rolf

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 17 Apr 2012 22:43:20 +0000

Hi Rolf,

If we do decide we want to pontificate on the "why" in terms of adoption choices people made, I would prefer to collect that information in an appendix and not in the "meat" of the document.  Although it might be interesting to revisit all of that, the question before us doesn't necessarily include that analysis, and I think it only complicates the question quite a bit because we'd have to be sure somehow that we've covered everything.

Instead, I think it suffices for us to demonstrate which one has wider adoption after six years.

-MSK

From: [email-masked] [mailto:[email-masked]] On Behalf Of Rolf E. Sonneveld
Sent: Tuesday, April 17, 2012 3:19 PM
To: Murray S. Kucherawy
Cc: [email-masked]
Subject: Re: [spfbis] I-D Action: draft-ietf-spfbis-experiment-03.txt

-----Original Message-----

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]] On Behalf Of [email-masked]<mailto:[email-masked]>

To: [email-masked]<mailto:[email-masked]>

Cc: [email-masked]<mailto:[email-masked]>

of the IETF.

 Pages           : 10

 Date            : 2012-04-16

[...]

Summarizing the differences between SPF and Sender-ID is OK. Leave in section 3, but rephrase it: s/ substantially// for both occurrences in the second paragraph of 3.

Another aspect / difference between SPF and Sender-ID is that the post-DATA processing prevents the SMTP server from per-recipient SMTP responses. Most of the time _in the context of SPF/Sender-ID_ this will not be a problem, but suppose a recipient has a whitelist entry for example.com, where the SPF policy of example.com would instruct the server to reject the message, the user policy would be overruled by the system-level decision.

I agree with you that bullet 6 of section 4.1 does not belong in the list of 'From the Evidence', as it merely poses a possible reason for the low adoption rate of Sender-ID, so I'd suggest to leave it out, or add it to section 3 with a text like:

/rolf

---

From: "Rolf E. Sonneveld" <[email-masked]>
Date: Wed, 18 Apr 2012 09:24:53 +0200

On 4/18/12 12:43 AM, Murray S. Kucherawy wrote:
>
> Hi Rolf,
>
> If we do decide we want to pontificate on the "why" in terms of 
> adoption choices people made, I would prefer to collect that 
> information in an appendix and not in the "meat" of the document.  
> Although it might be interesting to revisit all of that, the question 
> before us doesn't necessarily include that analysis, and I think it 
> only complicates the question quite a bit because we'd have to be sure 
> somehow that we've covered everything.
>

Agreed.

/rolf

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 18 Apr 2012 16:20:53 +0200

On Wed 18/Apr/2012 15:37:06 +0200 Murray S. Kucherawy wrote:
>> From: ietf.org On Behalf Of Alessandro Vesely
>
>> I'd like to run an extra test [...]
> 
> My concern with this approach is that it's not accurate.  A 5yz reply to
> MAIL FROM isn't guaranteed to be caused by SPF because there isn't a
> specific SMTP reply code or enhanced status code that guarantees the
> rejection is caused by SPF.  For all you know, a DNSBL could be the cause
> of the error.

Yeah, it won't bring any absolute certainty.  It would contribute some facts,
which might help getting item #12 slightly more on focus --I see that
discussing doesn't seem to bring any change of opinion.

> By contrast, the survey data currently represented in the experiment
> document don't have the "maybe" sort of quality to them.

Well, test data is test data.  If one wanted to be picky, he could claim that
"v=spf1" is equivalent to "spf2.0/mfrom,pra" but shorter, so that we are
unable to conclude anything by surveying that data...

---

From: Scott Kitterman <[email-masked]>
Date: Wed, 18 Apr 2012 12:42:28 -0400

On Monday, April 16, 2012 09:11:38 PM [email-masked] wrote:
> directories. This draft is a work item of the SPF Update Working Group of
> the IETF.
> 

Comments:

section 1 says in part:

   Because Sender-ID supported use of the same policy statement as SPF,
   the IESG at the time was concerned that an implementation of
   Sender-ID might erroneously apply that statement to a message and,
   depending on selected recipient actions, could improperly interfere
   with message delivery.  

I think this is close but not quite right.  My proposal is replace "that 
statement" with "a statement intended for use only with SPF".

section 2.2 says in part:

   o  A web site [SID-IMPL] dedicated to highlighting Sender-ID
      implementations last updated in late 2007 listed 13
      implementations, which we assume means they implement the PRA
      checks.  At least one of them is known no longer to be supported
      by its vendor.

I think it's worth noting that there are no FOSS SenderID implementations 
(that I know of).  While I don't want to drag the document down the IPR 
rathole, it's an important enough part of the ecosystem that I think it's 
worth mentioning.

My proposal is to add two new sentences between the current two:

"All listed implementations are proprietary vendor implementations.  There are 
no known Free Software/Open Source implementations."

section 3 says in part:

   Another notable difference between the two is that the PRA part of
   Sender-ID cannot be determined without proceeding to the end of the
   DATA phase of an SMTP session, while SPF only requires the HELO/EHLO
   and MAIL FROM command parameters.  Because MTAs historically consume
   substantially more resources for a transaction once past the MAIL
   FROM command in the SMTP sequence, this means SPF has a substantially
   lower cost to evaluate.

Shouldn't this be after RCPT TO?  I know that SPF cares about HELO/Mail From, 
but it's DATA that is generally expensive, not RCPT TO (in fact Postfix by 
default defers all pre-DATA checks to RCPT TO due to problems with other buggy 
MTAs if it didn't).

Section 4.1.2.2 says:

   2.  Of the records retrieved, fewer than 5% requested processing of
       messages using the PRA algorithm, which was an integral part of
       Sender-ID.

I would propose changing integral to essential.  Without PRA, SenderID is 
SPF2.0/mfrom which is a subset of SPF.  I think essential is correct and makes 
the point better.

Section 4.1.2.6 says:

   6.  The overall lack of Sender-ID adoption likely has as a
       contributing factor the additional cost of having to accept and
       process the entire message rather than just a portion of its
       envelope.

Is there data to support this or is this speculation?  If it's speculation, I 
would remove it (personally, I think the lack of marginal benefit over SPF and 
the patent issues are more important - I don't propose we add that though).

Section 4.2.3 says:

   3.  that the absence of significant adoption of the type 99 DNS
       resource record, the [SUBMITTER] extension, [SENDER-ID], and
       [PRA], indicates that they are de facto obsolete.

I know this got a lot of discussion already.  My proposal is, instead of 
"...are de facto obsolete" say "are not worthy of consideration for the 
Standards Track".  This mirrors exactly the previous positive statement about 
SPF.  I think it's sufficient for our purpose and will be less controversial.

Appendix A, first paragraph says:

   SPF was originally developed by a community of interested developers
   outside the IETF.  It was brought to the IETF for standardization
   only after the specification was relatively mature and ready for the
   rigors of the IETF publication process.

Please add to start of the second sentence, "As intended from the start of the 
project, ...".  The current wording makes it sound to me like coming to the 
IETF was a late revelation.  It was the plan from the beginning (as evidence, 
the SPF spec from very early in the project were published in the format of 
Internet Drafts).

The second paragraph mentions a working group.  RFC 4408 was an individual 
submission.  There was no working group, so that should be reworded.  Also in 
that paragraph "... who encouraged doing so as the preferred path toward using    
the DNS for storing such things as policy data" would be more accurately put 
"... who threatened to block publication of the document unless it contained a 
dedicated RR type, even though there was no transition strategy".  I'll leave 
it to the author to determine if the current text is just a polite version of 
that or if the text needs changing.  Finally, the word perceived ("was 
perceived to have high barriers to entry") bothers me.  Yes.  It was perceived 
to have high barriers to entry, but that was because it did.  The perception 
was correct, so I'd rather just say that.

Once again in Section A.1:

   1.  there was hesitation to make the transition because of concerns
       that nameservers (and, in fact, DNS-aware firewalls) would drop
       or reject requests for unknown RR types (see Section 2 for
       evidence of this), which means successful rollout of a new RR
       type is contingent upon widespread adoption of updated
       nameservers and resolver functions;

It wasn't just concerns.  There were real problems (and still are).  At least 
fix this one.  Please change by removing "of concerns that".

These two parts as written make it sound like we were wringing our hands over 
nothing.  It was a huge problem then and it's still a problem now.

I think we're pretty close.

Scott K

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 18 Apr 2012 17:31:42 +0000

Thanks for the review, Scott.

> From: [email-masked] [mailto:[email-masked]] On Behalf Of Scott Kitterman
> Sent: Wednesday, April 18, 2012 9:42 AM
> To: [email-masked]
> I think this is close but not quite right.  My proposal is replace
> "that statement" with "a statement intended for use only with SPF".

Is that an important distinction to make?  That is, given the goal of this document is merely to indicate which protocol got more adoption, making this point doesn't advance that goal.

> I think it's worth noting that there are no FOSS SenderID
> implementations (that I know of).  While I don't want to drag the
> document down the IPR rathole, it's an important enough part of the
> ecosystem that I think it's worth mentioning.

I agree that this is worth pointing out, especially without mentioning the "why".  People can do their own investigation on that point if desired.

> "All listed implementations are proprietary vendor implementations.
> There are no known Free Software/Open Source implementations."

I've added a slight variant of this, and also mentioned that the [OPENSPF] reference included a mix of the two.

> Shouldn't this be after RCPT TO?  I know that SPF cares about HELO/Mail
> From, but it's DATA that is generally expensive, not RCPT TO (in fact
> Postfix by default defers all pre-DATA checks to RCPT TO due to
> problems with other buggy MTAs if it didn't).

I struck the paragragh.  It speculates about why the results are what they are, rather than merely presenting results.

> I would propose changing integral to essential.  Without PRA, SenderID
> is SPF2.0/mfrom which is a subset of SPF.  I think essential is correct
> and makes the point better.

Done.

> Is there data to support this or is this speculation?  If it's
> speculation, I would remove it (personally, I think the lack of
> marginal benefit over SPF and the patent issues are more important - I
> don't propose we add that though).

Nope; also struck this one.

> I know this got a lot of discussion already.  My proposal is, instead
> of "...are de facto obsolete" say "are not worthy of consideration for
> the Standards Track".  This mirrors exactly the previous positive
> statement about SPF.  I think it's sufficient for our purpose and will
> be less controversial.

I actually prefer the current wording, because it is precisely correct.  The word "worthy" is a little bit subjective.  But, of course, I'm happy to defer to consensus.

> Please add to start of the second sentence, "As intended from the start
> of the project, ...".  The current wording makes it sound to me like
> coming to the IETF was a late revelation.  It was the plan from the
> beginning (as evidence, the SPF spec from very early in the project
> were published in the format of Internet Drafts).

I've reworked it slightly to include your point.

> The second paragraph mentions a working group.  RFC 4408 was an
> individual submission.  There was no working group, so that should be
> reworded.

Fixed; there was a WG, but its charter was to pick a proposal and advance it, which it never did.  Clarified to indicate such.

> Also in
> the DNS for storing such things as policy data" would be more
> accurately put "... who threatened to block publication of the document
> unless it contained a dedicated RR type, even though there was no
> transition strategy".  I'll leave it to the author to determine if the
> current text is just a polite version of that or if the text needs
> changing.

I'll do some worsdmithing that part of the appendix to cover this better.  I think this needs to relay relevant history, but without sounding too frustrated.

> Finally, the word perceived ("was perceived to have high
> barriers to entry") bothers me.  Yes.  It was perceived to have high
> barriers to entry, but that was because it did.  The perception was
> correct, so I'd rather just say that.

Fixed.

> It wasn't just concerns.  There were real problems (and still are).  At
> least fix this one.  Please change by removing "of concerns that".

Fixed.

Thanks again,
-MSK

---

From: John Leslie <[email-masked]>
Date: Thu, 19 Apr 2012 08:13:39 -0400

> 	Filename        : draft-ietf-spfbis-experiment-03.txt

   I want to congratulate Murray on his work here. I think it is nearly
ready to send to the IESG.

   However...

   I do believe the document would be stronger with a clearer separation
of data, analysis, and conclusions; and I believe the data would be much
easier for the IESG to assimilate were it in table form.

   (Did I just volunteer to help?) ;^)

--
John Leslie <[email-masked]>

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 19 Apr 2012 08:19:23 -0400

On Thursday, April 19, 2012 08:13:39 AM John Leslie wrote:
> > the IETF.> 

If you look at 04, I think he's a step ahead of you.

Scott K