Subject: [spfbis] FW: New Version Notification for draft-kucherawy-spfbis-experiment-00.txt

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 17 Feb 2012 11:42:32 -0800

As a straw man document for describing the experiment and its conclusions, I offer this to the WG and volunteer to act as editor.  It contains the data we've seen on the list so far (thanks, Paul and Phillip!), and I believe is an appropriate thing to submit given the posture we've seen from Pete about what the IESG would accept as resolution to the experiment.

I have deliberately avoided populating the Conclusions section on my own.  That's for the Working Group to do by consensus.

I've polled a number of sources both on this list and off for data to include in this memo, or whatever other memo we decide to advance.  I'll encourage them to post to this list, or I'll forward what they give me.

As for my previous message, I did the interoperability report for DKIM so I'm happy to do one for SPF as well, although I'd also be happy to have another volunteer.  :-)

-MSK

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: Friday, February 17, 2012 11:39 AM
To: Murray S. Kucherawy
Cc: Murray S. Kucherawy
Subject: New Version Notification for draft-kucherawy-spfbis-experiment-00.txt

A new version of I-D, draft-kucherawy-spfbis-experiment-00.txt has been successfully submitted by M. Kucherawy and posted to the IETF repository.

Filename:	 draft-kucherawy-spfbis-experiment
Revision:	 00
Title:		 The SPF/Sender-ID Experiment
Creation date:	 2012-02-17
WG ID:		 Individual Submission
Number of pages: 6

Abstract:
   In 2006 the IETF published a suite of protocol documents comprising
   SPF and Sender-ID, two proposed email authentication protocols.
   Because of interoperability concerns and the inability of the working
   group producing them to converge on a single specification, the IESG
   required them to have Experimental status and invited the community
   to observe their deployments for a period of time, hoping convergence
   would be possible later.

   After six years, sufficient experience and evidence have been
   collected that the experiment thus created can be considered
   concluded, and a common path toward can be selected.  This memo
   presents those findings.

The IETF Secretariat

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 17 Feb 2012 18:20:49 -0500

>    to observe their deployments for a period of time, hoping convergence

I know we don't want to reopen old wounds, but I think that it's important to 
get the history correct.

MARID had a single specification.  It was shut down without warning by the IETF 
powers that be without a lot of public discussion in advance.  This issues 
(that I believe) prevented a single result in the MARID working group were not 
technical.

The controversial feature in Sender-ID of using SPF records was not part of 
the MARID design.  It was added by Microsoft in post-MARID drafts.

Scott K

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 17 Feb 2012 15:24:47 -0800

> From: [email-masked] [mailto:[email-masked]] On Behalf Of Scott Kitterman
> Sent: Friday, February 17, 2012 3:21 PM
> To: [email-masked]
> Subject: Re: [spfbis] FW: New Version Notification for draft-kucherawy-spfbis-experiment-00.txt
> 
> MARID had a single specification.  It was shut down without warning by
> the IETF powers that be without a lot of public discussion in advance.
> This issues (that I believe) prevented a single result in the MARID
> working group were not technical.
> 
> The controversial feature in Sender-ID of using SPF records was not
> part of the MARID design.  It was added by Microsoft in post-MARID
> drafts.

I was basing my text on the content of the IESG Note in each of the documents.

In fact, interestingly, the document history of RFC4408 suggests it too was never a working group item, meaning all four of them were individual submissions.  I wasn't part of MARID so I'm learning this all after-the-fact.

Anyway, it looks like you're saying deleting the clause about working group convergence would suffice, correct?

-MSK

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 17 Feb 2012 18:51:11 -0500

On Friday, February 17, 2012 03:24:47 PM Murray S. Kucherawy wrote:
> > From: [email-masked] [mailto:[email-masked]] On Behalf
> > Of Scott Kitterman Sent: Friday, February 17, 2012 3:21 PM
> > To: [email-masked]
> In fact, interestingly, the document history of RFC4408 suggests it too was
> never a working group item, meaning all four of them were individual
> submissions.  I wasn't part of MARID so I'm learning this all
> after-the-fact.
> 

I think that would make it not definitely wrong.  Personally I don't think the 
interoperability concerns were the primary reason for what happened, but 
that's all speculation.  

Scott K

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 17 Feb 2012 15:56:38 -0800

> Sent: Friday, February 17, 2012 3:51 PM
> To: [email-masked]
> > Anyway, it looks like you're saying deleting the clause about working
> > group convergence would suffice, correct?
> 
> I think that would make it not definitely wrong.  Personally I don't
> think the interoperability concerns were the primary reason for what
> happened, but that's all speculation.

Done in my working copy.  Thanks!

-MSK

---

From: Scott Kitterman <[email-masked]>
Date: Sat, 18 Feb 2012 01:25:20 -0500

On Friday, February 17, 2012 11:42:32 AM Murray S. Kucherawy wrote:
> As a straw man document for describing the experiment and its conclusions, I
> offer this to the WG and volunteer to act as editor.  It contains the data
> we've seen on the list so far (thanks, Paul and Phillip!), and I believe is
> an appropriate thing to submit given the posture we've seen from Pete about
> what the IESG would accept as resolution to the experiment.
> 
> I have deliberately avoided populating the Conclusions section on my own. 
> That's for the Working Group to do by consensus.
> 
> I've polled a number of sources both on this list and off for data to
> include in this memo, or whatever other memo we decide to advance.  I'll
> encourage them to post to this list, or I'll forward what they give me.
> 
> As for my previous message, I did the interoperability report for DKIM so
> I'm happy to do one for SPF as well, although I'd also be happy to have
> another volunteer.  :-)
> 
> -MSK
> 
> A new version of I-D, draft-kucherawy-spfbis-experiment-00.txt has been
> successfully submitted by M. Kucherawy and posted to the IETF repository.
> 

I set out to review this document.  Since it's an early draft, I'll give you 
general comments.

Abstract: Already discussed.

Introduction: As I discussed in my comments about the abstract, none of the 
drafts ultimately published related to SPF or Sender ID were MARID working 
group documents.  The Sender ID documents are closest to the MARID documents, 
but significant post-MARID changes were made that affected interoperability.  I 
think it's important that these weren't working group document (the SPF 
documents were developed in public with input handled in a manner very much 
like an IETF working group on spf-discuss and the Sender ID documents were 
developed in private by Microsoft).

I think it's difficult to know what the IESG of the time was concerned with.  
Instead of guessing, I would recomend just copying the IESG note that was 
stamped on the front of all the RFCs without amplification.  That's all we 
really know about what the IESG was concerned about.

Finally, this working group wasn't formed to advance "a" protocol.  It was 
formed to advance SPF.  There's no point in pretending neutrality where it 
doesn't exist.

"Need for convergence": There isn't any convergence going on, so I'm confused 
by the title.  We're advancing SPF.  We aren't folding any Sender ID 
functionality into SPF.  That's out of scope, so I don't think we should be 
talking about convergence.

Evidence of deployment: I think it's worth mentioning that given the paltry 
level of type SPF deployment there are multiple applications that don't even 
bother to query for it by default.  It may be used in the sense that some 
small number of records are published, but it's not used in any way that 
confers interoperability.

Appendix A: Since Hotmail doesn't do SPF checks (nor Sender ID mail from 
checks) I'm somewhat confused about how they could offer about the difference 
between SPF and Sender ID evaluations?  Perhaps they offered data on the 
frequency of Mail From and From being the same?  I suspect this is true for 
some of the other providers as well.  I think we should be careful to call 
data, data and distinguish that from what we are inferring from this.

Scott K

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 17 Feb 2012 23:33:00 -0800

> Sent: Friday, February 17, 2012 10:25 PM
> To: [email-masked]
> Introduction: As I discussed in my comments about the abstract, none of
> the drafts ultimately published related to SPF or Sender ID were MARID
> working group documents.  The Sender ID documents are closest to the
> MARID documents, but significant post-MARID changes were made that
> affected interoperability.  I think it's important that these weren't
> working group document (the SPF documents were developed in public with
> input handled in a manner very much like an IETF working group on spf-
> discuss and the Sender ID documents were developed in private by
> Microsoft).

I've already made that correction in my working copy, at the same time I responded to your comments about the Abstract.

> Instead of guessing, I would recomend just copying the IESG note that
> was stamped on the front of all the RFCs without amplification.  That's
> all we really know about what the IESG was concerned about.

OK, but rather than copying it, I'll simply refer to it with a "for more detail, go there" kind of thing.

> Finally, this working group wasn't formed to advance "a" protocol.  It
> was formed to advance SPF.  There's no point in pretending neutrality
> where it doesn't exist.

This document (or whichever version the WG adopts) is in essence a reply to the IESG Note, which as I read it also (rightly) remained agnostic as to the eventual outcome.  It doesn't need to declare the conclusion up top.  It also doesn't need to repeat the working group's charter, which already says exactly what you're saying.

Essentially with the IESG Note, the IESG of the time posed a challenge.  I think this document is strongest when its reply follows this form or a similar one:

1) This is what we understand the question to be.
2) This is the evidence we have collected since then to answer that question.
3) Here, then, is our answer.

> "Need for convergence": There isn't any convergence going on, so I'm
> confused by the title.  We're advancing SPF.  We aren't folding any
> Sender ID functionality into SPF.  That's out of scope, so I don't
> think we should be talking about convergence.

I'm attempting to respond directly to the IESG Note, which called for "consensus in the future".  If you prefer, I can rename that section to "The Need for Consensus", but it seems to me it says almost exactly the same thing.  Perhaps that's because you're reading it as convergence of technologies, where I mean it as convergence of support or opinion in favour of one technology.

> Evidence of deployment: I think it's worth mentioning that given the
> paltry level of type SPF deployment there are multiple applications
> that don't even bother to query for it by default.  It may be used in
> the sense that some small number of records are published, but it's not
> used in any way that confers interoperability.

I don't think the SPF RR type is germane to the question of which of the two proposals will advance.  That's more a property of the implementation report being discussed on another thread, because it goes to the question of whether or not RFC4408bis should drop that part of the specification and declare the use of the SPF RR type deprecated or historic.

> checks) I'm somewhat confused about how they could offer about the
> difference between SPF and Sender ID evaluations?  Perhaps they offered
> data on the frequency of Mail From and From being the same?  I suspect
> this is true for some of the other providers as well.  I think we
> should be careful to call data, data and distinguish that from what we
> are inferring from this.

Unless I'm mistaken (Paul, please do correct me if I'm wrong), Hotmail provided a statement that their data shows the RFC5322.From domain and the RFC5321.MailFrom domain are different in fewer than 5% of the cases they looked at.  Actual numbers are forthcoming.

It was TDP's data that claimed that for messages where both results are available (129,000 at the moment), they agreed (both pass or both fail) in over 95% of the cases.

-MSK

---

From: Alessandro Vesely <[email-masked]>
Date: Sat, 18 Feb 2012 10:55:50 +0100

On 17/Feb/12 20:42, Murray S. Kucherawy wrote:
> I have deliberately avoided populating the Conclusions section on
> my own.  That's for the Working Group to do by consensus.

Is that going to get merged with
draft-moonesamy-senderid-spf-conclusion or are they bound to stay
separated?

---

From: Scott Kitterman <[email-masked]>
Date: Sat, 18 Feb 2012 09:44:31 -0500

On Friday, February 17, 2012 11:33:00 PM Murray S. Kucherawy wrote:
> This document (or whichever version the WG adopts) is in essence a reply to
> the IESG Note, which as I read it also (rightly) remained agnostic as to
> the eventual outcome.  It doesn't need to declare the conclusion up
> top.  It also doesn't need to repeat the working group's charter, which
> already says exactly what you're saying.
> 
> Essentially with the IESG Note, the IESG of the time posed a challenge.  I
> think this document is strongest when its reply follows this form or a
> similar one:
> 
> 2) This is the evidence we have collected since then to answer that
> question. 3) Here, then, is our answer.

I think it's fair and reasonable to say that a task of the working group is to 
document the results of the IESG experiment.  

If I were writing this, I'd have something very much like an executive summary 
in the introduction, so I'd have the high level result up front, but that's an 
editorial preference.

My major point is that if it's going to say anything about the work of the 
working group, it should say it correctly and the language I referenced 
doesn't do that.

For this document though I'm not sure it needs to say anything about the 
working group beyone "this document is a product of the SPFbis working group".

Scott K