Subject: [spfbis] I-D Action: draft-ietf-spfbis-experiment-05.txt

From: [email-masked]
Date: Thu, 19 Apr 2012 22:55:40 -0700

A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the SPF Update Working Group of the IETF.

	Title           : Resolution of The SPF/Sender-ID Experiment
	Author(s)       : Murray S. Kucherawy
	Filename        : draft-ietf-spfbis-experiment-05.txt
	Pages           : 11
	Date            : 2012-04-19

   In 2006 the IETF published a suite of protocol documents comprising
   SPF and Sender-ID, two proposed email authentication protocols.
   Because of possible interoperability issues, particularly but not
   only those created by simultaneous use of the two protocols by a
   receiver, the IESG was unable to determine technical consensus and
   decided it was best to publish all of RFC4405, RFC4406, RFC4407 and
   RFC4408 as Experimental documents.  The IESG invited the community to
   observe their deployments for a period of time, and expressed hope
   for later convergence of opinion.

   After six years, sufficient experience and evidence have been
   collected that the experiment thus created can be considered
   concluded, and a single protocol can be advanced.  This document
   presents those findings.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-spfbis-experiment-05.txt

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

This Internet-Draft can be retrieved at:
ftp://ftp.ietf.org/internet-drafts/draft-ietf-spfbis-experiment-05.txt

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 20 Apr 2012 05:58:29 +0000

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of [email-masked]
> Sent: Thursday, April 19, 2012 10:56 PM
> To: [email-masked]
> Cc: [email-masked]
> Subject: I-D Action: draft-ietf-spfbis-experiment-05.txt
> 
> A New Internet-Draft is available from the on-line Internet-Drafts
> directories. This draft is a work item of the SPF Update Working Group
> of the IETF.
> 
> [...]

Incorporates co-chair review comments.  No changes to Appendix A yet as discussion is ongoing.

-MSK

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 20 Apr 2012 16:11:14 +0200

> yet as discussion is ongoing.

Why does it have to be an Appendix?

But I hadn't commented on -04, so I take my chance now.

*Abstract*

   and a single protocol can be advanced.

Although "there can be only one" would more closely recall that movie,
I'd also strike "single" from the last paragraph.

*DNS Resource Record Types*

     | TXT replies      |   397,511 | 39.8% |
     | SPF replies      |     6,627 | <0.1% |
     | SPF+TXT replies  |     6,603 | <0.1% |
     | spf2.0/* replies |     5,291 | <0.1% |

Kudos for the tables.  However, confusion may arise because row names
are informal and the number of TXT records that are neither v=spf1 nor
spf2.0 is missing.  I'd suggest the following column names, or
equivalent, where the leading phrase "domains having one or more RRs
such that" is omitted:

     v=spf1 TXT
     any TXT
     any SPF
     (v=spf1 || spf2.0/*) both SPF && TXT
     spf2.0/* (TXT || SPF)
     google-site-verification TXT (for comparison)

Or would it be better to put "TXT", "SPF", "either", or "both" in a
separate column?

In addition, it would help having a few words for characterizing how
the domain names in the two surveys were selected/collected.

*Analysis*

  2.  Of the records retrieved, fewer than 3% requested processing of
      messages using the PRA algorithm, which was an essential part of
      Sender-ID.

This is formally incorrect, as the PRA algorithm can be applied to
v=spf1 records by virtue of the compatibility equivalence of Section
3.4 of RFC 4406.  What we can derive is the percentage of domains
whose admins determined that PRA deserves a different treatment than
MFROM.  That is the percentage of domains that expect there to be a
difference, and corroborates the analysis of differences carried out
in Section 4.

We can also infer, by comparing the orders of magnitude, that a good
deal of domains didn't care about spf2.0 at all, but that's not
related to the PRA specifically.

*Experiences Developing SPF*

I'd change the title to something containing the word "DNS", and
possibly move it to a numbered section (i.e. not an appendix.)

Furthermore, I'd mention the following:

* The format of the SPF RRTYPE was defined to be identical to the
  corresponding TXT.  Therefore, only one of the two advantages
  highlighted by RFC 5507 would have been attained.

* Using the TXT RRTYPE in parallel should have prevented defining an
  SPF RRTYPE in the first place.  [That's according to Section 3.5/1
  of RFC 5507.  What part of RFC 4020 leads to the same conclusion?
  The more DNS references we cite the better, IMHO.]

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 20 Apr 2012 14:32:35 +0000

> From: [email-masked] [mailto:[email-masked]] On Behalf Of Alessandro Vesely
> Sent: Friday, April 20, 2012 7:11 AM
> To: [email-masked]
> Subject: Re: [spfbis] I-D Action: draft-ietf-spfbis-experiment-05.txt
> 

It is orthogonal to the main question being answered.

> I'd also strike "single" from the last paragraph.

Apart from the cult movie reference, why is "single" harmful here?  It's correct.

> spf2.0 is missing.

They also aren't relevant to answering the question of which of the two protocols is in use.

I could add a note saying only valid replies were tabulated, just to be clear.

> I'd suggest the following column names, or

How do those additional details go toward answering the main question?

> the domain names in the two surveys were selected/collected.

The -05 version does say that.

> 3.4 of RFC 4406.  What we can derive is the percentage of domains whose
> admins determined that PRA deserves a different treatment than MFROM.
> That is the percentage of domains that expect there to be a difference,
> and corroborates the analysis of differences carried out in Section 4.

A sender that wants its mail to be subjected specifically to PRA evaluation, and not to SPF, has to use an "spf2.0/*" record.  A sender that is ambivalent about which of the two methods is to be used by receivers can use "v=spf1" and roll the dice.  The only way to state "I want SPF only and not PRA" is to include an "spf2.0/pra ?all" record in the RRset, but we saw only a couple hundred of those across the entire survey.  There's no obvious explanation for that number, because it could well be that word about this tool never made it to most people, or that provisioning tools don't allow for multi-record replies to TXT queries.

So I believe what's cited is correct, though we could be more precise and say "specifically requested" instead of just "requested" if you like.  Relying on the compatibility stuff in RFC4406 doesn't constitute a specific request.

> deal of domains didn't care about spf2.0 at all, but that's not related
> to the PRA specifically.

This is really getting at why the IESG did what it did in 2006.

>   The more DNS references we cite the better, IMHO.]

Comments from others on these, since we're talking a lot about the appendix?  There's an obvious point that 5507 came long after 4408...

-MSK

---

From: Andrew Sullivan <[email-masked]>
Date: Fri, 20 Apr 2012 12:45:24 -0400

Dear colleagues,

No hat.

> are informal and the number of TXT records that are neither v=spf1 nor

Why would anyone care about that?  

>   highlighted by RFC 5507 would have been attained.

Well, if we really want open an immense can of worms, the fact is that
defining two RRTYPEs such that their RDATA has to be the same is by
definition preposterous, at least on the consumer side: there is
absolutely no way to test the requirement.  According to my casual
investigations last night, that was a problem of which dnsext
participants were actuely aware at the time this was being specified,
but there seems to be a general assumption in SPF that DNS entries are
by and large stable -- an assumption that is formally false but
practically true enough to be useful.

I don't think that the document ought to spend a lot of time
investigating all the various ways that the DNS can be abused.  I
think it's a good idea to say, "Here's what happened when we did
this."  I think we are asking for an argument the more we start to try
to explain those historical facts.

>   The more DNS references we cite the better, IMHO.]

I'm not sure what RFC 5507 has to do with this.  Also, I'm not sure
what "the more DNS references we cite" is for here, since I can't see
what (for instance) Dynamic Update or DNAME has to do with any of it.

A

-- 
Andrew Sullivan
[email-masked]

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 20 Apr 2012 19:23:59 +0200

On Fri 20/Apr/2012 18:08:53 +0200 Murray S. Kucherawy wrote:
>> From: ietf.org On Behalf Of Alessandro Vesely
>

I assume you mean the main question being asked.  Doesn't it get less
emphasis than it deserves, that way?

> Apart from the cult movie reference, why is "single" harmful here?

It is correct after the charter assumptions, but it emphasizes them
unnecessarily.  Anyway, it was just an aesthetic consideration.

> They also aren't relevant to answering the question of which of the

No, they aren't.  They just help evaluate the background noise.

> I could add a note saying only valid replies were tabulated, just

Yes please.  If you run a syntax check, or a record selection
according to Section 4.5 of RFC 4408, please say so.

> How do those additional details go toward answering the main question?

The "any TXT" and "google-site-verification TXT" parts are only
relevant for the DNS usage appendix.

> The -05 version does say that.

You don't mean:

   Specifically, these surveys pulled a large number of domain names
   from recent activity logs and queried their nameservers for both
   RRTYPEs that can be used for SPF and/or Sender-ID.

Do you?  It doesn't tell the difference between the two.

> A sender that wants its mail to be subjected specifically to PRA
> evaluation, and not to SPF, has to use an "spf2.0/*" record.
^^^^^^^^^^^^^^^^^^^^^^^^^MFROM?

Ditto reversing MFROM and PRA.

> A sender that is ambivalent about which of the two methods is to be
> used by receivers can use "v=spf1" and roll the dice.  The only way
> to state "I want SPF only and not PRA" is to include an "spf2.0/pra
> ?all" record in the RRset, but we saw only a couple hundred of
> those across the entire survey.  There's no obvious explanation for
> that number, because it could well be that word about this tool
> never made it to most people, or that provisioning tools don't
> allow for multi-record replies to TXT queries.

I see no rational explanation either.  With or without tool, they are
complicated stuff to write and to maintain.  An irrational feeling
could be derived from the tendency to associate MFROM with v=spf1, and
it's easier to feel than to ratiocinate :-/

There are thousands "spf2.0/pra" but only hundreds "spf2.0/mfrom".
Would that mean that PRA is perceived as more important than MFROM by
those who opt for specific treatments?

> So I believe what's cited is correct, though we could be more
> precise and say "specifically requested" instead of just
> "requested" if you like.  Relying on the compatibility stuff in
> RFC4406 doesn't constitute a specific request.

Adding "specifically" makes the sentence correct at least until the
comma.  I'm not sure whether the ability to differentiate the two is
(or was) an essential part of Sender-ID.

> This is really getting at why the IESG did what it did in 2006.

And we'd probably better not making it explicit.  Numbers speak by
themselves.

---

From: Douglas Otis <[email-masked]>
Date: Fri, 20 Apr 2012 10:25:18 -0700

> Incorporates co-chair review comments.  No changes to Appendix A yet as discussion is ongoing.
Dear Murray,

Thank you for your excellent efforts.

Would you consider expanding DNS resource deployment tables to include 
contained macro use?  Some of these are at thresholds used to justify 
deprecation of other items, such as RR types.  This could help shape 
future cruft ridding efforts.

Regards,
Douglas Otis

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 20 Apr 2012 17:35:05 +0000

> Sent: Friday, April 20, 2012 10:24 AM
> To: [email-masked]

Yes, and rightly so.  The document isn't supposed to be a treatise on SPF history.  It's just meant to answer a question.  As it happens we have a lot of information relevant to IETF current practices with respect to RRTYPE allocation, and it seems like the community could learn from the SPF experience if it were to be summarized someplace, especially since it's a recurring topic in the community.  The "as it happens" bit means it shouldn't go in the main body of the document.

> No, they aren't.  They just help evaluate the background noise.

True, but that goes to SPF implementation advice (i.e., "be prepared to process a ton of crap while you're getting DNS answers"), not to the question of adoption rates.

> Yes please.  If you run a syntax check, or a record selection according
> to Section 4.5 of RFC 4408, please say so.

Added such a note for the next version.

> relevant for the DNS usage appendix.

Only if we want the appendix also to discuss arbitrary junk an SPF verifier will have to handle.  That seems to be to be the kind of thing to emphasize in RFC4408bis itself as implementation advice though, not for the experiment document.

> Do you?  It doesn't tell the difference between the two.

Yes, that's what I meant.  That text indicates how the domains were selected and exactly how they were queried.  What's missing?

> >> whose admins determined that PRA deserves a different treatment than MFROM.
> >> That is the percentage of domains that expect there to be a

Why?  If I want MFROM processing only, I can use "spf2.0/mfrom" or "v=spf1".  If I want PRA processing only, I can't say "v=spf1" or I might get MFROM processing from SPF-only verifiers.  So the reverse is not true.

> those who opt for specific treatments?

It would mean there are more requests for PRA processing than MFROM processing directed at verifiers that understand Sender-ID.  It contains no information for SPF-only verifiers.

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 20 Apr 2012 17:36:03 +0000

> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of Douglas Otis
> Sent: Friday, April 20, 2012 10:25 AM
> To: [email-masked]

We have those data in both of the DNS survey tables (which you can access yourself; see previous posts), but they are immaterial for this document.  They can be used when we crack open RFC4408bis if we need to.

-MSK

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 20 Apr 2012 20:13:53 +0200

On Fri 20/Apr/2012 19:52:19 +0200 Murray S. Kucherawy wrote:
> Only if we want the appendix also to discuss arbitrary junk an SPF
> verifier will have to handle.  That seems to be to be the kind of
> thing to emphasize in RFC4408bis itself as implementation advice
> though, not for the experiment document.

The two main issues RFC 5507 brings on why adding a new RRTYPE are the
collision with other uses of TXT, and space considerations in DNS
messages.  The second can be considered a consequence of the first,
and we can provide some numbers on how the first goes in our case.

> Yes, that's what I meant.  That text indicates how the domains were
> selected and exactly how they were queried.  What's missing?

Any clue on how/why they differ from one another?  Anything that may
characterize one sample w.r.t. the other?

> Why?  If I want MFROM processing only, I can use "spf2.0/mfrom" or
> "v=spf1".

No, the compatibility stuff says:

   Sender ID implementations SHOULD interpret the version prefix
   "v=spf1" as equivalent to "spf2.0/mfrom,pra"

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 20 Apr 2012 18:16:42 +0000

> Sent: Friday, April 20, 2012 11:14 AM
> To: [email-masked]
> messages.  The second can be considered a consequence of the first, and
> we can provide some numbers on how the first goes in our case.

We could include a "See RFC5507 for further related discussion" if you like.

> characterize one sample w.r.t. the other?

What's "they"?  I don't know what you're comparing.

>    "v=spf1" as equivalent to "spf2.0/mfrom,pra"

"spf2.0/mfrom,pra" is not MFROM only, nor is it PRA only.

-MSK

---

From: Douglas Otis <[email-masked]>
Date: Fri, 20 Apr 2012 11:25:29 -0700

On 4/20/12 10:23 AM, Alessandro Vesely wrote:
>  I assume you mean the main question being asked. Doesn't it get
>  less emphasis than it deserves, that way?
>
> >> *Abstract*
> >>
> >> Although "there can be only one" would more closely recall that
> >> movie, I'd also strike "single" from the last paragraph.
> >
> >> | TXT replies | 397,511 | 39.8% | | SPF replies |
> >> 6,627 | <0.1% | | SPF+TXT replies | 6,603 | <0.1% | |
> >> spf2.0/* replies | 5,291 | <0.1% |
> >>
> >> Kudos for the tables. However, confusion may arise because row
> >> names are informal and the number of TXT records that are neither
> >> v=spf1 nor spf2.0 is missing.
> >
> >> I'd suggest the following column names, or equivalent, where the
> >> leading phrase "domains having one or more RRs such that" is
> >> omitted:
> >>
> >> v=spf1 TXT any TXT any SPF (v=spf1 || spf2.0/*) both SPF && TXT
> >> spf2.0/* (TXT || SPF) google-site-verification TXT (for
> >> comparison)
> >
> >> In addition, it would help having a few words for characterizing
> >> how the domain names in the two surveys were selected/collected.
> >
> >> 2. Of the records retrieved, fewer than 3% requested processing
> >> of messages using the PRA algorithm, which was an essential part
> >> of Sender-ID.
> >>
> >> This is formally incorrect, as the PRA algorithm can be applied
> >> to v=spf1 records by virtue of the compatibility equivalence of
> >> Section 3.4 of RFC 4406. What we can derive is the percentage of
> >> domains whose admins determined that PRA deserves a different
> >> treatment than MFROM. That is the percentage of domains that
> >> expect there to be a difference, and corroborates the analysis of
> >> differences carried out in Section 4.
> >
Dear Alessandro,

Agree/Disagree.

Agree:
Reasons for using spf2.0 were declared in the IESG note and clarified in 
Section 3.4 of RFC4406 which defaults "v=spf1" as being equivalent to 
"spf2.0/mfrom,pra" unless preempted by an existing "spf2.0/..." record.  
The IESG note created a devious problem for subsequent efforts in 
deciphering publisher intent.  Only when "spf2.0/mfrom" records have 
been published (that many considered either a sacrilege or waste of 
resources) would intent be clear as intent to avoid application of 
Sender-ID.  RFC4406 defaults can lead to misleading results when the 
"spf2.0" version is tallied as evidence of Sender-ID intent when it 
might imply the opposite, and absence of "spf2.0" records doesn't 
exclude Sender-ID intent.  You clarified this again later.  Heads you 
lose, tails I win.

Disagree:
SPF offers no scope and may apply against either MFROM or the HELO.

Regards,
Douglas Otis

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 20 Apr 2012 21:23:28 +0200

> What's "they"?  I don't know what you're comparing.

The two surveys.  They are derived from activity at real sites, not
after statistical sampling methods, hence they are skewed.  I think
that including some random notices about the sites may help guessing
how representative they might be of the global server population.  I
don't know what data may be relevant, though.  Anything that can
depict in a few words what is the real traffic of the sites would
probably do, including the amount of time it took to collect the
sample, the nations or states, mailing lists, monkey butter and
fruits...  anything you'd feel like disclosing.  No?

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 20 Apr 2012 16:34:17 -0400

>    "v=spf1" as equivalent to "spf2.0/mfrom,pra"

There is actually no way to avoid PRA processing due to the broken way Sender 
ID is designed.  If I do:

v=spf1 [some stuff here] -all
spf2.0/pra ?all

That's actually a request for PRA processing.  The so called opt-out isn't 
really an opt-out, it's an opt-in with a hard coded result.  Since use of SPF 
(and Sender ID) results is a matter of local policy, there's no knowing what 
the effect of publishing such a record might be (It could turn results that 
would have been pass to neutral if there was no spf2.0/pra ?all record).

If Microsoft had stuck with the MARID consensus and not reused SPF records for 
Sender ID, this would all be much cleaner now.

I'd recommend we don't dig any deeper than the current text does.  Going 
farther down doesn't make it any prettier.

Scott K

---

From: Douglas Otis <[email-masked]>
Date: Fri, 20 Apr 2012 13:58:39 -0700

>>     "v=spf1" as equivalent to "spf2.0/mfrom,pra"
Dear Scott,

The RFC4406 "equivalence" has an additional requirement: "provided no 
record starting with "spf2.0" exists".
Alessandro's suggestion to note cases where "spf2.0/mfrom" supplant pra 
is appropriate when assessing intent. This sidesteps RFC4406 advise in 
how to formulate Sender-ID avoidance (for the win).

Regards,
Douglas Otis
> farther down doesn't make it any prettier.
Don't ignore obvious attempts depicted by "spf2.0/mfrom" sans pra.

Regards,
Douglas Otis

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 20 Apr 2012 21:28:30 +0000

> Sent: Friday, April 20, 2012 12:23 PM
> To: [email-masked]
> don't know what data may be relevant, though.  Anything that can depict
> in a few words what is the real traffic of the sites would probably do,
> including the amount of time it took to collect the sample, the nations
> or states, mailing lists, monkey butter and fruits...  anything you'd
> feel like disclosing.  No?

I would think it's enough to indicate (as we have) that the domain names were selected from recent mail logs, meaning they reflect domain names of actual mail traffic that was evaluated with SPF and/or Sender-ID.  However, I can add a time range to the second one.  If the people who gave me data for the first and third can respond, I'll do the same for theirs.

If we believe, however, that 90% or more of current mail traffic is spam, I would imagine it doesn't matter who collects the domain names; they will be largely random, with some of the more popular ones (gmail.com, for example) common to all of them.

-MSK

---

From: Paul Midgen <[email-masked]>
Date: Fri, 20 Apr 2012 14:31:00 -0700

(note: my email address is no longer at microsoft.com, this is because 
i've left the company, however for the purpose of this discussion i'll 
base my comments on my experience at hotmail)

Hi Murray,

Thanks for shepherding this, by the end you'll have earned some kind of 
medal for valor. Having contributed the Hotmail-based data to this 
effort, given that it is in general agreement with the data collected by 
others, I think the document is drawing the right high-level conclusions.

I would like to say for the record that Hotmail's decision to support 
SPF was driven by the following conclusions based on stable observations 
of hundreds of billions of inbound messages from millions of domains:

    1. for nearly 80% of messages the final authentication state (after
    spf, sender-id, dkim, local policy, etc.) was identical when
    evaluating spf and sender-id for the same message using the same record.

    2. domains publishing both spf2.0/pra and spf1 records almost always
    "said" exactly the same things with those records.

    3. the cases where PRA offered some advantage over MFROM (e.g.
    forwarders, MLMs), where "advantage" means "more likely to generate
    a pass/fail authentication result" are easily handled in local
    policy by making the decision to look at 5322.From and Sender on the
    basis of information external to SPF, such as IP and domain reputation.

It's worth noting that #3 does in fact say that, for Hotmail, the 
experience of using PRA was valuable and in the end our email 
authentication system will be stronger for it. That learning does not 
include a statement about the security and/or rightness of said 
practice, and therefore doesn't argue for or support modification of 
SPFbis to include PRA, SUBMITTER, etc. The point is that for a majority 
of cases SPF will perform its intended function, and when it doesn't we 
can compensate in our local policy system in a manner that preserves the 
integrity of SPF.

I noted no mention of message volume anywhere in the document, if there 
continues to be debate about DNS and connection log data being 
sufficient to prove the necessary pointswe may consider overlaying the 
existing studies with volume information from a couple of receivers 
ranging from small to large to see if the evidence remains consistent.

 From section 6:
>     2.  The absence of significant adoption of the RRTYPE 99 DNS Resource
>         Record suggests that it has not attracted enough support to be
>         useful.
I'd suggest that given the design of DNS, the point here is not about 
usefulness of RRTYPE 99, rather that it "has not attracted enough 
support to be mandated for SPFbis compliance" or a statement similar to 
that.

>     3.  The absence of significant adoption of the [SUBMITTER] extension,
>         [SENDER-ID], and [PRA], indicates that there is not a strong
>         community prepared to develop those mechanisms beyond
>         experimental status.
There is clearly a community of *some* size prepared to develop these 
things, but I wonder what the point would be. Why advance two standards 
when one is clearly good enough to start with && can be deliberately 
evolved over time if it proven lacking.

It isn't that Sender-ID lacks proponents or has zero value, it's that 
Sender-ID's deployment and use will never approach that of SPF, the two 
have essentially the same result (in observed practice, not strictly 
hypothetically/on paper), advancing SPFbis to standard status 
establishes a starting point to evolve from, if a proven need emerges.

-p

On 20-Apr/12 11:16 AM, Murray S. Kucherawy wrote:
> _______________________________________________
> spfbis mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/spfbis

---

From: Hector Santos <[email-masked]>
Date: Sat, 21 Apr 2012 07:34:22 -0400

Hi Paul,

I would like to express a POV below, I believe may provide some 
insights to help produce the issues we are dealing with today. see 
inline comments.

Paul Midgen wrote:

>    evaluating spf and sender-id for the same message using the same record.

This is same observation experienced over 80% or more. It was expected 
the PRA would only be useful for certain use cases.  I believe I was 
the first to highlight the high equality, indicating our stats ~84%, 
thus sparking the submitter proposal as an optimizer. But when I did 
the scanning of mail archives of already accepted mail, including 
spam, it was only possible to get the percentage when they did not 
match but not possible to get a payoff value given the 2005 dearth of 
published SPF records. You can only pick and choose by domains you 
knew had records.  In other words, the percentage was clear, but not 
possible to see if it meant anything without having SPF1 and/or SPF2 
records.

>    "said" exactly the same things with those records.

I suspect this only shows that a deployment supported SenderID however 
not have the network use cases that required a different 
IP::PRA.DOMAIN relationship.  I also suggest without a more detail 
analysis of paths and whether their clients supported SUBMITTER, its 
hard to tell what the receiver will be doing here.  If the receiver 
support SUBMITTER, then extracting the SPF2.0/PRA sufficed.  If the 
receiver did not support submitter, then the SPF1.0 record was 
available.  So its pretty much attempt to cover all bases. Its the 
same with DKIM, if you sign only with SHA256, you might find older 
DKIM receivers not supporting SHA256.  But if you sign with SHA1, you 
guaranteed the widest coverage.  Some sign with both to get the 
highest strength and the widest coverage.

Just a interesting file I just found. I have an old June 2005 AOL.COM 
file capture of their SPF/SPF2 records, with the following:

spf2.0/pra
    ip4:152.163.225.0/24
    ip4:205.188.139.0/24
    ip4:205.188.144.0/24
    ip4:205.188.156.0/23
    ip4:205.188.159.0/24
    ip4:64.12.136.0/23
    ip4:64.12.138.0/24
    ptr:mx.aol.com ?all

v=spf1
    ip4:152.163.225.0/24
    ip4:205.188.139.0/24
    ip4:205.188.144.0/24
    ip4:205.188.156.0/23
    ip4:205.188.159.0/24
    ip4:64.12.136.0/23
    ip4:64.12.138.0/24
    ptr:mx.aol.com ?all"

The same records, compared to today:

     spf2.0/pra ptr:mx.aol.com ?all
     v=spf1 ptr:mx.aol.com ?all

Reduced but the same.  Now whether the logistic of this has some merit 
I can't tell, but I do know one the main concerns I had with all this 
new DNS related query stuff was the waste and redundancy in the lookup 
with neutral policy results.

> can compensate in our local policy system in a manner that preserves the 

See my comment below regarding a very important consideration that was 
on the table.

> establishes a starting point to evolve from, if a proven need emerges.

There were many who question PRA in the SPF camp, after all, plain and 
simple, you only needed the SMTP envelope domains to apply SPF.  The 
concept of a payload based 3rd identity called PRA was the monkey wrench.

To people who were interested in the integrated solutions that not 
only covered SMTP but the highly subjective content analysis methods 
with heuristic ideas and promising 3rd party trusted service 
reputation services, was all part of the strategic views people presented.

And Paul, that included the high possibility that end users, our 
ultimate customers and one we wish to protect with lower exposed risk, 
did not have a backend host that was yet offering or up to par with 
all the new email technology with backend filtering or separating of 
good vs bad streams.

So as the RFC4406 (SenderID) describes in its introduction:

    This document describes a mechanism such that receiving Mail Transfer
    Agents (MTAs), Mail Delivery Agents (MDAs), and/or Mail User Agents
    (MUAs) can recognize mail in the above category and take appropriate
    action.  For example, an MTA might refuse to accept a message, an MDA
    might discard a message rather than placing it into a mailbox, and an
    MUA might render that message in some distinctive fashion.

It addresses all components within a total mail system including the 
compliant backend MTA/MDA dealing with problematic messages we were 
concerned with or non-compliant backend and the possibility the MUA 
could fill that niche or gap by using the protocols or perhaps use the 
results of a MARK ONLY backend to help the users.  The same issue 
carried over to DKIM as well where the backend was not yet ready or 
compliant with DKIM verification.,    In fact, in the MUA area, there 
were some some 3rd party add-ons, I recall an  ActiveX component for 
Outlook that marketed SenderID support and I personally recall a 
ThunderBird XUL plug-in that also did DKIM verification.   Related to 
the idea that a backend server may not yet be ready or is still 
exploring deployment, I wrote an I-D that offered a Partial Support 
idea to help with the migration and time-shifted delay problem of 
users getting unverified expired signed mail.

        http://tools.ietf.org/html/draft-santos-dkim-rcvd-00

Finally, just consider the POV of the total integrated aspect, we have 
five to six domains on DNS "lookup" table to evaluate either separated 
or in some optimized combined together:

    - 5321.EHLO     (SPF)
    - 5321.MAILFROM (SPF
    - 5321.SUBMITTER (SIDF)
    - 5322.PRA      (SIDF)
    - 5322.FROM     (DKEYS/POLICY support or DKIM/ADSP/ATPS signature 
protection policies)
    - 5322.DKIM.D   (DKIM Signer ID)

This is why I sympathize with Mr. Levine long time belief that a 
trusted signature is all that is needed and it help trumps many of the 
known practical transport ills and hiccups known to exist.  Its an 
great idea, but unfortunately its only for filtering good mail, can 
only work consistently and persistent across the board with everyone 
using the same 3rd party trust or reputation "Batteries" and does 
nothing to address the faults of a transaction with/or a faulty 
signature and the #1 problem of legacy abuse on domains and receivers 
continues.

Does PRA play a role somewhere?  At the backend or the MUA? I don't a 
feel for it, but we haven't really shown it does not.  We only 
repeated what was already known, low volume on per site average, 
specific use cases and I personally do not believe this justifies 
getting rid of it.

-- 
Hector Santos, CTO
http://www.santronics.com
http://hector.wildcatblog.com

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sun, 22 Apr 2012 06:13:42 +0000

Hi Paul, comments inline.

From: [email-masked] [mailto:[email-masked]] On Behalf Of Paul Midgen
Sent: Friday, April 20, 2012 2:31 PM
To: [email-masked]
Subject: Re: [spfbis] I-D Action: draft-ietf-spfbis-experiment-05.txt

1. for nearly 80% of messages the final authentication state (after spf, sender-id, dkim, local policy, etc.) was identical when evaluating spf and sender-id for the same message using the same record.

[MSK: Thanks, I'll add that alongside the existing data in the document.]

2. domains publishing both spf2.0/pra and spf1 records almost always "said" exactly the same things with those records.

[MSK: Do people find this an interesting point to make?  I'm not sure it solidifies our conclusions further.]

3. the cases where PRA offered some advantage over MFROM (e.g. forwarders, MLMs), where "advantage" means "more likely to generate a pass/fail authentication result" are easily handled in local policy by making the decision to look at 5322.From and Sender on the basis of information external to SPF, such as IP and domain reputation.

It's worth noting that #3 does in fact say that, for Hotmail, the experience of using PRA was valuable and in the end our email authentication system will be stronger for it. That learning does not include a statement about the security and/or rightness of said practice, and therefore doesn't argue for or support modification of SPFbis to include PRA, SUBMITTER, etc. The point is that for a majority of cases SPF will perform its intended function, and when it doesn't we can compensate in our local policy system in a manner that preserves the integrity of SPF.

[MSK: The work of the experiment document, at least, doesn't advocate for requiring anyone to turn off Sender-ID (or in Hotmail's case, one particular piece of the PRA algorithm).  It's merely an effort to determine and compare the uptake of both protocols and select one to promote to and along the Standards Track.  So while this is interesting, and possibly input for an "SPF experience" document or section in terms of later work, I don't think we should go into it here.]

I noted no mention of message volume anywhere in the document, if there continues to be debate about DNS and connection log data being sufficient to prove the necessary points we may consider overlaying the existing studies with volume information from a couple of receivers ranging from small to large to see if the evidence remains consistent.

[MSK: Certainly more data are welcome, although I expect as you've found that those data will further corroborate what we have.  We can go do that if asked.]

       useful.
I'd suggest that given the design of DNS, the point here is not about usefulness of RRTYPE 99, rather that it "has not attracted enough support to be mandated for SPFbis compliance" or a statement similar to that.

[MSK: I think "useful" as it's used here means use in terms of a standards track version of RFC4408.  Including the type 99 stuff there will only continue to confuse matters.  In the long view I think a dedicated RR type would've been a good thing to have from the start, but various factors made that not happen; see Appendix A (which may itself evolve as the Working Group Last Call progress).]

       experimental status.
There is clearly a community of *some* size prepared to develop these things, but I wonder what the point would be. Why advance two standards when one is clearly good enough to start with && can be deliberately evolved over time if it proven lacking.

[MSK: If there is any kind of community that still wants to do the work to advance Sender-ID in any way, it has yet to appear.  I think there exists a handful of people that do, but they aren't writing documents or having the conversations that make these things happen.  The work isn't there.  But it is for SPF.]

It isn't that Sender-ID lacks proponents or has zero value, it's that Sender-ID's deployment and use will never approach that of SPF, the two have essentially the same result (in observed practice, not strictly hypothetically/on paper), advancing SPFbis to standard status establishes a starting point to evolve from, if a proven need emerges.

[MSK: Precisely.]

-p

-----Original Message-----

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]] On Behalf Of Alessandro Vesely

To: [email-masked]<mailto:[email-masked]>

You don't mean:

"v=spf1".

-MSK

spfbis mailing list

[email-masked]<mailto:[email-masked]>

---

From: Paul Midgen <[email-masked]>
Date: Sun, 22 Apr 2012 02:15:06 -0700

Just a couple of comments-on-comments:

> it solidifies our conclusions further.]
I don't actually suggest adding text regarding this, I just pointed it 
out as background.
> forwarders, MLMs), where "advantage" means "more likely to generate a 
> pass/fail authentication result" are easily handled in local policy by 
> making the decision to look at 5322.From and Sender on the basis of 
> information external to SPF, such as IP and domain reputation.
>
>
> SPFbis to include PRA, SUBMITTER, etc. The point is that for a 
> majority of cases SPF will perform its intended function, and when it 
> doesn't we can compensate in our local policy system in a manner that 
> preserves the integrity of SPF.
>
> [MSK: The work of the experiment document, at least, doesn't advocate 
> for requiring anyone to turn off Sender-ID (or in Hotmail's case, one 
> particular piece of the PRA algorithm).  It's merely an effort to 
> determine and compare the uptake of both protocols and select one to 
> promote to and along the Standards Track.  So while this is 
> interesting, and possibly input for an "SPF experience" document or 
> section in terms of later work, I don't think we should go into it here.]
>
>
Agreed.