Subject: [spfbis] What unused means

From: "John Levine" <[email-masked]>
Date: 19 Jul 2012 16:29:55 -0000

We sure are talking past each other here.

As far as I can tell, the main disagreement here is what it means for
a feature to be unused.  Does it mean that nobody uses at all
anywhere, or does it mean that the benefit to the small set of people
who use it is outweighed by the cost of keeping it in for everyone
else.

If you look at the history of mail standards, it's quite clear that
the working definition is the latter one.  If it were the former, we'd
still be insisting that SMTP servers support SOML and source routes.

With respect to localpart macros, beyond the problem that they're ill
specified now, they're a time bomb.  If EAI gets significant uptake,
which I think is fairly likely in view of the level of interest from
Chinese ISPs, we're stuck because there is no way to make them work
with EAI -- by design you cannot turn an EAI localpart into a domain
name.

So the world will thank us if we get rid of them now rather than later.

R's,
John

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 19 Jul 2012 12:40:28 -0400

On Thursday, July 19, 2012 04:29:55 PM John Levine wrote:
...
> with EAI -- by design you cannot turn an EAI localpart into a domain
...
Not quite right.  Some you can.  Some you can't.  Saying something like "You 
can only use EAI local-parts that convert to a legal domain name in local-part 
macros" means that senders have a choice.  This is, admittedly, a more complex 
solution than removal (but then that's true of everything).  It would also 
lead to, if EIA takes off, local-part macros becoming less useful over time.  I 
don't see that as a problem (making it easy to remove later if we deprecate it 
now).

While I agree that EAI is one of the issues around local-part macros, I don't 
think it's an issue that can only be resolved through removal.

Scott K

---

From: Andrew Sullivan <[email-masked]>
Date: Thu, 19 Jul 2012 12:52:10 -0400

Mostly no hat.  See inline for exception.

> still be insisting that SMTP servers support SOML and source routes.

And if you look at the history of DNS, the working definition is very
nearly the former.  This is even more true in deployed practice than
it is in the specifications.  Since SPF interacts with both of these
systems, appeal to the traditions in one or the other isn't going to
help us.  And as I've already argued, it seems to me that if one wants
to make changes to the protocol for little-used features and ones we
think are too complicated, the charter should _say that_ instead of
promising something else. 

If I put my hat on for a moment, either SM or I would have to shepherd
this document.  We would have to explain to the IESG that, yes, our
charter said we wouldn't remove any features we knew to be in use, and
yes, we had evidence that these features were sometimes, only rarely,
in use by some people; but we decided to ditch them anyway, because we
didn't care about those uses and thought they were dumb and decided
not to count those people because there were too few of them.  As a
simple process matter, that's a problem.  (Hat off again.)

> specified now, they're a time bomb.
[…]
> with EAI -- by design you cannot turn an EAI localpart into a domain

First, the latter claim is flat-out false.  DNS labels are octets,
period.  We could put anything we wanted in there.  We could also
specify ways of processing SPF records so that they process such UTF-8
data. 

One might ask why IDNA doesn't use UTF-8.  That's because a large
amount of the deployed software out there -- consuming software, not
the serving software -- can't handle anything outside the ASCII range,
and we didn't want the answer for "deploy IDNs" to depend on the step,
"Upgrade the Internet."

But for the very same reasons EAI decided that the mail case was
different to the DNS case, because in email the people affected are
already interested parties, we could decide that supporting these
rules using UTF-8 in the DNS is acceptable in a way that it wouldn't
have been for IDNs.

More importantly, however, if the document says that localpart macros
are deprecated, then why doesn't that defuse the time bomb anyway?  We
maximise interoperability with actually deployed software on the
Internet today, and yet tell people that this particular feature isn't
likely to work forever.  I would be perfectly comfortable if the draft
included words about how the feature is probably completely broken in
the face of some software that might, someday, be widely deployed on
the Internet.  (I also note that your argument is that we should break
interoperation with existing systems known to have been deployed for
some time in order that we maybe won't have a problem with software
that maybe will be deployed in the future.  I think that has the
burden of proof wrong.)

I've said enough on this topic that I won't post on it again without a
hat.

Best,

A

-- 
Andrew Sullivan
[email-masked]

---

From: "John Levine" <[email-masked]>
Date: 19 Jul 2012 20:43:03 -0000

>are deprecated, then why doesn't that defuse the time bomb anyway?

That doesn't address the security issues.  I think you're schrod with
the IESG either way.

R's,
John

PS: If someone suggests that we should tidy up localpart macros to
tell servers to defend against localparts with spaces and backslashes
and ampersands, we will of course jump up and down and demand that you
prove that nobody uses addresses with those characters.

PPS: With respect to DNS history, it's not all upward compatible.  The
restriction of host names to ldh probably broke some names with other
punctuation.

---

From: Dave Crocker <[email-masked]>
Date: Thu, 19 Jul 2012 15:08:48 -0700

Andrew,

On 7/19/2012 9:52 AM, Andrew Sullivan wrote:
> nearly the former.

For example, SMTP clients are still required to process the earlier 
alternatives to the DNS' MX RR?

Also, are you saying that the incremental, Internet-wide/systemic cost 
of that retention is comparable to support for SPF macros?

> This is even more true in deployed practice than
> it is in the specifications.

An issue with something like macros is that it can't be ignored.  It 
affects the core, receiver-side SPF engine.  This makes macros 
fundamentally more expensive, system-wide, than an RR feature falling 
out of use.  If a receiver plays SPF, it /must/ support that component 
of it.

> appeal to the traditions in one or the other isn't going to
> help us.

The point being made was not about "tradition" but of an interpretation 
for the word "unused".

The example provided demonstrates an existence proof for deciding that 
it does /not/ require using a definition employing mathematical zero.

Some of us are used to this looser definition; indeed it has a long 
history in the IETF.  That some of us choose the tighter definition 
justifies a debate between the two positions, rather than automatically 
requiring that the tighter definition apply to the SPFbis effort.

That someone at some time might have required mathematical zero usage 
for some specification is not the point.  The point is that there is 
plenty of IETF precedence for permitting the definition to be "great 
than zero".

Also, there is a difference between determining whether a feature has 
gained traction, versus deciding how to handle ones that have not. 
Debating the definition of 'unused' is the former, not the latter.

> period.  We could put anything we wanted in there.

Hostnames are not pure octets.

> We could also

Now you appear to be describing a significant enhancement to the SPF 
specification and it is in anticipation of established practice for 
dealing with UTF-8 in domain names, rather than after the practice is 
established.

> and we didn't want the answer for "deploy IDNs" to depend on the step,

It's probably better not to pursue the topic of why (or even what) 
things have been done in the IDN or EAI space.  That space has a very 
long and very messy history -- up through the present -- and touching 
that tar baby is very likely to get us stuck in its muck.

> rules using UTF-8 in the DNS is acceptable in a way that it wouldn't

And how exactly does this fit within the current working group charter?

> are deprecated, then why doesn't that defuse the time bomb anyway?

It retains support for macros.  There is cost in that retention, 
including an argument for later problems.  It also invites interaction 
between macros and any emerging EAI usage.

> We
> likely to work forever.

Because it is going to be easier to drop it when moving from Proposed to 
Full than it is now, going from Experimental to Proposed?

>  I would be perfectly comfortable if the draft
> the Internet.

Offhand, I'd expect text like that in a specification to be considered 
rather odd, at best.

d/

-- 
  Dave Crocker
  Brandenburg InternetWorking
  bbiw.net

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 20 Jul 2012 17:55:47 +0200

Please let mi chime in.  As Andrew said he'll reply on this subject
wearing his hat, I'd like that to be a successful call for consensus.

> out of use.  If a receiver plays SPF, it /must/ support that component

As Scott said, a receiver who plays SPFbis MAY want to omit SPF
checking for that particular record.  If the claim that those
dangerous macros are unused is correct, that is a holdable choice for
new or upgraded implementations.  OTOH, older software and records can
continue to work undisturbed and still be compliant, which is a good
interpretation of "backward compatibility".

>> […]
>> We could also specify ways of processing SPF records so that they
>> process such UTF-8 data.
> 

EAI is now a proposed standard, so it would be an error not to
consider the possibility of UTF-8 names.  draft-ellermann-spf-eai was
first written in 2008, so this is not exactly a new feature.  OTOH,
removing local-part macros will not disallow UTF-8 in domain names.

> And how exactly does this fit within the current working group charter?

Could we pass it as an error correction?  It does look like a due
upgrade to me.

> including an argument for later problems.

See above for verifiers that omit checking in case a record contains
unused features.  Of course, we cannot force existing software to
upgrade immediately.

> It also invites interaction between macros and any emerging EAI
> usage.

For some of Dave's reasons that I didn't quote, I'd avoid linking
macro removal to EAI issues.  Sanitizing labels before looking them up
has to be done also for US-ASCII names.

>> We maximise interoperability with actually deployed software on
>> the Internet today, and yet tell people that this particular
>> feature isn't likely to work forever.

Moreover, deprecated features may gradually stop working as upgraded
verifiers drop support of macros.  Developers may choose between
security costs and support for unused features.  That's the essence of
the compromise that Scott proposed[1&2].  Like him, I'm not enthusiast
of this solution, but that's the nature of compromises.

[1] http://www.ietf.org/mail-archive/web/spfbis/current/msg01922.html
[2] http://www.ietf.org/mail-archive/web/spfbis/current/msg01940.html

> Because it is going to be easier to drop it when moving from Proposed
> to Full than it is now, going from Experimental to Proposed?

Hm... will that be in 2021?  And if that will use some SPF3 RRTYPE,
possibly binary encoded, it will have to be recycled at Proposed.
That's OT here, except for considering that 8 years from 4408 to
4408bis are such a long time already that giving up this charter
because we fear the cost of Chinese macro expansion is probably not
the best service we can do to the IETF :-/

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 20 Jul 2012 13:05:43 -0400

> interpretation of "backward compatibility".

Note: I didn't say I thought this was a good idea.  I don't.  I merely think 
it's better than partial implementations of the protocol issuing errors for 
things that are valid, but they've chosen not to implement.

Scott K

---

From: Hector Santos <[email-masked]>
Date: Fri, 20 Jul 2012 13:52:55 -0400

> things that are valid, but they've chosen not to implement.

If I read this correct, then this a broken protocol by BIS design 
which is not what BIS is suppose to be about.

IMTO, we should spit the issues up.  PTR vs %p are two different 
things. PTR is used, %p is rare.  Alternatively, if the ITCH is that 
big to have deprecated, removed, obsolete features, then do a SPF3.0 
but do it properly so there is logic backward and upward path for both:

    v=spf1  continues supporting PTR, %p
    v=spf3  does not support PTR, %p and whatever else you wish to strip.

You're text describing v=spfX can simply note what 3.0 does not offers 
and this allows new developers to decide.

But an graceful error will be needed for a v=spf3 receiver seeing an 
PTR, which to me, means "I don't support SPF" for the session.  But of 
the v=spf3 receiver wants to support v=spf1 features, he could - it 
would have a wider coverage.

The current v=spf1 receiver can now take his sweet time to add support 
for v=spf3 which in this case, itn't much because it already supports 
PTR.

If other good things are added, this also gives SPF3.0 some payoff and 
incentive for other to support this version.

Nothing breaks which is what is bring proposed now.

-- 
HLS

---

From: S Moonesamy <[email-masked]>
Date: Sat, 21 Jul 2012 03:50:33 -0700

Hello,

Here is a rough summary of this thread.

John Levine mentioned that "the main disagreement here is what it 
means for a feature to be unused.  Does it mean that nobody uses at 
all anywhere, or does it mean that the benefit to the small set of 
people who use it is outweighed by the cost of keeping it in for 
everyone else.  If you look at the history of mail standards, it's 
quite clear that the working definition is the latter one" [1].

Andrew Sullivan commented that "and if you look at the history of 
DNS, the working definition is very nearly the former.  This is even 
more true in deployed practice than it is in the 
specifications.  Since SPF interacts with both of these systems, 
appeal to the traditions in one or the other isn't going to help us" [2].

Dave Crocker mentioned that 'The point being made was not about 
"tradition" but of an interpretation for the word "unused".  The 
example provided demonstrates an existence proof for deciding that it 
does /not/ require using a definition employing mathematical 
zero.  Some of us are used to this looser definition; indeed it has a 
long history in the IETF. That some of us choose the tighter 
definition justifies a debate between the two positions, rather than 
automatically requiring that the tighter definition apply to the 
SPFbis effort.  That someone at some time might have required 
mathematical zero usage for some specification is not the point. The 
point is that there is plenty of IETF precedence for permitting the 
definition to be "great than zero".  Also, there is a difference 
between determining whether a feature has gained traction, versus 
deciding how to handle ones that have not. Debating the definition of 
'unused' is the former, not the latter' [3].

Alessandro Vesely commented that "developers may choose between 
security costs and support for unused features" [4].

My comment is that the problem being discussed is the following in 
the SPFBIS charter:

   "Changes to the SPF specification will be limited to the correction
    of errors, removal of unused features, addition of any enhancements
    that have already gained widespread support, and addition of
    clarifying language."

There is a message [5] and some comments on other threads about 
"deprecated" not being the same as "removal".  The bone of contention 
is the "removal of unused features" in the paragraph quoted 
above.  So what's my opinion about the meaning of "unused"?  I would 
go for simple; it's not used.    You would ask me whether it is 
zero.  I would say that it is unlikely that it is zero.

My opinion about the meaning of the word does not matter.  What 
matters is that the WG may decide to remove unused features and the 
WG Chairs might say that's not what it was chartered to do.  In my 
opinion it is not worth going there if there may be a way to resolve 
the problem.  I don't know what that way is.  I would not use any of 
the above arguments, whether for or against, in an explanation to an 
Area Director or an IETF participant during the Last Call.  This is 
more about spending less energy to achieve the same outcome.

Regards,
S. Moonesamy

1. http://www.ietf.org/mail-archive/web/spfbis/current/msg01946.html
2. http://www.ietf.org/mail-archive/web/spfbis/current/msg01948.html
3. http://www.ietf.org/mail-archive/web/spfbis/current/msg01952.html
4. http://www.ietf.org/mail-archive/web/spfbis/current/msg01953.html
5. http://www.ietf.org/mail-archive/web/spfbis/current/msg01874.html

---

From: Alessandro Vesely <[email-masked]>
Date: Sat, 21 Jul 2012 13:28:16 +0200

On Fri 20/Jul/2012 19:52:55 +0200 Hector Santos wrote:
>>> As Scott said, a receiver who plays SPFbis MAY want to omit
>>> SPF checking for that particular record.  If the claim that
>>> those dangerous macros are unused is correct, that is a
>>> holdable choice for new or upgraded implementations.  OTOH,
>>> older software and records can continue to work undisturbed and
>>> still be compliant, which is a good interpretation of "backward
>>> compatibility".
>>
>> Note: I didn't say I thought this was a good idea.  I don't.  I
>> merely think it's better than partial implementations of the
>> protocol issuing errors for things that are valid, but they've
>> chosen not to implement.

draft-schlitt-spf-classic-00 discouraged PTR in 2004, so dropping it
in 2012 shouldn't come as much of a surprise.

> things. PTR is used, %p is rare.

I'm unable to guess the meaning of that "T" in "IMTO".  True?  Tough?
 Typical?  Tergiversatory?

> Alternatively, if the ITCH is that big to have deprecated, removed,
> obsolete features, then do a SPF3.0 but do it properly so there is
> logic backward and upward path for both:
> 

I don't see how this would help anything, since the publisher doesn't
have a say about what SPF version, if any, receivers apply.  Let's
consider Stuart's example: "v=spf1 ?ptr:isp.com -all".  That client
wouldn't know what to write in a v=spf3 record, so I'm unable to guess
what advantages might result from a version change.

More on that example:

On Thu 19/Jul/2012 06:49:48 +0200 Stuart Gathman wrote:
> the ISP provides no SPF record of its own, but the smtp servers all
> end in "isp.com".

IMHO, the trick lays in the problem statement:  How can one know that
all their smtp servers end in "isp.com", given that the most obvious
place to make such a statement would be an SPF record?  Obviously,
ISPs can change host names without notice.

How meaningful is that name suffix?  RFC 5451 formalizes an "iprev"
authentication method, which is similar to the PTR mechanism, but
easier to pass, as it doesn't take such name.  A receiver who
implements both methods could mark a message from that client like so:

   Authentication-Results: SPFbis.example;
      iprev=pass policy.iprev=192.0.2.30;
      spf=neutral smtp.mailfrom=client-of-isp.com
   Received-SPF: neutral mechanism=ptr client-ip=192.0.2.30

I note that the speculated "isp.com" string is not reported.  In case
the message is abusive, I wouldn't know where to send a complaint.

RFC 5451 doesn't use the term "discouraged", but says:

 There is some contention regarding the wisdom and reliability of this
 test.  For example, in some regions it can be difficult for this test
 ever to pass because the practice of arranging to match the forward
 and reverse DNS is infrequently observed.  Therefore, the actual
 implementation details of how a verifier performs an "iprev" test are
 not specified here.  The verifier MAY report a successful or failed
 "iprev" test at its discretion having done some kind of check of the
 validity of the connection's identity using DNS.  It is incumbent
 upon an agent making use of the reported "iprev" result to understand
 what exactly that particular verifier is attempting to report.

 Extensive discussion of reverse DNS mapping and its implications can
 be found in [DNSOP-REVERSE].  In particular, it recommends that
 applications avoid using this test as a means of authentication or
 security.  Its presence in this memo is not an endorsement, but is
 merely acknowledgement that the method remains common and provides
 the means to relay the results of that test.

NB: [DNSOP-REVERSE] is the "blancmange" document that Andrew referred
in a parallel thread a couple of days ago.  An interesting reading,
rich of historical details, that document is alas also an example of a
topic which can yield not-published-as-an-RFC.  There's plenty of
different topics to discuss in 4408bis.

[DNSOP-REVERSE]
http://tools.ietf.org/html/draft-ietf-dnsop-reverse-mapping-considerations-06

---

From: Hector Santos <[email-masked]>
Date: Sat, 21 Jul 2012 10:43:52 -0400

And Hector stated PTR is in heavy use, by his measures of 16-19%.  It 
can posted if so desired.  See inline.

S Moonesamy wrote:
> John Levine mentioned that "the main disagreement here is what it means 
> for a feature to be unused.  Does it mean that nobody uses at all 
> else.  If you look at the history of mail standards, it's quite clear 
> that the working definition is the latter one" [1].

It is not quit clear and if this was the case in history, you would 
gave major anti-trust issues, legal issues and what would be quite 
clear, no one would trust standards any more if businesses were 
beholding of technocrats making decisions that can cause major chaos.

The devil is in the details but it was never as quite clear as it been 
stated.

> Andrew Sullivan commented that "and if you look at the history of DNS, 
> the working definition is very nearly the former.  This is even more 
> true in deployed practice than it is in the specifications.  Since SPF 
> interacts with both of these systems, appeal to the traditions in one or 
> the other isn't going to help us" [2].

> "tradition" but of an interpretation for the word "unused".  The example 
> provided demonstrates an existence proof for deciding that it does /not/ 
> require using a definition employing mathematical zero.  

Only one problem - PTR is in heavy used.

> Alessandro Vesely commented that "developers may choose between security 
> costs and support for unused features" [4].

Business also decide on strategic plans and operation and using a 
standard is one of them. PTR is in used.  It is not a unused feature.

> My comment is that the problem being discussed is the following in the 
> SPFBIS charter:
> 
> "deprecated" not being the same as "removal".  The bone of contention is 
> the "removal of unused features" in the paragraph quoted above.  So 
> what's my opinion about the meaning of "unused"?  I would go for simple; 
> it's not used.    You would ask me whether it is zero.  I would say that 
> it is unlikely that it is zero.

This is unethical for attempting to change the charter that was 
altered in the beginning for these these very concerns.  You can not 
change the rules of the games in order to get what you want.

Besides PTR is in used.

> My opinion about the meaning of the word does not matter.  What matters 
> is that the WG may decide to remove unused features and the WG Chairs 
> might say that's not what it was chartered to do.  In my opinion it is 
> not worth going there if there may be a way to resolve the problem.  I 
> don't know what that way is.  I would not use any of the above 
> arguments, whether for or against, in an explanation to an Area Director 
> or an IETF participant during the Last Call.  This is more about 
> spending less energy to achieve the same outcome.

Surreal.

-- 
HLS

---

From: Hector Santos <[email-masked]>
Date: Sat, 21 Jul 2012 11:21:51 -0400

> in 2012 shouldn't come as much of a surprise.

If that was the case, then aren't you surprise that is still in use 
and you still have wizards creating them?  OpenSPF.org wizard isn't 
the only one.

Yes it is surprise because the concerns back then was just like it 
always was for DNS PTR usage in general - it was a highest expense of 
queries and in an age where bandwidth was lower but increasing, 
arguably lesser optimized DNS servers, networks, hardware but getting 
better of course, using PTR was not par for the course, it was 
discouraged and it didn't take rocket science to know this.

Today it is used and the fact the major players and growing among many 
servers who are following the "market leaders" are using PTR as a 
requirement for SMTP clients.

In 2010, with performance concerns, we finally added support for rDNS 
in our scripting languages for SMTP operators to use:

     Aug 2010 Update History Summary:
     http://www.winserver.com/public/aup/aup453-4.htm#WCSAP

That was only done after the market has increasing began to do this, 
so we "followed" the market place.

For the most part, if you want to get "serious" about the "big boys" 
doing the right thing, having a better DNS cache is a prerequisite, 
and more so today because of all the DNS related protocols, especially 
in areas of AVS:

    CBV
    RBL
    SPF
    DKIM
    ADSP
    VBR

Did I forget any?  And we support all of them.  With all these 
increase amount of DNS related queries and it is now prudent to have 
an improved cache arrangement.  We completed that effort in July 2011:

     July 2011 Update History Summary:
     http://www.winserver.com/public/aup/aup454-1.htm#WCDNS

In summary, for a large multi-internet hosting application product 
framework, we consolidated and single source all the DNS queries from 
all the different components into a single DNS API/local cache layer. 
   With all the different queries for a complete coverage, this is 
very necessary for all scales.

> I'm unable to guess the meaning of that "T" in "IMTO".  True?  Tough?

oy vey Alessandro. try 'In my Technical Opinion' :)

> have a say about what SPF version, if any, receivers apply. 
> 
> ...

> Let's

A publisher of v=spf1 with a PTR would only be covered by v=spf1 
receivers and by "cross your fingers" v=spf3 receiver sites who want 
to offer original v=spf1 feature support.

This is really not to hard, but is what hard and IMEO borderline 
unethical is to pull the rug from under the feet of existing 
businesses and operations with published records where an new "Upgrade 
SPFBIS" is now promising new developers and also current who follow 
it, to remove PTR support - their end work of deprecation.

I can't believe we are really trying to alter the protocol with this BIS.

-- 
HLS

---

From: Hector Santos <[email-masked]>
Date: Sat, 21 Jul 2012 16:13:34 -0400

Hector Santos wrote:

> can post [it] if so desired. 

Back in early April 2012, data was posted by Murray and Philip:

   http://www.ietf.org/mail-archive/web/spfbis/current/msg01032.html

This contained 1/2 million records. The main focus back then was type 
99/SPF analysis.

Rather than work on my own data (since Murray doesn't honor it anyway, 
we are "too small" I guess), lets use the data he posted.

Imported into MySql, an ad hoc query to search for PTR policies:

   select distinct domain, data from spfdb
     where data like "%ptr%"

Produced over 42,874 unique domains/data pairs or about 8% of the half 
million. It  covers the big to small boys to the random unknowns out 
there.  Many domains show up twice to reflect two records - one SPF1 
and one SPF2.0 (Sender ID Policy).

This is real technical and practical field information that the PTR 
command is in use and it covers  all market size segments. I will 
venture most sites will have a near statistical average all depending 
on the type of domains they attack. Below I posted a sample of the 
first 100+ domains from the 42 thousand.

I really don't think there is any engineering merit to attempt to 
deprecate PTR. However, I agree there could be a BIS BCP section to 
discuss its usage along with others performance factoring deployments 
such as using INCLUDES at multiple levels.

But specifically with PTR, in my SPF engineering opinion there are 
good engineering recommendations:

  - Don't use it if you don't need to.
  - Place it as the last or near last directive.

    BAD    --> v=spf1 ptr a mx [-|~ALL]
    BETTER --> v=spf1 a mx ptr [-|~ALL]

and things of that nature.

Regard wizards, there are many SPF wizards out there and I have not 
checked them all, but all were mostly started using the early OpenSPF 
wizard model which did include the PTR as part of this "SPF expert 
system" modeling for creating SPF records.  A simple GOOGLE search for:

     spf wizard

will show you it is extensive.

We don't have the justification nor the engineering merits of a real 
problem to consider removing PTR.   This will create issues for 
existing publishers I will venture many have done a "Set it and Forget 
it" for their SPF policy - we are one of them and I don't have the 
time nor looking forward to be changing our policies without any real 
urgency to do so.  IMO, we will find many people in the same boat - 
unless there is a obvious problem, not many are going to change or 
bother to allocate time/resource to address this come looking at 
supporting SPFBIS time.

I believe there are other more useful, more critical, higher payoff 
items we can do to make SPFBIS a useful end product.  Removing items 
and breaking the SPF1 is not one of them (again, when there is no 
technical basis to do so).

Thanks

PS: Personally I won't like to end my input on this, doctor's orders, 
and I am hoping using Murray's and I guess Philip's own data is "Good 
Enough" there is no technical merit to deprecate/remove PTR creating 
an immediate issue for SPFBIS supporters against all current SPF1 
policy holders with an PTR directive.

Sample 100 domains with PTR SPF policies, sorted, no searching for 
"high value" or recognized domains done. Please there is no need to 
begin a "Domain Quality" debate arguing whether the list "represents" 
spammers and therefore don't count.

1000gems.com
4ssl.us
abbyynews.com
abelssoft.de
abelssoft.net
aceindustries.com
acore.org
aeecenter.org
ahmdirect.com
aikema.nl
aim.com
albawheelsup.com
alertsite.com
alwayskit.info
amazingabsformula.com
americanhomeinspectordirectory.com
americanstandard.com
angeliquedeparis.com
annfry.com
aol.com
apps.jobserve.com
artbizcoach.com
balinbestdesigns.net
bankatfirst.com
bankersinsurance.com
beallsflorida.com
beallsinc.com
biddingforgood.org
bizx.com
blairworldwidehunting.com
bombingdragg.com
bookit.com
bootignite.com
broadjam.com
brokeragejobs-mail.com
buffalospree.com
buildcentral.com
burns-wilcox.com
bustamante.com.ec
buyermax.com
cabelas.com
cadlink.com
cdiemail.com
celanaeasyweb.net
census.gov
cesa6.org
chartercom.com
cheappetes.com
cheatcodes.com
chicagohopeacademy.org
chickenbrickstudios.com
claimyourpowernow.com
cokesburycommunity.com
connectionstosuccess.org
coupons.com
couponsurfer.com
courts.state.ny.us
cpmsnewsletter.com
creativesocialmedia.org
creditrights.org
csba.org
customercare.publicstorage.com
da.org
danc.org
davidakaminsky.com
deflowth.info
delong.com
dimensioni.net
diskeepermail.com
docmagic.com
docutemp.com
drgnewsletters.com
easternshoreparent.com
easyecm.xerox.com
egroups.njscpa.org
egroups.nsba.org
elderpagesonline.com
email.joann.com
emailer.scholastic.com
enamor.com
enews.aeglive.com
enews.opticsplanet.com
ere.net
esellerate.net
ethereal.net
exfactory.net
farmprogress.com
fbacchog.com
firstmedianc.com
ftdreminders.com
fulfillmentprocessing.com
fullcontactgeek.com
gandi.net
geeks.com
geeks.org
golffacility.com
golflink.com.au
goosync.com
govdelivery.com
greatamerican.com
greencard.by
harborfreight22.com
harthosp.org
hinghamweather.com
hjortnesscpa.com
hnseek.com
hoptoitkidsparties.com
hosting.com
hotwire-travel.com

-- 
HLS

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sat, 21 Jul 2012 21:02:37 -0700

On Sat, Jul 21, 2012 at 1:13 PM, Hector Santos <[email-masked]> wrote:

> Hector Santos wrote:
>
>  And Hector stated PTR is in heavy use, by his measures of 16-19%.  It can
>> post [it] if so desired.
>>
>
>   http://www.ietf.org/mail-**archive/web/spfbis/current/**msg01032.html<http://www.ietf.org/mail-archive/web/spfbis/current/msg01032.html>
>
> Rather than work on my own data (since Murray doesn't honor it anyway, we
> are "too small" I guess), lets use the data he posted.
>

That appears not to be the case, since your SUBMITTER data was cited in
RFC6686 and you got credit for providing it.

I'm happy to honour anyone's data, provided we all get access to it and to
a description of the way the survey was done so that anyone else can
reproduce the results.  As for PTR, you haven't provided any data other
than a sentence saying it's in use, which doesn't provide solid evidence.
Data others can look at, such as what Philip and I have provided, are
really want's needed to prove a point in a working group and to the
satisfaction of reviewers and the IESG.  It helps if someone independently
could repeat what you did and come to the same conclusion.

> million. It  covers the big to small boys to the random unknowns out there.
>  Many domains show up twice to reflect two records - one SPF1 and one
> SPF2.0 (Sender ID Policy).
>
>
It's actually even higher than that.  The most recent data set says 284,475
domains surveyed, 154,090 responding with what look like valid SPF records,
54,385 appear to include a "ptr" mechanism.  That's 35% of responding
domains with valid records, and 19% of all domains.

However, the Cisco survey's numbers are 1,000,000, 359,659, 409, 0.11% and
0.004%, respectively.

As a reminder, the difference between the two surveys is that the TDP
survey includes domains that are actually sending data, pulled from over 20
live mail streams all but one of which is outside of my control, while the
Cisco survey is based on Alexa rankings that may or may not reflect what's
seen in live mail streams.

But I don't think anyone's arguing that PTR is unused.  The point is that
we believe it shouldn't be used any longer, because it's a bad idea.

-MSK

---

From: Hector Santos <[email-masked]>
Date: Sun, 22 Jul 2012 02:00:43 -0400

> we believe it shouldn't be used any longer, because it's a bad idea.

Thanks. Its much easier to deal with the question based on technical 
merits.

On the unused issue, as a side point and case in point, we could 
"argue" redirect and exist should be deprecated.  Perform a sql query 
on other commands like the following and I get:

    include   46,735
    redirect  1,202
    exist     309

For comparison,

    ptr       42,874

On system performance related, in my engineering opinion, there is 
good argument that using redirect, include and exist are also "bad 
ideas" because they add additional DNS queries and I would suggest 
good APIs use a recursive functional or class object. That adds stack 
and memory overhead and to resolve it requires complex non-recursive 
coding - of course, a piece of cake for the paid computer science 
major, programmer.

But looking at just one domain with multiple DNS query levels:

Query1: abc.nc.gov

         v=spf1 redirect=its.nc.gov

Query2: its.nc.goc

         v=spf1 ip4:149.168.220.0/24 include:spf.securence.com ?all

Query3: spf.securence.com

         v=spf1 ip4:216.17.3.0/24 ip4:204.11.209.64/26 ip4:216.17.14.0/24
                ip4:216.17.62.0/24 ip4:216.17.6.0/24 ip4:216.17.63.0/24
                ip4:216.17.12.0/24 ip4:216.17.20.0/24 ip4:216.17.24.0/24
                ip4:50.93.240.0/22 ip4:216.17.54.0/24 ~all

What makes this wasteful is its three potential DNS queries. Its a 
softfail, be great if it was a hard fail to provide a useful payoff 
for such a long spf policy.

Other similar policies where its not fail (-ALL), its not softfail 
(~ALL) and end up with a Unknown (?ALL) like gmail.com are very wasteful:

query1: gmail.com

         v=spf1 redirect=_spf.google.com

query2: _spf.google.com

         v=spf1 ip4:216.239.32.0/19 ip4:64.233.160.0/19 ip4:66.249.80.0/20
                ip4:72.14.192.0/18 ip4:209.85.128.0/17 ip4:66.102.0.0/20
                ip4:74.125.0.0/16 ip4:64.18.0.0/20 ip4:207.126.144.0/20
                ip4:173.194.0.0/16 ?all"

Only 2 queries, but it makes it even more wasteful for receivers to 
process the non-matching result with no value.  What makes this worst, 
is its high spoofing attraction and the volume of gmail.com 
transactions - this adds to the overhead with there is no payoff for 
negative results.

So in my view, this and many others like it can be also argued as a 
"Bad Idea" for SPF policies to have these complex, multilevel DNS 
queries, especially for high value domains who are targets by 
spoofers, ending up with a neutral or unknown SPF result.

I believe we can definitely benefit from a BCP section or even 
separate document, but SPFBIS can cover it I think.   This would be 
better and productive than debating on what is "unused" or what is 
considered a bad idea for DNS or the overall system.  SPF is a very 
wasteful protocol and BCP can help BIS lower the waste/overhead.

Finally, PTR for SPF and rDNS usage, although we all know it has an 
overhead, it really depends on how poor the site DNS server 
arrangement is.  Poor = slow, Fast = no problem.  It is very popular, 
like Levine pointed out, for logging purposes.  A cache is important 
to these to address redundancy and IMV, that is why, we know its the 
"slowest" among the queries, but it has a high utility and purpose. 
For SPF, I can't speak for others, but for us, its possible that we 
may temporarily have a xyz.winserver.com subdomain used for some 
special email purpose.  As long as we have an PTR record for its up, 
we don't have to change our SPF record.  So it serves a very useful 
purpose with no extra SPF DNS maintenance required.

-- 
HLS