Subject: [spfbis] A standards track document defining SPF

From: S Moonesamy <[email-masked]>
Date: Wed, 18 Jul 2012 13:09:36 -0700

Hello,

The second and last deliverable for the working group is to produce a 
standards track document defining SPF by December 2012.  The SPFBIS 
Charter mentions features.  Instead of commenting about features I'll 
look at this from another angle.

  (i)   RFC 4408 has not been reviewed by the IETF

  (ii)  There are multiple implementations of RFC 4408

  (iii) The Errata for RFC 4408 does not mention Issue #9

draft-ietf-spfbis-4408bis is unusual work as Experimental RFCs 
generally go through (i).  It can be argued that the draft can be 
published as a RFC with (iii) as the significant change and some 
editorial changes.  Murray Kucherawy mentioned the following [1]:

    "If I were sitting on SECDIR, I'd make some noise about this 
during Last Call,
     for example."

draft-ietf-spfbis-4408bis will be reviewed by the IETF Security 
Directorate (SECDIR).  It is expected that the working group will 
give some serious thought to the security considerations aspect 
before the Last Call.

Scott Kitterman raised an interesting point: backward incompatibility 
[2].  The question that comes to my mind is whether 
draft-ietf-spfbis-4408bis has to be backward compatible with RFC 4408.

Section 4 of RFC 4408 has the following requirement:

   "Mail receivers that perform this check MUST correctly evaluate
    the check_host() function as described here."

There is a reference in that Section 4 to Section 2.5.  In Section 
2.5.5, there is:

   'A "SoftFail" result should be treated as somewhere between a "Fail"
    and a "Neutral".'

Depending on how one interprets RFC 2119 there may or may not be a 
recommendation in that sentence.  Assuming that it is a 
recommendation (SHOULD) the "somewhere between" two possible results 
is ambiguous.

Section 5 defines the mechanisms.  Section 6 defines the 
modifiers.  I'll assume that a change in these sections can affect 
the results (Section 4).  The macros defined in Section 8 indirectly 
affects mechanisms and modifiers.

In my individual opinion RFC 4408 is not ready for publication as a 
Proposed Standard as an IESG Evaluation would generate DISCUSSes 
(issues the working group must fix).

Kurt Andersen commented about why "management elected to discontinue 
the use of all SPF records [3].  Derek Diget mentioned that 'as a 
email admin, I am starting to feel that the more that is deprecated 
or removed, the more that I will run into issues with a sender saying 
one thing (either a "legacy" 4408 SPF record or an updated 4408bis 
SPF record) and my (receiver) software evaluating it differently than 
the publisher wanted' [4].

draft-ietf-spfbis-experiment did not generate any DISCUSSes as the 
working group discussed the issues and possible issues.  My suggestions are:

   1. Demonstrate that a RFC 4408 implementation and a 
draft-ietf-spfbis-4408bis
      implementation will produce the same results

   2. If (1) is not possible, provide an explanation for the change

Regards,
S. Moonesamy

1. http://www.ietf.org/mail-archive/web/spfbis/current/msg01879.html
2. http://www.ietf.org/mail-archive/web/spfbis/current/msg01888.html
3. http://www.ietf.org/mail-archive/web/spfbis/current/msg01889.html
4. http://www.ietf.org/mail-archive/web/spfbis/current/msg01902.html

---

From: Commerco WebMaster <[email-masked]>
Date: Wed, 18 Jul 2012 16:01:18 -0600

On 7/18/2012 2:09 PM, S Moonesamy wrote:
> draft-ietf-spfbis-4408bis is unusual work as Experimental RFCs generally
> go through (i). It can be argued that the draft can be published as a
> RFC with (iii) as the significant change and some editorial changes.
> Murray Kucherawy mentioned the following [1]:
>
> "If I were sitting on SECDIR, I'd make some noise about this during Last
> Call,
> for example."
>
> Directorate (SECDIR). It is expected that the working group will give
> some serious thought to the security considerations aspect before the
> Last Call.
>

Yow!

As I understood the charter, the idea here was to spend the time and 
energy of all involved in formalizing and correcting any errors and / or 
omissions in the specification for surviving branch of SPF as 4408bis.

In that charter, I understood that the goal would be to do things like 
clean up incorrect or invalid wording, take the experience we have with 
some 8 years of SPF in the wild and apply that to the 4408bis document 
in order to have a stronger standard from which to work for future 
implementers and perhaps clean up some misunderstandings by those who 
are existing implementers (both from a code provider and DNS publisher 
standpoint).

What I did not understand as a charter goal was the redefinition of SPF 
to the point that it would not become interoperable with that which 
correctly exists in the wild.

If we were to do that, the the point raised recently about SPF V3 on 
this list is probably valid, but I am fairly sure that our charter is 
not to create SPF V3 (a future version which may or may not be backwards 
compatible, but then that is a whole other working group... I think).

I guess the question for me really is are we fixing something that is 
really broken or breaking something that actually works?

I am certainly hopeful that it is the former rather than the latter.

Alan M.

---

From: Dave Crocker <[email-masked]>
Date: Wed, 18 Jul 2012 15:09:24 -0700

> correctly exists in the wild.

Removing unused features is an essential part of this type of exercised. 
  Unused feature add cost, complexity and bugs.

d/
-- 
  Dave Crocker
  Brandenburg InternetWorking
  bbiw.net

---

From: Hector Santos <[email-masked]>
Date: Wed, 18 Jul 2012 18:21:57 -0400

This is an generality that has not been shown to be the case for PTR, 
include the %p macro.

All this has been tested by implementators and deployments long ago. 
No problem, no bugs and that cost has already been spent.  What is 
being suggested to create change, and with change chaos begins - time, 
energy, cost all to reach equilibrium again.  We are at equilibrium 
now.

That said, I have yet to see a legit reason to remove a USED feature 
in PTR.

-- 
HLS

---

From: Hector Santos <[email-masked]>
Date: Wed, 18 Jul 2012 18:38:51 -0400

+1 to all you stated.

Commerco WebMaster wrote:
> _______________________________________________
> spfbis mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/spfbis
> 
> 

-- 
HLS

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 18 Jul 2012 18:56:40 -0700

Hi Alan,

On Wed, Jul 18, 2012 at 3:01 PM, Commerco WebMaster
<[email-masked]>wrote:

> What I did not understand as a charter goal was the redefinition of SPF to
> the point that it would not become interoperable with that which correctly
> exists in the wild.
>

The two surveys I referenced show that both the "ptr" mechanism and macros
in general are in use in the wild, but by a very tiny percentage of the
deployed base.  It's true that removing them entirely would render those
few sites incompatible with the Proposed Standard.  In my view (and I
believe Dave's and the two John Ls, at least), that is a cost offset by the
fact that the two features we're talking about removing are costly: "ptr"
is expensive in terms of performance and can be accomplished by other
extant, less expensive means, and macros are expensive in terms of their
complexity and thus high maintenance cost and risk of misimplementation
("dangerous", as John Levine put it), not to mention the DNS issues it
raises.

The TDP survey shows a fraction of a percent using "ptr" or macros.  The
Cisco survey shows exactly one out of a million domains using "ptr", and
none using macros.  Hector claims heavier "ptr" use in his data, but has
not provided the details.

Based on that, I maintain the spec is "broken" (as you've used the term)
and needs fixing.  I also maintain that the charter allows such removals
under its "unused" clause.

Further, I don't much like the idea of going onto the Standards Track for
the first time with two features we know see very little use, have known
costs, and have basically agreed should at least be deprecated.  That just
feels completely sloppy to me.  I recognize there's a big deployed base for
SPF, but approximately none of that base uses the stuff we're talking
about.  Keeping it creates more of a burden than it solves.

The counter-argument is basically "It's only a 'proposed' standard, and we
can remove these things before it goes along the RFC6410 route."  Those of
us who have been around long enough know that such advancement is rare
indeed (which is why we recently went from three maturity levels down to
two), so on the premise that such a revision is also statistically
unlikely, I think the proper thing to do is get it right up front.

As I said in an earlier post, if something were to land in front of me, as
a SECDIR reviewer or an IESG member, with two sections that basically say
"We know this is broken, please don't use it in TXT records but receivers
MUST support it anyway", I'd be asking questions about why it's going for
the Standards Track in the first place..

-MSK

---

From: Scott Kitterman <[email-masked]>
Date: Wed, 18 Jul 2012 22:09:39 -0400

>unlikely, I think the proper thing to do is get it right up front.

Up front was in 2005.  

>the Standards Track in the first place..

Was the ip6 mechanism used more or less than ptr?

Scott K

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 18 Jul 2012 19:24:41 -0700

> Up front was in 2005.

As SM has pointed out, RFC4408 didn't undergo the rigors of formal IETF
review back then.  It is now.

In the TDP survey, "ptr" had 93 distinct domains using it; ip6 had 0.

In the Cisco survey, "ptr" had 1 domain using it; ip6 had 0.

If the move here is to ask me if I'd also deprecate ip6 based on the data,
the answer is "no" because IPv6 match logic is far simpler than macros, for
example.  I don't find that notion to be complex or dangerous.

-MSK

---

From: Scott Kitterman <[email-masked]>
Date: Wed, 18 Jul 2012 22:55 -0400

> review back then.  It is now.

Sure, but you can't have it both ways.  You can't both claim that this is a 
normal transition from experimental to standards track because by the letter 
of the IETF process, that's what it is, but then on the other hand claim that 
the letter of the IETF standards advancement process isn't realistic and we 
can't consider it reasonable that features we deprecate now will be removed 
when SPF is advanced.

I think you either get to pick the letter of the IETF process or a realist 
view of it, not whichever is convenient.

> example.  I don't find that notion to be complex or dangerous.

OK.  So to be clear, being unused is not a criteria for removal?  It's being 
unused and problematic in some manner?

Scott K

---

From: Andrew Sullivan <[email-masked]>
Date: Wed, 18 Jul 2012 23:15:08 -0400

No hat.

> the Standards Track in the first place..

We have a glib answer to such questions, though: "It's what the IETF
chartered us to do.  If you think that needs changing, then the WG
needs to be rechartered to fix these problems."

Moreover, there is an interoperability argument available, which is
that we have positive evidence that some people publish these sorts of
records, and that therefore receivers need to be able to process them.
This kind of issue is one we face in DNS-land all the time: once
something is out there, it is all but impossible to get rid of it.
Why do you think IDNA is the miserable hack it is?  Why do you think
we haven't introduced any new CLASSes since CHAOS?  Why do you think
the absurd and conceptually broken underscore convention has been
embraced instead of just using new RRTYPEs?  

The counter-argument available to us, of course, is that we are moving
from the experimental track.  But for that argument to work, the
charter would need to be more permissive in what it says we can
remove.  Moreover, suppose you are an SPF receiver and you get one of
these records.  What are you going to do with it?  Drop it on the
floor?  No, of course not.  You're going to use your hoary old 4408
code to handle it anyway.  If that's true, then we should document it.
(Note that deprecation _does_ mean that publisher-side tools can drop
support for these things immediately.)

I am having a really hard time accepting "nobody _we_ know uses this"
as the meaning of "unused".  To my mind, that's what these claims of
"it's not an important use" amount to.  The charter says quite
explicitly that we're not allowed to remove "existing features that
are in current use."  The attempt to redefine "in current use" so that
it doesn't include cases that are observed examples of the feature
actually being deployed (even if it's a stupid feature) makes me very
uneasy.

The arguments from increased simplicity are good and convincing ones,
but they are _irrelevant_ given our charter.  The charter doesn't say,
"You can't remove things in use, unless it makes the protocol better
by virtue of simplicity."[1] .  It says, "You can't remove things in
use."  The charter also doesn't say, "You can't remove things in use,
unless you think the use is a bad idea," or, "You can't remove things
in use, unless only a few people are using it," or anything of the
kind.  It is crystal clear.

If we don't like that, the answer is not to try to weasel around the
problem by stretching the meaning of "unused" past the breaking point
even of English.  The answer is to go back to the community and say,
"This charter is stupidly restrictive for something coming from the
experimental track and moving to the standards track."  I am
personally disinclined to lead that charge; but if people really want
to do it, I won't try to stop anyone.

Alternatively, we can embrace the "deprecated" route that is already
in the draft.  This has the conisderable benefit that it gets us
around both the charter and interoperation issues while yet
broadcasting the point that these mechanisms already don't work
reliably and shouldn't be used.  If interoperability is really our
core concern, that approach addresses it while still saying the
features need to go away.

Best,

A

[1] It does of course say, "You can remove things in use if they're
errors."  That's the way we get away with removing RRTYPE 99 support:
4408 has an error that causes interoperation failure between two
conforming implementations.  There is more than one way to fix this,
but an obvious one is to remove TYPE 99, so we chose that partly based
on observed use.  If people want to try out that line of attack on
local-part macros, go nuts.

-- 
Andrew Sullivan
[email-masked]

---

From: Hector Santos <[email-masked]>
Date: Wed, 18 Jul 2012 23:19:44 -0400

> In the Cisco survey, "ptr" had 1 domain using it; ip6 had 0.

Either its flawed or they have an extremely different set and unique 
of Personal Network Community (PCN) of domains that don't use it. 
Highly unique.

I can easily post 361 domains from a cache of 1 day with SPF policies 
having PTR directives, but I won't.  BTW, for IP6, 15 domains had ip6 
in their SPF policies.

-- 
HLS

---

From: Commerco WebMaster <[email-masked]>
Date: Wed, 18 Jul 2012 21:21:15 -0600

Hi Murray,

On 7/18/2012 7:56 PM, Murray S. Kucherawy wrote:
> <[email-masked] <mailto:[email-masked]>> wrote:
>
>     What I did not understand as a charter goal was the redefinition of
>     SPF to the point that it would not become interoperable with that
>     which correctly exists in the wild.
>
>
> The two surveys I referenced show that both the "ptr" mechanism and
> macros in general are in use in the wild, but by a very tiny percentage
> of the deployed base.  It's true that removing them entirely would
> render those few sites incompatible with the Proposed Standard.  In my
> view (and I believe Dave's and the two John Ls, at least), that is a
> cost offset by the fact that the two features we're talking about
> removing are costly: "ptr" is expensive in terms of performance and can
> be accomplished by other extant, less expensive means, and macros are
> expensive in terms of their complexity and thus high maintenance cost
> and risk of misimplementation ("dangerous", as John Levine put it), not
> to mention the DNS issues it raises.
>
> Further, I don't much like the idea of going onto the Standards Track
> for the first time with two features we know see very little use, have
> known costs, and have basically agreed should at least be deprecated.
> That just feels completely sloppy to me.  I recognize there's a big
> deployed base for SPF, but approximately none of that base uses the
> stuff we're talking about.  Keeping it creates more of a burden than it
> solves.
>
> The counter-argument is basically "It's only a 'proposed' standard, and
> we can remove these things before it goes along the RFC6410 route."
> Those of us who have been around long enough know that such advancement
> is rare indeed (which is why we recently went from three maturity levels
> down to two), so on the premise that such a revision is also
> statistically unlikely, I think the proper thing to do is get it right
> up front.
>
> As I said in an earlier post, if something were to land in front of me,
> as a SECDIR reviewer or an IESG member, with two sections that basically
> say "We know this is broken, please don't use it in TXT records but
> receivers MUST support it anyway", I'd be asking questions about why
> it's going for the Standards Track in the first place..
>
> -MSK
>

The primary reason I piped up was in response to our co-chair's remark 
about removing SPF backward compatibility in the specification.  Perhaps 
it was my conditioning by a former employer (who was a well known 
"Measurement and Computation" company) in my younger days that cause the 
reaction.  The group I worked with viewed backward compatibility as one 
of the key selling features of a line of their business computers 
because it preserved client's investments in software development.

Actually, I don't really disagree with the idea of depreciating support 
for PTR (because of reasons discussed by John Levine and others re the 
recent SPFbis list posts). I also don't have any irons in the fire as 
regards macros (we don't publish them), so I am pretty neutral there.

My thinking is to perhaps consider these features depreciated but still 
usable as extensions to the MUST part of the specification, if they 
don't break SPF.

Perhaps something like:
---
Over a number of years preceding RFC 4408bis, SPF operated as an 
experimental protocol.  During those years, SPF software implementers 
and publishers both observed cases where issues were encountered 
surrounding the use of both PTR and support for SPF macros used in SPF 
software implementations and published DNS TXT SPF records.

Such issues include <list of issues with PTR and Macros>.

In order to preserve backward compatibility, implementers and publisher 
MAY choose to implement these features, but they should be considered as 
depreciated or alternatively as optional extensions some may choose to 
adopt at their own peril, as their use has even been deemed dangerous by 
some.
----

The above may only work if the statements can be considered true.  I 
guess I would now ask the list, can they be considered true and does it 
even make sense to look at the PTR and Macros issue in this way to try 
to find consensus to move forward?

My mind drifts back to RFC 821 (yes, I know depreciated by 5321, but 
even before it was depreciated, this thought might have applied) and the 
concept of a minimum implementation with other commands (e.g., SAML / 
SOML) which I believe were not widely implemented in circa 1995 and 
beyond SMTP servers.

That is the source of my hope for finding a compromise that works for 
SPF.  In other words, can we find a workable middle ground to move us 
ahead in this process?

Best,

Alan M.

---

From: Hector Santos <[email-masked]>
Date: Wed, 18 Jul 2012 23:27:18 -0400

> the Standards Track in the first place..

hmmmm, the same was done for DKIM with its SHA1 vs SHA256.  There are 
other examples I am sure we can find in the annals for IETF protocol 
development and updates.

I'm lost as to why this is a big issue. PTR is in used. It shouldn't 
be deprecated.  But if technocrats prefer to recommend PTR not be 
published, that shouldn't be basis for promoting removing from 
processors because it is a high utility item already in use.   This is 
no different from recommending EHLO over HELO. HELO still must be 
supported otherwise your software will not worth much.

-- 
HLS

---

From: Hector Santos <[email-masked]>
Date: Wed, 18 Jul 2012 23:53:09 -0400

> because it preserved client's investments in software development.

Most Product Vendors would go out of business if they didn't provide 
backward compatibility, especially in an customer environment where 
there is coding and/or operator settings, configurations involved. 
You can do more with "Canned" Applications, but if the customer has 
settings, coding, i.e. DNS management or even a tertiary market of 3rd 
party developers, the barriers are higher to change an existing 
protocol especially when there is very little basis to do so, and the 
'unused' statements are patently incorrect (in regards to PTR).

Scott has a legitimate technical reason such as Slowness, but I submit 
this was more of a concern early on but no such much today.  First, 
more and more SMTP servers are already requiring a PTR for the 
connecting client, so this result will already be in server's DNS 
resolver cache by the time SPF is checked and processed.  Second, if 
"Slowness" is a problem, there are other parts of SPF that are either 
as much or more expensive than PTR - any directive that starts 
additional SPF or recursive lookups (i.e., INCLUDE, REDIRECT, EXIST) 
has a "Slowness" attribute to it.

I submit having INCLUDE (and related commands), especially those SPF 
domains with multiple INCLUDES, can be very expensive and complex 
compared to a PTR.

Why don't we deprecate INCLUDE on the basis of Slowness?

-- 
HLS

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 18 Jul 2012 21:19:13 -0700

> can't consider it reasonable that features we deprecate now will be removed

I never made the latter claim.  I think it is realistic, and I also
understand the nuances of it as I was part of the whole debate that
resulted in RFC6410.

In fact I believe we could (if we agree to it) go down the path of
deprecating these features now and removing them later.  What I'm saying is
that often enough, "later" never actually happens, which means these warts
on the protocol will never actually go away; calling them "deprecated" is
merely hand-waving and ultimately ineffective at solving the problem.

If we're trying to protect people from themselves by discouraging use of
these features, attaching an easily ignored labels will not achieve that
goal.

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 18 Jul 2012 21:29:10 -0700

> embraced instead of just using new RRTYPEs?

Isn't it desirable, then, to get rid of this stuff while it's Experimental
and not Proposed Standard?

Would it make people satisfied if I were to chase down all 203 domains that
we found publishing SPF records with macros in them and ask them if they
would complain if that aspect of the protocol were removed?  I'd be happy
to do that.

> support for these things immediately.)

It's not true for me.  Given what I know, I wouldn't implement ptr or
macros.  They'd get permerror or something.

> "it's not an important use" amount to.

That's not the intent I have in mind.  I have in mind the same logic we had
with DKIM when we removed "g=": Nine domains out of a million-plus were
using it, so we dumped it.

> to do it, I won't try to stop anyone.

I'd support that, if it breaks this logjam.  I suspect though that it would
be split along party lines, so to speak.

> features need to go away.

I don't find interoperability when moving from Experimental to Proposed
Standard to be as critical as others do.  I would find that argument far
more compelling if we were moving from Proposed Standard to Internet
Standard.  Yes, I recognize the ample deployed base, but it's not like the
word "Experimental" in RFC4408 was concealed.

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 18 Jul 2012 21:35:07 -0700

On Wed, Jul 18, 2012 at 8:19 PM, Hector Santos <[email-masked]> wrote:

> having PTR directives, but I won't.  BTW, for IP6, 15 domains had ip6 in
> their SPF policies.
>
>
If you won't provide data, your claim isn't meaningful.

The TDP and Cisco surveys amount to reproducible experiments.  Your say-so
isn't.

-MSK

---

From: Dave Crocker <[email-masked]>
Date: Wed, 18 Jul 2012 21:59:02 -0700

On 7/18/2012 9:29 PM, Murray S. Kucherawy wrote:
> Isn't it desirable, then, to get rid of this stuff while it's
> Experimental and not Proposed Standard?

This strikes me as the single-most basic and compelling question, and it 
seems to be getting ignored.

Everything ought to flow from the answer to that question.

Debating the meaning of "unused", for example, should only come after 
agreeing whether a feature is in fact desirable, based on the experience 
of more than a few years in the field.

If someone is claiming that PTR or macros are indeed practically 
desirable, based on the community's extensive experience, they should 
make that case.  We have, I think, plenty of evidence to the contrary.

d/

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 19 Jul 2012 01:01:39 -0400

> would complain if that aspect of the protocol were removed?  I'd be happy

If it's going to be gotten rid of, I think it's appropriate to do so after 
providing a warning/transition time (which is why I think deprecate is 
reasonable).

If you're willing to contact the 203 domain owners, I think the more useful 
thing to do would be find out why they are using macros and what problem they 
solve for them.  Part of my own rationale for deprecating PTR was that I've 
helped quite a few people convert records that used PTR to those that didn't 
and it has never been very hard (despite other claims that it's essential, my 
personal experience is that it isn't and the reason - when asked "why did you 
use PTR to start with" - was generallly "that's what the SPF wizard said to 
use).  Maybe if we understood why, we could understand reasonable 
alternatives.

BTW, I'm sure altavista.com didn't show up on your list, since it sends no 
mail, but it's got one of these records too (in fact it's similar or identical 
to the DNS logging record in RFC 4408).  I can put you in touch with the 
person that put that up in 2003/2004 if you want.

> macros.  They'd get permerror or something.

If you aren't going to implement the entire thing, I'd recommend treating 
these as None.  Raising errors on thing you don't like in the protocol isn't 
good citizenship.

> using it, so we dumped it.

If those nine domains kept using it, what would happen (I don't recall)?

> be split along party lines, so to speak.

No doubt.  I think the charter is constraining us from doing precisely stuff it 
was meant to constrain us from doing.

> word "Experimental" in RFC4408 was concealed.

How many people outside IETF professional pay attention to such labels?  It's 
not many.  My reading of the IETF note has always been that SPF was 
experimental because MARID failed and there wasn't a single protocol to 
advance.  The reasons were not related to the technical maturity of the 
protocol.  Among the group of people that know what an experimental RFC is and 
have been interested in RFC 4408, a large fraction (IME, almost all) know that 
the reasons it's experimental aren't limited to technical issues.  As a 
result, the IETF's label got pretty soundly ignored.

Concluding the deployed based doesn't count because RFC 4408 was 
'experimental' is a great way to reinforce the degree of connection a lot of 
people think the IETF has with the real world.

Scott K

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 18 Jul 2012 22:04:34 -0700

> was meant to constrain us from doing.

As the charter's author, I can assure you that wasn't the intent.

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 18 Jul 2012 22:11:35 -0700

> people think the IETF has with the real world.
So "Experimental", like "deprecated", is ultimately a meaningless label?

Seems like there's not a lot of solid ground left around here.

-MSK

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 19 Jul 2012 01:13:01 -0400

On Wednesday, July 18, 2012 10:04:34 PM Murray S. Kucherawy wrote:
> > > I'd support that, if it breaks this logjam.  I suspect though that it
> > 
> > would be split along party lines, so to speak.
> > 
> > No doubt.  I think the charter is constraining us from doing precisely
> > stuff it was meant to constrain us from doing.
> 

As one of the people that reviewed it, I can assure you I thought it was.  If 
I thought the current charter allowed us to toss out backward compatibility on 
the basis of "oh, it seems simpler this way", I'd have never supported it.

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 19 Jul 2012 01:18:41 -0400

On Wednesday, July 18, 2012 10:11:35 PM Murray S. Kucherawy wrote:
> > How many people outside IETF professional pay attention to such labels?
> > 
> >  It's not many.  My reading of the IETF note has always been that SPF was
> > 'experimental' is a great way to reinforce the degree of connection a lot
> > of people think the IETF has with the real world.
> 

I think the operative bits around the definition of deprecated in the current 
draft are SHOULD NOT.  It's that versus removal (MUST NOT).

I think the IESG of the day used Experimental to dodge it's way out of a 
problem with Microsoft.  Given the reality of 2005, I don't know that they 
could have realistically done any different, but RFC 4408 was never 
'experimental' in the same say an experimental RFC written up after a few 
people have spent some time in a lab is 'experimental'.  People understood 
that.  I think it's not appropriate to go back now and pretend it was other 
than it was.

Scott K

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 18 Jul 2012 22:24:11 -0700

> the basis of "oh, it seems simpler this way", I'd have never supported it.
And you really believe that's a fair characterization of the arguments
being made?

-MSK

---

From: Commerco WebMaster <[email-masked]>
Date: Wed, 18 Jul 2012 23:25:30 -0600

Dave,

> make that case. We have, I think, plenty of evidence to the contrary.

This seems a pretty reasonable approach.  As much as I personally 
dislike anything that breaks compatibility with earlier work product, 
the reasoning founded upon the experimental status of SPF is a pretty 
good argument.

Sometimes beta software when it becomes released product has some 
removed features which proved themselves as unproductive for the 
majority of the consumers who might use the software during the 
shakedown of the beta.  Perhaps there is a parallel.

I think that if we were to setup a list of bullet point reasons pro and 
con for the disputed PTR and Macro features, perhaps that might help us 
figure out and make the best choices in resolving the matter.

Alan M.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 18 Jul 2012 22:39:33 -0700

On Wed, Jul 18, 2012 at 9:29 PM, Murray S. Kucherawy <[email-masked]>wrote:

> Would it make people satisfied if I were to chase down all 203 domains
> that we found publishing SPF records with macros in them and ask them if
> they would complain if that aspect of the protocol were removed?  I'd be
> happy to do that.
>
>
I was worried I'd put my foot in it when I said that, but I just discovered
that more than half of that 203 is a single ESP that uses a subdomain for
each customer.  So the actual number of "users" is far smaller than I'd
originally thought.

I'll try to contact all of them, though probably not until next week due to
other obligations.

-MSK

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 19 Jul 2012 01:47:28 -0400

On Wednesday, July 18, 2012 10:24:11 PM Murray S. Kucherawy wrote:
> > As one of the people that reviewed it, I can assure you I thought it was.
> > 
> >  If I thought the current charter allowed us to toss out backward
> > compatibility on

I'm not convinced that the arguments against macros are anything other than an 
interesting theory.  A slightly longer version of what I wrote above would be:

1.  Usage is very low.  Close enough to zero that we can call it zero.
2.  We have a number of theories why keeping it is bad:

a.  It adds complexity to the protocol
b.  The code behind these protocol elements won't be well tested and is buggy.
c.  Local-part macros in particular make EAI harder.
d.  The impact on DNS cachability of some macros (including local-part) is a 
problem.

I don't argue 2.a.  

The only real evidence for 2.b is one report of a problem with one provider.  
For 2.c, I think we'd agreed that there's a subset of international local-
parts that could be represented in the local-part macro and the rest can't.  
It's true that getting full EAI support for local-part macros would be hard, 
but no one has argued it's needed (in fact it's been suggested we might be 
best off to ignore it entirely for now).  For 2.d, there's not a lot of 
evidence that SPF records of any type are substantially cached for any but the 
largest domains.  The practical impact of some macros inhibiting cachability 
seems to be nil.

So while the language I used was a bit flip, I think it's a reasonable short 
form.  It would have been more complete to say "Oh, we have some theories and 
in any case it's simpler this way", but I don't see that any case based on 
anything other than simplicity and lack of use has really been made.

Scott K

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 18 Jul 2012 23:10:04 -0700

> a.  It adds complexity to the protocol
b.  The code behind these protocol elements won't be well tested and is
> buggy.

These are the three arguments I'm focused on, so I'll ignore the rest or
let others carry them forward. But it's derisive and plain wrong to dismiss
any of them, much less all of them, as theoretical.

> anything other than simplicity and lack of use has really been made.
You find those last two points to be uninteresting, while I find them
critical.  I could be equally dismissive and characterize your position as
"Oh, even one user shown to be using this stuff means (to use Alan's
example) this broken Beta feature that we don't really want to support has
to stay in the released product for the foreseeable future."  To me, that
line of thinking makes neither good protocol design sense nor good business
sense.

-MSK

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 19 Jul 2012 02:14:06 -0400

> line of thinking makes neither good protocol design sense nor good business

I think the comparison between a protocol that's been in Internet scale 
production for close to a decade and a beta software release is far fetched, 
but that's not important.

I got lost in the issue counting and the line wrapping above, so would you 
please restate the arguments you're focused on?

Scott K

---

From: Alessandro Vesely <[email-masked]>
Date: Thu, 19 Jul 2012 09:28:30 +0200

On Wed 18/Jul/2012 22:09:36 +0200 S Moonesamy wrote:
> generally go through [IETF revision].  It can be argued that the
> draft can be published as a RFC with (iii) as the significant
> change and some editorial changes.  Murray Kucherawy mentioned the
> following [1]:
> 
>     during Last Call, for example."
> 

I'm rather lost.  What are we talking about?

We agreed to drop historical questions and just clean up the protocol.
 We devised a path to deprecate unwanted features.  While that path
can be ameliorated (see below), even Dave and Murray seem to agree
that it may be a reasonable compromise.  Thus, the question boils down
to specific considerations, e.g. on the status of %{l}:  Its usage in
mechanisms is discouraged with the same normative strength as
deprecation, but it's not formally deprecated.  Thus, it can still be
used for explanations.  What are we going to do with the latter use?

I don't know what SECDIR will say, but don't understand why we should
get ahead of ourselves trying to guess what they'll object and fix
unbroken stuff based on conjectures.  Can't we cross that bridge when
we get to it?

> recommendation in that sentence.  Assuming that it is a recommendation
> (SHOULD) the "somewhere between" two possible results is ambiguous.

Hm...  "SHOULD NOT be treated more severely than fail NOR more mildly
than neutral"?  Part of that belongs to the cleanup.

Some ambiguity originates from the fact that the intended treatment is
underspecified in general.  I trust we'll explicate it better, but
that's issues #12, #13, and #29.

> My suggestions are:
>      draft-ietf-spfbis-4408bis implementation will produce the same
>      results
> 

I don't think they need to produce the same results.  It is enough
that they are interoperable and produce compatible results.

In particular, I like Scott's idea to stretch the meaning of "none" in
order to allow (new) implementations to skip processing of some
features.  I'm not sure about header fields saying "none", as the
usual interpretation of that is "the sender publishes no record".
Alternatively, they could pretend not to be carrying out any SPF check
--which is already possible, obviously.

The explanations should go in their own section/appendix.  Scott is
currently keeping a log of changes, and I propose that it remains that
way until the document is almost ready.  At that point we can add
detailed explanations of the reasons why we made those changes.

For PTR, besides what Hector already recalled, there is also the
underspec that Murray noted (xebay.com), and the fact that .arpa
delegations are often omitted or not in sync.

--

---

From: S Moonesamy <[email-masked]>
Date: Thu, 19 Jul 2012 03:25:24 -0700

>to do it, I won't try to stop anyone.

The working group has a charter.  It was cleared spelled out that it 
will work on moving a specification from Experimental to the 
Standards Track.  Even if I am convinced about a change or non-change 
to the specification I cannot overlook the restrictions in the 
charter.  I cannot go back to the community and say that the charter 
needs to be changed.  I am ok if a WG participant decides to go back 
to the community to ask for a change to the charter.  The only advice 
I can provide is to be prepared to spend the time and energy to 
convince the IETF community.

In a message posted on December 1, 2011, backward compatibility was 
mentioned [1].  If Andrew Sullivan asks me about that I'll quote the 
following paragraph:

   "Changes to the SPF specification will be limited to the correction
    of errors, removal of unused features, addition of any enhancements
    that have already gained widespread support, and addition of
    clarifying language."

>I'm rather lost.  What are we talking about?

What I was talking about is to take security considerations 
seriously.  It is up to the working group to do that work.

>that it may be a reasonable compromise.  Thus, the question boils down

I like the word "compromise".  Here are equivalent terms:

   - compromis
   - kompromiss
   - compromesso
   - compromiso
   - kompromiss

>unbroken stuff based on conjectures.  Can't we cross that bridge when

Yes, as long as the working group carefully considers the issues.

>that's issues #12, #13, and #29.

It is up to the working group to identify the ambiguities.

At 15:01 18-07-2012, Commerco WebMaster wrote:
>I guess the question for me really is are we fixing something that 
>is really broken or breaking something that actually works?

The working group might want to think very carefully about that.  I 
understand that the issues are not always clear-cut.  Any WG 
participant who breaks something may be asked to fix it.

At 18:56 18-07-2012, Murray S. Kucherawy wrote:
>Further, I don't much like the idea of going onto the Standards 
>Track for the first time with two features we know see very little 
>use, have known costs, and have basically agreed should at least be 
>deprecated.  That just feels completely sloppy to me.  I recognize 
>there's a big deployed base for SPF, but approximately none of that 
>base uses the stuff we're talking about.  Keeping it creates more of 
>a burden than it solves.
>
>The counter-argument is basically "It's only a 'proposed' standard, 
>and we can remove these things before it goes along the RFC6410 
>route."  Those of us who have been around long enough know that such 
>advancement is rare indeed (which is why we recently went from three 
>maturity levels down to two), so on the premise that such a revision 
>is also statistically unlikely, I think the proper thing to do is 
>get it right up front.
>
>As I said in an earlier post, if something were to land in front of 
>me, as a SECDIR reviewer or an IESG member, with two sections that 
>basically say "We know this is broken, please don't use it in TXT 
>records but receivers MUST support it anyway", I'd be asking 
>questions about why it's going for the Standards Track in the first place..

The above is the IETF angle.  I am excluding "features" in my comment 
or else I would not be able to comment.  Two of the questions which 
the document shepherd has to answer are:

   1. What type of RFC is being requested (BCP, Proposed Standard,
      Internet Standard, Informational, Experimental, or Historic)?

   2. Why is this the proper type of RFC?"

>view of it, not whichever is convenient.

I am in favor of a practical approach.  It entails taking all the 
factors into consideration including IETF process.

At 20:21 18-07-2012, Commerco WebMaster wrote:
>The primary reason I piped up was in response to our co-chair's 
>remark about removing SPF backward compatibility in the 
>specification.  Perhaps it was my conditioning by a former

It was an individual comment.  I posted an argument to find some way 
to reach the goal.

At 20:27 18-07-2012, Hector Santos wrote:
>hmmmm, the same was done for DKIM with its SHA1 vs SHA256.  There 
>are other examples I am sure we can find in the annals for IETF 
>protocol development and updates.

I am more interesting in hearing about other examples.

At 21:19 18-07-2012, Murray S. Kucherawy wrote:
>If we're trying to protect people from themselves by discouraging 
>use of these features, attaching an easily ignored labels will not 
>achieve that goal.

There are four instances of the word "discouraged" in RFC 4408.

Regards,
S. Moonesamy

1. http://www.ietf.org/mail-archive/web/spfbis/current/msg00020.html

---

From: Hector Santos <[email-masked]>
Date: Thu, 19 Jul 2012 07:25:42 -0400

> The TDP and Cisco surveys amount to reproducible experiments.  Your say-so

And thats the whole problem with someone and my original compliant 
with this group where someone who was a militant anti-SPF, public 
opponent of SPF with no SPF history, no SPF deployment field 
experiences, with his own site/company and/or thousands of customers, 
a extremely disturbing high propensity to ignore participants input 
COULD NOT be the chair and author of SPFBIS.

We have 9 years of LMAP and SPF experience, in implementation for 3rd 
party usage, site deployment and field deployment and support 
experiences with customers.   You do not. The years of facts there 
existed wizards producing PTR statement is unknown to you, all the 
examples and documentation that exist. The facts there there are PTR 
usage is unknown to you from all angles, the high utility it offers 
especially practical experience is all missing.

You're wasting people time with this nonsense.

How do the chairs what the PTR data?  I can post a small sample here, 
a full attachment (waste of bandwidth) or a provide a link?

-- 
HLS

---

From: Andrew Sullivan <[email-masked]>
Date: Thu, 19 Jul 2012 07:38:22 -0400

No hat.

Dave,

On Wed, Jul 18, 2012 at 09:59:02PM -0700, Dave Crocker wrote:
> This strikes me as the single-most basic and compelling question,
> and it seems to be getting ignored.

I was not ignoring it.  I was arguing instead that, given our charter,
it's not the most basic and compelling question, even if one believes
it should be.

> desirable

Irrelevant.  It doesn't matter whether they are desirable, given the
charter.  It matters whether they are used.

Best,

A

-- 
Andrew Sullivan
[email-masked]

---

From: Andrew Sullivan <[email-masked]>
Date: Thu, 19 Jul 2012 07:54:32 -0400

No hat.

> would complain if that aspect of the protocol were removed?  I'd be happy

No.

The surveys check a tiny percentage of the domains on the Internet.  We
don't even know what percentage, because we don't have a practical way
of actually enumerating the domains on the Internet.

What the surveys show is that these features are in use by a very
small percentage of the population.  If we assume that our sample is
representative of the Internet (and that's a big leap, but for the
sake of this argument it doesn't matter), we may conclude that the
features are in use in a tiny percentage of the Internet population's
case.

Now, since we don't have a way of contacting every operator of every
domain on the Internet, even if you contacted those 203 people and
found that they were using these features in a way such that they
could easily adapt to elimination of the feature, that would in no way
provide you with evidence that the features are "unusued".  Your only
evidence is still your starting evidence, which is that the feature is
used by a tiny percentage of the population.

Again, to me this is a charter issue.  If we want literally to make
this removal, then the charter needs to change.

> > code to handle it anyway.
> 

That's an even better response than, "I'll do what 4408 does," because
if we are to remove the features knowing that they used to be
published by people, we will need to say what one does when confronted
with use of that feature.  (The analogy with RRTYPE 99 fails here,
because by removing it we actually ensure that it doesn't get looked
up; so no software needs to accommodate running into it.)

Best,

A

-- 
Andrew Sullivan
[email-masked]

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 19 Jul 2012 06:11:53 -0700

> used by a tiny percentage of the population.
If we're going to accept that these are representative samples, then
discovering that the 203 would be willing to adapt to elimination of the
feature also means we have to accept that the same is likely true of those
domains the survey didn't explicitly discover.

As you said, and I agree, the best we could ever hope to achieve is a
representative sample.  That logic worked for reaching the conclusions in
draft-ietf-spfbis-experiment, and I don't see why it doesn't apply now.

-MSK

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 19 Jul 2012 09:23:08 -0400

> There are four instances of the word "discouraged" in RFC 4408.

Actually it's two "discouraged" and two "discourage".  The former are relevant 
to this discussion.  Use of PTR is "discouraged".  Use of wildcards and SPF 
records is "discouraged".  Use of macros that can't be cached (including the 
dreaded local-part macro) is something that one "should avoid".

I find it interesting that the precise features RFC 4408 pushed people away 
from are among the ones we find little uptake for.  I think that's at least 
indicative of the idea that people actually read the RFC and take what it says 
into account.

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 19 Jul 2012 09:27:18 -0400

On Thursday, July 19, 2012 09:28:30 AM Alessandro Vesely wrote:
...
> --which is already possible, obviously.
...
If you aren't implementing all the functionality of the protocol, you aren't 
performing an SPF check, so some variant of "Decided not to check this one" is 
correct.  

I actively don't like my idea, I just think it's slightly better than 
implementations doing a subset of the specification and then claiming domains 
that use the parts they chose not to implement are somehow in error.

Scott K

---

From: Andrew Sullivan <[email-masked]>
Date: Thu, 19 Jul 2012 09:54:50 -0400

Still hatless.

> domains the survey didn't explicitly discover.

I think that's false for two reasons.

1.  It is one thing to assume that things we look up in the DNS are
representative of what DNS operators are doing generally.  It is quite
another thing to assume that the motivations and trade-offs people
have made in configuring their DNS zones that way are also all the
same, or that the same considerations would apply uniformly if
everyone faced the same stimuli.  (In fact, Kenneth Arrow made a name
for himself investigating these matters.  Cf. Arrow's impossibility
theorem.  A key insight is that, even if individual preferences are
transitive, group preferences are not.  Empirically, however, it turns
out that individual preferences aren't transitive, either, which has
some nasty implications for what we think of as reason.  This is
rather off topic, though :) )

2.  By contacting the operators of the 203, you give them an advantage
that nobody else who is using the features has: you're warning them
explicitly.  If you ask someone, "Given that we're going to change
this, can you adapt?" they automatically know to look for a change.
Anyone else who is using these features won't get that warning, and so
they will necessarily be affected differently than these 203.  (In
effect, you are introducing a change to the sample that you aren't
introducing to the population, and then concluding that the
sample's response to that change is indicative of the population's
response.  This might be true, but the population doesn't get that
change, so it doesn't matter.)

In general, the "unused" argument isn't going to work _given our
charter_ for the case where we persuade known users to change their
minds.  Having evidence of use in hand, I think we can't argue
"unused", even if we persuade some people to stop using.

Let me make a different argument.  Perhaps I am being heavily
influenced by my experiences in DNS-land.  In the DNS community, we
simply accept that anything that gets turned on anywhere is something
we're going to have to live with for a very long time.  When we
deprecate something, it takes years to get rid of it.  Similarly in
this case, 4408 has some features we (or anyway, I) don't like.  We
know that people are using them.  Stating outright that the feature is
deprecated is actually _better_ than just removing the feature,
because we explicitly interoperate with the code in the wild while yet
saying what we think people should be doing.  The argument for "just
getting rid of" these features leaves us with the problem of what to
do when a 4408bis system encounters a 4408 system using these
features.  We need to say something about that, in my opinion, or
we're not doing our job -- and that's true even if we say that systems
MUST NOT publish such records and MUST NOT interpret them.  I prefer
Scott's version of deprecation for this because I think it maximizes
interoperability while yet directing people on the right path.

Best,

A

-- 
Andrew Sullivan
[email-masked]

---

From: Andrew Sullivan <[email-masked]>
Date: Thu, 19 Jul 2012 10:02:39 -0400

No hat.

On Wed, Jul 18, 2012 at 11:53:09PM -0400, Hector Santos wrote:

> First, more and more SMTP servers are already requiring a PTR for
> the connecting client, so this result will already be in server's
> DNS resolver cache by the time SPF is checked and processed.

Whoa. 

Some time ago, DNSOP failed to reach consensus on a draft, which died
as draft-ietf-dnsop-reverse-mapping-considerations-06.  There were
plenty of things wrong with that draft (not least of which was the way
it became a soggy blancmange in an attempt to address everyone's
objections), but one distinction it made that is useful is between
_existing_ reverse mapping data and _matching_ reverse mapping data.

In my experience, most SMTP servers that are checking for a reverse
mapping are in fact checking that there is a reverse mapping entry at
all.  They are checking for existing reverse data.

Many spam-scoring systems also check for match, but a
forward-reverse match is _much_ harder to ensure, and that can lead to
a high false positive rate.

The ptr mechansim in 4408 checks for matching reverse data.  This is
both a higher bar to leap, and more expensive to process, than a mere
existence check.

Best regards,

A

-- 
Andrew Sullivan
[email-masked]

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sat, 21 Jul 2012 21:17:45 -0700

On Wed, Jul 18, 2012 at 6:56 PM, Murray S. Kucherawy <[email-masked]>wrote:

> The TDP survey shows a fraction of a percent using "ptr" or macros.
>

Mea culpa.  I got the first part of this claim wrong.

Hector's analysis of my data is roughly right about the PTR data in our
survey.  There's a nuance of the selection phrase I was using in SQL that I
wasn't aware of, which produced a very restricted result.  His query was
also not quite accurate, but his result is a lot closer to the truth.

Others should feel free to download Philip's and my databases to verify
these claims for themselves.  Again, they are available at
http://www.blackops.org/~msk/spfbis.  The "live" TDP database is more
current than the one available there but only by a relatively small number
of records.

To be clear: TDP's survey claims about 35% of SPF-aware sites are using
"ptr", not a fraction of a percent.

However, as I said in another thread, non-use of PTR is not (as I
understand it) the basis for lobbying for its deprecation or removal, but
rather its very design.

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sat, 21 Jul 2012 21:25:51 -0700

> all.  They are checking for existing reverse data.

At least sendmail checks for both.  If the matching check fails, it claims
the value returned in the PTR payload "may be forged" and logs it as such,
and in fact won't inform attached filters of the name it found just to be
safe.

I don't know about others, though I imagine postfix is the same since it
tries to conform to that same filtering API.

-MSK

---

From: "John Levine" <[email-masked]>
Date: 22 Jul 2012 05:00:31 -0000

>tries to conform to that same filtering API.

For those of us running qmail and its many variants, it's optional to
check that the rDNS resolves forward to the connecting IP, but my
impression is that everyone turns the option on.

If the check fails, it doesn't log any name in the Received: line,
just the IP.

R's,
John

---

From: Scott Kitterman <[email-masked]>
Date: Sun, 22 Jul 2012 02:02:43 -0400

> tries to conform to that same filtering API.

Postfix can reject mail that fails either the existence check or the matching 
check, depending on how it's configured.  It doesn't write unverified PTR names 
into received headers.

Scott K

---

From: Andrew Sullivan <[email-masked]>
Date: Sun, 22 Jul 2012 23:08:10 -0400

> and in fact won't inform attached filters of the name it found just to be

That'll teach me to write quickly.

Yes, most servers check for both IME.  But in terms of actually
rejecting mail, they check for existence.  When checking for matches,
they generally accept mail with a warning, log message, &c.  This is
because the matching-reverse test is just too strong, and there's no
reason to expect it to work reliably in practice.

Or anyway, that's what all the mail people I talked to (when I was
working on the dnsop reverse-mapping draft) told me.

Best,

A

-- 
Andrew Sullivan
[email-masked]

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 23 Jul 2012 11:18:24 +0200

On Sun 22/Jul/2012 06:17:45 +0200 Murray S. Kucherawy wrote:
> 
> Hector's analysis of my data is roughly right about the PTR data in
> our survey.  There's a nuance of the selection phrase I was using in
> SQL that I wasn't aware of, which produced a very restricted result. 
> His query was also not quite accurate, but his result is a lot closer
> to the truth.

Oh well, query bugs had been eluding our analysis of statistical data
reliability for too long...  Thanks to Hector for pointing that out.

I'd suggest we use `mysql --tee query.txt' and attach (part of) the
resulting file, if at all possible.

> To be clear: TDP's survey claims about 35% of SPF-aware sites are
> using "ptr", not a fraction of a percent.

Using an older version of your data I get 31%, but only 0.8% specify a
target-text to be matched.  Many of the records that use ptr have the
form "v=spf1 a mx ptr ~all".  Disregarding four of those idioms, I'd
reckon the responsible use of ptr is somewhere between the former 0.8%
and 6% of v=spf1 records.

(The two "named" idioms can easily be traced back to
https://my.bluehost.com/cgi/help/txt_record and
https://my.hostmonster.com/cgi/help/txt_record .)

The widespread use of (bad) idioms may indicate that a deployment
document --specifically for wizards and verification sites-- would not
be a bad idea after all.

> understand it) the basis for lobbying for its deprecation or removal,
> but rather its very design.

Yes, but PTR is now back to "unneeded".

---

From: Hector Santos <[email-masked]>
Date: Tue, 24 Jul 2012 03:48:15 -0400

> Yes, but PTR is now back to "unneeded".

Up there in our support incident history, and often also upon 
unsolicited observation "Hey, you web site can be faster if..." during 
an unrelated support incident, our customers report their web site is 
"slower" than they preferred, page rendering is not "snappy."  The 
answer is simple - check your DNS performance and/or disable rDNS 
which is only required to log the domains in their HTTP logs. Serves 
no other purpose.

Does that mean they really don't need it to the extenr that we can now 
consider deprecating rDNS support and maybe, perhaps in a future 
version remove the rDNS checkbox option in the name of removing all 
future support calls and incidents?

I don't think so.

The fact is that was a higher issue in years past, but in the modern 
hosting world, everything is faster, including DNS server access.  In 
fact, I know a number of customers who switched from there a 
relatively fast ISP uplink provider to a free super faster cache 
servers.  Nothing is always free, they get you with the tracking and 
redirection of bad urls, etc, but no doubt, they are large and super 
fast.  I believe Google is among them with free DNS IP server addresses.

The PTR "Slowness" issue IME has become a "thing of the past' with 
faster hardware, high speed connectivity, better DNS databases, better 
DNS setups, records, automated caching and/or record assignments, 
temporary or otherwise makes the rDNS more feasible today and probably 
this is reflective in the greater amount of application enabling it, 
like many MTAs now requiring clients to have a PTR record.  For me, 
this was a real case where we did follow the market trend, the market 
leaders because the overhead was there for the customer base we 
served. I pay very careful attention to optimizing our product because 
it pays in the long run to have a high QA out of the box with little 
support overhead, and in the two years since we implemented rDNS for 
our customer Anti-Spam scripts, I haven't heard one complaint yet.

-- 
HLS

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 24 Jul 2012 17:13:30 +0200

> our customer Anti-Spam scripts, I haven't heard one complaint yet.

It is not so much the performance as the effectiveness and reliability
of the results.  No doubt, some domains are able to use PTR in a
reasonable way.  However, rDNS is already a "derived" datum, in the
sense that it should be set after the corresponding DNS entries.
Errors may remain unnoticed for a long time.  As a third-hand tool,
PTR can give a false sense of soundness.

In addition, SERVFAIL cases seem to be more frequent when using PTR,
possibly because secondary servers are not set up equally well for
rDNS, if those zones are delegated at all.

It is true that EXISTS and IP6 are even less used than PTR, but if I
were to strike mechanisms from the spec --in order to simplify it--
I'd choose PTR and MX:  They seem to be more suitable for heuristic
reasoning than for conveying precise policy statements.

---

From: Hector Santos <[email-masked]>
Date: Tue, 24 Jul 2012 12:40:20 -0400

Hi, this is probably all true, no reason to doubt it, but unless there 
is an real cost impact/problem analysis to warrant deprecation, we are 
simply speculating about something  we may or may not still believe is 
a problem today as it was more sure of yesteryears.

Consider, this change was done on assumption, not based on any report 
of a real measured problem existing costing anyone a problem or even 
the costing the network.  We are making  an intelligence engineering 
guess since it is "overhead" therefore its a problem, and we were 
wrong that it wasn't in use, therefore its open for deprecation. 
Thats a big impact.

Consider, we all knew about the cost of PTR and yet, I don't see any 
resistance or barriers to usage.  As you noted, it was mentioned in 
the old draft, yet, it still happen. In other words, if we had annual 
data collection, if this old recommendation was followed, then we 
would expect the usage of PTR to be on a down curve, and not only for 
SPF but for MTAs and other applications - less applied usage.  In 
fact, I submit we are seeing the opposite - an annual increase usage 
(at the more applied usage level).

I call it becoming more feasible today.   Look, we can say the same 
for some many things that the purist will tell ya is high overhead, 
for example XML, Javascripting, etc.  XML today is viewed as a 
"Language" for god sake by application developers, and it took a while 
but even system level developers have given in.  It all has to do with 
the increase speed and throughput of processing allowing its 
"productivity" value to be realized and feasible to use. Would many 
even consider using XML in years past for XYZ applications? I say no. 
Today, Yes. People don't even sweat it.  Same with JSON, same with 
depending on interpretative languages as opposed to compiled or even 
p-code, why? Well, interpretative feeds into dynamic thinking, 
artificial intelligence creatively, code creating code, etc.   Ok with 
JIT, pcode, but not as dynamic if compiled code/data was done for the 
purpose of obtaining high speed.  Its a balance of size vs speed.

--
HLS

-- 
HLS