Subject: Re: [spfbis] #9: RFC 4408 SPF RR type

From: S Moonesamy <[email-masked]>
Date: Mon, 20 Feb 2012 09:00:39 -0800

Hello,

We would like to start the discussions about the issues.  Here are 
some points from the SPFBIS Charter as guidelines:

   Changes to the SPF specification will be limited to the correction
   of errors, removal of unused features, addition of any enhancements
   that have already gained widespread support, and addition of
   clarifying language.

   Specifically out-of-scope for this working group:

     * Revisiting past technical arguments where consensus was reached in
       the MARID working group, except where review is reasonably
       warranted based on operational experience.

     * Discussion of the merits of SPF.

     * Discussion of the merits of Sender-ID in preference to SPF.

     * Extensions to the SPF protocols.

     * Removal of existing features that are in current use.

Issue #9 is up for discussion.  It is based on the following:

"This may be a can of worms,  but since we're chartered to tidy up 
where appropriate, I wonder if this might be such a spot.

RFC4408 registered a new DNS RRtype, SPF, code 99.

Are we able to tell how much deployment it's received?  If only 
little (which is what I suspect, but I've no data to back that up), 
would it be appropriate to relegate its discussion to an appendix and 
call it "historic"?"

The WG Chairs request your comments.  Please keep the subject line so 
that it is easy for us to keep track of the discussion.

Regards,
Andrew Sullivan & S. Moonesamy, SPFBIS co-chairs

---

From: "John Levine" <[email-masked]>
Date: 20 Feb 2012 21:23:46 -0000

I'm OK with the list.

>RFC4408 registered a new DNS RRtype, SPF, code 99.

I think it's worth looking at the SPF RR and deciding whether it
provides an overall benefit.

R's,
John

-- 
Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. http://jl.ly

---

From: Commerco WebMaster <[email-masked]>
Date: Mon, 20 Feb 2012 11:53:47 -0700

John,

Historical observation.  I think you may find that the SPF RR was done 
in part to support version SPF V1 in addition to using the TXT RR, but 
the major perceived benefit of having the SPF RR in place and available 
was for any future SPF versions (e.g., V3 the next logical future 
version of SPF V1).

As I recall, in those days there were some concerns about overusing the 
TXT RR, making a separate RR for SPF a logical step for both the 
applications developers and the user implementers (those who put the RRs 
in their DNS host files).

As such, I think its existence does provide a benefit for both V1 and 
any future SPF implementations.  Arguably, far less user implementers 
have added SPF RR records to their host files than TXT RR records, but 
then the V1 experimental does support both.  My guess is that a future 
SPF V3 will probably have only want to support the SPF RR.

Best,

Alan M.

---

From: Scott Kitterman <[email-masked]>
Date: Mon, 20 Feb 2012 14:19:11 -0500

> that it is easy for us to keep track of the discussion.

I would like to suggest a nuance about the removal aspect of the charter.  I 
think there is an implicit expectation in the way that the charter is drafted 
that removal of features that are in use will inevitably harm 
interoperability.  In this one case, I don't think it does.

We can tell based on DNS surveys that there is very little Type SPF 
deployment.  In cases where there is a type SPF record, there are three 
possible cases:

1.  Identical Type TXT record published

2.  Non-identical type TXT record published

3.  No TXT record published.

Since virtually all SPF records are only published as Type TXT, there is no 
ineroperability benefit to also publishing Type SPF.  

If there is a non-identical record published the publishing domain will get 
inconsistent results since not all receivers of SPF records look up Type SPF. 
Publishing the same information in two places places an operational burden 
(not just the dual publishing, but also keeping the information in sync).

If there is no TXT record and just a Type SPF record then the results are 
going to be poor.  Only Type SPF checkers will see the record.

The RFC 4408 language (SHOULD check SPF first) also causes additional DNS 
lookups that will almost always be fruitless.

I think that Type SPF was created based on fear of what might happen if Type 
TXT were overused.  Those fears have not come to pass and we should drop Type 
SPF as harmful to interopability.  There is no operational case for a 
transition now and I think it's safe to assume there won't be.

There is no reasonable case where dropping Type SPF will cause SPF record 
publishers to change their records (I don't consider publishing only a Type 
SPF record a reasonable case - it just doesn't work due to the predominance of 
Type TXT in SPF records).

I don't object to reserving Type SPF for some future non-compatible SPFvX, but 
for v1, I think we can and should drop it.

Scott K

---

From: Andrew Sullivan <[email-masked]>
Date: Mon, 20 Feb 2012 14:31:03 -0500

No hat.

On Mon, Feb 20, 2012 at 11:53:47AM -0700, Commerco WebMaster wrote:

> As I recall, in those days there were some concerns about overusing
> the TXT RR, making a separate RR for SPF a logical step for both the
> applications developers and the user implementers (those who put the
> RRs in their DNS host files).

Not only "in those days".  Some of us who have spent a little time
working on the DNS continue to have reservations about overuse of the
TXT RRTYPE on the grounds that the DNS is a typed database, and the
overloading of TXT violates the typing system.  I am aware of the
reasons why people use TXT records, and I'm not the least bit
surprised that the SPF RRTYPE turns out to have been a waste of bits.
But TXT subtyping has its problems (IMO not least of which is that
we're seeing three layers of underscore, with no principled reason why
it shouldn't go deeper).  Someone will accuse me of empty or, worse,
academic formalism, I'm sure; but I still think the TXT records are a
dirty hack that mostly works because there are still few enough of
them that it doesn't hurt yet.  I don't believe that will be true
forever (although I've certainly been wrong before), and when there
are enough TXT records with in-RNAME subtyping that we have a problem,
it will be much harder to clean up.

> RR records, but then the V1 experimental does support both.  My
> guess is that a future SPF V3 will probably have only want to
> support the SPF RR.

While I pray to the DNS fairy that your prediction comes true, I don't
believe it will.  SPF v3 will need to be backward compatible with v2
and v1, which means that it'll use TXT too.

-- 
Andrew Sullivan
[email-masked]

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 20 Feb 2012 11:38:08 -0800

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of S Moonesamy
> Sent: Monday, February 20, 2012 9:01 AM
> To: [email-masked]
> Cc: Andrew Sullivan
> Subject: Re: [spfbis] #9: RFC 4408 SPF RR type
> 
> Are we able to tell how much deployment it's received?  If only little
> (which is what I suspect, but I've no data to back that up), would it
> be appropriate to relegate its discussion to an appendix and call it
> "historic"?"
> 

These data were submitted earlier by Philip Gladstone in response to this question:

http://www.ietf.org/mail-archive/web/spfbis/current/msg00354.html

Are there others on the list that can produce similar reports?  My own data roughly match the large number of SPF-publishing domains, but don't break it down by RR type.  I can work on creating such a report but it will take me a week or two to do so due to other commitments.

-MSK

---

From: Commerco WebMaster <[email-masked]>
Date: Mon, 20 Feb 2012 14:03:56 -0700

> No hat.
:-)
> it will be much harder to clean up.

+1 agree entirely.

> and v1, which means that it'll use TXT too.

Perhaps a way to get around the issue of TXT RRs backward compatibility 
with SPF V1 in future version may be found in how SPF V3 and other 
future versions will specify how they handle such requests.

In other words, in any future versions, specify a check for SPF RR as 
the first request and then, failing an SPF RR, go look for a TXT RR for 
compatibility with V1 in yet another request.

With enough requests for SPF RR preceding the TXT RR requests, as the 
newer implementation pattern become more frequently observed, hopefully 
most DNS administrators will get the hint and establish the new SPF RRs 
in their authoritative host zone files and ultimately be able to get rid 
of the SPF V1 TXT RRs.

Best,

Alan M

---

From: Dave CROCKER <[email-masked]>
Date: Mon, 20 Feb 2012 13:13:16 -0800

On 2/20/2012 11:19 AM, Scott Kitterman wrote:
> interoperability benefit to also publishing Type SPF.

This is probably the essential, objective observation.

If correct, it means that there is both no benefit in retaining the record and 
no detriment in removing it.[1]

Even better is that removing it makes the specification that much simpler.

Considering how it "might" be used or how it "should" be used or any other 
hypothetical or preference is not productive.

d/

[1] FWIW I both would expect it to be true and believe it actually is.

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

---

From: Scott Kitterman <[email-masked]>
Date: Mon, 20 Feb 2012 16:13:21 -0500

On Monday, February 20, 2012 02:03:56 PM Commerco WebMaster wrote:
> :
> :

We aren't here to debate some theoretical future SPF version, but the one we 
have.  Type 99 (SPF) is not serving a useful purpose right now, so we should 
mark it deprecated (don't use) and quit worrying about it.  The DNS Type can't 
be reused for something else, so it can always be revived in the event there's 
some future effort that can effectively use it.

Scott K

---

From: Dave CROCKER <[email-masked]>
Date: Mon, 20 Feb 2012 13:17:16 -0800

On 2/20/2012 11:31 AM, Andrew Sullivan wrote:
> overloading of TXT violates the typing system.

Use of a TXT directly under the domain name doesn't scale, because it takes the 
equivalent of deep packet inspection to distinguish among different apps' use of 
the record.[1]  It also means that all of those records get return together, 
potentially hitting some maximums.

However, SPF uses the record already and doesn't use the new RR.  No precedent 
is being set by acknowledging this reality.  In spite of the fact that an SPF RR 
is a cleaner approach, enough time has passed to make clear that it's not being 
adopted and, therefore, won't be.

d/

[1]  Just to be thorough, it's worth noting that use of an underscore nodename 
fixes the scaling and ambiguity concerns, since it defines the /one/ application 
using the record.

-- 

   Dave Crocker

---

From: Andrew Sullivan <[email-masked]>
Date: Mon, 20 Feb 2012 16:34:55 -0500

On Mon, Feb 20, 2012 at 01:17:16PM -0800, Dave CROCKER wrote:

> However, SPF uses the record already and doesn't use the new RR.  No
> precedent is being set by acknowledging this reality. 

Yes, I think that's what I was saying.

> [1]  Just to be thorough, it's worth noting that use of an
> underscore nodename fixes the scaling and ambiguity concerns, since
> it defines the /one/ application using the record.

Only sort of, and it brings with it additional costs; but I don't
think this is the right list on which to have that debate.  We don't
have to agree about why some dumb thing has managed to establish
itself (or even whether it's dumb) in order to agree _that_ it has
established itself, and the latter is the only question here.

A

-- 
Andrew Sullivan
[email-masked]

---

From: Dave CROCKER <[email-masked]>
Date: Mon, 20 Feb 2012 13:40:15 -0800

On 2/20/2012 1:34 PM, Andrew Sullivan wrote:
> think this is the right list on which to have that debate.

Agree that pursuing this isn't for this list; however in practical terms, that 
change in how the record has used seems to have fundamentally altered the 
acceptability of using TXT (and SRV) records.

I'll also mention that I hadn't heard the "additional costs" concern as a 
serious concern.  To the extent it has meaningful constituency, it would be 
worth finding the right venue for resolving it.

-- 

   Dave Crocker

---

From: "John Levine" <[email-masked]>
Date: 20 Feb 2012 21:40:02 -0000

>is being set by acknowledging this reality.

That was my thinking.  But at the moment, all we need to do is to put
the type 99 RR on the list of issues to settle before we're done.

R's,
John

---

From: Hector Santos <[email-masked]>
Date: Mon, 20 Feb 2012 17:38:11 -0500

As I recall, the main MARID reason why we could just use a new RR was 
because around that time frame, there were still many DNS servers that 
did not support the recursion requirement to support new RR record SRV 
queries.   That was the stated reason I recall and I believe there was 
general agreement about that.

Microsoft DNS 4.0 was still a dominant server and it did not support 
recursion requirement nature, and if I recall, there was a patch but 
it as a hot request (not downloadble).

Even AD suffered as well, as well as JABBER and other protocols that 
had a specific RR record but  the fall back to common A/TXT was necessary

Windows 2000 was still just 3 years old, and DNS 5.0 came with Windows 
2003 so there was some migration period there.  Today, I will state 
that we would be very or more successful as evidence of better 
operations with AD and XMPP/Jabber.

FTR, there were also other non-MS DNS servers that also had issues so 
even if I got the patch for DNS 4.0 back then to explore new RR types 
(not just for SPF), it worked at the DNS 4.0 with manual updating of 
zone records and with direct queries in a LAN, but not always with 
queries across a WAN/internet.  I gave up on these "create a SRV 
protocol" projects because of this fact - simply too early for prime 
time reliability.  I think it is ready today.

S Moonesamy wrote:
> Hello,
> 
> We would like to start the discussions about the issues.  Here are some 
> points from the SPFBIS Charter as guidelines:
> 
> "This may be a can of worms,  but since we're chartered to tidy up where 
> appropriate, I wonder if this might be such a spot.
> 
> (which is what I suspect, but I've no data to back that up), would it be 
> appropriate to relegate its discussion to an appendix and call it 
> "historic"?"
> 
> _______________________________________________
> spfbis mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/spfbis
> 
> 

-- 
Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com

---

From: Hector Santos <[email-masked]>
Date: Mon, 20 Feb 2012 17:56:52 -0500

I believe we could be jumping the guns because we need to consider the 
reasons why wasn't used and that include the fact that there was still 
a large population of DNS servers that didn't support the 
passthru/recursion requirements to make new RR named work without 
flaws and not requirement a fallback to common support record.

I don't think that dearth of RR records is an issue today.  The 
experience the same issue when I installed my Jabber Server a few 
years back. It had a SRV record for discovery and a fallback.  I 
recall believing just using the SRV record was enough and depending on 
the query path, it works very nicely, but for other paths it didn't 
and I had no choice but the add the other records. And finally when I 
expanded the jabber network with peers, maybe 1.5/2 years ago, I 
recall how smoothly it all worked not with these other paths.    Same 
experience with we installed a AD/LMAP server, but that was more 
successful because it was done more recently.

I suggest we talk about it and if we believe and conclude that it 
technical feasible and will have a high reliability and it improve the 
SPF protocol existence within DNS for SPF-BIS era and years to come, 
we should not just blatantly abandon it the original ideas we had for 
it - which was a long term support concept.  We all knew it wasn't 
going to be reliable back then - not all DNS servers were ready for 
it, but we want to get ready for it.

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 21 Feb 2012 17:59:38 +0100

>> in their authoritative host zone files and ultimately be able to get rid

Grepping "view external: query: $helo_name IN SPF", I found their
count came out at 85% of the corresponding TXT.  Yet I have no SPF RR,
so 85% is obviously not enough.

> mark it deprecated (don't use) and quit worrying about it.

+1, although we should praise those who were so clever to use SPF, and
suggest to concede plenty of compatibility support.  However, as it's
already possible that a 4408-compliant checker ignores the SPF RR
type, we should recommend that new applications ignore it too.

---

From: Scott Kitterman <[email-masked]>
Date: Wed, 22 Feb 2012 11:32:59 -0500

...
> would it be appropriate to relegate its discussion to an appendix and
...

Since discussion on this seems to have died down and we've a dearth of other 
on topic topics to talk about, I'll attempt a summary (I know there is still 
data collection going on, but I think it's reasonable to assume that they will 
confirm the current sense that deployment is miminal).

It seems to me that the group is in favor of deprecating type SPF.  We can 
call it historic, but since the RRtype will still be assigned, it could be 
used if there were ever a non-backward compatible update to SPF.

Any objections to that as a sense of the group?

Scott K

---

From: Hector Santos <[email-masked]>
Date: Wed, 22 Feb 2012 11:47:20 -0500

> Any objections to that as a sense of the group?

Then why bother marking as deprecated and cause more work in the future?

I object to the idea SPF can not use a special RR nor the idea it 
can't be further explored with high reliability that will exist today. 
  -1 on dropping it.

---

From: Dave CROCKER <[email-masked]>
Date: Wed, 22 Feb 2012 08:58:04 -0800

> used if there were ever a non-backward compatible update to SPF.

I think the summary is accurate.

I don't know whether the registry can be changed to deprecate the type, but I 
suspect it can.

d/
-- 

   Dave Crocker

---

From: Scott Kitterman <[email-masked]>
Date: Wed, 22 Feb 2012 12:00:14 -0500

On Wednesday, February 22, 2012 08:58:04 AM Dave CROCKER wrote:
> > It seems to me that the group is in favor of deprecating type SPF.  We
> > can call it historic, but since the RRtype will still be assigned, it
> > could be used if there were ever a non-backward compatible update to
> > SPF.
> I don't know whether the registry can be changed to deprecate the type, but
> I suspect it can.

Even if it can't, we can mark it SHOULD NOT use in 4408bis.

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Wed, 22 Feb 2012 12:00:29 -0500

On Wednesday, February 22, 2012 11:47:20 AM Hector Santos wrote:
> > Since discussion on this seems to have died down and we've a dearth of
> > other on topic topics to talk about, I'll attempt a summary (I know
> > there is still data collection going on, but I think it's reasonable to
> > assume that they will confirm the current sense that deployment is
> > miminal).
> > 

The reason to mark it deprecated is there's no value in using it and retaining 
it adds complexity.  

There's no "high reliability that will exist today".  Many implementations 
either default to not checking RRtype SPF by default or don't support it at 
all.

I took a quick look at pyspf and dropping type SPF only saves about 3 dozen 
lines of code.  Most of the complexity that's avoided by removing it is the 
operational complexity of having to maintain identical data in two different 
DNS records.

Scott K

---

From: Dave CROCKER <[email-masked]>
Date: Wed, 22 Feb 2012 09:30:50 -0800

> The reason to mark it deprecated is there's no value in using it and retaining

+1

d/
-- 

   Dave Crocker

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 22 Feb 2012 17:34:57 +0000

> From: [email-masked] [mailto:[email-masked]] On Behalf Of Scott Kitterman
> Sent: Wednesday, February 22, 2012 9:00 AM
> To: [email-masked]
> The reason to mark it deprecated is there's no value in using it and
> retaining it adds complexity.

+1.

-MSK

---

From: Paul Midgen <[email-masked]>
Date: Wed, 22 Feb 2012 17:43:12 +0000

+1

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Murray S. Kucherawy
Sent: Wednesday, February 22, 2012 9:35 AM
To: [email-masked]
Subject: Re: [spfbis] #9: RFC 4408 SPF RR type

+1.

-MSK

---

From: Andrew Sullivan <[email-masked]>
Date: Wed, 22 Feb 2012 13:23:28 -0500

Colleagues,

With all due respect, I think people might reign in their enthusiasm for making peremptory consensus calls. Thanks. 

-- 
Andrew Sullivan 
Please excuse my clumsy thumbs. 

On 2012-02-22, at 11:58, Dave CROCKER <[email-masked]> wrote:

> I don't know whether the registry can be changed to deprecate the type, but I suspect it can.
> 
> d/
> -- 
> 
>  Dave Crocker

---

From: John Leslie <[email-masked]>
Date: Wed, 22 Feb 2012 13:33:23 -0500

Andrew Sullivan <[email-masked]> wrote:
> 
> Colleagues,
> 
> With all due respect, I think people might reign in their enthusiasm
> for making peremptory consensus calls. Thanks. 

   +1

   We should trust our WGCs to make consensus calls: it's not helpful
for others to usurp that role.

   (BTW, I disagree with calling type 99 "deprecated" or "historic",
though I'm perfectly happy for it to be "MAY" status on both publishing
and checking.)

--
John Leslie <[email-masked]>

---

From: S Moonesamy <[email-masked]>
Date: Wed, 22 Feb 2012 10:36:32 -0800

Hello,

In case there is any confusion, I would like it to be clear that the 
SPFBIS WG Chairs have not made any determination of consensus about 
Issue #9.  It is premature to draw any conclusion at the moment.  We 
are here to listen to the views expressed by all WG 
participants.  The working group will be given ample time to think 
about the issues and provide comments.

When there is a consensus call on this issue, or any other issue, the 
message about it will be sent by the SPFBIS WG Chairs.  If you have 
any objections about a determination of consensus, you are welcome to 
raise them.

Issue #9 remains open.  If you have not commented on it, please do 
so.  If you have already provided comments on Issue #9, there is no 
need to restate them again as Andrew and I will read all the messages 
that have been posted about this issue.

Regards,
S. Moonesamy
SPFBIS WG Chair

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 22 Feb 2012 19:20:11 +0000

> From: [email-masked] [mailto:[email-masked]] On Behalf Of John Leslie
> Sent: Wednesday, February 22, 2012 10:33 AM
> To: Andrew Sullivan
> Cc: [email-masked]

Yes, that's right of course.  Apologies for contributing.

There's an apparent undercurrent of being in a hurry to progress or complete this work.  I'm not sure why.

-MSK

---

From: "John Levine" <[email-masked]>
Date: 22 Feb 2012 19:36:41 -0000

Looking at my DNS server logs, I am surprised to see nearly as many
requests for type 99 records as for SPF TXT records.  I'm collecting
data, we can talk about it once we're done putting the issues list
together.

I am NOT arguing that there's any particular value in publishing type
99 records, but the common perception that nobody checks them seems to
be wrong.

R's,
John

---

From: Scott Kitterman <[email-masked]>
Date: Wed, 22 Feb 2012 14:41:19 -0500

> 99 records, but the common perception that nobody checks them seems to

Mail::SPF (which is used by Spamassassin) checks for Type SPF first by default.  
I would imagine that accounts for a large fraction of what you're seeing.

Myself I've never argued (or never intended to) that no one checked it.  I 
know they are.  I also know that they are not consistently checked, so they 
can't be relied on.

The "almost no" arguement I've made is about publishing.

I think the fact that there are so many pointless queries for Type SPF argues 
for getting rid of it.  It's a waste.

Scott K

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 22 Feb 2012 19:50:19 +0000

> Sent: Wednesday, February 22, 2012 11:41 AM
> To: [email-masked]
> Myself I've never argued (or never intended to) that no one checked it.
> I know they are.  I also know that they are not consistently checked,
> so they can't be relied on.
> 
> I think the fact that there are so many pointless queries for Type SPF
> argues for getting rid of it.  It's a waste.

My data so far agree with this.  There's lots of software that checks, but fewer than 5% of the SPF or Sender-ID records out there actually publish them.

If for example the world was full of DKIM verifiers but nobody ever signed, I think it would be reasonable to conclude that DKIM is also, effectively, unused.  To use another DKIM example, when we revised DKIM to Draft Standard, we dropped the "g=" tag because our data showed almost nobody was using it, even though most or all DKIM verification software knew how to apply it.

-MSK

---

From: Dave CROCKER <[email-masked]>
Date: Wed, 22 Feb 2012 11:52:27 -0800

If it's not to much trouble, it will probably help to have empirical data like 
this correlate sources and timing.

That is, is a query for a type 99 coupled to (and preceding) a query to the 
corresponding SPF TXT?

d/

-- 

   Dave Crocker

---

From: "John Levine" <[email-masked]>
Date: 22 Feb 2012 19:57:29 -0000

>That is, is a query for a type 99 coupled to (and preceding) a query to the 

Give me a few days, I'll see what I can find.

Many of my domains actually do publish type 99 records, so I'll have
to separate the ones that do and the ones that don't.

R's,
John

---

From: Andrew Sullivan <[email-masked]>
Date: Wed, 22 Feb 2012 15:08:26 -0500

No hat.

> for getting rid of it.  It's a waste.

That would be as strong an argument in favour of _promoting_ the SPF
record and saying that implementations SHOULD check for the SPF record
unless they already know there is no SPF record present.

After all, lots of people didn't used to use MX records either.

Best regards,

A

-- 
Andrew Sullivan
[email-masked]

---

From: Scott Kitterman <[email-masked]>
Date: Wed, 22 Feb 2012 15:21:23 -0500

> After all, lots of people didn't used to use MX records either.

RFC 4408 already says that type SPF should be checked and that it should be 
published.  Despite doubts about if a transition to type SPF could ever be 
made (IMO both those SHOULD statements were statements of bold optimism) the 
SPF community has solidly supported type SPF.

Within a week of Type 99 being allocated to Type SPF there were open source 
implementations of SPF checking libraries that would check it.  In fact, 
community testing between the IANA assignment and the author's 48 hour review 
prior to RFC 4408 publication identified some interoperability issues with 
multiple DNS RR types carrying the same information that were corrected prior 
to RFC 4408's publication.

Even with this support, Type SPF has little traction operationally and there 
is no basis to expect this to change.  Given that, the best thing we can do is 
encourage receivers not to do these pointless queries.

There is an operational benefit to using MX records.  As we've discussed, there 
isn't one for Type SPF.

Scott K

---

From: Andrew Sullivan <[email-masked]>
Date: Wed, 22 Feb 2012 15:44:00 -0500

No hat.

> There is an operational benefit to using MX records.  As we've discussed, there 

That was actually not the conclusion I reached from that discussion.
What I recall concluding was that, in the absence of people actually
doing the lookup, it was going to be pretty hard to argue that there
was any benefit to it.

If the actual deployed software does the SPF lookup first, then there
certainly is an operational benefit from deploying SPF records: you
cut your negative queries out dramatically.  DNS admins like to cut
needless queries.  Moreover, if the bulk of the deployed code sends
SPF first, then just adding an SPF record would be a pain-free way to
cut down all those extra queries.  You'd reduce the pointless SPF
validation queries that your consumers send, too.  And you'd have a
guarantee that other semantics couldn't trample on your RR's
semantics.

Seems like operational benefit to me.  What do you see differently?

Best,

A

-- 
Andrew Sullivan
[email-masked]

---

From: Scott Kitterman <[email-masked]>
Date: Wed, 22 Feb 2012 15:49:49 -0500

On Wednesday, February 22, 2012 03:44:00 PM Andrew Sullivan wrote:
> > There is an operational benefit to using MX records.  As we've
> > discussed, there isn't one for Type SPF.
> 

I was looking at it from the other end.  There's no operational benefit to 
doing the query and so the fact that a lot of software does it wastes 
resources on both ends since so few records are published.

Even if one has the necessary infrastructure support for publishing type SPF 
records (I'm not aware of any commercial DNS providers that support or nor do 
I think standard control panel packages support it) there is the added 
complexity of managing two sets of records that have to be kept identical.

I think, from a record publisher POV that more than offsets not having the 
additional TXT lookup.

In any case, I think the market has spoken.  Despite broad support for the new 
RR type, it just hasn't caught on and I don't think it's going to.

Scott K

---

From: Dave CROCKER <[email-masked]>
Date: Wed, 22 Feb 2012 12:54:13 -0800

> After all, lots of people didn't used to use MX records either.

A history of non-use rarely provides a strong argument for coercive legislation. 
  (The OSI folks misunderstood this.)

Adoption is driven by need.  The MX record and port 587 are good examples of 
this.  Their adoption increased when the community started seeing benefits.

The record has yet to show a benefit in adoption of the special SPF RR and, 
hence, failed to gain that adoption.

The questions should be: what will make that change, and it that change viable?

d/
-- 

   Dave Crocker

---

From: Andrew Sullivan <[email-masked]>
Date: Wed, 22 Feb 2012 16:13:47 -0500

No hat.

On Wed, Feb 22, 2012 at 12:54:13PM -0800, Dave CROCKER wrote:
> The record has yet to show a benefit in adoption of the special SPF
> RR and, hence, failed to gain that adoption.

We agree about this. 

> The questions should be: what will make that change, and it that change viable?

I was suggesting that the question should be, "Why make this change if
already-available operational benefit depends on not making the
change?"

According to what people have said lately in this thread, all of the
following things appear to be true (some are waiting on data for confirmation):

    1.  The bulk of the clients send SPF queries first.

    2.  SPF records and the SPF-use TXT records, when they are
    maintained correctly (and in the latter case maintained to the
    exclusion of other things), have the same use value.

    3.  Not putting an SPF record in your zone causes you to
    experience extra query traffic.

    4.  Not putting an SPF record in your zone causes your clients to
    have to send extra traffic.  

Those who are arguing, "SPF records should be deprecated," are
effectively arguing that all the client software ought to be upgraded
in order to make the extra traffic go away.  I was merely noting that
people could get an immediate operational benefit to themselves simply
by adding the SPF record to their zone; this requires no change in
software in at least a large number of cases, and causes a reduction
in needless DNS traffic.  Seems to me like an operational benefit
available this afternoon to anyone who cared to do it.  Those who want
to deprecate the RRTYPE are arguing that greater operational benefit
would come if we made a specification change and everybody on the
Internet using SPF upgraded their software.  From my perspective, this
sounds like the burden of proof is being put on the wrong party.

Best,

A

-- 
Andrew Sullivan
[email-masked]

---

From: Hector Santos <[email-masked]>
Date: Wed, 22 Feb 2012 16:27:28 -0500

Scott, we adding the type to our DNS resolver and script base SPF 
software.

The only reason it was disabled by default was 100% because their was 
still a large network of DNS servers that simply did not support it. 
  Do you dispute this was not the issue when it was first proposed for 
any protocol pushing a new RR type?

See RFC3597 "Handling of Unknown DNS Resource Record (RR) Types"

   Abstract

    Extending the Domain Name System (DNS) with new Resource Record (RR)
    types currently requires *changes to name server software.*  This
    document specifies the changes necessary to allow future DNS
    implementations to handle new RR types transparently.

It well known it would not work well back then at least in the short 
term until DNS server were updated, especially the Microsoft DNS 4.0 
to 5.0 level.  The SPF type was added to the spec with what I believe 
was a full knowledge it had a low reliable to start and an expectation 
to be used in the future as these DNS servers were updated.

So it is by now surprise the early testing after it was tested would 
have a high failure rate.

Am I off base?

Besides the point evidence is arriving showing the type is already 
used, I suspect people just didn't bother for no other reason perhaps 
their infrastructure was already setup around TXT records.

But with SPF-BIS and knowing today, DNS servers do handle this, I 
believe it is worth a short.

---

From: Hector Santos <[email-masked]>
Date: Wed, 22 Feb 2012 16:34:56 -0500

Why isn't RFC3597 relevant here?

       Handling of Unknown DNS Resource Record (RR) Types

   Abstract

    Extending the Domain Name System (DNS) with new Resource Record (RR)
    types currently requires changes to name server software.  This
    document specifies the changes necessary to allow future DNS
    implementations to handle new RR types transparently.

It was well known when the decision to the type, it would be to 
prepare it for the future as the DNS servers get updated.  If 
anything, the industry got established and comfortable with TXT but 
IMO, the original ideals should not be lost.

Only with a SPF-BIS recommendation, would I go back to the software 
and being enabling it with a high expectation that it will be reliable 
today to a high degree.

---

From: Scott Kitterman <[email-masked]>
Date: Wed, 22 Feb 2012 16:44:02 -0500

> sounds like the burden of proof is being put on the wrong party.

If it's all so obvious that it makes sense to publish SPF records, why aren't 
more people doing it?

My main argument for removing the type is that it simplifies things and does no 
harm.

What's the benefit of not doing away with it?  Nothing as far as I can tell.

I guarantee you that as soon as there's an RFC that deprecates use of type SPF 
I'll be simplifying some code.

Scott K

---

From: S Moonesamy <[email-masked]>
Date: Wed, 22 Feb 2012 13:45:35 -0800

Hello,

I have been following the discussion about SPF RR Type.  It seems 
that the issue is being read as a binary question, i.e. should the 
SPR RR record be kept or not.  Some of the questions that come to my mind are:

  (a) How many domains are publishing SPF RR records?

  (b) How many domains are publishing  SPT TXT records?

  (c) How many domains are querying for SPR RR records?

  (d) How many domains are querying for SPR TXT records?

  (e) What are the implementations out there?

  (f) What DNS records do the implementations query for?

The above is not an exhaustive list of questions.  If I missed any 
questions, let me know.  Without empirical data, it is not possible 
to answer these questions.  Is there anyone who would like to 
volunteer to find the answers to these questions?

Regards
S. Moonesamy
SPFBIS WG co-chair

---

From: Scott Kitterman <[email-masked]>
Date: Wed, 22 Feb 2012 16:48:16 -0500

> any protocol pushing a new RR type?

It was certainly an issue in 2005.  I've seen no evidence to indicate it's not 
an issue now.

> believe it is worth a short.

I've not seen any evidence to support the notion that it's reliable now.  It 
may be.  In the code I support where it's not enabled by default, I've gotten 
zero complaints.  The reverse is not true.

So you think that if 4408bis retains the recommendation to use type SPF, 
adoption would suddenly take off?

I don't.

Scott K

> > RFC 4408 already says that type SPF should be checked and that it should
> > be published.  Despite doubts about if a transition to type SPF could
> > ever be made (IMO both those SHOULD statements were statements of bold
> > optimism) the SPF community has solidly supported type SPF.
> > 
> > Within a week of Type 99 being allocated to Type SPF there were open
> > source implementations of SPF checking libraries that would check it. 
> > In fact, community testing between the IANA assignment and the author's
> > 48 hour review prior to RFC 4408 publication identified some
> > interoperability issues with multiple DNS RR types carrying the same
> > information that were corrected prior to RFC 4408's publication.
> > 
> > Even with this support, Type SPF has little traction operationally and
> > there is no basis to expect this to change.  Given that, the best thing
> > we can do is encourage receivers not to do these pointless queries.
> >

---

From: Scott Kitterman <[email-masked]>
Date: Wed, 22 Feb 2012 16:55:56 -0500

> volunteer to find the answers to these questions?

I think questions a and b have to be linked to some degree.  I'd ask them this 
way:

1.  How many domains publish only Type TXT SPF records
2.  How man domains publish only Type SPF SPF records
3.  How many domains publish both Type SPF and Type TXT SPF records
4.  Of the domains in question 3, how many have identical records.

WIthout knowing the degree of overlap/non-overlap you won't be able to draw 
conclusions about the effect of deprecating support for Type SPF.  Also, I'd be 
interested in the list of domains publishing only Type SPF since I doubt they 
understand what they are doing.

Scott K

---

From: Andrew Sullivan <[email-masked]>
Date: Wed, 22 Feb 2012 17:14:03 -0500

No hat.

> It was certainly an issue in 2005.  I've seen no evidence to indicate it's not 

I think that's oversimplifying things some.

There remain difficulties in some systems with managing unknown
RRTYPEs.  This includes some widely-deployed ones, including at least
some Microsoft libraries.  But any DNS _server_, including any caches,
that has any business responding to queries on the Internet today can
certainly cope with unknown RRTYPEs.  If they couldn't, we'd almost
certainly have heard loud noises when the root of the DNS was signed
last year, and we didn't.

Unlike many people in the DNS community -- a community that is
sometimes tarred with a broad brush -- I believe quite strongly that
there remain serious practical difficulties in some cases with new
RRTYPEs, and I have no interest in pretending otherwise.  But I don't
think it does anyone any service to pretend that it's impossible to
add new RRTYPEs.  The question is one of costs and benefits.

You are arguing that the benefit here isn't worth the additional
cost.  I'm pointing out that, where the DNS is concerned, it takes
years to upgrade software -- indeed, this is exactly the problem that
you're using as a premise.  Therefore, if you say, "Didn't work, rip
it out," you're effectively saying that you'll need to support it for
some period of time -- more than 10 years, is my guess -- anyway.  If
we have at least a 10 year horizon, it's anyone's guess which
direction would be preferable. 

Best,

A

-- 
Andrew Sullivan
[email-masked]

---

From: S Moonesamy <[email-masked]>
Date: Wed, 22 Feb 2012 14:20:09 -0800

Hi Scott,
>I think questions a and b have to be linked to some degree.  I'd ask them this

Ok.

>conclusions about the effect of deprecating support for Type 

I am not drawing any conclusions about the effects of deprecating 
support for Type SPF.  What I am looking for is people who can come 
to the working group with data that answers the questions.  The more 
data the working group can get (e.g. to answer the questions you 
mentioned), the better.

Would you like to volunteer?

Regards,
S. Moonesamy
SPFBIS WG co-chair

---

From: Scott Kitterman <[email-masked]>
Date: Wed, 22 Feb 2012 17:22:34 -0500

On Wednesday, February 22, 2012 05:14:03 PM Andrew Sullivan wrote:
> > It was certainly an issue in 2005.  I've seen no evidence to indicate
> > it's not an issue now.
> 

I don't think I am.  All I said is I don't know, because I don't.

There is an added operational complexity associated with publishing the same 
information multiple places, so there is a transitional cost to support both.

I don't think that SPF record publishers are going to accept that cost without 
a worthwhile benefit.  I believe the evidence we've seen shows that even with 
widespread SPF checking, people aren't publishing.  What's going to change?

Scott K

---

From: Pete Resnick <[email-masked]>
Date: Wed, 22 Feb 2012 16:24:27 -0600

Hatless.

> If it's all so obvious that it makes sense to publish SPF records, why aren't

Maybe they set them up long ago when software wasn't available to create 
SPF records and they could easily create TXT records. Maybe they don't 
realize that failure to set them up causes extra traffic because the 
cost is non-obvious or externalized (i.e., the folks who deploy SPF are 
not the ones that see the cost). It would be good to start asking these 
questions to find out why.

> My main argument for removing the type is that it simplifies things and does no

The harm is that everyone has to upgrade they're querying software to 
stop querying for the SPF record.

> What's the benefit of not doing away with it?  Nothing as far as I can tell.

No software upgrades, and *if* people (due to the publication of the 
standards track document) realize that they should be deploying the SPF 
record, they will and that will cut down DNS traffic.

> I'll be simplifying some code.

New code (even if it is simplifying code) is always risky code. Not a 
huge risk, but a risk nonetheless. Anyway, I'm pretty sure that you are 
one of the more motivated SPF developers. :-)

Now that I know about this issue, I'm going to go check my own server 
and add a 99 record, since I don't think I have one. But I think that's 
because I did it so long ago and never looked to see how to enter a new 
record type into my little server. But I'm pretty sure I'm not an 
average SPF deployer myself.

More questions to be asked.

pr

-- 
Pete Resnick<http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

---

From: Scott Kitterman <[email-masked]>
Date: Wed, 22 Feb 2012 17:34:32 -0500

On Wednesday, February 22, 2012 02:20:09 PM S Moonesamy wrote:
> >WIthout knowing the degree of overlap/non-overlap you won't be able to
> >draw conclusions about the effect of deprecating support for Type
> >SPF.  Also, I'd be
> 

You also asked about the appropriateness of the questions.  If you don't ask 
questions in a way that will allow you to answer the question that's being 
asked, there's no point in collecting the data.

A couple of people have already volunteered to do surveys of published DNS 
records.  I could do it, but I don't know that we need more.  If we do, I can 
help out.

I don't have access to a good way of helping out with the questions about how 
many domains are doing queries.

I can tell you that the major FOSS SPF libraries that I'm familiar with 
(libspf2 (C), Mail::SPF (Perl), and pyspf (Python)) all support Type SPF 99 
(although it's disabled by default in pyspf and Mail::SPF provides a way for 
applications to avoid calling it - I'm not a C programmer, so I'm not sure 
exactly what libspf2 does).

Scott K

---

From: S Moonesamy <[email-masked]>
Date: Wed, 22 Feb 2012 14:57:32 -0800

Hi Scott,
>records.  I could do it, but I don't know that we need more.  If we do, I can

The data that have been posted to this mailing list does not answer 
all the questions I had in mind.

Ok.

>exactly what libspf2 does).

Thanks for the input.  That is not enough for "implementations out 
there".  We need to cover as much as possible and that includes 
non-"FOSS" implementations.

Regards,
S. Moonesamy
SPFBIS WG co-chair

---

From: Scott Kitterman <[email-masked]>
Date: Wed, 22 Feb 2012 18:16:20 -0500

On Wednesday, February 22, 2012 02:57:32 PM S Moonesamy wrote:
> >(libspf2 (C), Mail::SPF (Perl), and pyspf (Python)) all support Type SPF
> >99 (although it's disabled by default in pyspf and Mail::SPF provides a
> >way for applications to avoid calling it - I'm not a C programmer, so I'm
> >not sure exactly what libspf2 does).
> 

Of course.  I'm just giving you what I have.

Scott K

---

From: Dave CROCKER <[email-masked]>
Date: Wed, 22 Feb 2012 15:28:42 -0800

On 2/22/2012 2:24 PM, Pete Resnick wrote:
> The harm is that everyone has to upgrade they're querying software to stop
> querying for the SPF record.

They do not have to.  The /may/ do it, if they want the benefit.

> No software upgrades, and *if* people (due to the publication of the standards
> track document) realize that they should be deploying the SPF record, they will
> and that will cut down DNS traffic.

Right.  No software upgrade and no reduction in wasted queries.

An expectation that people will "realize" they "should" do the SPF RR 
establishes a technical requirement based on hypothetical human behavior, rather 
than established community need.  On the average, the track record for such 
adoption predicates is quite poor.

> New code (even if it is simplifying code) is always risky code. Not a huge risk,
> but a risk nonetheless. Anyway, I'm pretty sure that you are one of the more
> motivated SPF developers. :-)

That invokes a rather unusual argument, for -bis efforts, namely to retain 
everything that is not known to be highly problematic.

Whereas typical -bis efforts are encouraged to drop unused features, to simplify 
the spec and simply implementations and -- often -- simplify operations.

> Now that I know about this issue, I'm going to go check my own server and add a
> 99 record, since I don't think I have one. But I think that's because I did it
> so long ago and never looked to see how to enter a new record type into my
> little server. But I'm pretty sure I'm not an average SPF deployer myself.

d/

-- 

   Dave Crocker

---

From: Hector Santos <[email-masked]>
Date: Wed, 22 Feb 2012 19:10:03 -0500

Knicks/J-LIN cap on.

> direction would be preferable. 

The SRV type also had its similar issues with DNS server support.  The 
server might have supported the record, but there potentially were 
recursion issues along the path.

Testament of increasing protocols usage with successful SRV discovery 
supports will suggest that the 3597 issues are must less than in 2005 
and prior.  But protocols still required to have a fallback for those 
paths that have a legacy DNS server.

Related to SPF, what was interesting to me with my Jabber Server 
installation (OpenFire) was noticing how they also went about the same 
question with DNS record installations for discovery and noticing that 
they hung on to the idea that using a SRV record would be preferable 
and even though it may be duplicate record setup, eventually the SRV 
record will be begin to work more reliability.  It really help with 
networking a few in faster and less query discoveries. Felt more 
"snappier."  I think we should hang on to ours :)

Point is, unlike circa 2005, today, I would not be hesitant to use new 
RR types for SPF and any new protocol work we do. But I think for SPF, 
we still need the fallback until more type 99 records are published 
and I think people will start to published them if we sell it to them 
that its better, more reliable today and it does seem faster - might 
be an illusion. :)  But technical people also know it will help with 
DNS network wide overhead as well and I think that is a big plus for 
helping its standardization.

---

From: Douglas Otis <[email-masked]>
Date: Wed, 22 Feb 2012 17:03:28 -0800

> Any objections to that as a sense of the group?
Dear Scott,

Don't know what calling SPF RR Historic would mean.  It would have made 
sense to deprecate (ab)use of TXT RR, but that opportunity was lost with 
termination of Marid. :^(

Deprecating SPF RR is logical, although not having this alternative 
means TXT RR collisions with other uses can not be avoided.  TXT RR at 
the base domain is a logical location for many uses that must be shared 
with various versions of SPF and Sender-ID.

Regards,
Douglas Otis

---

From: Pete Resnick <[email-masked]>
Date: Wed, 22 Feb 2012 19:21:19 -0600

Let me open by saying: I am absolutely fine with either outcome on this 
particular issue. My only reason for posting was because I thought 
Andrew and Scott were talking past each other, and this is an important 
issue to decide one way or the other. Do take whatever I say here as 
trying to elucidate; I am absolutely ambivalent about the outcome.

So, still trying to be hatless:

> They do not have to.  The /may/ do it, if they want the benefit.

Well, the query for 99 would then be "out of spec", but yes, you are 
correct.

>> standards

:) I take your point that my "if" might be equivalent to "if the moon is 
made of green cheese."

> establishes a technical requirement based on hypothetical human 
> behavior, rather than established community need.  On the average, the 
> track record for such adoption predicates is quite poor.

Yup. The reason to keep it would be entirely *if* you think there's a 
chance that people might start using it. If not, the change to client 
software to eliminate the second query would be reasonable.

>> huge risk,
> That invokes a rather unusual argument, for -bis efforts, namely to 
> retain everything that is not known to be highly problematic.
>
> Whereas typical -bis efforts are encouraged to drop unused features, 
> to simplify the spec and simply implementations and -- often -- 
> simplify operations.

Actually, I think both are true: In moves along the standards track, we 
get rid of unused things, but we also don't dork with things that are 
used if they're not causing problems. My only point was this is an 
unusual situation in that the client side of this piece protocol *is* 
demonstrably used, in that clients *are* sending the queries, and the 
server side is excruciatingly minimally used, meaning that the clients 
are not getting any use out of doing so. Add to that the unusualness of 
the fact that this is *not* progress along the standards track (which is 
what we normally think of as the -bis effort), but getting something 
onto the standards track.

If we decide that the feature is essentially unused except for some 
outliers who won't be bothered if we remove the feature, peachy, we 
should remove it.
If we decide that there is some use, but it's complicating the protocol 
and possibly causing problems, also peachy.
If we decide that it is getting at least some use *and* there is a 
reasonable possibility that increased use will occur and that will 
improve things, the we should go the other way.

I think those are the sorts of things folks should be looking at.

Back to watching from the corner. :-)

pr

---

From: Scott Kitterman <[email-masked]>
Date: Wed, 22 Feb 2012 22:16:32 -0500

> Back to watching from the corner. :-)

I think this is an important, but not absolutely critical, issue for the WG.  
I've made my preference clear (I'm sure), but I'm not going to get 
extraordinarily worked up over it.  This is a nice opportunity to simplify the 
protocol with no interoperability harm, but it doesn't touch on the critical 
issues such as backward compatiblity.

One additional data point that might be of interest ...

I suspect a large fraction of the Type SPF queries people are seeing are due 
to Spamassassin using Mail::SPF and Mail::SPF has Type SPF queries enabled by 
default.  The reason they are enabled by default is that the Mail::SPF author 
aimed it to be suitable to be a reference implementation for RFC 4408, so he 
followed it's guidance as closely as he could.  I believe it's his intent to 
do the same for 4408bis.

It's a configuration option to disable and a one liner to change the default, 
so I suspect the estimates of ten years to change this are a bit long.

Scott K

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 23 Feb 2012 04:20:32 +0000

> From: [email-masked] [mailto:[email-masked]] On Behalf Of Douglas Otis
> Sent: Wednesday, February 22, 2012 5:03 PM
> To: [email-masked]

What collision problem is not solved by discarding those TXT records in the reply that don't start with "v=spf1"?

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 23 Feb 2012 04:30:30 +0000

> Sent: Wednesday, February 22, 2012 1:46 PM
> To: [email-masked]
> I have been following the discussion about SPF RR Type.  It seems that
> the issue is being read as a binary question, i.e. should the SPR RR
> record be kept or not.  Some of the questions that come to my mind are:
> 
> questions, let me know.  Without empirical data, it is not possible to
> answer these questions.  Is there anyone who would like to volunteer to
> find the answers to these questions?

We can get answers to (a) and (b) just by querying some large list of domains.  Philip from Cisco did one batch and I'm running another which can at least answer those two, and I'm running my own similar report.

To do (c) and (d) we need to look at nameserver logs.  John Levine is conducting that experiment.

I don't know how we could come up with an exhaustive list of (e) for Sender-ID, but it seems that OpenSPF has a fairly comprehensive list for SPF.

For (f), again we're split between the two proposals, and now we need documentation and/or source code for each.

I suggest that the answers to (e) and (f) might reasonably be approximated by the answers to (c) and (d).

-MSK

---

From: Dave CROCKER <[email-masked]>
Date: Wed, 22 Feb 2012 22:09:27 -0800

On 2/22/2012 5:21 PM, Pete Resnick wrote:
> Well, the query for 99 would then be "out of spec", but yes, you are correct.

1. They'd perceive no change.  No added overhead.  No change in error message. 
Nothing.

2. In formal terms, yes, the behavior would no longer be conformant, but I guess 
my reaction is, so what?  What's unusual and quite nice about the current 
opportunity is that dropping this has no (new) downside for those who happen to 
publish the record or do the query.

>> establishes a technical requirement based on hypothetical human behavior,
>> rather than established community need. On the average, the track record for
>> such adoption predicates is quite poor.
>
> Yup. The reason to keep it would be entirely *if* you think there's a chance
> that people might start using it. If not, the change to client software to
> eliminate the second query would be reasonable.

If there were any indication of an adoption vector or a change in incentives (or 
value proposition or anything else that would drive an adopter pull) I'd say 
keep it.  But we've had plenty of time for the trajectory to be established and 
nothing that I know of is happening that changes where things currently are.

>> Whereas typical -bis efforts are encouraged to drop unused features, to
>> simplify the spec and simply implementations and -- often -- simplify operations.
>
> Actually, I think both are true: In moves along the standards track, we get rid
> of unused things, but we also don't dork with things that are used if they're
> not causing problems.

So, in this case, there's some ambiguity to the use of what it means to be 
"used".  We have almost no one publishing the record; this means "unused".  We 
have a significant number of failed queries being made for the record; querying 
is a form of "being used".  Mumble.  Querying and having no problems falling 
back to the alternative record means "/trying/ to be used, but fine with 
failing".  Foo.

> My only point was this is an unusual situation in that the
> client side of this piece protocol *is* demonstrably used, in that clients *are*
> sending the queries, and the server side is excruciatingly minimally used,
> meaning that the clients are not getting any use out of doing so. Add to that
> the unusualness of the fact that this is *not* progress along the standards
> track (which is what we normally think of as the -bis effort), but getting
> something onto the standards track.

Well, not surprisingly, we agree this is an odd one, for analysis.

If one takes an end-to-end view of things -- and I realize that's an unusual 
perspective these days -- it has no real-world, system-level end-to-end utility.

> If we decide that the feature is essentially unused except for some outliers who
> won't be bothered if we remove the feature, peachy, we should remove it.

There is not evidence that it will magically become useful.  So even though this 
is a perspective from which it can legitimately be claimed it it is used, there 
is not current perspective from which it can be claimed to be useful.

In the face of that, I find the benefit of simplification to be a strong 
argument in favor of dropping it.

d/
-- 

   Dave Crocker

---

From: S Moonesamy <[email-masked]>
Date: Wed, 22 Feb 2012 22:20:20 -0800

Hi Murray,
At 20:30 22-02-2012, Murray S. Kucherawy wrote:
>We can get answers to (a) and (b) just by querying some large list 
>of domains.  Philip from Cisco did one batch and I'm running another 
>which can at least answer those two, and I'm running my own similar report.

Ok.

>I don't know how we could come up with an exhaustive list of (e) for 
>Sender-ID, but it seems that OpenSPF has a fairly comprehensive list for SPF.

It's a best effort.  It is not be possible to come up with a list of 
every implementation out there.

>For (f), again we're split between the two proposals, and now we 
>need documentation and/or source code for each.

Ok.

>I suggest that the answers to (e) and (f) might reasonably be 
>approximated by the answers to (c) and (d).

The WG decisions will be reviewed by the IETF Community during the 
IETF Last Call.  I would like to see some effort on (e) and (f) so 
that the SPFBIS WG can say that it carefully considered the issue.

Regards,
S. Moonesamy
SPFBIS WG co-chair

---

From: "John Levine" <[email-masked]>
Date: 23 Feb 2012 06:46:15 -0000

>that don't start with "v=spf1"?

The "so many text records that the response doesn't fit in a UDP packet" problem,
I guess, although I must admit I haven't run into that one in practice.

There are other reasons to want a specific record type such as playing nicely
with wildcards, but that is, shall we say, water under the dam in this particular
case where the practice is well established.

R's,
John

---

From: Hector Santos <[email-masked]>
Date: Thu, 23 Feb 2012 06:59:29 -0500

Hi Pete,

Don't wish to fall trap to generalization, a 'No Surprise Principle' 
<tm>, perhaps a conservative approach is probably a good (specific) 
BIS policy model for SPF/SIDF/SUBMITTER.

Optional reading below.

> I think those are the sorts of things folks should be looking at.

There is much conflict of interest (and not all in negative sense) but 
in end goals; IETF desires, the Technology Leader (SPF Council) 
desires, existing implementations desires which include:

    - Pure SPF advocates,
    - Pure SIDF advocates (the corporate image),
    - And system integrators of both.

A historical fact is that SPF was fast becoming a widely adopted 
NON-IETF protocol in all market segments. I generally never use 
non-standard technology, and it was only when AOL announced SPF 
support, did I finally stop other work and began to support it as 
well.  Soon Microsoft came with  its XML version of SPF (CEP) and 
throw in the SPF wench with the suggestion and use cases for the 
existence of a 3rd payload based Identity - Purported Responsible 
Author (PRA) or Submitter.

Yet too many, including myself, believed it was just not going to past 
IETF reviews. Of course, with all its DNS dependency, higher overhead 
and usage of TXT - yet still an less preferred idea to many to use for 
policies, the limits concerns, etc.

I only point this out because the only way to fast track the 
consolidation effort into IETF controlled RFCs was to give it a 
Experimental Status.  That is want people wanted - IETF based open 
source once MS came.

But I would be among those that would suggest, this (at least SPF) was 
already well beyond an "Experiment", already added and part of 
production vender code and operations, and today, IMV, IETF needs to 
consider that its not something that will offer a lot of room to 
drastically change it which can be very easy to do from a pure IETF 
"Experimental Status" perspective.  I am just saying, IMV, this is 
just one of those well entrenched fast tracked protocols less open to 
typical and valid "Experiment Status" stripped down efforts.

---

From: Alessandro Vesely <[email-masked]>
Date: Thu, 23 Feb 2012 17:40:36 +0100

On 22/Feb/12 23:24, Pete Resnick wrote:
> and add a 99 record, since I don't think I have one. But I think
> that's because I did it so long ago and never looked to see how to
> enter a new record type into my little server. But I'm pretty sure I'm
> not an average SPF deployer myself.

IMHO, cycles available for SPF maintenance would be better spent
adding SPF records for helo names, than keeping SPF and TXT types in sync.

If, after a seven-year experiment, we think we are going to need two
different RR types for another ten years or so, perhaps we should
offer some CTYPE directive that allows a DNS server to send /that/ TXT
record in response to SPF queries, or vice-versa.

---

From: Philip Gladstone <[email-masked]>
Date: Thu, 23 Feb 2012 13:42:41 -0500

> To do (c) and (d) we need to look at nameserver logs.  John Levine is conducting that experiment.
I did a quick check over my logs. Over the last 24 hours or so, 372 name 
servers have requested SPF or TXT records. 278 requested TXT records. 
238 requested SPF records. I.e. 144 requested both. I serve up the same 
data in both the SPF record and the TXT record. These name servers were 
in 122 different IPv4 /24 networks. [It seems that my nameserver doesn't 
have ipv6 connectivity]

The /24 with the most DNS servers in it ( 216.69.186.* -- 26 dns 
servers) was also one which requested both SPF and TXT records the same 
number of times. This network happens to belong to godaddy (their 
secureserver.net email service).

This is a small sample, but I think it indicates that the deployment of 
implementations that request SPF records is significant.

Philip

-- 
Philip Gladstone
Distinguished Engineer
Product Development
[email-masked]
Phone: +1 978-ZEN-TOAD (+1 978 936 8623)
Google: +1 978 800 1010
Ham radio: N1DQ

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 23 Feb 2012 14:02:09 -0500

On Thursday, February 23, 2012 01:42:41 PM Philip Gladstone wrote:
> > We can get answers to (a) and (b) just by querying some large list of
> > domains.  Philip from Cisco did one batch and I'm running another which
> > can at least answer those two, and I'm running my own similar report.
> > 

I think it's interesting that even if you send a Type SPF answer some 
implementation(s) is/are requesting the Type TXT record anyway (that's how I 
read you note). 

I wonder if they are asynchronously requesting both and then applying whatever 
answer they get?

For the ones that request both, can you tell if the sequence is consistent 
(e.g. SPF then TXT)?

Scott K

---

From: Dave CROCKER <[email-masked]>
Date: Thu, 23 Feb 2012 11:07:54 -0800

> implementations that request SPF records is significant.

While it's important to have empirical of data of this, I think there is not 
debate about this specific fact.

I believe the question is about the penetration of published SPF records, and 
their relevance to the overall system.

d/
-- 

   Dave Crocker

---

From: Hector Santos <[email-masked]>
Date: Thu, 23 Feb 2012 14:08:32 -0500

Thanks Philip.

Hopefully the chairs and SK can put this issue to bed.  If we drop 
support, this will cause a a surprise and new waste on SPF clients 
supporting the SPF RR.  Dropping would of gone against what was the 
original intention with initial expected but short term faults, but as 
long as we had a TXT fall back, it will still functionally work.  This 
should help SPF within DNS.  This was always an implementation 
decision anyway.  I have great interest now to start a project to 
enabled it and begin a gamma field testing cycle.

Philip Gladstone wrote:
>> domains.  Philip from Cisco did one batch and I'm running another

---

From: "John Levine" <[email-masked]>
Date: 23 Feb 2012 19:43:35 -0000

>I wonder if they are asynchronously requesting both and then applying whatever 

Eyeballing my log extracts, I see a lot of queries for both records at
the same time.  Places like Yahoo frequently ask for both via
different DNS caches, so it's hard to tell mechanically that they're
the same query.  But just off the top of my head, I'd say it's between
1/4 and 1/3 of the queries.

---

From: Philip Gladstone <[email-masked]>
Date: Thu, 23 Feb 2012 15:06:05 -0500

> I wonder if they are asynchronously requesting both and then applying whatever
It might be worth running the experiment and returning subtly different 
records and seeing which they execute. Putting exists:inspfrecord.%{d} 
in one and exists:intxtrecord.%{d} in the other would allow one to 
determine what they were doing.
> For the ones that request both, can you tell if the sequence is consistent
There are certainly some implementations which do asynchronous requests 
and request both at the same time, and the order that the requests 
arrive is not consistent.

For example:

216.69.186.13: TXT -> SPF: 0.000032 secs
216.69.186.14: SPF -> TXT: 0.000010 secs
216.69.186.15: SPF -> TXT: 0.000007 secs
216.69.186.16: SPF -> TXT: 0.000007 secs
216.69.186.16: SPF -> TXT: 0.000031 secs
216.69.186.17: SPF -> TXT: 0.000018 secs
216.69.186.18: TXT -> SPF: 0.038848 secs
216.69.186.19: SPF -> TXT: 0.000007 secs

This shows a particular set of name servers. The first line means that 
216.69.186.13 requested a TXT record and then an SPF record and that 
these requests arrived 32usecs apart (much less than the RTT).

WIth this nameserver set it is less clear what is going on:

74.6.109.17: SPF -> TXT: 2.239237 secs
74.6.109.17: TXT -> SPF: 3.206624 secs
74.6.109.18: SPF -> TXT: 0.106386 secs
74.6.109.18: TXT -> SPF: 2.930859 secs
74.6.109.20: SPF -> TXT: 0.014469 secs
74.6.109.20: SPF -> TXT: 0.038548 secs
74.6.109.20: SPF -> TXT: 0.043084 secs
74.6.109.20: SPF -> TXT: 3.557841 secs
74.6.109.20: TXT -> SPF: 0.021898 secs
74.6.109.20: TXT -> SPF: 0.042948 secs
74.6.109.20: TXT -> SPF: 0.125548 secs
74.6.109.25: SPF -> TXT: 0.000187 secs
74.6.109.25: SPF -> TXT: 0.104728 secs
74.6.109.25: SPF -> TXT: 0.114294 secs
74.6.109.25: SPF -> TXT: 0.318294 secs
74.6.109.25: TXT -> SPF: 0.000081 secs
74.6.109.26: SPF -> TXT: 0.109614 secs
74.6.109.27: SPF -> TXT: 0.107179 secs
74.6.109.27: TXT -> SPF: 2.477460 secs

The RTT to this nameserver is around 100ms. I *suspect* that the large 
time intervals are actually from separate messages (I have a cutoff of 5 
seconds for this log).

Philip

p.s. I'm actually quite impressed how quickly this can serve a request. 
This particular server is written in perl and uses mysql as the backend. 
I guess that the caching works!

---

From: "Rolf E. Sonneveld" <[email-masked]>
Date: Thu, 23 Feb 2012 21:40:43 +0100

> We can get answers to (a) and (b) just by querying some large list of domains.  Philip from Cisco did one batch and I'm running another which can at least answer those two, and I'm running my own s…

as sets (a) and (b) may overlap, and as RFC4408 uses a SHOULD for 
publishing both types, the interesting question here is: how many 
domains are publishing SPF but no TXT.

/rolf

---

From: Hector Santos <[email-masked]>
Date: Thu, 23 Feb 2012 16:11:10 -0500

> domains are publishing SPF but no TXT.

Good question and I'll be surprise if anyone doing this since it may 
show a lack of understanding there was an high unreliability factor 
just using a new named type.

You know, I suspect withing the reported DNS analysis, it includes a 
factor of what DNS query path was taken or perhaps dominate with a PCN 
(Personal Community Network) or BCN (Business Community Network).

I am a firm believer that every one of us has a unique PCB or BCN, 
with many perhaps a common subset, but nonetheless each is unique.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 23 Feb 2012 21:13:33 +0000

> From: [email-masked] [mailto:[email-masked]] On Behalf Of Rolf E. Sonneveld
> Sent: Thursday, February 23, 2012 12:41 PM
> To: Murray S. Kucherawy
> Cc: [email-masked]

Phillip's data and mine attempt to answer that question.  (His data are available, my report is still churning.)

---

From: "John Levine" <[email-masked]>
Date: 23 Feb 2012 21:40:10 -0000

>74.6.109.27: TXT -> SPF: 2.477460 secs

Those are all Yahoo.  I have other logs that strongly suggest they're
intending to do simultaneous queries.

R's,
John

---

From: Stuart Gathman <[email-masked]>
Date: Fri, 24 Feb 2012 09:36:25 -0500

Long ago, Nostradamus foresaw that on 02/20/2012 04:17 PM, Dave CROCKER
would write:
> precedent is being set by acknowledging this reality.  In spite of the
> fact that an SPF RR is a cleaner approach, enough time has passed to
> make clear that it's not being adopted and, therefore, won't be.
>
The SPF RR will likely never be widely adopted for SPFv1.  However, the
SPF RR should remain reserved for the next generation protocol (v=spf3
or whatever).  In the meantime, we should encourage publishing both RR
types for v=spf1, if for no other reason than so we can keep complaining
to operators of braindead DNS servers (that timeout with no response on
SPF queries) so they will eventually get fixed in time for v=spf3 - so
the next generation is not force to roll back to TXT records again.

---

From: Philip Gladstone <[email-masked]>
Date: Fri, 24 Feb 2012 13:19:04 -0500

> intending to do simultaneous queries.
The data from a bunch of nameservers in 72.167.238.* which are doing 
(apparently) simultaneous lookups indicates that they then pick 
whichever record comes back first.

I modified my TXT/SPF records by adding a  'exists:SPF.qtype.%{d}' to 
the SPF record and TXT.qtype to the TXT record. This allows me to tell 
(by the subsequent lookup) which they process.

Indeed, whichever query I process first (TXT or SPF), this is the record 
that they actually process as shown by their processing of the exists: 
mechanism (these domains returns NXDOMAIN so as not to interfere with 
processing).

Philip

---

From: Dan Schlitt <[email-masked]>
Date: Tue, 28 Feb 2012 12:55:30 -0500

It seems to me that Andrew has the reasonable analysis of the situation 
here.

It seems that some of the disfunction that surrounded the initial work 
in this area has persisted. That is really unfortunate. Reading the 
discussion brings to mind the disfunction of the usenet WG. It would be 
good to avoid that.

I was not a participant in the earlier work but I did have some private 
discussion with some folks about the overloading to the TXT record. I 
think it was a bad idea then and still is. Having a record like the TXT 
record that has no defined proptocol function was a good idea on the 
part of the designers of the DNS.

Back in the dark past there were suggestions about how to use the TXT 
record for keeping useful information about the network in the DNS. I 
used it for that kind of thing in the network I managed. I wouldn't do 
it in that way in today's Internet, but that is another discussion. 
There are people that still use the TXT record for that purpose today.

My understanding of why it was used as it was had three parts. It was 
hard to get a new RRTYPE requiring lots of effort and delay. Some 
servers in use curled up and died when they encountered a "unknown" 
record type. And folks wanted to get on with using spf right away.

If one was starting out with the design of spf today it might use a 
different approach for the function that the SPF RR and the specially 
designed TXT RR fill. But we need to deal with the way things are now.

/dan

-- 

Dan Schlitt
[email-masked]

---

From: Dave Crocker <[email-masked]>
Date: Tue, 28 Feb 2012 11:58:33 -0800

On 2/28/2012 9:55 AM, Dan Schlitt wrote:
> If one was starting out with the design of spf today

but we aren't.

the issue today is not what should have been or what was/is a good or bad idea.

the constraints imposed by the charter are quite clear.

d/
-- 

   Dave Crocker