Subject: [spfbis] Possible new issue: Be more liberal with the syntax?

From: Arthur Thisell <[email-masked]>
Date: Wed, 7 Mar 2012 06:19:34 -0800 (PST)

I realize I'm very late in suggesting issues to be discussed, so I don't know  if the ADs will even consider my two possible issues.   Their ruling isn't that important to me because I don't think these are critical issues and I can live with them not being considered.   I'm breaking up the two issues into two posts for ease of threading...

The first issue:   Should the syntax checking of SPF records be relaxed at all?

The robustness principle says we should be conservative with what we send, but liberal with what we accept.   During the design of the SPF spec, it was considered to be very important that all receivers interpret the domain owner's policy the same way, so choices were taken such as requiring all syntax errors to be detected in all cases.  I'm not sure if that was the correct choice.

One of the things that Caller-ID for Email did right is that Microsoft recognized that you can unambiguously tell a domain name from an IPv4 address from an IPv6 address, so they had one data field for all of them.  With SPF, we have a: vs ip4: vs ip6:, and as a result, there used to be a lot of malformed SPF records with things like a:1.2.3.4, or ipv4:example.org.

Since it usually is possible to unambiguously determine the intent of the publishers, should we relax the syntax of SPF records so that you can put any kind of addressing in any of the a/ip mechanisms?   This would effectively make a:, ip4:, and ip6: aliases of each other, and the same processing would be done on all three.   Throw in the new aliases of ip:, ipv4: and ipv6:, and I think you could cover a lot of the malformed SPF records out in the wild.

Basically, I'm advocating that RFC4408bis say that domain owners MUST publish in the same format as specified in RFC4408, but implementations MAY choose to consider this extended syntax.   Later on, another WG might consider relaxing the MUST publish to a SHOULD publish, or making the aliases officially blessed.

Now, I haven't looked at an actual list of SPF records in several years, so I don't know if things have gotten better or worse.  I don't know what percentage of SPF records have syntax errors, nor what percentage of those  errors would be fixed by this change.  I'm not even sure if relaxing the syntax would be a good thing, but it is something I think is worth at least considering.

---

From: Scott Kitterman <[email-masked]>
Date: Wed, 07 Mar 2012 09:49:11 -0500

On Wednesday, March 07, 2012 06:19:34 AM Arthur Thisell wrote:
> I realize I'm very late in suggesting issues to be discussed, so I don't
> know  if the ADs will even consider my two possible issues.   Their ruling
> isn't that important to me because I don't think these are critical issues
> and I can live with them not being considered.   I'm breaking up the two
> issues into two posts for ease of threading...
> 
> The robustness principle says we should be conservative with what we send,
> but liberal with what we accept.   During the design of the SPF spec, it
> was considered to be very important that all receivers interpret the domain
> owner's policy the same way, so choices were taken such as requiring all
> syntax errors to be detected in all cases.  I'm not sure if that was the
> correct choice.
> 
> One of the things that Caller-ID for Email did right is that Microsoft
> recognized that you can unambiguously tell a domain name from an IPv4
> address from an IPv6 address, so they had one data field for all of them. 
> With SPF, we have a: vs ip4: vs ip6:, and as a result, there used to be a
> lot of malformed SPF records with things like a:1.2.3.4, or
> ipv4:example.org.
> 
> Since it usually is possible to unambiguously determine the intent of the
> publishers, should we relax the syntax of SPF records so that you can put
> any kind of addressing in any of the a/ip mechanisms?   This would
> effectively make a:, ip4:, and ip6: aliases of each other, and the same
> processing would be done on all three.   Throw in the new aliases of ip:,
> ipv4: and ipv6:, and I think you could cover a lot of the malformed SPF
> records out in the wild.
> 
> Basically, I'm advocating that RFC4408bis say that domain owners MUST
> publish in the same format as specified in RFC4408, but implementations MAY
> choose to consider this extended syntax.   Later on, another WG might
> consider relaxing the MUST publish to a SHOULD publish, or making the
> aliases officially blessed.
> 
> Now, I haven't looked at an actual list of SPF records in several years, so
> I don't know if things have gotten better or worse.  I don't know what
> percentage of SPF records have syntax errors, nor what percentage of those 
> errors would be fixed by this change.  I'm not even sure if relaxing the
> syntax would be a good thing, but it is something I think is worth at least
> considering. 

 My gut feel is that the frequency of these types of errors has been 
decreasing.  Since I maintain the openspf.net/org email test reflector, I've 
got the recent logs.  I checked the most recent just to get a sense of how 
frequent this is.  There were 56 permanent errors in the log (many were 
duplicates as people often send more than one test message without fixing 
anything).  Only one of them was in this catagory:

SPF Permanent Error: Invalid IP4 address: ip4:smptout.secuereserver.net

The vast majority of errors were including domains that didn't have SPF 
records or trivial recursion errors in include where they included themselves.

While I agree that, in principle, a:, ip4:, ip6: could have been combined, I 
think it would be a mistake to do so in 4408bis.  I think anything like such a 
change should be reserved for a future effort that does not have backwards 
compatibility as a goal.

There are implementations that do this sort of thing for private use (pyspf 
has a "relaxed" mode) and I think that for people who know what they are doing 
it's OK, but it should not be standardized.

Scott K

---

From: Andrew Sullivan <[email-masked]>
Date: Wed, 7 Mar 2012 10:55:05 -0500

Dear colleagues,

On Wed, Mar 07, 2012 at 06:19:34AM -0800, Arthur Thisell wrote:
> I realize I'm very late in suggesting issues to be discussed, so I
> don't know if the ADs will even consider my two possible issues.

I don't know what the ADs will say about it, but the chairs have been
clear that we will not rule out of scope issues that come up and that
meet the charter.  A technical issue is a technical issue, and we'll
deal with them as long as they're on-charter. There may, of course,
also be issues that would not be on charter, but that could be the
basis for future work.

> One of the things that Caller-ID for Email did right is that
> Microsoft recognized that you can unambiguously tell a domain name
> from an IPv4 address from an IPv6 address

No hat.  It's funny you should say that; there's currently a great
deal of gnashing of teeth over how you can do that.  How can you
unambiguously tell a domain name from an IPv4 address?  What if the
list of TLDs changes dramatically?

Best,

A

-- 
Andrew Sullivan
[email-masked]

---

From: "John Levine" <[email-masked]>
Date: 7 Mar 2012 17:13:48 -0000

>The first issue:   Should the syntax checking of SPF records be relaxed at all?

Heck, no.  Implementations are always allowed to work around bugs if
they want to, but the best way to interoperate is to provide a clear spec
and encourage people to implement it correctly.

I suppose it would be mostly harmless to add an appendix noting common
errors and how not to make them, but I fear that some people would
insist on misreading it as if if's in the RFC it must be OK.

R's,
John

---

From: S Moonesamy <[email-masked]>
Date: Wed, 07 Mar 2012 10:21:11 -0800

>The first issue:   Should the syntax checking of SPF records be 

This is being tracked as Issue #26.  There is also a ticket for the 
second issue that was raised today.

Could we defer the discussion on this issue until the thread is open?

Regards,
S. Moonesamy
SPFBIS WG co-chair