Subject: Re: [spfbis] #14: RFC 4408: local-part macro prep for deprecation

From: S Moonesamy <[email-masked]>
Date: Tue, 13 Mar 2012 22:17:22 -0700

At 17:53 17-02-2012, spfbis issue tracker wrote:
>#14: RFC 4408: local-part macro prep for deprecation
>
>  Message from Doug Otis on 17 Feb 2012:
>
>  Use of local-part macros in SPF record sets necessitates repeating all
>  invoked DNS transactions.  DNS caching only limits DNS overhead when no
>  local-part macros are used within SPF record sets.  It is not uncommon for
>  the use of random local-part names in messages to be sent by malefactors.
>  Such use significantly impacts the utility of SPF without burdening the
>  authoritative servers employed by malefactors.  Not only will local-part
>  macros increase the gain of a reflected attack, it also prevents resolving
>  addresses of a set of servers for a specific domain.
>  Permitting local-part assessments will return results based upon entire
>  email addresses and not just the email-address domain.  It is common to
>  use SPF to resolve the outbound addresses used by a domain to check
>  reputation listings against these addresses.  This use assumes a specific
>  domain has not defined local-part macros in their record set or that it
>  defaults to "postmaster".
>
>  SPF local-part macros should not be used as a method to vet valid local-
>  part names, as this would permit the use of high speed dictionary attacks
>  to discover which are valid.  A domain better protects confidentiality of
>  valid local-parts by postponing acceptance or refusal to the Data command.
>  A reasonable practice for recipients to use to ensure effective caching
>  and to retain the desired utility is to supplant any local-part with that
>  of "postmaster".  Otherwise, when SPF processing follows other types of
>  vetting, expansion of the local-part would leak its status via DNS
>  requests.
>
>  Add:
>
>  A method for mitigating a reduction in utility or risks associated with
>  local-part macros would be to replace any local-part with that of
>  "postmaster".
>
>  http://www.ietf.org/mail-archive/web/spfbis/current/msg00412.html

[snip]

>Ticket URL: <http://trac.tools.ietf.org/wg/spfbis/trac/ticket/14>

Please comment on the Issue #14.  As a reminder, please keep the subject line.

Regards,
S. Moonesamy
SPFBIS WG co-chair

---

From: Scott Kitterman <[email-masked]>
Date: Wed, 14 Mar 2012 07:11:11 -0400

On Tuesday, March 13, 2012 10:17:22 PM S Moonesamy wrote:
> >  local-part macros are used within SPF record sets.  It is not uncommon
> >  for the use of random local-part names in messages to be sent by
> >  malefactors. Such use significantly impacts the utility of SPF
> >  without burdening the authoritative servers employed by malefactors. 
> >  Not only will local-part macros increase the gain of a reflected
> >  attack, it also prevents resolving addresses of a set of servers for
> >  a specific domain.
> >  reputation listings against these addresses.  This use assumes a
> >  specific domain has not defined local-part macros in their record set
> >  or that it defaults to "postmaster".
> >  
> >  part names, as this would permit the use of high speed dictionary
> >  attacks to discover which are valid.  A domain better protects
> >  confidentiality of valid local-parts by postponing acceptance or
> >  refusal to the Data command. A reasonable practice for recipients to
> >  use to ensure effective caching and to retain the desired utility is
> >  to supplant any local-part with that of "postmaster".  Otherwise,
> >  when SPF processing follows other types of vetting, expansion of the
> >  local-part would leak its status via DNS requests.
> >  
> >  Add:
> >  

This is a used, but uncommonly used feature of the protocol.  It is uncommonly 
used as most domains will not need per-user policies.  Removing it would break 
backward compatiblity.  Arbitrarily replacing the actual localpart with 
postmaster would do the same.

The fact that localpart macros affect cachabilty is not news.  4408 8.1 
includes:

   Note: Domains should avoid using the "s", "l", "o", or "h" macros in
   conjunction with any mechanism directive.  Although these macros are
   powerful and allow per-user records to be published, they severely
   limit the ability of implementations to cache results of check_host()
   and they reduce the effectiveness of DNS caches.

There's no evidence provided that these macros are, or ever have been, used by 
"malefactors" to attempt to cause harm and even less evidence that such 
attempts have ever succeeded.

The point about outbound checks is odd, since that's within an ADMD's control 
how to manage such things and so it's a simple case of "if it hurts, don't do 
that".

It's been a best practice for the better part of a decade for MTAs not to 
accept mail for invalid recipients, so there are other ways of accomplishing 
the exact idea of extracting valid localpart names.  There is no news about 
this.

Fundamentally the changes requested in this issue are outside the charter and 
there's no valid rationale for pursuing them in any case.

Scott K

---

From: Philip Gladstone <[email-masked]>
Date: Thu, 15 Mar 2012 12:18:25 -0400

On 3/14/2012 1:17 AM, S Moonesamy wrote:
>> specific
>> attacks
I don't understand this paragraph. The local part macro is used to 
validate the *sender* of the email. This cannot be done by postponing 
acceptance or refusal to the Data command. I suspect that there is some 
confusion between sender local part and recipient local-part.
>>  local-part macros would be to replace any local-part with that of
This would break backward compatibility. There is no reason that the 
string "postmaster" would pass validation. Note that an MTA may serve 
several domains, and it may arrange that all outgoing mail from 
postmaster comes from a specific domain. In this case, if it was doing 
local-part verification, then the use of postmaster could easily cause 
SPF failures.

In conjunction with issue #12 and forwarders that do not do SRS, the 
forwarder will be sending mail *from* the original local-part, but from 
the *wrong* MTA. At least checking the local part allows this to be 
validated. [I could see systems which tried to distinguish between valid 
forwarders and spammers by using a set of heuristics based on the 
local-part and the timing of SPF checks]

In other words, I do not support the deprecation of the local-part macro.

Philip

-- 
Philip Gladstone
Distinguished Engineer
Product Development
[email-masked]
Phone: +1 978-ZEN-TOAD (+1 978 936 8623)
Google: +1 978 800 1010
Ham radio: N1DQ

-- 
Philip Gladstone
Distinguished Engineer
Product Development
[email-masked]
Phone: +1 978-ZEN-TOAD (+1 978 936 8623)
Google: +1 978 800 1010
Ham radio: N1DQ

---

From: Douglas Otis <[email-masked]>
Date: Thu, 15 Mar 2012 18:23:36 -0700

On 3/15/12 9:18 AM, Philip Gladstone wrote:
> acceptance or refusal to the Data command. I suspect that there is 
> some confusion between sender local part and recipient local-part.
Dear Philip,

Thank you for the question.  Valid local-parts are exposed with 
immediate rejections of individual recipients for inbound messages, and 
at anytime when resource records in DNS determine sender validity when 
SPF records contain local-part macros.  Exposing valid local-parts in 
this manner asks for additional abuse and should be considered a bad 
practice on that basis alone, but there is more...

> local-part verification, then the use of postmaster could easily cause 
Host checks default to "postmaster" when local-parts are not included 
for various reasons such as with NDNs or EHLO checks.  Use of local-part 
macros relinquishes sender's control over authorized sources.  
Substituting "postmaster" protects against possible abuse of the 
recipient's resources, which should be considered a good practice.
> forwarder will be sending mail *from* the original local-part, but 
> from the *wrong* MTA. At least checking the local part allows this to 
> be validated. [I could see systems which tried to distinguish between 
> valid forwarders and spammers by using a set of heuristics based on 
> the local-part and the timing of SPF checks] 
It seems safe to conclude SRS has not reached a level of adoption 
offering benefit. The The Authorized Third-Party Signatures (ATPS) 
draft, http://tools.ietf.org/html/rfc6541 should impose fewer obstacles, 
especially when supported by various service providers.

Basing acceptance on local-part macros allows undetectable spoofing of 
senders.  Again, this asks for abuse and places recipients at risk which 
should be considered an extremely bad practice.
> In other words, I do not support the deprecation of the local-part macro.
It should be mentioned, support of local-part macro vetting by 
recipients allows any malefactor the ability to hammer victims using 
recipient's resources, and lacking limits on no-data, still have 
messages accepted based upon SPF results.

Would you advocate for MFROM and EHLO checks?  Would you advocate for 
spf/txt being done in parallel?  Lacking no-data limits can result in 
hundreds of needless queries aimed at any victim that only needs to 
support DNS, and not even email.  It is also very likely no one will 
notice except victims who may lack any relationship with email 
services.  It should also be mentioned, these victims will be unable to 
play any role in mitigating the attacks.

Regards,
Douglas Otis

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 15 Mar 2012 23:39:03 -0400

On Thursday, March 15, 2012 06:23:36 PM Douglas Otis wrote:
...
> recipient's resources, which should be considered a good practice.
...

After the first sentence, this is incorrect.

The use of local-part macros does exactly the opposite of that.  It enables 
the sender to set per-user SPF policies.

There's no abuse of recipient resources here.  Yes, SPF records that use 
local-part macros can't be globally cached, so there might be some increased 
DNS usage, but there's no abuse.  Should this ever prove to be a problem in 
practice, receivers are free to skip SPF checks on such messages.

Limiting local-part macros to always use postmaster is functionally equivalent 
to removing the macro.  This would break backwards compatibility with a 
feature that's in use and there's no reason to go there.

Scott K

---

From: Douglas Otis <[email-masked]>
Date: Fri, 16 Mar 2012 17:46:57 -0700

> the sender to set per-user SPF policies.
Dear Scott,

Senders should internally assert their per-user policies before sending 
and not place this burden on recipients.

The SPF left to right process concludes when a mechanism matches and 
qualifiers are returned.  To constrain possible sources in combination 
with possible local-parts, merged resource record references must be 
used.  This expects recipients to query unique resource records for 
every source address prefix, times the number of local-parts handled.

Local-part macros used in this fashion will place a large burden on 
recipient resources.  A domain sourcing 15 million local-parts from N 
number of unique prefixes represents a _significant_ increase in the 
consumption of recipient resources.  The level of this consumption is 
further exacerbated by the SPF process being unable to cache results 
based on domains since results may include local-part macros.

So even when exactly the same identifier is checked, the full set of DNS 
transactions must occur where a reduction of repeated transactions 
against authoritative servers relies fully upon prior transactions 
remaining in the recipient's cache.  But when transactions return 
no-data, caching durations may be fairly short and not comply with the 
authoritative domains expectations.  Malefactors are able to send 
messages continuously from different sources and domains while still 
exploiting the same evil MX resource records having non-existent targets 
within the victim's domain.  While the MX record itself gets cached, the 
contained targets may have already expired from the cache.  SPF's lack 
of no-data errors makes such strategies extremely easy to configure.
> practice, receivers are free to skip SPF checks on such messages.
There is no reasonable justification for continuing both a dangerous and 
unnecessary practice.  Those running MTAs are extremely unlikely to know 
whether they are participating in a distributed denial of service 
attack.  Victims of such attacks will likely see thousands of legitimate 
sources for an attack while being unsure whether sources have been 
spoofed.  All thanks to SPF local-part macros.
> feature that's in use and there's no reason to go there.
Substituting postmaster will not break compatibility.  This simply 
requires restrictions on local-parts be done by senders prior to sending.

In the few cases I have seen this mechanism used, it was to allow users 
to send from _any_ source.   When used in this fashion, SPF 
confirmations place recipients at risk of being deceived.  There is no 
reasonable justification for continuing support for this seldom used and 
poorly considered "feature".

Regards,
Douglas Otis

---

From: Philip Gladstone <[email-masked]>
Date: Sat, 17 Mar 2012 00:21:56 -0400

On 3/16/2012 8:46 PM, Douglas Otis wrote:
> Dear Scott,
>
> Senders should internally assert their per-user policies before 
> sending and not place this burden on recipients.
This makes no sense. The *purpose* of SPF is to allow the origin domain 
to express policies about which MTAs are authorized to send mail 
purporting to come from the origin domain. Can you please explain 
exactly why an unauthorized sender (aka spammer) should assert my (the 
purported origin domain) per-user policies before sending? If the 
spammers obeyed my policies, then we wouldn't have spam any more!
> consumption of recipient resources.

I also don't understand what 'resources' you are referring to when you 
claim a _significant_ increase in resources. Are you referring to CPU 
time, Disk bandwidth, Memory consumption, Network bandwidth, electricity 
use or what? My feeling is that the use of local-part macros increases 
the number of DNS lookups by (probably) 1 per mechanism that uses a 
local part macro. Is this increase significant in terms of CPU? I doubt 
it as spam filtering is computationally very expensive. Disk bandwidth 
-- I doubt that the DNS lookup by the receiving MTA involves any disk 
IO. For memory, there may be an increase of the order of 1K in a DNS 
cache. Sensible policies on the TTL of the record will mean that this 
record only remains cached for a short period of time. In terms of 
network bandwidth, the consumption is probably under 1K travelling from 
receiving MTA to authoritative DNS server managed by the origin domain.
> The level of this consumption is further exacerbated by the SPF 
> process being unable to cache results based on domains since results 
> may include local-part macros.
SPF can still cache domain results in exactly the same way. All that the 
use of local part macros does is to reduce the hit rate of the cache 
somewhat. In the records that appear to be in use that use local-part 
macros, there are three groups:

1) Those which appear to use it as a fallback after a/mx/ptr records. 
Presumably, most valid mail gets through the first checks and the local 
part macro is only tested for forged mail

2) Those which do all the checks using one big macro record. E.g. 
'v=spf1 exists:_h.%{h}._l.%{l}._o.%{o}._i.%{i}._spf.%{d} ?all'

3) Those which have rather different policies for each local-part. These 
records use redirect or include to fetch a per-local-part policy.

Having said all that, there are very few records that use macros at all:

     45: %{d}
     30: %{h}
     28: %{ir}
   1386: %{i}
    126: %{l1r+}
      1: %{l1r-}
     41: %{l}
     19: %{o}
     11: %{r}
     40: %{s}
      2: %{v}

The first column is the count, and the second is the macro. This is from 
my 280k SPF record dump. I.e. less than 1% use macros, but the 
local-part macro is second only to the source IP in usage.

Is there an increase in consumption if I add an 
exists:%{l1r-}.user.spf.%{d) clause. Yes. Is it significant. No. [Note 
that I can cause *exactly* the same increase in consumption if I add 
a:mumble.spf.%{d} and always return a record with a TTL of 0]

I just don't see that the resource increase is noticeable by comparison 
with the cost of handling the email. I don't know about you, but my 
email feed runs at around 95% spam and most of the resource is spent 
discarding it.

Philip

---

From: Douglas Otis <[email-masked]>
Date: Sun, 18 Mar 2012 23:30:57 -0700

On 3/16/12 9:21 PM, Philip Gladstone wrote:
> This makes no sense. The *purpose* of SPF is to allow the origin 
> domain to express policies about which MTAs are authorized to send 
> mail purporting to come from the origin domain. Can you please explain 
Dear Philip,

SPF authorized sources should allow recipients to assume user level 
controls have already been applied!  Processing local-part macros 
entails additional DNS transactions requiring a substantial increase in 
DNS caching and SMTP session durations.  This expects a local-part macro 
process will be applied against sub-domains illustrated by your example 
whenever a different user is seen from the SPF domain.

Fortunately few use this method so there is no good record supporting 
the utility or scalability of local-part macros.  The use of local-part 
or i macros also prevents use of SPF to identify the range of IP 
addresses used by the domain, which represents SPF's greatest utility.
> time, Disk bandwidth, Memory consumption, Network bandwidth, 
> electricity use or what? 
All of the above, in addition to increasing the potential amplification 
for reflected attacks off of recipients resolving local-part macros.  
This represents a significant concern since the exists and every other 
mechanism supporting local-part macros are assumed to potentially occur 
against non-existent resources any number of times without necessarily 
affecting the results returned.
> My feeling is that the use of local-part macros increases the number 
> of DNS lookups by (probably) 1 per mechanism that uses a local part 
> macro. Is this increase significant in terms of CPU? I doubt it as 
> spam filtering is computationally very expensive. Disk bandwidth -- I 
> doubt that the DNS lookup by the receiving MTA involves any disk IO. 
> For memory, there may be an increase of the order of 1K in a DNS 
> network bandwidth, the consumption is probably under 1K travelling 
> from receiving MTA to authoritative DNS server managed by the origin 
> domain.
http://tools.ietf.org/html/draft-otis-spf-dos-exploit-01

Ignore the text and just review the log.  Assuming only the Mail From 
identity is checked, the log shows 34.8K bytes from victim's name 
servers responding to 28.8K bytes of requests by recipients for every 
message containing a different local-part.  A different local-part for 
every message is not unusual in spam campaigns.  Spammers may minimize 
their traffic by waiting to repeat local-parts after typical negative 
cache durations.  Once their evil MX record sets supporting their 
campaign have been cached, nearly the entire follow-on DNS traffic will 
be carried almost exclusively by recipients.  These durations normally 
override any requested negative cache duration and are also likely to be 
flushed anyway.

You would need to assume local-part macros are not used abusively to 
arrive at perhaps a 1K byte/user increase in the DNS traffic.  This 
issue must still consider there are millions of local-parts used per 
domain non-abusively.  Once the number of users per domain are 
considered this quickly becomes a problem without there being any 
malicious use of this feature.
> SPF can still cache domain results in exactly the same way. All that 
> the use of local part macros does is to reduce the hit rate of the 
> cache somewhat. In the records that appear to be in use that use 
> local-part macros, there are three groups:
>
> Presumably, most valid mail gets through the first checks and the 
> local part macro is only tested for forged mail
>
> 3) Those which have rather different policies for each local-part. 
> These records use redirect or include to fetch a per-local-part policy.
>
> The first column is the count, and the second is the macro. This is 
> from my 280k SPF record dump. I.e. less than 1% use macros, but the 
An SPF record such as this would allow anyone to spoof messages from the 
domain.  It would need to include some form of the i macro as well.
> I just don't see that the resource increase is noticeable by 
> comparison with the cost of handling the email. I don't know about 
> you, but my email feed runs at around 95% spam and most of the 
> resource is spent discarding it.
When you know 95% of SPF records processed are on behalf of malefactors, 
every transaction that might be directed against any domain, especially 
those allowed to return no data,  creates an unnecessary risk especially 
for worthless overhead supporting local-part macros.   A safe and 
pragmatic solution would be to swap all local-parts with "postmaster".

Regards,
Douglas Otis

---

From: Philip Gladstone <[email-masked]>
Date: Mon, 19 Mar 2012 09:21:59 -0400

On 3/19/2012 2:30 AM, Douglas Otis wrote:
> controls have already been applied!

One question:   What entity is supposed to have applied these user level 
controls? Note that in the case that it is not the authorized domain 
sending traffic, then the authorized domain does not have any chance to 
apply user level controls.

What am I missing?

Philip

---

From: Scott Kitterman <[email-masked]>
Date: Mon, 19 Mar 2012 09:32:05 -0400

> controls have already been applied!  

This doesn't make any sense.  The point of per-user policies is that they are, 
in fact, per user.

All you are saying in this thread is you don't think SPF should have per-user 
policies.  I get that, but the fact is it does and the feature is in use.

Scott L

---

From: Douglas Otis <[email-masked]>
Date: Mon, 19 Mar 2012 11:01:42 -0700

Dear Scott,

SPF offers authorizations for outbound domains ensuring return of NDNs 
or to declare the IP addresses used by domain's outbound servers.  These 
addresses can then construct white-lists or resolve reputation services 
blocking the domain.  This utility represents easily more than 99% of 
current uses for SPF not compatible with User level control without a 
general agreement "postmaster" as a local-part always enumerates these 
addresses.

Expecting recipients to impose user level controls on behalf of senders 
is highly ill considered nor has such use been widely implemented except 
in rare cases.  The current macro processing structure ignores the 
100/200/400 transactions that ignore the occurrences of no-data errors 
(depending upon use of spf/txt resource records and Mail From/EHLO 
identity options).

The EHLO processing is still able to exploit the 10 x gain offered by 
the 10 MX mechanisms simply by spoofing EHLO sub-domains.  Based upon 
current SPF specification, the use of macros and insensitivity to 
no-data errors permits a 1 KB message to generate (34.8KB +28.8 KB) x 4 
or 254KB of victim traffic against any domain that simply supports DNS.  
The victim domain does not even need to offer MX/TXT/spf resource records.
> policies.  I get that, but the fact is it does and the feature is in use.
This is a poorly considered and rarely used feature nor does User level 
control based upon local-part macros serve a valid purpose.  SPF should 
not authorize untrusted outbound servers!  DKIM provides a safer and 
more practical means to ensure control of email header fields.  SPF was 
never intended to ensure an identity used in email header fields nor can 
this mechanism be expected to scale for such use even when extended by 
another protocol that attempts to restrict these fields based upon the 
unseen Mail From parameter.

Regards,
Douglas Otis

---

From: Andrew Sullivan <[email-masked]>
Date: Mon, 19 Mar 2012 14:15:44 -0400

Hi Doug,

On Mon, Mar 19, 2012 at 11:01:42AM -0700, Douglas Otis wrote:

> senders is highly ill considered nor has such use been widely
> implemented except in rare cases. 

Just as a point of information: is that a positivice claim that it has
been implemented (though rarely); or are you really saying that as far
as you know it's unimplemented, but you're not willing to say "never"?

> SPF should not authorize untrusted outbound servers!  DKIM provides
> a safer and more practical means to ensure control of email header
> fields.

Please check our charter; we have some pretty strict limits on what we
are allowed to do:

      Changes to the SPF specification will be limited to the correction
      of errors, removal of unused features, addition of any enhancements
      that have already gained widespread support, and addition of
      clarifying language.

Are you claiming that the feature is unused, or that it is an error?

Best,

A

-- 
Andrew Sullivan
[email-masked]

---

From: Douglas Otis <[email-masked]>
Date: Mon, 19 Mar 2012 12:47:47 -0700

> as you know it's unimplemented, but you're not willing to say "never"?
Dear Andrew,

The local-part feature is used by a fraction of a percent of exchanges 
supported by SPF.  Even so, in these rare cases, it serves no valid 
purpose and conflicts with the majority of SPF uses.  This mechanism 
expects recipients to impose restrictions on Mail From parameters used 
by a domain's authorized servers.  A parameter not seen by recipients 
and no protocol imposes local-part restriction on other email header 
fields that might be visible.  This is a feature that clearly should be 
removed as not being used, or at least not gainfully used while imposing 
additional resource risks.

To ensure compatibility in those few instances where SPF records contain 
local-part macros, the specification should indicate use of "postmaster" 
can be alternatively applied to enumerate a domain's outbound servers.  
This would ensure SPF's general utility and compatibility, while also 
guarding against unscalable abuse of recipient resources.
>        that have already gained widespread support, and addition of
The specification is in error by not ensuring a local-part mechanism 
will not interfere with the vast majority of utilities normally offered 
by SPF.  In that sense, it would be an error to not ensure against such 
compatibility related interference.  This interference is corrected by 
supplanting local-part with "postmaster" when assessing a domain's 
outbound server address set.

The specification is also in error by ignoring accumulations of no-data 
errors that may occur during processing of SPF related macros.  This is 
a different error that must also be resolved.

Likely due to its overhead and unreliable results, few large domains 
dynamically employ SPF as a basis for rejecting messages.  As such, it 
presents fewer exploit opportunities.  It would be naive to assume that 
no evidence being published of SPF macro mechanisms being exploited in 
attacks against some victim domain offers assurances against current or 
future exploits or that SPF will continue to be generally ignored as a 
basis for acceptance.  It is likely exploitation of a recipient's 
resources will go unnoticed by email recipients.  Since email is able to 
make a distributed attack extremely easy to instrument, it is important 
recipient resources not fall into the wrong hands.
> Are you claiming that the feature is unused, or that it is an error?

Both.  To ensure compatibility in those few instances where SPF records 
contain local-part macros, the specification should indicate use of 
"postmaster" can be alternatively applied to enumerate a domain's 
outbound servers.  This would ensure SPF's general utility and 
compatibility, while also guarding against unscalable abuse of recipient 
resources.

Regards,
Douglas Otis

---

From: Stuart Gathman <[email-masked]>
Date: Mon, 19 Mar 2012 23:03:22 -0400

Long ago, Nostradamus foresaw that on 03/14/2012 01:17 AM, S Moonesamy
>>  the use of random local-part names in messages to be sent by
While Doug is correct that the localpart macro defeats DNS caching in
the hands of a malefactor (and rfc4408 mentions this), so does an IP6
connection with PTR (or ip address macro) (sender has a 64 bit address
space to randomize for a unique IP on each connection).  Should we
remove IP6 support?

The postmaster idea is not backward compatible.  But I have a better
suggestion: if a receiver finds their DNS cache thrashing from localpart
macros (or ip6), or is concerned that that *might* happen, then they
should simply *ignore* SPF records with localpart macros.  Even better,
the libspf2 library has a "cache able" test that determines whether a
policy can be cached.  A receiver concerned about the attacks Doug is
talking about can simply ignore uncache able SPF records. 

We could even be helpful and include the algorithm for cacheability
testing in the security section, and suggest that receivers MAY ignore
uncache able policies if they suspect or wish to prevent a Doug style
attack (although there are easier ways to get the same amplification, so
I don't see the motivation for a bad guy to use such an attack).  This
avoids breaking policies,  and provides an effective mitigation when needed.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 20 Mar 2012 03:49:58 +0000

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of Stuart Gathman
> Sent: Monday, March 19, 2012 8:03 PM
> To: [email-masked]
> Subject: Re: [spfbis] #14: RFC 4408: local-part macro prep for deprecation
> 
> suggestion: if a receiver finds their DNS cache thrashing from
> localpart macros (or ip6), or is concerned that that *might* happen,
> then they should simply *ignore* SPF records with localpart macros.
> Even better, the libspf2 library has a "cache able" test that
> determines whether a policy can be cached.  A receiver concerned about
> the attacks Doug is talking about can simply ignore uncache able SPF
> records.
> 
> attack (although there are easier ways to get the same amplification,
> so I don't see the motivation for a bad guy to use such an attack).
> This avoids breaking policies,  and provides an effective mitigation
> when needed.

Assuming we decide we want to say anything at all about the perceived problem, this seems like a reasonable compromise versus Doug's preference of removing or disabling the local part macros outright.

-MSK

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 20 Mar 2012 09:46:16 +0100

On 20/Mar/12 04:49, Murray S. Kucherawy wrote:
>> From: ietf.org On Behalf Of Stuart Gathman
>
> Assuming we decide we want to say anything at all about the
> perceived problem, this seems like a reasonable compromise versus
> Doug's preference of removing or disabling the local part macros
> outright.

That variation looks similar to the best-guess case.  That is, for
domains having no or unacceptable SPF record receivers can synthesize
a policy and evaluate that.  (There are similar approaches for ADSP.)

IIRC, we decided that the reporting header fields
(Authentication-Results and Received-SPF) should not conflate such
surrogate results with true ones.

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 20 Mar 2012 06:53:31 -0400

>surrogate results with true ones.

No. It's rather the opposite.

In that case a receiver is making up a record and applying it.  Inventing records and applying them is not SPF.

In this case, it's a matter of not doing SPF processing if resources are not available. Nothing forces a receiver to check SPF for a particular message.

To further clarify, (thinking in terms of an RFC 5451 Authentication Results header field) I suspect you are thinking the result reported would be SPF none. What we're discussing is reporting no authentication check done.

Scott K

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 20 Mar 2012 13:16:15 +0100

On 20/Mar/12 11:53, Scott Kitterman wrote:
> 
> To further clarify, (thinking in terms of an RFC 5451
> Authentication Results header field) I suspect you are thinking the
> result reported would be SPF none. What we're discussing is
> reporting no authentication check done.

Any check done may be worth being reported.  Personally, I don't see
much added value in reporting that a check was skipped, but if there
is a (valid) reason to skip it and if alternative checks have been
carried out, it might still make sense to report that.  Consider:

Authentication-Results: somehost.example;
  spf=skip reason=uncacheable;
  spf-best-guess=pass smtp.mailfrom=sender.example;
  spf-quirk=neutral (using [email-masked])
     smtp.mailfrom=sender.example;

Would any part of such a result be useful?

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 20 Mar 2012 08:53:44 -0400

> Would any part of such a result be useful?

I was thinking the case under discussions would be more like:

Authentication-Results: test.example.org; none (SPF skipped due to non-
cacheable record)

Someone might (I question the value, but someone might) define spf-best-guess 
as a new email authentication method, in which case spf-best-guess=pass 
smtp.mailfrom=sender.example; would be reasonable.  If I were the one defining 
it, I would only permit pass and none for results as the other possibilities 
are at best meaningless and at worst actively dangerous.

I've no idea what spf-quirk would be?

In one open source implementation where I've implemented RFC 5451, I've also 
done things like:

Authentication-Results: test.example.org; none (SPF skipped for whitelisted 
relay domain - client-ip=%s; helo=%s; envelope-from=%s; receiver=%s)

I think that recording that no authentication test was done and why for post-
delivery analysis is an important thing, so in general, I think it's useful to 
report both what's done and not done and why.  It's not going to be a lot of 
help for automatic downstream processing, but it can be invaluable for 
troubleshooting.

Scott K

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 20 Mar 2012 13:40:49 +0000

> From: [email-masked] [mailto:[email-masked]] On Behalf Of Alessandro Vesely
> Sent: Tuesday, March 20, 2012 1:46 AM
> To: [email-masked]
> domains having no or unacceptable SPF record receivers can synthesize a
> policy and evaluate that.  (There are similar approaches for ADSP.)
> 
> IIRC, we decided that the reporting header fields (Authentication-
> Results and Received-SPF) should not conflate such surrogate results
> with true ones.

The difference is that Best-Guess SPF imposes a record where there is none, which is a complete policy fabrication.  In this case we're ignoring all or part of a record that (in theory) causes an operational hardship, which receivers are always free to do anyway.

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 20 Mar 2012 13:42:38 +0000

> From: [email-masked] [mailto:[email-masked]] On Behalf Of Scott Kitterman
> Sent: Tuesday, March 20, 2012 5:54 AM
> To: [email-masked]
> [...]

+1 to all of this.

-MSK

---

From: "John Levine" <[email-masked]>
Date: 20 Mar 2012 22:47:42 -0000

>Assuming we decide we want to say anything at all about the perceived problem, this seems like a reasonable
>compromise versus Doug's preference of removing or disabling the local part macros outright.

Based on the limited data I've seen, there seems to be remarkably
little cacheing of any of the DNS records typically looked up for
e-mail, such as rDNS, DNSBLs, and SPF.

As far as I know, nobody's ever seen cache thrashing due to SPF, so
nobody's ever tried to deal with it, and we have no idea how well
various solutions would work.  I would strongly discourage us from
recommending untried solutions to hypothetical problems in a standards
track document.

R's,
John

---

From: Douglas Otis <[email-masked]>
Date: Tue, 20 Mar 2012 20:16:53 -0700

> e-mail, such as rDNS, DNSBLs, and SPF.
Dear John,

Those processing SPF records while running an MTA are unlikely to notice 
their participation in a DDoS attack orchestrated with SPF.  Those 
impacted by SPF will be unable to implement a remedy.
> recommending untried solutions to hypothetical problems in a standards
The goal is not to make changes in how SPF is normally used.  The goal 
is to ensure SPF retains its widely used utility for enumerating 
addresses of a domain's outbound servers.  SPF has never been widely 
used as intended.  There is no way to predict what revival of this 
flawed specification may have on its use.  It is prudent to consider 
what increased processing of SPF may have with its related security 
concerns.  Security concerns that extend beyond those that may 
inadvertently abuse DNS with hundreds of transactions ignoring no-data 
errors initiated with recipient resources by unknown entities.

Regards,
Douglas Otis

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 21 Mar 2012 10:27:06 +0100

>> e-mail, such as rDNS, DNSBLs, and SPF.

At least, responses originating from the same message should get
cached during the message processing, in case various tools running at
the same MTA are unable to reuse one another's results.

> addresses of a domain's outbound servers.

Do we have utilities for counting how many IPv4/6 addresses does an
SPF record mean to "pass"?  Comparing the number of (connected
components of) good addresses to the number of queries required to
determine it should measure the efficacy of SPF records.  IMHO, the
boundary between poor efficacy and malicious attack is subjective, but
such is any measure of domain reputation.

---

From: Douglas Otis <[email-masked]>
Date: Wed, 21 Mar 2012 17:31:33 -0700

> the same MTA are unable to reuse one another's results.
Dear Alessandro,

Agreed, several messages may exist within an SMTP session.  When just 
the right-hand-side of email-addresses are used for SPF record checks, a 
single check would apply for all messages of a domain.  Precluding 
"postmaster" local-parts from authorizing the server could impair NDN 
acceptance and degrade interchange compatibility.

Reusing checks for messages of the same domain would be ideal.  Users 
should be authenticated prior to messages being sent anyway.  After all, 
left-hand-side restrictions were recently deprecated in DKIM where it 
did not cause the additional overhead seen with SPF.  Expecting 
recipients to confirm local-parts has serious scaling issues.  
Fortunately this rarely attempted mechanism can be "nipped in the bud" 
before it becomes a serious problem.
> such is any measure of domain reputation.
Such measurements will be difficult when SPF records contain "i" or "l" 
macros.  Domain reputations for SPF will also immediately confront 
latency issues.  Reputation checks are normally based upon source IP 
addresses used by TCP connections.  This occurs well before SPF domains 
can be verified as authorized.  Often these checks only occur prior to 
sending NDNs.   With latency being a problem, reputation checks are 
unlikely able to preclude problematic SPF record domains.   However, 
problematic macro elements can be proactively precluded instead.  With 
use of problematic macros being rare, careful preclusion should not 
prove disruptive especially when specifications offer guidance for both 
senders and receivers.

Regards,
Douglas Otis