Subject: Re: [spfbis] #12: RFC 4408 Section 9 - Forwarding and helo identities

From: Stuart Gathman <[email-masked]>
Date: Wed, 04 Apr 2012 02:12:16 -0400

Long ago, Nostradamus foresaw that on 03/24/2012 12:52 AM, Murray S.
Kucherawy would write:
> This an interesting tack, but I don't think I agree. SPF takes as
> input a domain name and an IP address, does some crunching, and
> returns a result. That result is deterministic. At the level of that
> interface, the IP address of a forwarder is indistinguishable from any
> other, so to say one class of IP address is valid while another is not
> seems a non-sequitur to me. Any result produced by taking those two
> pieces of input and following the SPF algorithm is an SPF result. If
> two different systems have differing outputs given the same inputs,
> then we have an interoperability problem and/or a broken
> specification. Requiring different treatment of forwarders based on a
> heuristic would not be an SPF result, in the same way
rfc4408 says:

9.5. MTA Relays

   The authorization check generally precludes the use of arbitrary MTA
   relays between sender and receiver of an E-Mail message.

   Within an organization, MTA relays can be effectively deployed.
   However, for purposes of this document, such relays are effectively
   transparent.  The SPF authorization check is a check between border
   MTAs of different domains.

   For mail senders, this means that published SPF records must
   authorize any MTAs that actually send across the Internet.  Usually,
   these are just the border MTAs as internal MTAs simply forward mail
   to these MTAs for delivery.

So, alias forwards that are requested/configured by the recipient are a
border MTA for the recipients organization.  If the alias forwarder does
not do the SPF check, any SPF check must use the Received header from
the forwarder and not the forwarders IP (or simply don't do the check at
all).  See paragraph 9.3 which also explains these options.

Therefore, I believe you are incorrect.  Simply plugging the alias
forwarders IP and MAIL FROM domain into the SPF algorithm does NOT
produce an SPF result according to rfc4408.

---

From: Stuart Gathman <[email-masked]>
Date: Wed, 04 Apr 2012 02:36:07 -0400

Long ago, Nostradamus foresaw that on 03/24/2012 12:57 AM, Murray S.
Kucherawy would write:
> I would bet real money that this is a non-starter. By doing so, you
> would declare a big piece of the deployed email infrastructure
> non-compliant. If I may paraphrase John Levine about MLMs: Forwarders
> have been doing what they do for decades without complaints, so who
> are we to suddenly tell them that they're wrong?
Forwarders are not doing it wrong.  If they are authorized by the
sender, the sender needs to include them in his SPF record as another
"border MTA".  If authorized by the recipient, then the recipient needs
to treat the forwarder as another "border MTA".  (This is easier if a
large forwarder publishes SPF so the recipient knows in an automatically
updated manner which MTAs belong to the forwarder.)

---

From: Stuart Gathman <[email-masked]>
Date: Tue, 17 Apr 2012 22:05:57 -0400

Kucherawy would write:
>>   Forwarding is only broken when receivers
>> confuse pseudo-SPF results from an alias forwarder with real SPF
>> results, and then reject on FAIL.
>> [...]
> This an interesting tack, but I don't think I agree.  SPF takes as input a domain name and an IP address, does some crunching, and returns a result.  That result is deterministic.  At the level of that interface, the IP address of a forwarder is indistinguishable from any other, so to say one class of IP address is valid while another is not seems a nonsequitur to me.
Ok, so the SPF result is a real SPF result.  But it is insane for a
receiver to pay any attention to that SPF result when *they themselves*
have requested the alias forward, and they *know* (or ought to know)
from rfc4408 9.3 that the SPF result from such forwards are not useful
(if you don't agree with my stronger term of "not valid").  The IP
address of a forwarder *is* distinguishable to the recipient.  The
recipient requested the forward!  Translating that to an IP set
representing forwarders is especially easy when the forwarder publishes
SPF (and the revised RFC ought to recommend this - it is not mentioned
in rfc4408).  Conceptually, apply the check() algorithm to each
forwarder domain in your list (the domains will not appear in MFROM, of
course) with the connect IP.

All I'm saying is, don't check SPF, or at least don't reject on SPF for
alias forwarders.  This is an *easy* requirement for the vast majority
of domains, which have a relatively small number of users.

    A Specific Request for 4408bis

In section 9.3:

  2. The middle, when E-Mail is forwarded.
     ...
     3.  Forwarder services SHOULD publish SPF for a domain representing
their forwarding service, so that recipients can conveniently skip SPF
checks for the forwarder, in a manner that automatically incorporates IP
changes on forwarders MTAs (assuming they update their policy properly).

---

From: "Rolf E. Sonneveld" <[email-masked]>
Date: Wed, 18 Apr 2012 10:13:10 +0200

On 4/18/12 4:05 AM, Stuart Gathman wrote:

[...]

> (if you don't agree with my stronger term of "not valid").

This typically scales to a company size where all employees still fit in 
one room and the mail admin can yell: thou shalt always tell me thy 
forwardings. It simply doesn't scale to the size of the Internet.

>    The IP

So if someone sets a forwarder at Gmail or Hotmail, you would exempt all 
mail with RFC5321.MailFrom containing gmail.com and hotmail.com from SPF 
checking?

> changes on forwarders MTAs (assuming they update their policy properly).

And how are you supposed to know the domainnames of all forwarding 
services world wide, to be able to _skip_ SPF checks? And wouldn't that 
be an incentive for spammers to start abusing those domainnames for 
sending their spam to circumvent SPF checks?

/rolf

---

From: "John Levine" <[email-masked]>
Date: 18 Apr 2012 14:37:51 -0000

>sending their spam to circumvent SPF checks?

Of course.  We've been through this a bazillion times already.
Domains cannot publish assertions that say to accept mail you wouldn't
otherwise accept, because bad guys will lie.

SPF's path authentication is funamendally broken, because SMTP mail is
designed to be delivered independent of the path it takes.  It happens
that a lot of mail is delivered via a predictable path, enough to make
SPF useful, but for the stuff that's not, it's a limitation of SPF,
not a fault of the mail system, and there is nothing to fix.

R's,
John

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 18 Apr 2012 17:14:15 +0200

On Wed 18/Apr/2012 17:05:41 +0200 John Levine wrote:
> 
> SPF's path authentication is fundamentally broken, because SMTP mail is

Hm...  IMHO the design of SMTP mail is fundamentally broken, as it provides
for no path authentication.  That way it puts billions of email users at the
mercy of a fistful of spammers who exploit such inability to ascertain
forwarding responsibilities.  That's what we have to fix.

---

From: Scott Kitterman <[email-masked]>
Date: Wed, 18 Apr 2012 11:25:38 -0400

> forwarding responsibilities.  That's what we have to fix.

The version of we that has to fix that is not this working group.

Scott K

---

From: Andrew Sullivan <[email-masked]>
Date: Wed, 18 Apr 2012 11:33:57 -0400

Dear colleagues,

On Wed, Apr 18, 2012 at 05:14:15PM +0200, Alessandro Vesely wrote:

> for no path authentication.

This is not the SMTPBIS WG, and we are not going to "fix" that
problem.  The extent to which it would be nice if SMTP had a radically
different design is simply out of scope for this WG, and any argument
that leads to the conclusion "so we need to fix SMTP" is
automatically, for our purposes, a _reductio ad absurdum_.  

Best,

A

-- 
Andrew Sullivan
[email-masked]

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 18 Apr 2012 16:09:49 +0000

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of Alessandro Vesely
> Sent: Wednesday, April 18, 2012 8:14 AM
> To: [email-masked]
> Subject: Re: [spfbis] #12: RFC 4408 Section 9 - Forwarding and helo identities
> 
> > SPF's path authentication is fundamentally broken, because SMTP mail
> > is designed to be delivered independent of the path it takes.  It
> > happens that a lot of mail is delivered via a predictable path, enough
> > to make SPF useful, but for the stuff that's not, it's a limitation of
> > SPF, not a fault of the mail system, and there is nothing to fix.
> 
> Hm...  IMHO the design of SMTP mail is fundamentally broken, as it
> provides for no path authentication.  That way it puts billions of
> email users at the mercy of a fistful of spammers who exploit such
> inability to ascertain forwarding responsibilities.  That's what we
> have to fix.

This is straying from the topic at hand.  However, I disagree with the characterization of SMTP mail as "fundamentally broken" merely because it can be abused, unless you're also prepared to put that label on anything else that can be abused or spoofed.  It's a pretty long list, and it includes IP, ICMP, etc.

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 18 Apr 2012 18:29:12 +0200

> automatically, for our purposes, a _reductio ad absurdum_.  

Isn't SPF itself an attempt at fixing SMTP?  It hijacks elements of the SMTP
protocol and confers them a meaning that they didn't originally bear.  John
holds that it is broken for this very reason.  If he is right, we should
somehow state that SMTP servers must not reject on spf fail, possibly except
for sites where the admin personally knows all mail users.  Otherwise, we may
indicate a safe way, if it exists, to perform rejection, possibly compelling
forwarders to publish extra SPF records as needed.

Please note that when I write "we should state" or "we may indicate", above,
I don't mean any formal language.  I'm confident we'll be able to word the
document acceptably well according to what we think.  I just want to know
what we think.  Isn't it possible to come to a consensus?  Leaving things in
a half-broken status isn't good for anyone.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 18 Apr 2012 16:34:27 +0000

> Sent: Wednesday, April 18, 2012 9:29 AM
> To: [email-masked]
> Isn't SPF itself an attempt at fixing SMTP?  It hijacks elements of the
> SMTP protocol and confers them a meaning that they didn't originally
> bear.

Would you say as well that greylisting is an attempt at fixing SMTP?

I see SPF and its ilk as layers on top of SMTP to provide additional services, not attempts at fixing it.

-MSK

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 18 Apr 2012 18:42:37 +0200

On Wed 18/Apr/2012 18:38:46 +0200 Murray S. Kucherawy wrote:
>> From: ietf.org On Behalf Of Alessandro Vesely
>

Yes, albeit less invasive.

> services, not attempts at fixing it.

That's a more polite way to convey the same meaning, if you consider the
nature of the service being provided.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 18 Apr 2012 16:48:56 +0000

> Sent: Wednesday, April 18, 2012 9:43 AM
> To: [email-masked]
> That's a more polite way to convey the same meaning, if you consider
> the nature of the service being provided.

The difference is that your view is that SMTP is broken; mine is that SMTP works as designed without it, but has some added protection with it, and the consumer gets to decide which of the two modes is desirable.

I'll let those more seasoned in protocol and email history take it from there if more information is needed.

-MSK

---

From: Andrew Sullivan <[email-masked]>
Date: Wed, 18 Apr 2012 12:51:11 -0400

No hat.

On Wed, Apr 18, 2012 at 06:29:12PM +0200, Alessandro Vesely wrote:

> protocol and confers them a meaning that they didn't originally
> bear.

No.  SPF is an attempt to express something about how receivers might
want to treat a messge received using SMTP.  The Internet works in
layers, and SPF sits atop SMTP.  I will cheerfully grant that it is
perched there somewhat precariously, partly because it attempts to
make assertions that are sometimes false.  But it is not itself
altering SMTP.  

There is an important difference between those two ways of looking at
things, and I want to attend to that difference.

Best,

A
-- 
Andrew Sullivan
[email-masked]

---

From: Commerco WebMaster <[email-masked]>
Date: Wed, 18 Apr 2012 11:06:13 -0600

> Isn't SPF itself an attempt at fixing SMTP?  It hijacks elements of the SMTP

In a sense, yes, though at least from the perspective of an implementer, 
the true value of SPF seems mostly linked to its primary reason for 
being.  That is, a mechanism to reasonably authoritatively tie domain 
names to specific outbound email servers, such that the overall 
integrity of the domain name's reputation can be preserved and so that 
others cannot misuse these domain names in spam email.  So in that 
spirit, it is sort of a wrapper to the way SMTP works, so more of an 
upgrade than a fix per se.  I think, kind of along the lines of what 
Murray was talking to in his point about greylisting.

We have had a lot of activity on this list over the past day or so which 
seems to exceed the WG charter scope.  Many of the things said appear as 
simply truisms which cannot be fixed in SPFbis.  I tend to agree with 
John Levine's statements which I interpret as indicating a desire to 
move on past some of the things we cannot address here, even though they 
may well be interesting to the technical minds here.

Best,

Alan M.

---

From: Hector Santos <[email-masked]>
Date: Wed, 18 Apr 2012 13:15:59 -0400

Hi,

SMTP is not "broken" - it has relaxed provisions by design.  It has 
all the requirements and expectations for validity, but it is relaxed 
enforcement.

You can see the mindset here, with 2821:

    7.1 Mail Security and Spoofing

    ..

    This specification does not further address the authentication issues
    associated with SMTP other than to advocate that useful functionality
    not be disabled in the hope of providing some small margin of
    protection against an ignorant user who is trying to fake mail.

With 5321, what changed is that the user is no longer ignorant! :)

The first thing to fix is this mindset! There are strong forces to 
ACCEPT all mail and let the user decide and passing the buck to the 
user is seriously abused and dangerous.    More system level filtering 
is required and SPF helps in that areas.  Its a domain policy, not a 
domain user policy.

For SPF, honor the domain policy for -ALL. Its means bad mail - get 
rid of it! Regardless if you don't believe the domain meant it, that 
is what the RULES is. -ALL means REJECT.

Instead what we see? More relaxed provisions and we wonder whether 
"anything" can work without abuse, but we do our best to make sure its 
abused?

Also, gmail.com is not a MODEL for everyone to use.  Not everyone is a 
ESP, plus google has more interest and seeing and accepting ALL 
traffic - good or bad for its form of BI (Business Intelligence) data 
warehousing.

-- 
Hector Santos, CTO
http://www.santronics.com
http://hector.wildcatblog.com

Alessandro Vesely wrote:
> _______________________________________________
> spfbis mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/spfbis
> 
>

---

From: Hector Santos <[email-masked]>
Date: Wed, 18 Apr 2012 13:51:56 -0400

I always classified this multi-network problem as Developer vs 
Administrator philosophical differences.  (note, that does not mean 
its distinct)

Adhering to strong SMTP compliancy vs relaxation.   When it comes to 
SPF, it simply tries to re-introduce a form of validation that for 
relaxed requirement that is is already there.  It raises the bar for 
expected compliancy and by following it, you remove the legacy 
problems related to relaxation.

But if we continue with even more relaxed provisions for a protocol 
designed to remove it, well, we defeat the propose of the protocol and 
the legacy problem remains - no incentive to legacy bad guys to adapt 
or even try to fake it.  Just do nothing and simply target systems 
that behave with relaxed provisions.

---

From: Alessandro Vesely <[email-masked]>
Date: Thu, 19 Apr 2012 17:47:16 +0200

> make assertions that are sometimes false.  But it is not itself

Reject-on-fail was the initially proposed use of SPF.  Later on, SPF
results started to be used to adjust messages' score.  Your
description above seems to refer to the latter mode only.

> things, and I want to attend to that difference.

As a matter of fact, RFC 4408 openly considers legitimate to reject on
fail.  Thus, SPF can (and does) affect SMTP functioning.  IMHO, moving
from experimental to standard track must imply clarifying such issue.

---

From: Hector Santos <[email-masked]>
Date: Thu, 19 Apr 2012 12:12:30 -0400

Alessandro Vesely wrote:
> Reject-on-fail was the initially proposed use of SPF.  

And still is.

> Later on, SPF
> results started to be used to adjust messages' score.  

Isolated solutions, heuristic methods not not univerisal standards 
material and more importantly only for relayed policies where there 
was no REJECT-ON-FAIL policy condition to process.

> Your description above seems to refer to the latter mode only.

And I hope that mindset does not continue to permeate the framework to 
alter the purpose of REJECT-ON-FAIL deterministic nature of SPF.  That 
would be a major show or endorsement and adoption stopper.

Please stop trying to make it into worthless protocol with relaxed 
provisions that will do nothing more but add more overhead to DNS and 
the receiver and foremost does NOTHING to solve the abuse of domains 
and the major abuse the receivers gets are targets of the junk mail.

> fail.  Thus, SPF can (and does) affect SMTP functioning.  

Only on the positive side in allowing what it could not do before. 
Its like 4409, if you connect to a SUBMIT server, on purpose via port 
587, knowing full well you are required to have an account to login, 
then that ACTION authorizes the MSA SMTP server to do a lot more it 
could not do before on a public port 25 server.