Subject: [spfbis] RFC 4408 Section 9 - Forwarding and helo identities

From: Scott Kitterman <[email-masked]>
Date: Thu, 16 Aug 2012 11:40:41 -0400

Alessandro:

Since you filed this issue:

http://trac.tools.ietf.org/wg/spfbis/trac/ticket/12

I would appreciate it if you would review the current -04 draft and comment on 
if additional changes are needed (text always welcome).  In particular, in 
this revision there is more discussion about redirect and why it is useful.

I don't think this is addressed in your comments you already sent today.

Scott K

---

From: Alessandro Vesely <[email-masked]>
Date: Thu, 16 Aug 2012 20:58:31 +0200

Hi Scott,

On Thu 16/Aug/2012 17:40:41 +0200 Scott Kitterman wrote:
> I would appreciate it if you would review the current -04 draft and
> comment on if additional changes are needed (text always welcome).

Briefly, adopting SPF implies a conundrum of endlessly repeated
philosophical principles like "No, it is the {sender|receiver} who
should do so and so."  A protocol that people can adhere to should not
cause such issues, or at least indicate a quick way to solve them.

We need to discuss this subject, IMHO.

> In particular, in this revision there is more discussion about
> redirect and why it is useful.

The redirect modifier has nothing to do with issue #12.

> I don't think this is addressed in your comments you already sent

No, that was too long already...

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 16 Aug 2012 15:01:01 -0400

> The redirect modifier has nothing to do with issue #12.

In issue #12 it says:

"We should encourage the use of _spf labels and redirect= for designing sound 
and maintainable SPF compliance."

So I'm a bit confused why you say it's unrelated.

Scott K

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 17 Aug 2012 09:32:00 +0200

On Thu 16/Aug/2012 21:01:01 +0200 Scott Kitterman wrote:
> "We should encourage the use of _spf labels and redirect= for
> designing sound and maintainable SPF compliance."

A sound SPF compliance is a prerequisite.  Redirect allows
maintainability, hence is necessary.

> So I'm a bit confused why you say it's unrelated.

Yes, you're right, sorry.  I should have said "has /little/ to do with
issue #12".

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 17 Aug 2012 15:38:17 -0700

> cause such issues, or at least indicate a quick way to solve them.

I appreciate the fact that there are people out there who want the
silver bullet.  For some people, SPF is that.  That simply is not true
for everyone.  I have yet to work for any company (and I now work for
a pretty huge one in terms of mail flows) where either it or its
customers enabled SPF to reject on "fail", for example.

The common point to both of these groups is that SPF's results are
input to an overall evaluation of a message.  For people that think
SPF completely solves the problem, their evaluation system consists
solely of SPF, and they do reject-on-fail.  It works for them and
that's fantastic.  There exists, however, a substantial population
that wants more than one piece of information to decide upon the fate
of a message. The most common reason for that appears to involve SPF's
known failure modes, but it's not for us to speculate on what those
people have in mind.

However, it's important to note that the basic premise I stated above
holds true for both camps.  Accordingly, the best this document (or
any other that specifies something which isn't a silver bullet) can do
is make those data available to the operator, accompanied by some
pedagogy on the surrounding subject matter, and let the operator
decide what to do with it.  I think that's what we're doing here, as
we did with other email authentication work, and as 4408 itself did.
It's the right approach.

Thus, it is absolutely the case that this lands us in a space of
presenting what you call philosophical principles.  But I don't think
we can or should do any better than that.  So we simply need to do as
good a job of that as possible, and then stop.

So back to the topic of writing a protocol document: The protocol,
then, comprises the steps to collect the inputs to the algorithm,
execution of the algorithm, the output of the algorithm, and how to
deal with exceptions.  As my high school English teachers used to say:
precise and concise.  I think we're doing that here.

All this stuff about doing some things as system settings and others
as user settings, picking different evaluation points, rewriting
envelopes and HELO, etc., really complicates what's supposed to be a
protocol document.  If we're seriously inclined to push all of that
architecture advice on the reader, I suggest we consider moving all of
that to a separate document that we develop once this one publishes
and we re-charter to cover it.  Or if it's already written down in an
existing document, just refer to that.

On the particular topic of forwarding, I think RFC5321 Section 3.9.1
is just fine.  SMTP works fine the way it is with respect to alias
expansion; SPF is in effect an overlay on top of it in this regard.
We can say that an alias expander wishing to ensure SPF passes can do
what's suggested in ticket #12, but go no further.  RFC5598 talked
about alias expansion in more detail if we're in need of references to
more material on the topic.

-MSK

---

From: Alessandro Vesely <[email-masked]>
Date: Sat, 18 Aug 2012 18:17:35 +0200

> customers enabled SPF to reject on "fail", for example.

I wouldn't strive for a silver bullet, but regular ammo, at least,
rather than a naked gun, would seem to be advisable.

> known failure modes, but it's not for us to speculate on what those

Agreed.  Scott mentioned the pros and cons of both approaches, recently.

> we did with other email authentication work, and as 4408 itself did.

Agreed, except that "the document defines a protocol" is a little more
than making the data --the protocol's results-- available.  In
particular, SPF defines both the policy and the algorithm to get the
results.

> good a job of that as possible, and then stop.

I read that as trying to avoid opening cans of worms larger than what
we are prepared to deal with.

> precise and concise.  I think we're doing that here.

Yes.

> existing document, just refer to that.

I'd like the idea of a deployment guide.  But we cannot say "a
receiver can shoot itself in the foot if it wishes to do so", whatever
the language (and bullets) we use.

The workaround I suggested, helo-pass prevents rejection, isn't really
that complicate.  It can be explained precisely and concisely within a
single short paragraph.  But different participants dislike it, for
different reasons that I never fully understood.

There's plenty of other ways.  For example, the receiver could check a
DNSWL before rejecting, and advertize which one it uses.  Instead,
Section 9.3.2 suggests that "the sender [...] might then be able to
resolve the issue".  What if it is not able to do so, discuss for ever?

> more material on the topic.

Neither Section 3.9.1 of RFC 5321 nor Section 5.1 of RFC 5598 consider
it makes a difference whether the expanded aliases land within the
same ADMD or to external domains.  It makes a relevant difference for SPF.

If we don't update that, we should make it clear that applying local
policy is an arbitrary and unilateral action that interferes with mail
delivery.  At least, that will save people useless discussions.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sat, 18 Aug 2012 09:39:51 -0700

> particular, SPF defines both the policy and the algorithm to get the

Yes, but SPF cannot demand enforcement of that policy.

> we are prepared to deal with.

That's certainly true.

> the language (and bullets) we use.

We certainly can say that, by presenting the landscape in which this
operates.  Hopefully the reader then makes an intelligent decision,
and we can encourage this in the way we write.

> different reasons that I never fully understood.

What I want us to avoid is presenting a pile of things that,
individually, are not that complicated, but as a group they simply
give the reader far too many variants of basic SPF to consider.

> resolve the issue".  What if it is not able to do so, discuss for ever?

That's further added complication.  This shouldn't be in the protocol document.

> same ADMD or to external domains.  It makes a relevant difference for SPF.

SMTP is relevant to SPF, but SPF is irrelevant to SMTP.  That's why an
update is the wrong thing to do.

> delivery.  At least, that will save people useless discussions.

I'm sure it's not arbitrary to the receiver, but the rest of that
sounds right to me.

-MSK

---

From: Scott Kitterman <[email-masked]>
Date: Sat, 18 Aug 2012 13:00:28 -0400

> I'm sure it's not arbitrary to the receiver, but the rest of that

Right.  It's supposed to interfere with mail delivery.

Scott K

---

From: Alessandro Vesely <[email-masked]>
Date: Sat, 18 Aug 2012 19:52:44 +0200

> Yes, but SPF cannot demand enforcement of that policy.

Of course not.  It just says a receiver may enforce it, albeit it
currently doesn't spell "MAY".

> and we can encourage this in the way we write.

Yes, but we cannot write "you can shoot without drawing" bald and bold
in the intro, and "take care of you feet, though" using font size 6
down in Section 9.  It sounds fraudulent.

> give the reader far too many variants of basic SPF to consider.

OTOH, if we were able to define a compound result for receivers who
check both identities, that would be a relevant simplification.  If an
added paragraph could save endless discussions times implementers,
it'd be worth.

> That's further added complication.

"As simple as possible, but not simpler."

> This shouldn't be in the protocol document.

Sorry...  "this" refers to?

> update is the wrong thing to do.

You know this better than I, so I'd bet you're right.  However, RFC
6409 warns against message modifications that can affect signatures,
while pre-DKIM RFC 4409 didn't.  RFC 4871 has no "updates: 4409".  Is
there a hidden update?

I recently mentioned a similar example in the passage from 2821 to
5321.  I don't mind not having "updates: 5321" if that's the way it works.

---

From: Hector Santos <[email-masked]>
Date: Sat, 18 Aug 2012 13:52:47 -0400

> Agreed.  Scott mentioned the pros and cons of both approaches, recently.

Disagreed with the introduction of a "Guessing Game" when there is no 
need to be guessing.

When it comes to -ALL policy there is a SILVER BULLET view by domains 
who go to that level and if a receiver is going to claim SPF 
compliance, it is not its job to 2nd guess that 100% exclusive nature 
of a -ALL policy, even with ~ALL (SoftFail), but definitely with -ALL 
- there is no dispute what the domains desires and if they felt that 
receivers are too stupid to know what they wanted with -ALL, well, 
they could a) easily not use SPF and/or B) use a weaker policy such as 
~ALL or the the wasteful ?ALL.

We need to keep the eye on the prize of what SPF is and it was 
definitely NOT about reputation modeling or even DKIM.  People wanted 
SPF for a single purpose and those who could use -ALL, did and a 
growing amount of domains who started with ~ALL or ?ALL, did 
eventually learn what a waste it was with little value in protecting 
against spoofing. So if they could do so, they switched to a HARDFAIL 
-ALL for the sole purpose of remove any guessing on the receiver part.

The key thing to remember about a domain public announcement of a 
security policy, and for many businesses, a CORPORATE MANDATE, trhe 
SPF compliance receiver who ignores the -ALL is at risk of harming a 
DOMAIN with an uninvited uncertainty it introduces on its own. The 
DOMAIN is no longer legally responsible. So its now a legal disclaimer 
for the SPF domain.  If facebook.com says -ALL for its policy, and the 
so-called SPF compliant receivers ignores a detectable SPOOF based on 
the -ALL policy, legally, a mal-practice claim can be made against the 
receiver if there is measurable harm caused on the domain or user. 
Facebook will not be at fault here. The Receiver will.  The only 
lawsuit here would be against the  Receiver  So the best legal way to 
get around it, is to stop playing guessing games the domain did not 
want you to do in the first place.  If he says -ALL, if a spoof is 
detectable, then rejecting it is the safest bet across the board. 
Marking it is fine, but it better have a near equivalent security 
outcome for the end-users and that pretty much means stream separation.

-- 
HLS

---

From: SM <[email-masked]>
Date: Sat, 18 Aug 2012 13:28:53 -0700

Hi Alessandro,
>while pre-DKIM RFC 4409 didn't.  RFC 4871 has no "updates: 4409".  Is

No.  There was a discussion about message modifications during the 
update that lead to RFC 6409.

Regards,
-sm

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sat, 18 Aug 2012 20:44:45 -0700

> added paragraph could save endless discussions times implementers,

I believe we're constrained against adding things like this.  It would
constitute an extension to what check_host() currently does.

A receiver could decide to execute check_host() twice, once per
identifier, to get two results if it so chooses.  That's not an
extension to the algorithm, however.  I would argue it constitutes an
applicability statement.

To be entirely procedural, the SPFbis charter limits us to "addition
of any enhancements that have already gained widespread support", and
although of course the co-chairs are the abiters of such calls, I
personally don't think this qualifies.

> "As simple as possible, but not simpler."

Right.  Otherwise, why not also suggest correlating with DKIM results,
WHOIS data, content analysis, REPUTE results, etc.?  All plausible,
but all outside of the core of SPF, and I'd wager all relatively
untested in terms of protocol development.  And they also exceed the
charter, as above.

We need not to dive into ways to complicate matters.

> while pre-DKIM RFC 4409 didn't.  RFC 4871 has no "updates: 4409".  Is

Section 8 of RFC6409 refers not only to DKIM, but also to PGP and
S/MIME, technologies that easily pre-date RFC4409.  I read it as an
acknowledgement by the authors that RFC4409 should've done that in the
first place, and this is going back and fixing that problem.  DKIM
happened to come along in the interim.

Also, in my experience, "Updates" in terms of RFCs tends to be used
only to add to the normative language of a specification, not as a
means of providing more information to implementers.  Correcting ABNF,
adding normative constraints, applicability statements, adding
extensions, etc., are when you'd use it.  That's why RFCs 4871 and
6376 didn't do so.

-MSK

---

From: "Wayne Doust" <[email-masked]>
Date: Mon, 20 Aug 2012 14:34:05 +1000

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Murray S. Kucherawy
Sent: Saturday, 18 August 2012 8:38 AM
To: Alessandro Vesely
Cc: [email-masked]
Subject: Re: [spfbis] RFC 4408 Section 9 - Forwarding and helo
identities

...

I appreciate the fact that there are people out there who want the
silver bullet.  For some people, SPF is that.  That simply is not true
for everyone.  I have yet to work for any company (and I now work for a
pretty huge one in terms of mail flows) where either it or its customers
enabled SPF to reject on "fail", for example.

The common point to both of these groups is that SPF's results are input
to an overall evaluation of a message.  For people that think SPF
completely solves the problem, their evaluation system consists solely
of SPF, and they do reject-on-fail.  It works for them and that's
fantastic.  There exists, however, a substantial population that wants
more than one piece of information to decide upon the fate of a message.
The most common reason for that appears to involve SPF's known failure
modes, but it's not for us to speculate on what those people have in
mind.
----------------------------

At the risk of rehashing stuff that may have already been discussed....

To me (at least) the publication of "-all" is a very bold statement. The
sender is saying two things:

1) No email will originate from any address other than the one's listed
2) No email will be forwarded 

I include point 2 because the sender is not in control of the initial
receiver and therefore cannot guarantee that SRS is implemented
everywhere, nor can it guarantee that whitelists are in use at the final
destination.

Considering this, it would seem reasonable to assume that the SPF
"Silver Bullet" is determined by the sending domain, rather than a
policy of the receiver - that's why it's a "Sender Policy". Any domain
that publishes "-all" is saying that SPF is a silver bullet - for THEM.

Also considering this, one would think that "-all" would be less common
that "~all", however that does not appear to be the case from the email
that I receive.

Regards,

Wayne Doust

(disclaimer remove)

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sun, 19 Aug 2012 21:47:14 -0700

> that "~all", however that does not appear to be the case from the email

The survey I did shows "~all" over "-all" at better than 2:1.  Cisco's
survey is better than 3:1.

Undoubtedly there are plenty of ADMDs that think "-all" as you're
defining it is right for them.  Some of them are correct, but
experience has shown that many are unaware that they can't actually
make those assertions, leading to accidental rejection of important
messages.  That causes either (a) the sender to back down from "-all"
to "~all"; (b) the sender to split streams by creating a new domain
name; or (c) the recipient to treat "-all" as advisory.

That doesn't sound like a silver bullet to me.

-MSK

---

From: "Wayne Doust" <[email-masked]>
Date: Mon, 20 Aug 2012 15:32:19 +1000

-----Original Message-----
From: Murray S. Kucherawy [mailto:[email-masked]] 
Sent: Monday, 20 August 2012 2:47 PM
To: Wayne Doust
Cc: [email-masked]
Subject: Re: [spfbis] RFC 4408 Section 9 - Forwarding and helo
identities

Undoubtedly there are plenty of ADMDs that think "-all" as you're
defining it is right for them.  Some of them are correct, but experience
has shown that many are unaware that they can't actually make those
assertions, leading to accidental rejection of important messages.  That
causes either (a) the sender to back down from "-all"
to "~all"; (b) the sender to split streams by creating a new domain
name; or (c) the recipient to treat "-all" as advisory.

-MSK
-----------------------------

As with all things, YMMV and generalisations rarely fit with local
experience. My stats for yesterday were:

Fail      : 4672
SoftFail  : 814
PermError : 323
TempError : 134
None      : 12144
Neutral   : 758
Pass      : 10454

Regardless of whether people realise by adding "-all" they are making a
bold statement doesn't diminish that they ARE making a bold statement.
Perhaps this can be stated better in the rfc, however for me it does
seem clear from the text that it *is* a bold statement. 

So, the question is: Do we make allowances for the fact that
implementers are making bold statements they cannot back up? Now IMO
THAT is a separate question altogether - one I think should be lumped in
with those that cannot even publish a syntactically correct record to
start with.

It has been stated that accommodations should not be made for software
bugs. I think the same is true here - we shouldn't be lenient with
"-all" just in case the record publisher doesn't understand what it
means. Are wizards to blame for this?

FWIW - I TempFail (451 with detailed explanation and a phone number) on
SPF Fail with a whitelist. This gives the sender a chance to ring me and
be added to the whitelist.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 20 Aug 2012 00:49:04 -0700

> means. Are wizards to blame for this?

RFC4408 already leaves the option of what to do with "-all" to the
receiver.  That "leniency" is already well established as an
operational necessity.  We're not changing anything here, and I don't
think we should.

-MSK