Subject: Re: [spfbis] #12: RFC 4408 Section 9 - Forwarding and helo identities

From: S Moonesamy <[email-masked]>
Date: Tue, 13 Mar 2012 01:30:21 -0700

At 05:27 10-02-2012, spfbis issue tracker wrote:
>#12: RFC 4408 Section 9 - Forwarding and helo identities
>
>  Message from Alessandro Vesely on 10 Feb 2012:
>
>  The advice given in the non-normative Section 9 is misleading.  On the
>  one hand, it provides overly cumbersome techniques based on macros
>  that are not universally considered safe.  On the other hand, it fails
>  to state the obvious point that a return-path is only needed if
>  someone is going to care about delivery failures.
>
>  When an email alias refers to an external domain, the forwarder must
>  change it.  This has to be normative, and update Section 3.9.1 of
>  [RFC5321] --not one of the best written sections of that paper.
>
>  Forwarding Ã  la MAIL FROM:<> should be mentioned as the default,
>  easiest way to re-inject messages.  The implied interoperability is
>  that the owner of the target mailbox must be aware of a given
>  "forwarding recipe", and be able to modify or delete it.  The
>  variation MAIL FROM:<[email-masked]> is necessary when a
>  postmaster's action is needed for modifying/deleting a given recipe.
>  SRS should be mentioned as a general purpose variation for extended
>  forwarding needs.
>
>  The spec should say that an SPF record must be defined for every A or AAAA
>  record, irrespectively of whether its host label also appears in some MX
>  RRs.  We should encourage the use of _spf labels and redirect= for
>  designing sound and maintainable SPF compliance.
>
>  http://www.ietf.org/mail-archive/web/spfbis/current/msg00374.html
[snip]
>Ticket URL: <http://trac.tools.ietf.org/wg/spfbis/trac/ticket/12>

Please comment on Issue #12.  As a reminder, please keep the subject line.

Regards,
S. Moonesamy
SPFBIS WG co-chair

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 13 Mar 2012 14:22:54 -0400

> >  someone is going to care about delivery failures.

It is a given that one only need care about return-path being accurate if one 
cares about delivery failures, but also irrelevant.  Random dropping of email 
(either original messages or bounces) interferes with the overall reliability 
of email on the internet.  I don't think the group should add anything 
indicating that discarding bounces or directing mail from to a non-existant 
mailbox because the sender doesn't care is a good idea.   -1 on the concept.

More detailed comments below.

> >  The spec should say that an SPF record must be defined for every A or
> >  AAAA record, irrespectively of whether its host label also appears in
> >  some MX RRs.  We should encourage the use of _spf labels and
> >  redirect= for designing sound and maintainable SPF compliance.

This issue encompasses a number of disparate recommendations and Alessandro 
has a number of comments that relate to different parts of the section.  I 
think it's useful to talk about each part in turn:

   9. Implications
      9.1. Sending Domains

> >  that are not universally considered safe.  

This appears to refer to 9.1.  

Nothing is "universally considered safe".  It's a known fact that drinking too 
much water can kill you.  That doesn't make drinking water a bad idea.  It is 
true that the type of record suggested in 9.1 is uncachable, so domains that 
use it should be prepared for more DNS traffic than is usual for an SPF record 
that doens't use uncachable macros.  I think it would be reasonable to mention 
this.

Additionally, there are mechanisms that exist today that did not exist when 
RFC 4408 was drafted that provide alternative mechanisms to collect the 
relevant data.  draft-ietf-marf-spf-reporting, currently in IETF wide last 
call, is one mechanism that can support exactly this use case.  Outside the 
IETF, DMARC (dmarc.org) is another approach.  

References to at least draft-ietf-marf-spf-reporting should be added to 9.1, 
but the current text about a special SPF record should remain.  The advantage 
of the special SPF record/DNS log approach is that it requires no cooperation 
from receivers beyond doing normal SPF checks.

      9.2. Mailing Lists

It does not appear that any of the comments address 9.2.  I think it would be 
useful (and a good preamble to the forwarding discussion) to update this text 
to make use of the email architecture terminology that didn't exist when RFC 
4408 was written.  The references should be updated as well.  I don't think 
any functional changes to 9.2 are needed.

      9.3. Forwarding Services and Aliases 

> >  SRS should be mentioned as a general purpose variation for extended

I disagree with the comments.  First, I think 3.6.2 of RFC 5321 is more 
relevant (it even mentions SPF by name).  I don't think 4408bis should update 
5321.  I think it would make consensus substantially harder to achieve and 
there is no technical need for it.

In email architecture terms, a forwarder ("relay SMTP server" in RFC 5321 
terms, not an alias expander) is part of the receiving ADMD.  Forwarding is 
configured by the receiver for the receiver's convenience (unlike mailing lists 
where there is a joint participation model).

It would be useful to explicitly say in this section that THE place to do SPF 
checks is at the transition point between the sending ADMD and the receiving 
ADMD.  When a message is relayed within the recieving ADMD, SPF checks, per se 
are not appropriate, all that can be done is to use the results of checks done 
on the ADMD border if they can be trasmitted in a reasonably trustworthy 
manner.

There are lots of ways problems in this area can be addreesed, a couple are:

1.  Check at the border and internal relays whitelist border relays from SPF 
checks

2.  Rewrite Mail From (e.g. SRS) when mail is transmitted between external and 
internal parts of the ADMD so the check can validate that the mail 
legitimately came a trusted part of their ADMD.

Doing wild tricks like null mail from forwarding are not appropriate here.  
They will result in mail loss and there's no need.

In the terms that 9.3 is written, I think this is mostly a middle/end problem, 
but the methods listed for beginning/middle/end that are listed are all 
potentially useful.  This could use some cleanup to use email arch terms and 
such, but it's a pretty decent list of options to use.

      9.4. Mail Services 
      9.5. MTA Relays 

It doesn't appear that there are parts if this issue related to the last two 
sub-sections.

> >  The spec should say that an SPF record must be defined for every A or
> >  AAAA record, irrespectively of whether its host label also appears in
> >  some MX RRs.  We should encourage the use of _spf labels and
> >  redirect= for designing sound and maintainable SPF compliance.

I think it would be useful to add something in this section that addresses 
domains that send no mail.  For hosts that really send no mail, "v=spf1 -all" 
is a great record to be publishing as it's got a zero percent chance of false 
positive.  I think SHOULD publish for all domains is plenty.  I don't see a 
MUST.

redirect/include and _spf are nice tools for domains that need them.  They are 
not, however, a general solution or requirement.  I think 5.2 already give 
adequate coverage to the use of include:.  I think it would be useful (as part 
of the processing limits discussion) to include an indication that SPF records 
SHOULD NOT require TCP fallback to retrieve (and this includes the answers for 
other TXT records published for that domain) and offer the _spf example of how 
to work around packet size limitations.

Scott K

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 13 Mar 2012 20:41:51 +0000

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of S Moonesamy
> Sent: Tuesday, March 13, 2012 1:30 AM
> To: [email-masked]
> Subject: Re: [spfbis] #12: RFC 4408 Section 9 - Forwarding and helo identities
> 
> >  The advice given in the non-normative Section 9 is misleading.  On
> > the  one hand, it provides overly cumbersome techniques based on
> > macros  that are not universally considered safe.  On the other hand,
> > it fails  to state the obvious point that a return-path is only needed
> > if  someone is going to care about delivery failures.

If we have something to say about the safety of use of macros, doing so in Section 8 or adding a section above 9.1 would be the place to do it.  But I don't think I agree that what's in 9.1 is overly cumbersome.  It may not be syntactic beauty, but if you're willing to accept what's defined in Section 8, then Section 9.1 seems okay to me.

> > [RFC5321] --not one of the best written sections of that paper.

"it" being the RFC5321.MailFrom?

I don't agree that this document is the right place to make a correction to RFC5321 Section 3.9.1 outright.  If we want to say something like "SPF-sensitive implementations might want to ...", then I might go for the idea of saying "Updates RFC5321", but I don't think we'll get away with (and don't agree with) a blanket statement.

> >  Forwarding Ãƒ  la MAIL FROM:<> should be mentioned as the default,

So the empty string SHOULD be the envelope sender used for forwarded mail?  That effectively disables SPF, making forwarding into an attack vector.

I can't envision the infrastructure you're talking about here.  Whatever it might be, I'm confident guessing ahead of time that it would exceed the constraints of the charter, but I'd still like to hear what you have in mind.

SPF stands on its own without SRS, but it has seen only very little adoption as far as I know, so I think at best we could mention SRS in an appendix.

> > redirect= for  designing sound and maintainable SPF compliance.

I don't think we can encourage use of _spf in this document as it's a substantial departure from the deployed base, and we're thus charter-constrained against doing so.  Informative text about encouraging use of redirect= is fine if we can come to some consensus on it, especially if it's based on data that shows it's wise.

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 13 Mar 2012 20:55:14 +0000

Scott did all the work, but...

> From: [email-masked] [mailto:[email-masked]] On Behalf Of Scott Kitterman
> Sent: Tuesday, March 13, 2012 11:23 AM
> To: [email-masked]

An informative reference to that draft might be a good idea, though I don't have strong feelings either way.

> References to at least draft-ietf-marf-spf-reporting should be added to
> 9.1, but the current text about a special SPF record should remain.

+1

> It does not appear that any of the comments address 9.2.  I think it
> would be useful (and a good preamble to the forwarding discussion) to
> update this text to make use of the email architecture terminology that
> didn't exist when RFC
> 4408 was written.  The references should be updated as well.  I don't
> think any functional changes to 9.2 are needed.

+1

> relevant (it even mentions SPF by name).  I don't think 4408bis should
> update 5321.  I think it would make consensus substantially harder to
> achieve and there is no technical need for it.

+1 (I need a +1 button)

> It would be useful to explicitly say in this section that THE place to
> do SPF checks is at the transition point between the sending ADMD and
> the receiving ADMD.  When a message is relayed within the recieving
> ADMD, SPF checks, per se are not appropriate, all that can be done is
> to use the results of checks done on the ADMD border if they can be
> trasmitted in a reasonably trustworthy manner.

We can make liberal use of or reference to Appendix C of RFC5451 to back this up.

> redirect/include and _spf are nice tools for domains that need them.
> They are not, however, a general solution or requirement.  I think 5.2
> already give adequate coverage to the use of include:.  I think it
> would be useful (as part of the processing limits discussion) to
> include an indication that SPF records SHOULD NOT require TCP fallback
> to retrieve (and this includes the answers for other TXT records
> published for that domain) and offer the _spf example of how to work
> around packet size limitations.

Sounds reasonable.

-MSK

---

From: S Moonesamy <[email-masked]>
Date: Tue, 13 Mar 2012 17:46:37 -0700

Hello,

There has been comments about changes which should go in 4408bis.  It 
is up to the WG to reach agreement on whether it wants to document 
problems in the "conclusion" draft and address them in the 4408bis 
draft.  There isn't any mention of SRS in RFC 4408.  Currently, it is 
not possible to tell whether SRS will go into 4408bis or not.  I'll 
highlight a paragraph from the SPFBIS charter:

   "Changes to the SPF specification will be limited to the correction
    of errors, removal of unused features, addition of any enhancements
    that have already gained widespread support, and addition of
    clarifying language."

A narrow interpretation of that might come out as: if there isn't an 
error, there isn't any ground for making a correction.  If the 
someone argues for adding an enhancement, the person has to show that 
the enhancement has _already_ gained widespread support.

There is an IETF meeting coming up in a few weeks.  Is "forwarding" a 
problem which should be on the agenda?

Regards,
S. Moonesamy
SPFBIS WG co-chair

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 13 Mar 2012 21:17:23 -0400

> problem which should be on the agenda?

Forwarding (I think relaying is a better term as it's used in 5321) is not a 
problem, per se.  It is a case that SPF that it takes some care in an SPF 
deployment at accomodate.  Using SRS is one of many ways to do so, but since 
SRS has never been standardized, I don't think 4408bis should refer to it 
specifically.  It is a specific method of rewriting "Mail From" in order to 
support downstream SPF queries.  RFC 4408 9.3.2.1 describes this in general 
terms.

It is useful to have that in there, but I don't think defining a specific method 
is needed.  There is no interoperability benefit for doing so.

I'm quite confused about this mail as it seems to be concerned about reusing 
existing concepts from RFC 4408.  SRS is a shorthand (as one specific method) 
for doing what's described in 9.3.2.1.  There's no news here as far as I can 
tell.  Am I missing something?

Scott K

---

From: S Moonesamy <[email-masked]>
Date: Tue, 13 Mar 2012 18:41:13 -0700

Hi Scott,
>tell.  Am I missing something?

The message is about whether there is an issue which requires face to 
face discussion.  If so, time would have to be allocated to it in the 
agenda for the forthcoming WG session.

Regards,
S. Moonesamy

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 13 Mar 2012 21:58:13 -0400

On Tuesday, March 13, 2012 06:41:13 PM S Moonesamy wrote:
> >I'm quite confused about this mail as it seems to be concerned about
> >reusing existing concepts from RFC 4408.  SRS is a shorthand (as one
> >specific method) for doing what's described in 9.3.2.1.  There's no news
> >here as far as I can tell.  Am I missing something?
> 

What issue?

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 13 Mar 2012 22:41:13 -0400

> > agenda for the forthcoming WG session.

OK.  I wet back and read the original message again.  I gather this is your 

> problem which should be on the agenda?

I don't see the benefit.  RFC 4408 covers the range of possibilities reasonably 
well in terms of how senders and receivers can accommodate this attribute of 
the email architecture.  All we need to do is update the current text a bit to 
take into account the email arch document and it's terminology.  I don't think 
there's a real issue that needs to take up meeting time.

Scott K

---

From: "John Levine" <[email-masked]>
Date: 14 Mar 2012 03:58:45 -0000

>problem which should be on the agenda?

NO, Capital N, Capital O.

SPF is path authentication, it's well understood that it's not
designed to deal with forwarding.  People have been trying to invent
ways to shoehorn forwarded mail into SPF since long before RFC 4408,
and I think it's safe to say that nothing new in that regard has
happened since 4408.

There may still be some people who think that all SMTP mail should
change to match what SPF does.  (Not Scott, certainly.)  They are
wrong.

R's,
John

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 14 Mar 2012 11:17:09 +0100

> "it" being the RFC5321.MailFrom?

Yep.

> I don't agree that this document is the right place to make a
> correction to RFC5321 Section 3.9.1 outright.  If we want to say
> something like "SPF-sensitive implementations might want to ...",
> then I might go for the idea of saying "Updates RFC5321"

There are implementations that violate 3.9.1 by not leaving "the rest
of the envelope" (that is, RFC5321.MailFrom) unchanged.  As that
change was triggered by SPF, it should be up to SPF to spell the text
that 5321bis will hopefully adopt, whenever "the celestial omens align
correctly".

Scott correctly notes that RFC 5321 evades defining methods to verify
MailFrom, when it mentions SPF and relaying.  Forwarding (in the 821
sense of relaying while changing the envelope recipient) is not done
well in RFC 5321 because some sentences there are ambiguous (see also
http://trac.tools.ietf.org/wg/yam/trac/ticket/9).  In particular, the
term "alias" doesn't look like a synonym of intra-ADMD redirection/
expansion.

>>> Forwarding à la MAIL FROM:<> should be mentioned as the default,
>>> easiest way to re-inject messages.
> 
> So the empty string SHOULD be the envelope sender used for 
> forwarded mail?  That effectively disables SPF, making forwarding 
> into an attack vector.

SPF verification is limited to the helo identity in such cases.  In
the worst case that should deliver a neutral result, whereas keeping
RFC5321.MailFrom unchanged would deserve a fail.  The impasse whereby
most domains cannot afford -all is rooted in exactly this point.

> I can't envision the infrastructure you're talking about here.
> Whatever it might be, I'm confident guessing ahead of time that it
> would exceed the constraints of the charter, but I'd still like to
> hear what you have in mind.

I don't know how many MTAs changed their default forwarding behavior
in violation of Section 3.9.1 of RFC5321 in order to support SPF.  Is
that a widespread support?  I think it is an error that Section 3.9
does not circumscribe the use of alias expansion rules.  Should spfbis
correct that, or at least add clarifying language?

Forwarding has other shortcomings that ought to be fixed, but they are
not related to SPF.  See http://fixforwarding.org/.

> I don't think we can encourage use of _spf in this document as it's
> a substantial departure from the deployed base, and we're thus
> charter-constrained against doing so.

I though _spf was quite popular (_spf.microsoft.com, _spf.google.com,
_spf.yahoo.com, ...) but I'm willing to correct myself when I'm wrong.

---

From: Scott Kitterman <[email-masked]>
Date: Wed, 14 Mar 2012 07:22:01 -0400

> term "alias" doesn't look like a synonym of intra-ADMD redirection/

What is being discussed here isn't a (5321) 3.9.1 alias expansion, it's a 
3.6.1 relay SMTP server.  As such, your notion that Mail From must be changed 
is just incorrect.  5321 is silent on the issue and this isn't the place to fix 
it if it needs fixing.

> correct that, or at least add clarifying language?

No.  Section 9 of 4408 already beats the topic to death.  

> _spf.yahoo.com, ...) but I'm willing to correct myself when I'm wrong.

It is common use, but it's an arbitrary convention, not a feature of the 
protocol.  Such addresses are only used in SPF processing if they are pointed 
to by a redirect or include from elsewhere.  If you want to encourage this as 
a method to contrain DNS reply size in the initial query, I think it's fine, 
although using _bob.example.com would work just the same.  

BTW, not all those actually exist.

Scott K

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 14 Mar 2012 17:41:38 +0000

> Sent: Tuesday, March 13, 2012 5:47 PM
> To: [email-masked]
> draft.

I would say that the conclusion document is not a place to document problems except those that are germane to the resolution of the experiment.

> There isn't any mention of SRS in RFC 4408.  Currently, it is

I don't advocate adding SRS as any normative part of SPFbis.  I would, however, suggest in some informative section of the document that the WG consider including a reference to it as we describe the known limitations of SPF, assuming there's a stable definition for SRS somewhere out there we could use.

I also think SPF stands just fine on its own without SRS or a similar mechanism, so I'd be just fine leaving it out altogether.

I may need coffee to jar my memory, but I don't think I've seen anything in our deliberations so far that warrants more discussion of the forwarding problem than we already have in RFC4408.  We should definitely review it again though as we move forward.

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 14 Mar 2012 17:58:45 +0000

> From: [email-masked] [mailto:[email-masked]] On Behalf Of Alessandro Vesely
> Sent: Wednesday, March 14, 2012 3:17 AM
> To: [email-masked]
> > something like "SPF-sensitive implementations might want to ...", then
> > I might go for the idea of saying "Updates RFC5321"
> 
> of the envelope" (that is, RFC5321.MailFrom) unchanged.  As that change
> was triggered by SPF, it should be up to SPF to spell the text that
> 5321bis will hopefully adopt, whenever "the celestial omens align
> correctly".

Alluding to John Levine's post, I don't think SPF is sufficiently heavyweight to say that RFC5321 3.9.1 got it wrong.  SPF is effectively a layer on top of SMTP.  Environments that don't use SPF might rely on the opposite behavior.

> term "alias" doesn't look like a synonym of intra-ADMD redirection/

I disagree, the definition there reflects exactly my understanding of how aliases work in that or any context.  See also RFC5598, Section 5.1.

> > So the empty string SHOULD be the envelope sender used for forwarded
> > mail?  That effectively disables SPF, making forwarding into an attack
> > vector.
> 
> SPF verification is limited to the helo identity in such cases.  In the
> worst case that should deliver a neutral result, whereas keeping

There are some SPF implementations that don't check HELO, and they are compliant with RFC4408.

> I don't know how many MTAs changed their default forwarding behavior in
> violation of Section 3.9.1 of RFC5321 in order to support SPF.  Is that
> a widespread support?  I think it is an error that Section 3.9 does not
> circumscribe the use of alias expansion rules.  Should spfbis correct
> that, or at least add clarifying language?

Alias expansion doesn't change the envelope sender.  When you say "forwarding", that's what I think you mean.  And I can't see that SPF layered on top of SMTP and creating that need automatically means SMTP got it wrong and there's something we need to fix.

-MSK

---

From: Dotzero <[email-masked]>
Date: Wed, 14 Mar 2012 14:34:13 -0400

+1

---

From: "John Levine" <[email-masked]>
Date: 14 Mar 2012 19:28:02 -0000

>I don't advocate adding SRS as any normative part of SPFbis.  I would, however,
>suggest in some informative section of the document that the WG consider
>including a reference to it as we describe the known limitations of SPF,
>assuming there's a stable definition for SRS somewhere out there we could use.

There isn't really anything to standardize in SRS.  It's basically a
single user recipient of VERP, encoding stuff in the bounce address to
provide hints about what bounced.  Like VERP and BATV, the limit on
mailbox length gives it has fairly severe interop problems if you want
it work reliably as opposed to being a heuristic.  The only way I can
think of to avoid that problem is for forwarding systems to keep a
database of bounce addresses from forwarded mail and use a database
key rather than the address, but I don't know anyone who does that.
(Nor would I suggest anyone use it just to work around SPF.)

>I may need coffee to jar my memory, but I don't think I've seen anything in our
>deliberations so far that warrants more discussion of the forwarding problem
>than we already have in RFC4408. 

Yes.

> We should definitely review it again though as we move forward.

No.  It's a fact, not a problem.

R's,
John

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 14 Mar 2012 19:41:47 +0000

> From: [email-masked] [mailto:[email-masked]] On Behalf Of John Levine
> Sent: Wednesday, March 14, 2012 12:28 PM
> To: [email-masked]
> Cc: Murray S. Kucherawy

I mean we should review the text we're providing as information to make sure it's right, not we should try to fix forwarding vis a vis SPF.

-MSK

---

From: Stuart D Gathman <[email-masked]>
Date: Wed, 14 Mar 2012 17:33:29 -0400

Long ago, Nostradamus foresaw that on 03/14/2012 01:41 PM, Murray S. 
Kucherawy would write:
> I don't advocate adding SRS as any normative part of SPFbis. I would, 
> however, suggest in some informative section of the document that the 
> WG consider including a reference to it as we describe the known 
> limitations of SPF, assuming there's a stable definition for SRS 
> somewhere out there we could use. I also think SPF stands just fine on 
> its own without SRS or a similar mechanism, so I'd be just fine 
> leaving it out altogether. I may need coffee to jar my memory, but I 
> don't think I've seen anything in our deliberations so far that 
> warrants more discussion of the forwarding problem than we already 
> have in RFC4408. We should definitely review it again though as we 
> move forward.

The RFC should address one of the most common receiver mistakes: 
http://www.openspf.org/FAQ/Common_receiver_mistakes

          Checking SPF On Forwarded Mail

    Mail forwarding is set up by the receiver and so for forwarded mail,
    the border mail server (at which SPF should be checked) is the
    forwarder's mail server. If you check SPF on your mail server it is
    coming from your forwarder and not from a mail server authorized by
    the sending domain. Technically this is similar to checking SPF
    against mail relayed from your secondary MX like discussed in the
    previous item. Authorized forwarders should be whitelisted against
    SPF checks to avoid this problem.

In particular, if a receiver is unable to treat an authorized forwarder 
as an internal MX, and the forwarder does not rewrite MAIL FROM (via SRS 
or similar scheme), then you MUST NOT treat any alleged SPF result as a 
real SPF result.  It is only a guess, and not a very good one at that.  
Nevertheless, way too many shops do this.

---

From: Scott Kitterman <[email-masked]>
Date: Wed, 14 Mar 2012 17:38:13 -0400

> Nevertheless, way too many shops do this.

I think that RFC 4408 9.3.3.1 describes this, but it could be better written.

Scott K

---

From: "Stuart D. Gathman" <[email-masked]>
Date: Wed, 14 Mar 2012 19:41:20 -0400 (EDT)

> I think that RFC 4408 9.3.3.1 describes this, but it could be better written.

It describes several options, but the MUST NOT for the receiver end is missing.
There is a strong implication that if you can't handle your own forwarding,
then SPF results are invalid.  But that doesn't seem to be enough for
many email operators.

I also don't like the suggestion that senders should try to work around
forwarding controlled by the receiver.  You might do that to get mail through
to a particular recipient in a pinch, but the recipient is wrong, not
the sender.

-- 
 	      Stuart D. Gathman <[email-masked]>
"Confutatis maledictis, flamis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

---

From: Philip Gladstone <[email-masked]>
Date: Thu, 15 Mar 2012 09:34:04 -0400

> to a particular recipient in a pinch, but the recipient is wrong, not

The trouble (in my view) is that users set up forwarding at alumni 
associations, professional associations etc. They do not control 
anything about these systems except the forwarding tables.

The final destination of the email is some other mail server (gmail, 
hotmail, aol etc). The users do not control anything about those systems 
either except for their own email account.

What *exactly* is being suggested? That the recipient go to gmail and 
say 'please whitelist alumni.someuniversity.edu.somecountry for spf'? 
This doesn't seem workable to me. It also doesn't seem workable to have 
per-user SPF whitelisting rules on big mail receivers -- it would 
certainly make handling multi-recipient messages rather complex unless 
they all had the same set of SPF preferences.

This problem is, after all, what SRS was designed to solve.

Philip

-- 
Philip Gladstone
Distinguished Engineer
Product Development
[email-masked]
Phone: +1 978-ZEN-TOAD (+1 978 936 8623)
Google: +1 978 800 1010
Ham radio: N1DQ

---

From: "John Levine" <[email-masked]>
Date: 15 Mar 2012 17:57:53 -0000

>then SPF results are invalid.  But that doesn't seem to be enough for

I agree with Scott, the writing could be clearer, but the general discussion
in 9.3 is fine.

Unless your name is Paypal, a -ALL in your SPF record will screw up a
thousand innocent mail transactions for every random spoof it catches.
Mail forwarding has been around almost as long as mail has, it's not
broken, it's not going to change to deal with the well-known
limitations of SPF, and that's not a problem.

R's,
John

---

From: Commerco WebMaster <[email-masked]>
Date: Thu, 15 Mar 2012 13:08:29 -0600

On 3/15/2012 11:57 AM, John Levine wrote:
> _______________________________________________

We are hardly PayPal, yet we have had no real issues with SPF and 
forwarding (using -all) over the many years it has been in use here.

As to RFC 4408 9.3 it seems adequate.

Alan M.

---

From: Philip Gladstone <[email-masked]>
Date: Thu, 15 Mar 2012 17:47:14 -0400

> forwarding (using -all) over the many years it has been in use here.
This surprises me. Over the last week or so, these are my stats on which 
sites forward email from me without doing SRS and which would get 
dropped if I used a -all. This is just my home email server (and yes, I 
do run a reasonably active mailing list with maybe a thousand 
subscribers). I will agree that a lot of people use -all:

         +all   2145
         -all  54271
         ?all  54532
         ~all 166763

These are the counts from my record database.

Philip

The first column is the number of messages, then the IP address and then 
the reverse lookup if one exists.

     5:              10.51.0.5
     1:        128.103.229.181  postmail-s03.cadm.harvard.edu
     5:           128.186.98.3  mail.met.fsu.edu
     1:          128.59.28.172  sapodilla.cc.columbia.edu
     2:           129.170.6.23  mailhub23.Dartmouth.EDU
     3:         131.252.111.42  tomlinson.oit.pdx.edu
     5:         131.252.111.43  sluizer.oit.pdx.edu
     7:         131.252.111.44  postel.oit.pdx.edu
     1:           173.203.2.22  gate.forward.smtp.ord1a.emailsrvr.com
    12:          173.225.51.56  ganymede.interbridge.net
     2:        173.247.254.120  biz104.inmotionhosting.com
     2:             18.7.68.21  ALUM-MAILSEC-RELAY-1.MIT.EDU
     3:             18.7.68.28  ALUM-MAILSEC-RELAY-8.MIT.EDU
     2:             18.7.68.30  ALUM-MAILSEC-RELAY-10.MIT.EDU
     1:         180.151.57.187
    16:         198.82.165.108  rollins.bev.net
     2:           199.81.10.48
     1:           199.81.10.49  prh00393.prod.fedex.com
     1:           199.81.10.50  prh00394.prod.fedex.com
     1:         207.115.20.131  flpd121.sbcis.sbc.com
     7:          207.55.16.112  zmail-mta02.peak.org
     5:            209.68.2.48  shema.pair.com
     2:           216.139.86.1  mail01.mx.maxen.net
     4:           216.139.86.2  mail02.mx.maxen.net
     4:           216.139.86.3  mail03.mx.maxen.net
     3:           216.139.86.4  mail04.mx.maxen.net
     4:           216.40.42.17  forward.a.hostedemail.com
    14:          38.126.103.58  callisto.interbridge.net
     7:            64.14.73.19  s459.sureserver.com
     3:            64.18.0.119  exprod5mx233.postini.com
     2:            64.18.0.169  exprod5mx249.postini.com
     2:             64.18.0.35  exprod5mx189.postini.com
     1:             64.18.0.43  exprod5mx197.postini.com
     2:             64.18.0.54  exprod5mx258.postini.com
     6:             64.18.0.73  exprod5mx214.postini.com
     5:          64.40.145.244  mail2.websitesource.net
    18:           64.55.116.20  vhawiniis-002.wbi.mycspot.net
     1:          66.175.56.152  mail122c38.carrierzone.com
     2:          66.175.56.163  mail133c38.carrierzone.com
     5:          66.175.56.189  mail115c38.carrierzone.com
     1:          66.231.182.48  mx1-relay1.dnsmadeeasy.com
     3:          66.94.237.209  nm8.access.bullet.mail.mud.yahoo.com
    35:           74.208.4.194  mout.perfora.net
     5:           74.208.4.195  mout.perfora.net
     1:           76.96.27.228  qmta15.emeryville.ca.mail.comcast.net
     2:           76.96.59.228  qmta15.westchester.pa.mail.comcast.net
     1:            76.96.62.24  qmta02.westchester.pa.mail.comcast.net
     1:            76.96.62.56  qmta06.westchester.pa.mail.comcast.net
     3:           98.138.90.59  nm27-vm1.bullet.mail.ne1.yahoo.com
     3:         98.139.212.182  nm23.bullet.mail.bf1.yahoo.com
     3:         98.139.212.189  nm30.bullet.mail.bf1.yahoo.com
     6:         98.139.212.252  nm28-vm1.bullet.mail.bf1.yahoo.com
     3:         98.139.213.137  nm21-vm0.bullet.mail.bf1.yahoo.com
     6:         98.139.213.151  nm7-vm0.bullet.mail.bf1.yahoo.com
     2:          98.139.44.192  nm29-vm0.access.bullet.mail.sp2.yahoo.com
     1:          98.139.52.204  nm7.bullet.mail.ac4.yahoo.com
     6:          98.139.52.223  nm26.bullet.mail.ac4.yahoo.com
     1:          98.139.52.235  nm14-vm1.bullet.mail.ac4.yahoo.com
     3:          98.139.52.236  nm15-vm0.bullet.mail.ac4.yahoo.com
     6:          98.139.53.211  nm18-vm1.bullet.mail.ac4.yahoo.com
     6:          98.139.91.198  nm10-vm0.bullet.mail.sp2.yahoo.com
     3:          98.139.91.222  nm22-vm0.bullet.mail.sp2.yahoo.com
     1:          98.139.91.230  nm26-vm0.bullet.mail.sp2.yahoo.com
     6:          98.139.91.232  nm27-vm0.bullet.mail.sp2.yahoo.com
     6:          98.139.91.249  nm2-vm1.bullet.mail.sp2.yahoo.com

---

From: Stuart Gathman <[email-masked]>
Date: Thu, 15 Mar 2012 20:40:44 -0400

Long ago, Nostradamus foresaw that on 03/15/2012 05:47 PM, Philip
Gladstone would write:
> This surprises me. Over the last week or so, these are my stats on
> which sites forward email from me without doing SRS and which would
> get dropped if I used a -all. This is just my home email server (and
> yes, I do run a reasonably active mailing list with maybe a thousand
This is exactly what needs to be clearer.  You as the sender have no
control.  It is the *recipient* that sets up the forward, and is
responsible for handling it properly IF they want to check SPF.  That
does imply that gmail.com, aol.com, etc can't check SPF (except perhaps
on a per user basis as requested) because of exactly what you described:
people set up forwards from random locations to their gmail account and
expect it to work.  Their expectation is reasonable if they don't expect
SPF to screen forgeries.  But gmail MUST NOT consider SPF results
official (and probably shouldn't reject on an alleged SPF FAIL), unless
their terms of use forbid forwarding to a gmail account from random
locations, or they have some magic way of handling such forwards.

I suspect gmail actually has an internal list of trusted forwarders -
but I can't really tell.

If the random forwarder does SRS, then it all just works.
If the random forwarder does 551 with a forwarding address, it all just
works (appropriate for alumni, etc)

For recipients with a small private or company wide email domain,
requiring whitelisting of non-SRS forwarders is good policy.

Big email providers like aol and gmail generally don't need to reject on
SPF.  They have effective IP blacklists based on feedback from millions
of users.

Small email domains *do* need to reject on SPF, and it is a simple
matter of policy to either forbid random non-SRS forwards, or provide a
procedure to tell the company MTAs about the new semi-trusted MX.

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 15 Mar 2012 23:54:10 -0400

> I suspect gmail actually has an internal list of trusted forwarders -

I'm not sure how they manage it, but if you publish a DMARC record (see 
dmarc.org) the feedback you get from Google has forwarded messages marked as 
such.  They definitely have something.

Scott K

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 16 Mar 2012 12:32:25 +0100

> such.  They definitely have something.

Yes, a number of failed SPF checks for forwarded mail!

>> people set up forwards from random locations to their gmail
>> account and expect it to work.  Their expectation is reasonable
>> if they don't expect SPF to screen forgeries.  But gmail MUST NOT
>> consider SPF results official (and probably shouldn't reject on
>> an alleged SPF FAIL), unless their terms of use forbid forwarding
>> to a gmail account from random locations, or they have some magic
>> way of handling such forwards.

It seems that one of the following must be true:

1: gmail, or any other receiver, don't consider SPF results official,

2: senders never publish -all, or

3: users don't expect forwarding to work.

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 16 Mar 2012 08:34:26 -0400

On Friday, March 16, 2012 12:32:25 PM Alessandro Vesely wrote:
> > dmarc.org) the feedback you get from Google has forwarded messages
> > marked as such.  They definitely have something.
> 

4.  Gmail is a very large, sophisticated mail operator run by a company with 
lots of expertise in assessing and correlating large volumes of data and uses 
both the data and the capability to determine likely forwarding paths and 
doesn't reject forwarded mail (independent of  SPF) in order to avoid causing 
backscatter.

Scott K

---

From: Philip Gladstone <[email-masked]>
Date: Fri, 16 Mar 2012 10:03:25 -0400

> 3: users don't expect forwarding to work.

These options are *very* worrying for the future of SPF.

If the big players ignore SPF, then this reduces the effective adoption 
of SPF dramatically. Why publish an SPF record if the results of 
evaluation are ignored?

People do publish -all

Users do expect forwarding to work.

[In my case, I have an SPF record that rates limits the forwarding rate 
by any particular MTA]

Philip

---

From: John Leslie <[email-masked]>
Date: Fri, 16 Mar 2012 10:29:02 -0400

> of SPF dramatically. Why publish an SPF record if the results of 

   Philip is correct to talk of "effective adoption" in the eyes of the
beholder; but that's certainly not the whole picture.

   Some domains avoid publishing -all, specifically because of uncertainty
over possible forwarding.

   Some MTA managers _check_ SPF, but don't respect -all -- instead
recording the SPF results in a header.

   These and other workarounds are perfectly reasonable, so far as they
go...

> Users do expect forwarding to work.

   Let's be sure we're clear here: some sender domains do publish -all
when they know their email will never _originate_ from a different MTA;
some receivers enable forwarding and do not want emails discarded
because of an SPF -all.

   RFC 4408 has nothing to _resolve_ this tussle.

   I doubt we can _resolve_ it in 4408bis either; but I hope we can
explain it a bit better.

> [In my case, I have an SPF record that rates limits the forwarding rate 

   Sounds like one possible approach (though I don't fully understand it).
I can think of a half-dozen different approaches...

   I suggest we collect different approaches, and try to craft an
Appendix discussing their strengths and weaknesses.

--
John Leslie <[email-masked]>

---

From: Dotzero <[email-masked]>
Date: Fri, 16 Mar 2012 10:39:36 -0400

On Fri, Mar 16, 2012 at 10:03 AM, Philip Gladstone <[email-masked]> wrote:
> If the big players ignore SPF, then this reduces the effective adoption of
> SPF dramatically. Why publish an SPF record if the results of evaluation are
> ignored?
>
> People do publish -all
>
> [In my case, I have an SPF record that rates limits the forwarding rate by
> any particular MTA]
>
> Philip
>
>
>

Philip,

I'm not sure I understand your complaint. If I understand you
correctly, your complaint is that Gmail is not compiant with SPF
because it accepts mail coming through a maillist (which fails SPF
validation) when the original domain publishes a "-all".

I suggest you reread section 2.5.4 of RFC4408 more carefully. There is
no requirement to reject... and Gmail DOES mark the mail (adding a
header) as having failed SPF. As far as I can tell they are doing what
is required.

Mike

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 16 Mar 2012 12:17:22 -0400

> Users do expect forwarding to work.

I've published a -all SPF record since (IIRC) sometime in 2005.  There is very 
little forwarding in the set of people I send to.  In a few cases, I've gotten 
bounce messages due to post-forwarding rejections.  So far, they've always 
contained the target address of the forward and I just resent the message.

Forwarding does happen, but I think that it's rare enough that mail losses due 
to a combination of forwarding and SPF based rejection is a noise level issue 
in the scheme of the overall unreliability of email.  I'm, personally, a lot 
more worried about providers that break the reject or deliver paradigm and 
just drop email into a black hole for reasons that no doubt make sense to 
them.

Scott K

---

From: Commerco WebMaster <[email-masked]>
Date: Fri, 16 Mar 2012 10:43:15 -0600

> [In my case, I have an SPF record that rates limits the forwarding rate

In the early mid 2000s, when SPF was gaining momentum, its immediate 
appeal to me was that there was a defacto standard vehicle as a place to 
point others to when one of the company domains were abused by an email 
spammer.

Back then, very few systems supported SPF in the TXT RR (never mind the 
type 99 SPF RR, which was not even on the table at the time).  The issue 
was to have an authoritative place someone could see clearly where the 
domains we hold and control were allowed to send mail from.  I have 
always implemented the -all parameter and it has always worked for us.

So retrospect, it is really just a matter of perspective and 
expectations as to what SPF historically was really all about.  In our 
case that being, domain name reputation protection.

Alan M.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 16 Mar 2012 16:47:34 +0000

> Sent: Friday, March 16, 2012 5:34 AM
> To: [email-masked]

It could be that they do, but only use it as input to a larger evaluation algorithm.

> 4.  Gmail is a very large, sophisticated mail operator run by a company
> with lots of expertise in assessing and correlating large volumes of
> data and uses both the data and the capability to determine likely
> forwarding paths and doesn't reject forwarded mail (independent of
> SPF) in order to avoid causing backscatter.

That's kind of the same thing, I think.

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 16 Mar 2012 16:48:58 +0000

> From: [email-masked] [mailto:[email-masked]] On Behalf Of John Leslie
> Sent: Friday, March 16, 2012 7:29 AM
> To: Philip Gladstone
> Cc: [email-masked]

+1.

-MSK

---

From: John Leslie <[email-masked]>
Date: Fri, 16 Mar 2012 12:49:36 -0400

Scott Kitterman <[email-masked]> wrote:
> 
> Forwarding does happen, but I think that it's rare enough that mail
> losses due to a combination of forwarding and SPF based rejection is
> a noise level issue in the scheme of the overall unreliability of email.

   Is that damning with faint praise??

   Seriously, I don't know how to put bounds on such losses. It may be
"noise-level". Even at "noise-level" I think it deserves some discussion
in 4408bis (though I doubt we'll come up with any mandatory protocol
changes).

> I'm, personally, a lot more worried about providers that break the
> reject or deliver paradigm and just drop email into a black hole for
> reasons that no doubt make sense to them.

   It is rather hard to delimit such reasons...

   Nonetheless, backscatter is a real problem; and I have seen threats
of blacklisting due to backscatter -- that's arguably a sufficient
reason. :^(

   I'd like to believe that SPF can help the backscatter problem. Some
discussion of this in 4408bis seems like a good idea...

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 16 Mar 2012 13:23:52 -0400

> in 4408bis (though I doubt we'll come up with any mandatory protocol

I agree it's an issue, it just doesn't seem to be a major one.  It's covered 
in 4408 section 9 which, I think, has the right information, it just needs to 
be written a bit better and updated.

> discussion of this in 4408bis seems like a good idea...

The term backscatter does not appear in 4408.  It should probably come up in 
4408bis.

Scott K

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 16 Mar 2012 19:09:11 +0100

On 16/Mar/12 18:23, Scott Kitterman wrote:
> I agree it's an issue, it just doesn't seem to be a major one.

It implies that either SPF stays at limited use, or that forwarding
gets deprecated, so that their combination can be ignored and will
eventually vanish.

> It's covered in 4408 section 9 which, I think, has the right
> information, it just needs to be written a bit better and updated.

Section 9 explains it very well, but doesn't solve it.  Are we
standardizing SPF in the hope that people will spontaneously fix
forwarding, and at that point, maybe, dare say that they fixed it
because it was broken?

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 16 Mar 2012 14:25:38 -0400

> gets deprecated, so that their combination can be ignored and will

Forwarding is already a small niche.  From an SPF protocol design perspective 
I think it can be ignored.  As was said already somewhere in this thread, it's 
an attribute of SMTP that people who deploy SPF (both as record publishers and 
as checkers) will have to consider how to deal with.  Section 9 gives some 
advice on this.  We should improve that advice, but that's really all was can 
do.

> forwarding, and at that point, maybe, dare say that they fixed it

It is quite possible to check SPF at SMTP time and reject on SPF Fail without 
rejecting mail due to forwarding.  I've written software that gives 
administrators the tools to do it (tools are the easy part - knowing what data 
to put in the tools is harder), but mostly it doesn't scale for large 
operators.

At the other end of the scale, you have massive operators like Gmail who can 
use SPF and other inputs to figure out who the forwarders are and not reject 
based on SPF for such hosts.

None of it's easy, but it's fundamentally doable if the receiver cares to.  No 
matter what we write in an RFC, we can't make them care.

There are other possibilities that are clearly out of scope for SPFbis.  DMARC 
proposed to use SPF and DKIM in combination.  Their protocol 'handles' 
forwarding.

Perhaps, someday, someone will understand the situation well enough to write 
an applicability statement that explains all about how to integrate SMTP 
forwarding and SPF, but I don't think there's anything much beyone a better 
written section 9 we can do in 4408bus.

Scott K

---

From: "John Levine" <[email-masked]>
Date: 16 Mar 2012 22:12:27 -0000

>Forwarding is already a small niche.

Forwarding is a small niche, but people collecting and sending mail at sites separate
from their home domain (typically doing so at Gmail or Yahoo) is a growing niche.
It has all the same issues as forwarding, it's not going away, SPF does what it does.

R's,
John

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 23 Mar 2012 10:57:23 +0100

On 16/Mar/12 19:25, Scott Kitterman wrote:
> It is quite possible to check SPF at SMTP time and reject on SPF
> Fail without rejecting mail due to forwarding.  I've written
> software that gives administrators the tools to do it (tools are
> the easy part - knowing what data to put in the tools is harder),
> but mostly it doesn't scale for large operators.
> 
> At the other end of the scale, you have massive operators like
> Gmail who can use SPF and other inputs to figure out who the
> forwarders are and not reject based on SPF for such hosts.
> 
> None of it's easy, but it's fundamentally doable if the receiver
> cares to.  No matter what we write in an RFC, we can't make them
> care.

RFC 4408 repeatedly mentions the possibility to reject messages that
fail SPF.  Indeed, that was the original idea of the protocol, IIRC.

John confirms that forwarding is "not going away, SPF does what it
does."  Nevertheless, I think it is a moral obligation for a standard
writer to figure out and describe how to make things work.  Operators
who know better will do their way, under their responsibility.  But it
is insane to air a possibility --reject on fail-- which we know does
not work.

> There are other possibilities that are clearly out of scope for
> SPFbis.  DMARC proposed to use SPF and DKIM in combination.  Their
> protocol 'handles' forwarding.

They seem to be asking "Did you mean 'reject' when you said '-all'?"

> Perhaps, someday, someone will understand the situation well enough
> to write an applicability statement that explains all about how to
> integrate SMTP forwarding and SPF, but I don't think there's
> anything much beyond a better written section 9 we can do in
> 4408bis.

Shall we stay experimental until then?  Or will 4408bis be such an
irresponsible specification as to result in an SDO being unable to
comply with its own standards:  I see DMARC reporting messages failing
SPF with smtp.mailfrom=ietf.org because they are forwarded to Google
without changing smtp.mailfrom.  I see bounces from IETF's role
addresses ([email-masked]) due to the same reason.  (The
postmaster said:
  Yes.  Sight.  I've run into this particular problem before, it's
  been discussed on the tools-discuss list and on the wgchairs list,
  and I've asked if there's a clearcut recommendation I can follow,
  and the response has been such that I conclude that any changes I
  can make also has equally serious (different) consequences.  It's
  apparently (not that I know enough about SPF to be able to
  adjudicate this, but that's the way I've understood the responses
  I've received) an inherent problem with SPF....
)

The possible solutions I see are as follows:

* Recklessly pretend that the problem doesn't exist,
* prohibit rejection on SPF fail,
* specify under what conditions it is safe to reject on SPF fail,
* update SMTP to make forwarding SPF-compatible, or
* deprecate SPF altogether.

---

From: S Moonesamy <[email-masked]>
Date: Fri, 23 Mar 2012 02:35:22 -0700

This is a rough summary of the discussion about Issue #12.  As usual, 
if you are misquoted or misunderstood, please let me know.

Issue #12 is based on a message posted on February 10 [1].  Murray 
Kucherawy does not agree about making a correction to Section 3.9.1 
of RFC 5332 in 4408bis [2].  He also mentioned that the empty string 
effectively disables SPF, making forwarding into an attack 
vector.  He also pointed out that SPF stands on its own without SRS 
and there is very little adoption of the later.  He mentioned that 
encouraging the use of _spf is a substantial departure from the deployed base.

Alessandro Vesely pointed out that there are implementations that 
violate 3.9.1 by not leaving "the rest of the envelope" (that is, 
RFC5321.MailFrom) unchanged.  As that change was triggered by SPF, it 
should be up to SPF to spell the text that 5321bis will hopefully 
adopt [3].  He also mentioned that Forwarding has other shortcomings 
that ought to be fixed, but they are not related to SPF.

Scott Kitterman does not think that any functional changes to 9.2 are 
needed [4] or that 5321 should be updated by 4408bis.  He also 
mentioned in another message that _spf is in common use but it is an 
arbitrary convention, not a feature of the protocol [5].

Murray Kucherawy alluded to a message from John Levine and commented 
that SPF not is sufficiently heavyweight to say that RFC5321 3.9.1 
got it wrong.  SPF is effectively a layer on top of 
SMTP.  Environments that don't use SPF might rely on the opposite behavior.

John Levine mentioned that there isn't anything to standardize in 
SRS.  It's basically a single user recipient of VERP, encoding stuff 
in the bounce address to provide hints about what bounced.  Like VERP 
and BATV, the limit on mailbox length gives it has fairly severe 
interop problems if you want it work reliably as opposed to being a 
heuristic [6].

Stuart Gathman pointed out that the RFC should address one of the 
common receiver mistakes; checking SPF on forwarded mail [7].  He 
also mentioned that RFC 4408 9.3.3.1 describes several options but 
the MUST NOT for the receiver end is missing.
There is a strong implication that if you can't handle your own forwarding,
then SPF results are invalid [8].  Philip Gladstone pointed out that 
users do not control anything about these systems except the 
forwarding tables [9].

John Levine mentioned that the general discussion in 9.3 is fine 
[10].  Commerco WebMaster  reported not having any real issues with 
SPF and forwarding (using -all) [11].  Philip Gladstone  posted some 
statistics and pointed out that sites forward email from me without 
doing SRS and which would get dropped if I used a -all [12].  John 
Leslie mentioned that
some domains avoid publishing -all, specifically because of 
uncertainty over possible forwarding [13] and that some MTA managers 
_check_ SPF, but don't respect -all -- instead recording the SPF 
results in a header.

Scott Kitterman mentioned that Forwarding is already a small niche 
and from an SPF protocol design perspective he think it can be 
ignored [14].  John Levine pointed out that people collecting and 
sending mail at sites separate
from their home domain [15].

Alessandro Vesely suggested some possible solutions [16].

Regards,
S. Moonesamy

1. http://www.ietf.org/mail-archive/web/spfbis/current/msg00374.html
2. http://www.ietf.org/mail-archive/web/spfbis/current/msg00883.html
3. http://www.ietf.org/mail-archive/web/spfbis/current/msg00893.html
4. http://www.ietf.org/mail-archive/web/spfbis/current/msg00882.html
5. http://www.ietf.org/mail-archive/web/spfbis/current/msg00895.html
6. http://www.ietf.org/mail-archive/web/spfbis/current/msg00902.html
7. http://www.ietf.org/mail-archive/web/spfbis/current/msg00904.html
8. http://www.ietf.org/mail-archive/web/spfbis/current/msg00907.html
9. http://www.ietf.org/mail-archive/web/spfbis/current/msg00910.html
10. http://www.ietf.org/mail-archive/web/spfbis/current/msg00912.html
11. http://www.ietf.org/mail-archive/web/spfbis/current/msg00913.html
12. http://www.ietf.org/mail-archive/web/spfbis/current/msg00914.html
13. http://www.ietf.org/mail-archive/web/spfbis/current/msg00922.html
14. http://www.ietf.org/mail-archive/web/spfbis/current/msg00931.html
15. http://www.ietf.org/mail-archive/web/spfbis/current/msg00933.html
16. http://www.ietf.org/mail-archive/web/spfbis/current/msg00969.html

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 23 Mar 2012 07:00:32 -0400

>fail SPF.  Indeed, that was the original idea of the protocol, IIRC.

Yes, but no where is it mandated. With one specific exception, RFC 4408 is silent on receiver policy. That is not an accident. 

>is insane to air a possibility --reject on fail-- which we know does

Sure it does.  I've published a -all SPF record since before RFC 4408 was published and I know some receivers reject on fail because every now and then I get bounces due to forwarding (in these few cases, I just resend the message).

I've published software that has some reasonably wide use that defaults to reject on fail. No one has ever suggested this default was problematic. FWIW, defer on temperror (which used to be the default) did cause problems, did generate complaints, and I did change it.

>They seem to be asking "Did you mean 'reject' when you said '-all'?"
Among other things, yes.

>* deprecate SPF altogether.

Section 9 of RFC 4408 contains a veritable compendium of things that can be done at the beginning, middle, and end of the transit of a message. I don't see a lot more to be done.

I think the real issue here is you want receiver policy specified and RFC 4408 generally avoids that. Rather than argue about forwarding and receiver policy (which is only one aspect of it), perhaps it would be more useful for you to open a new issue on that more general topic.

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 23 Mar 2012 07:11:17 -0400

>16. http://www.ietf.org/mail-archive/web/spfbis/current/msg00969.html
I also mentioned not having problems with -all and forwarding. I disagree with the characterization of what Alessandro suggested in [16] as solutions.

Scott K

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 23 Mar 2012 18:07:40 +0100

On 23/Mar/12 12:00, Scott Kitterman wrote:
> Section 9 of RFC 4408 contains a veritable compendium of things
> that can be done at the beginning, middle, and end of the transit
> of a message. I don't see a lot more to be done.

I'd leave aside possible amendments to that text, for the time being.

> I think the real issue here is you want receiver policy specified
> and RFC 4408 generally avoids that.

>From a formal POV, it does so.  However, reject-on-fail is mentioned
in prominent passages of the spec, and not by mistake.  Many people,
including myself, have been advocating an even stronger position on
receiver policy.  As a result, reject-on-fail is widespread enough to
cause problems.

Of course, we can pretend there is no problem, 1st bullet above, but
that's not a useful answer to tools.ietf.org's postmaster and many
others like him.  It's a conceptual decision that I'm after, a
strategic resolution on what we /think/ is the best use of SPF.  I
think we'll work better, as a WG, if we share the same determination.
I'm open to any logical settlement, anything such that three hosts
implementing the spirit of the spec interoperate as expected.

> Rather than argue about forwarding and receiver policy (which is
> only one aspect of it), perhaps it would be more useful for you to
> open a new issue on that more general topic.

What more general topic?  This ticket is already too wide to be
practical for wordsmithing purposes, but I think it is just enough to
cover the issue at hand.  For example, we could plan, according to the
middle bullet, to trust the helo identity for forwarded messages, if
there were more than a few domains who cared to define an SPF record
for each helo identity they use...

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 23 Mar 2012 13:20:45 -0400

> for each helo identity they use...

The more general topic is "Should 4408bis be more prescriptive about receiver 
policy?"  With the exception of MUST treat Neutral and None the same there are 
no hard receiver policy requirements 4408.  All of the others are in the vain 
of "If you ... then ...".  

Getting beyond that to try and specify receiver policy would, IMO, be a 
mistake, but I think it should be generally discussed by the group rather than 
bolted onto this issue.  If we're going to specify it, we should broadly 
specify it, not just for dealing with forwarding.  If we're not going to 
generally specify receiver policy, then (obviously) we shouldn't.

Depending on how this discussion comes out, that will affect what it is 
reasonable to try and specify for forwarding.

Scott K

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 23 Mar 2012 17:32:10 +0000

> Sent: Friday, March 23, 2012 10:08 AM
> To: [email-masked]
> From a formal POV, it does so.  However, reject-on-fail is mentioned in
> prominent passages of the spec, and not by mistake.  Many people,

I'm quite sure at least Scott will tire of me holding up DKIM as an example of a better way to do things, but I'm talking here about the document, not the protocol.  :-)

As has been said, SPF tried at least in part to present a mechanism that was "safe" for reject-on-fail.  There's also common advice that says one shouldn't do this even when asked to so because of SPF's failure modes.  Indeed, this sentiment has even spilled over into the nascent DMARC work.

DKIM, on the other hand, retreated from that position some time ago and actually says that people shouldn't reject mail on failure specifically because of legitimate failure modes.  It should simply be input into a larger evaluation mechanism.

So I obviously don't advocate for "recklessly pretend the problem doesn't exist", but I do think we can only go so far in terms of giving people rope.  Being more assertive in our description of the problem and recommendations is probably quite appropriate.

We can't prohibit rejection on SPF fail, in the same way that we can't demand people discard on ADSP failures.

The last two options (update SMTP, deprecate SPF) sound like non-starters.

> implementing the spirit of the spec interoperate as expected.

Interoperability for SPF, in my view, only goes as far as the evaluation function produces the same result for everyone given the same inputs.  The actual action taken by the receiver is not our concern, just like with DKIM, although we can provide strong advice and explain why we think it's the right advice.

-MSK

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 23 Mar 2012 13:48:04 -0400

On Friday, March 23, 2012 05:32:10 PM Murray S. Kucherawy wrote:
> > From: [email-masked] [mailto:[email-masked]] On Behalf
> > Of Alessandro Vesely Sent: Friday, March 23, 2012 10:08 AM
> > To: [email-masked]
> I'm quite sure at least Scott will tire of me holding up DKIM as an example
> of a better way to do things, but I'm talking here about the document, not
> the protocol.  :-)
> 
> As has been said, SPF tried at least in part to present a mechanism that was
> "safe" for reject-on-fail.  There's also common advice that says one
> shouldn't do this even when asked to so because of SPF's failure modes. 
> Indeed, this sentiment has even spilled over into the nascent DMARC work.
> 
> DKIM, on the other hand, retreated from that position some time ago and
> actually says that people shouldn't reject mail on failure specifically
> because of legitimate failure modes.  It should simply be input into a
> larger evaluation mechanism.

I'm happy to learn lessons for improvement from wherever they come.  I'm not 
sure this is a good parallel, however.  You are correct that DKIM avoids any 
policy proscriptions, but I don't think that's a fair comparison because the 
scope of SPF is more like DKIM + ADSP than just DKIM.  It would be very odd to 
have something with policy as it's middle name that avoided policy.

I don't think that DKIM + ADSP is very generally useful.  It is far less 
useful than SPF with -all IMO.  It may be inherent or it may be because the 
DKIM working group ignored policy considerations when DKIM was being developed 
and only tried to bolt it on at the end.  There's really no way to know for 
sure (this isn't just looking in the rear view mirror - I argued when the DKIM 
working group was formed not to do it that way, but I lost out).

I think that the answer to the question of "Is reject on fail safe" and the 
question of "Is publishing a -all record safe" is very firmly "It depends".  
Sometimes it is and sometimes it isn't (in answer to both questions), so I 
think if we try to come up with a one size fits all for 4408bis we'll get one 
size fits approximately none.

I'm quite comfortable with the idea of a range of options for what receivers 
might do (and have written code that supports pretty much all of them I can 
think of) and only providing guidance in terms of "If you ..., then you 
SHOULD/MUST ...".

Scott K

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 23 Mar 2012 18:59:27 +0100

On 23/Mar/12 18:20, Scott Kitterman wrote:
> 
> The more general topic is "Should 4408bis be more prescriptive
> about receiver policy?"  With the exception of MUST treat Neutral
> and None the same there are no hard receiver policy requirements
> 4408.  All of the others are in the vain of "If you ... then ...".

Yes.  It is weakly expressed also due to standardization reasons that
I keep forgetting.  However, phrases like "local policy can take
stronger action against such E-Mail, such as rejecting it" are pretty
understandable even without MUSTard.

> Getting beyond that to try and specify receiver policy would, IMO,
> be a mistake, but I think it should be generally discussed by the
> group rather than bolted onto this issue.  If we're going to
> specify it, we should broadly specify it, not just for dealing with
> forwarding.  If we're not going to generally specify receiver
> policy, then (obviously) we shouldn't.

OTOH, predictable behavior makes for better interoperability.

> reasonable to try and specify for forwarding.

I agree forwarding is the ugliest case to discuss, but recipient
behavior is all downhill from there.

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 23 Mar 2012 14:01:42 -0400

> behavior is all downhill from there.

I'm of the opinion that receivers will do what they want and so it's pointless 
to try and tell them what to do.  What we should focus on is how to do the 
things they might choose in a reasonably standardized way (e.g. specify what 
result codes to use).

Scott K

---

From: "John Levine" <[email-masked]>
Date: 23 Mar 2012 18:44:11 -0000

>policy?"

Since it's unenforcable, heck no.

R's,
John

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 23 Mar 2012 20:57:08 +0100

On 23/Mar/12 18:32, Murray S. Kucherawy wrote:
>> From: ietf.org On Behalf Of Alessandro Vesely
>
>> reject-on-fail is widespread enough to cause problems.
> 
> I'm quite sure at least Scott will tire of me holding up DKIM as an
> example of a better way to do things, but I'm talking here about
> the document, not the protocol.  :-)
> 
> As has been said, SPF tried at least in part to present a mechanism
> that was "safe" for reject-on-fail.  There's also common advice
> that says one shouldn't do this even when asked to so because of
> SPF's failure modes.  Indeed, this sentiment has even spilled over
> into the nascent DMARC work.
> 
> DKIM, on the other hand, retreated from that position some time ago
> and actually says that people shouldn't reject mail on failure
> specifically because of legitimate failure modes.  It should simply
> be input into a larger evaluation mechanism.

This looks logical and safe, except that that larger evaluation
mechanism currently consists mostly of black magic and undisclosed
esoteric techniques.  We cannot say to accept messages failing SPF
without indicating some more or less equivalent protection against
domain-name spoofing.

> So I obviously don't advocate for "recklessly pretend the problem
> doesn't exist", but I do think we can only go so far in terms of
> giving people rope.  Being more assertive in our description of the
> problem and recommendations is probably quite appropriate.

+1, we should agree what to recommend, then assert it strongly enough.

> We can't prohibit rejection on SPF fail, in the same way that we
> can't demand people discard on ADSP failures.

That must be because of those standardization reasons that I'm unable
to get straight.  (ADSP did it so well that people have difficulties
even understanding what "discardable" means.)

> The last two options (update SMTP, deprecate SPF) sound like

Yeah; really, none of those bullets sound attractive.  Would it be
possible to check the feasibility of updating just a little bit of
SMTP?  For example, that one MUST NOT use alias mode to external
servers unless its address passes an SPF record with its label.  I
mean as a possibility, without committing ourselves to actually do so.
(Yes, I can imagine how would, say, John Klensin react if I hounded
him for such a question, but maybe there is a better way of asking...)

Deprecating SPF, to me, is not much more or less attractive than
recklessly ignoring the problem, since hurtful and deprecated are
nearly synonyms.  I hope the WG is good enough to avoid both.

> Interoperability for SPF, in my view, only goes as far as the
> evaluation function produces the same result for everyone given the
> same inputs.  The actual action taken by the receiver is not our
> concern, just like with DKIM, although we can provide strong advice
> and explain why we think it's the right advice.

One way or the other, we must say how recipients are supposed to
behave to make the protocol work.  The clearer the better.  It has to
be our concern, although it might be impermissible for us to
explicitly mandate it.

---

From: Stuart D Gathman <[email-masked]>
Date: Fri, 23 Mar 2012 16:25:24 -0400

Long ago, Nostradamus foresaw that on 03/23/2012 05:57 AM, Alessandro 
> is insane to air a possibility --reject on fail-- which we know does 
Reject on FAIL works perfectly well - WITH the constraint that receivers 
must not treat results from (alias/non-SRS) forwarders as SPF results.  
This is ridiculously simple for a small operator (configure a small list 
of authorized forwarders), doable for a very large operator like gmail 
or aol (using SPF results over millions of messages to figure out who 
the forwarders are), and very difficult for a medium operator (no, end 
users are *not* going to helpfully tell you when they set up a forward 
on some random MTA).  This probably means medium operators can't 
realistically reject on SPF.

spfbis should simply explain the constraint for correctly rejecting on 
SPF.  Rfc4408 already does, but not very explicity.  I think we should 
say that any SPF result MUST NOT be considered official (and therefore 
rejecting on FAIL is not recommended) when the client MTA is a non-SRS 
forwarder.  Note that for small operators, it is much more common in my 
experience for them to reject on FAIL from their own secondary MX, than 
to have forwarding problems.

SPF does not break forwarding.  It simply has the limitation that 
policies can't be (correctly) checked for alias/non-SRS forwarders.

Idea: instead of trying to detect alias forwarders, detect 
non-forwarders instead.  Start by assuming *all* MTAs are alias 
forwarders.  Whenever you get an SPF PASS, add the domain,IP tuple (MTA 
could send both forwarded an unforwarded emails) to a database of 
non-forwarders, for which you can reject on FAIL.  The database could 
get very large, however.

---

From: Stuart D Gathman <[email-masked]>
Date: Fri, 23 Mar 2012 17:11:08 -0400

Long ago, Nostradamus foresaw that on 03/23/2012 01:07 PM, Alessandro 
Vesely would write:
> Of course, we can pretend there is no problem,
There is, in fact, no problem.

The result of running the SPF algorithm on an alias forwarder is NOT an 
actual SPF result.  Rejecting on an actual SPF result does not cause 
problems.  The fact that connections from alias forwarders are invalid 
input to the SPF algorithm is a limitation of the protocol.  But it does 
not break forwarding.  Forwarding is only broken when receivers confuse 
pseudo-SPF results from an alias forwarder with real SPF results, and 
then reject on FAIL.

We should continue to encourage reject on *SPF* (vs pseudo-SPF) FAIL.  
We need to define valid inputs to SPF in a way that people won't confuse 
the two as easily.

There are a variety of ways to distinguish alias forwarders from direct 
MTAs, from the very simple "configure authorized forwarders in your SPF 
software", to webapps to allow users to enable reject on FAIL and enter 
authorized forwarders for a larger MTA, to methods that detect 
forwarders from a large set of connections.  If a receiver can't 
distinguish alias forwarders in some practical way, then SPF is not as 
useful to them.

Note that configuring authorized forwarders does not need to involve 
entering IP addresses if the forwarder publishes SPF for some domain 
that covers the MTAs used for alias forwarding.

As I mentioned in a previous message, I believe that if the SPF result 
is pass, we can assume that the MTA is *not* an alias forwarder for that 
domain, and this can be used to build a list of domain,ip where an SPF 
fail result is valid.  But this is a new idea (for me anyway), and needs 
to be vetted.

---

From: S Moonesamy <[email-masked]>
Date: Fri, 23 Mar 2012 14:45:40 -0700

Hi Alessandro,
>mean as a possibility, without committing ourselves to actually do so.

I don't see any justification at the moment for asking whether it is 
feasible to make such an update.  You could raise the question during 
the WG session next Friday if you wish.

Regards,
S. Moonesamy
SPFBIS WG co-chair

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sat, 24 Mar 2012 04:52:08 +0000

> From: [email-masked] [mailto:[email-masked]] On Behalf Of Stuart D Gathman
> Sent: Friday, March 23, 2012 2:11 PM
> To: [email-masked]
> input to the SPF algorithm is a limitation of the protocol.  But it
> does not break forwarding.  Forwarding is only broken when receivers
> confuse pseudo-SPF results from an alias forwarder with real SPF
> results, and then reject on FAIL.
> [...]

This an interesting tack, but I don't think I agree.  SPF takes as input a domain name and an IP address, does some crunching, and returns a result.  That result is deterministic.  At the level of that interface, the IP address of a forwarder is indistinguishable from any other, so to say one class of IP address is valid while another is not seems a non-sequitur to me.

Any result produced by taking those two pieces of input and following the SPF algorithm is an SPF result.  If two different systems have differing outputs given the same inputs, then we have an interoperability problem and/or a broken specification.

Requiring different treatment of forwarders based on a heuristic would not be an SPF result, in the same way that several have asserted that Best Guess SPF is not an SPF result.

> We need to define valid inputs to SPF in a way that people won't
> confuse the two as easily.

That might be a useful exercise, but I don't think it fits with the current protocol.

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sat, 24 Mar 2012 04:57:08 +0000

> Sent: Friday, March 23, 2012 12:57 PM
> To: [email-masked]
> servers unless its address passes an SPF record with its label.  I mean
> as a possibility, without committing ourselves to actually do so.

I would bet real money that this is a non-starter.  By doing so, you would declare a big piece of the deployed email infrastructure non-compliant.  If I may paraphrase John Levine about MLMs: Forwarders have been doing what they do for decades without complaints, so who are we to suddenly tell them that they're wrong?

> (Yes, I can imagine how would, say, John Klensin react if I hounded him
> for such a question, but maybe there is a better way of asking...)

Such as trying to get one of us to do it for you?  :-)

> One way or the other, we must say how recipients are supposed to behave
> to make the protocol work.  The clearer the better.  It has to be our
> concern, although it might be impermissible for us to explicitly
> mandate it.

When it comes time to crack open RFC4408bis itself, I imagine you should have clarifying text ready to propose so we can discuss it.  I haven't looked lately, but I believe Scott when he says it's already there.

-MSK

---

From: "John Levine" <[email-masked]>
Date: 24 Mar 2012 09:03:49 -0000

>> possible to check the feasibility of updating just a little bit of SMTP?

I think I know Klensin well enough to channel his answer to that
question.  To summarize three pages of detailed response: absolutely
none.

It is a specific design feature of SMTP that the contents of a message
are independent of the path it takes.  That isn't going to change.

The only reason SPF works at all is that a large fraction of mail
happens to take a predictable path.  But not all of it does, and the
part that doesn't can't be wished away.

R's,
John

---

From: Alessandro Vesely <[email-masked]>
Date: Sat, 24 Mar 2012 17:18:25 +0100

On 24.03.2012 05:52, Murray S. Kucherawy wrote:
>> From: ietf.org On Behalf Of Stuart D Gathman
>
> This an interesting tack, but I don't think I agree.  SPF takes as input a
> domain name and an IP address, does some crunching, and returns a result.
> That result is deterministic.  At the level of that interface, the IP
> address of a forwarder is indistinguishable from any other, so to say one
> class of IP address is valid while another is not seems a non-sequitur to
> me.

I didn't fully grasp Stuart's method.  If it works reliably, it is a
possibility.  It is true that it seems to add a complex IP database
maintenance to a simple algorithm, but I wouldn't consider it out of scope if
it enables us to spell under what condition it is safe to reject.

> Any result produced by taking those two pieces of input and following the
> SPF algorithm is an SPF result.  If two different systems have differing
> outputs given the same inputs, then we have an interoperability problem
> and/or a broken specification.

We can distinguish the algorithm from the conditions that enable reject on
fail.  Two systems may get the same results but different ability to reject
based on that.  I'm quite dubious about defining different behaviors for
small, medium, and very large systems, but, heck, whatever works!

> Requiring different treatment of forwarders based on a heuristic would not
> be an SPF result, in the same way that several have asserted that Best
> Guess SPF is not an SPF result.

Yes.

> That might be a useful exercise, but I don't think it fits with the

Yes, strictly speaking, it is not part of it.

---

From: Alessandro Vesely <[email-masked]>
Date: Sat, 24 Mar 2012 17:30:09 +0100

On 24.03.2012 05:57, Murray S. Kucherawy wrote:
>> servers unless its address passes an SPF record with its label.
> 
> I would bet real money that this is a non-starter.  By doing so, you would
> declare a big piece of the deployed email infrastructure non-compliant.

Isn't it a small piece?

> If I may paraphrase John Levine about MLMs: Forwarders have been doing
> what they do for decades without complaints, so who are we to suddenly
> tell them that they're wrong?

Not suddenly, every user and operator found out that email is too easily
abused.  IMHO, it is the smallest change that is compatible with plain reject
on fail.

> Such as trying to get one of us to do it for you?  :-)

Are you volunteering? :-)

---

From: Scott Kitterman <[email-masked]>
Date: Sat, 24 Mar 2012 12:31:37 -0400

On Saturday, March 24, 2012 04:52:08 AM Murray S. Kucherawy wrote:
> > Of Stuart D Gathman Sent: Friday, March 23, 2012 2:11 PM
> > To: [email-masked]

I think it's more useful to make recommendations about where SPF checks should 
be done.  My ascii art skills are minimal, so I made a PDF (attached - no 
comments on those skills please) to describe what I think are the different 
architectural cases and where SPF checks should be done in each.

The first one is the most common case of direct transfer from the sending ADMD 
to the receiving ADMD.  The check should be done on the border between the two 
ADMDs (that doesn't mean the results of those checks can't be used elsewhere 
if that's the design of the receiving ADMD, but that's where the measurement 
should be taken).

The second one is the traditional forwarder case.  In this case, the forwarder 
is an agent for the receiver (the sender has no idea their mail will be 
forwarded, it's the receiver that determines this).  The forwarder is 
effectively part of the receiver's ADMD.  The case is generically the same 
though.  SPF should be checked when it passes from the sender's ADMD to the 
receiver's.  The reason this case is so hard is that many receivers don't 
understand the boundaries of their ADMD to include this and argue they 
shouldn't have to.

The third one is the case of a rewriting forwarder (this could be SRS or some 
other method, that is an implementation detail).  In this case the forwarder 
proxies for the sender so that they are acting as both an agent for the sender 
and for the receiver.  The receiver is inside both ADMDs to some degree.

Finally, in the mailing list case, it's really very easy as normal mailing 
lists treat a submission as a completed delivery and then make a new message 
to the list.  In the first transaction they are the receiving ADMD and in the 
second they are the sending.  Lists are also different in that, unlike 
forwarding, both the sender and the receiver are party to the list (assuming 
appropriate controls to avoid random posts are in place) and so the list 
manager is an agent for both.

When I say section 9 needs to be re-written in modern email architecture 
terms, this is the kind of thing I had in mind.  I wouldn't go so far as 
Stuart does to declare that measurements taken in the wrong place are not SPF 
results.  That binds the protocol to architecture in a way that I think is 
problematic.  I do think we need to give advice on where/how to do it.

Scott K

---

From: Alessandro Vesely <[email-masked]>
Date: Sat, 24 Mar 2012 17:42:58 +0100

> question.  To summarize three pages of detailed response: absolutely

Probably yes.  Yet, those three pages may come handy if the WG needs to
justify a decision like banning reject on fail.

> are independent of the path it takes.

Not the content, just a piece of the envelope.  A varying return-path is not
such a novelty, after all.

> part that doesn't can't be wished away.

Thus, with SPF being a standard part of mail, we can say mail works in a
large fraction of cases.

---

From: Douglas Otis <[email-masked]>
Date: Sat, 24 Mar 2012 16:21:25 -0700

On 3/24/12 9:42 AM, Alessandro Vesely wrote:
> > happens to take a predictable path. But not all of it does, and
> > the part that doesn't can't be wished away.
>
>  Thus, with SPF being a standard part of mail, we can say mail works
>  in a large fraction of cases.

Dear Alessandro,

A piece missing in SMTP is a method to authenticate outbound and 
incoming servers, but email operations work well and likely better 
without SPF.  Dropping NDNs without an SPF pass lowers intended delivery 
integrity in a way not compliant with SMTP.

Dealing with NDNs containing spoofed return paths can be (and largely 
have been) supplanted by vetting prior to acceptance.  Domains that 
don't may find email not working as desired and their MTA's increasingly 
flooded with abuse.  DKIM provides more reliable defenses against 
phishing, where a return path is seldom visible anyway.

SPF may help facilitate the determination of IP address's reputation 
interference.  SPF may help facilitate white-listing of problematic 
domains that require too much support otherwise, but only when "i" and 
"l" macros have not been used.  Responding to problems created for even 
a small fraction of email is why SPF as a basis for acceptance or 
rejection is not practical for large operations who provide services for 
the majority of email.

Unfortunately, SPF remains problematic even for the EHLO such as what 
can be done for email coming from 10.64.0.0/10?

Regards,
Douglas Otis

---

From: Douglas Otis <[email-masked]>
Date: Sat, 24 Mar 2012 16:24:13 -0700

> can be done for email coming from 10.64.0.0/10? 
Correction: 100.64.0.0/10

Regards,
Douglas Otis

---

From: Alessandro Vesely <[email-masked]>
Date: Sun, 25 Mar 2012 09:56:20 +0200

> understand the boundaries of their ADMD to include this and argue they 

To include them, receivers should at least know.

There is no protocol whereby a forwarder can tell to a receiver that it is
going to forward mail.  (We may call it "consent-exchange protocol", as it
implies that the owners of the target addresses grant their consent for such
use; this is a legal requirement in some countries.)  Thus, operators on the
receiving side have to collect the relevant data by unspecified means; that
is to say, manual and fallible methods, irrespectively of the system size.

Of course, pretending to be a legitimate forwarder is a common attack path...

It would be illuminating to be able to estimate what fraction of legitimate
mail is (or fails to be) transferred that way.  Any report?

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sun, 25 Mar 2012 08:16:13 +0000

> Sent: Saturday, March 24, 2012 9:30 AM
> To: [email-masked]
> > I would bet real money that this is a non-starter.  By doing so, you
> > would declare a big piece of the deployed email infrastructure non-
> > compliant.
> 
> Isn't it a small piece?

I don't have such measurements.  Do they even exist?

> Not suddenly, every user and operator found out that email is too
> easily abused.  IMHO, it is the smallest change that is compatible with
> plain reject on fail.

Still, you're proposing adding a new requirement onto the entirety of a decades-old infrastructure.  I imagine for that trick to work, the benefit would have to be pretty enormous, and a huge majority would have to be on board with adopting it.

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sun, 25 Mar 2012 08:16:14 +0000

> Sent: Saturday, March 24, 2012 9:43 AM
> To: [email-masked]
> Thus, with SPF being a standard part of mail, we can say mail works in
> a large fraction of cases.

SPF isn't a standard part of mail.  It's an optional add-on.

-MSK

---

From: Alessandro Vesely <[email-masked]>
Date: Sun, 25 Mar 2012 12:14:13 +0200

> SPF isn't a standard part of mail.  It's an optional add-on.

Until we hold that reject-on-fail is valid, SPF is less optional than it
would seem at a first glance, because it affects a mail operation that used
to be possible without it.  A sender who does not opt for SPF can have its
forwarded messages bounced because both the originator and the target opted
to implement SPF in such a way to break that operation.  In this respect, SPF
has changed SMTP already, whether we're gonna document that as an update or not.

What's worse is that there's no guidance on how to avoid such incidents.  A
protocol worth its salt should allow to determine what parties are at fault
in case of malfunction.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 26 Mar 2012 05:59:36 +0000

Given Scott's last message, I think there's been enough discussion about deployment advice, caveats, etc., that we should decide whether there's enough material to warrant its own document.  If the volume of such pedagogy is significant, then packing it all into a really big appendix of RFC4408bis might be considered poor form.

It's obviously not one of our deliverables in the current charter, but I suspect it wouldn't be a big deal to add it since it doesn't affect the constraints on protocol changes.

Co-chairs, your thoughts?

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 26 Mar 2012 06:04:35 +0000

> Sent: Saturday, March 24, 2012 9:32 AM
> To: [email-masked]
> Finally, in the mailing list case, it's really very easy as normal
> mailing lists treat a submission as a completed delivery and then make
> a new message to the list.  In the first transaction they are the
> receiving ADMD and in the second they are the sending.  Lists are also
> different in that, unlike forwarding, both the sender and the receiver
> are party to the list (assuming appropriate controls to avoid random
> posts are in place) and so the list manager is an agent for both.

Do most list servers these days replace the envelope sender?  I can't remember.

Perhaps predictably, I want to avoid us telling MLMs what they have to start doing in terms of what they need to do to accommodate SPF.

---

From: Commerco WebMaster <[email-masked]>
Date: Mon, 26 Mar 2012 00:10:20 -0600

+1

Despite my not being a co-chair, I think the idea seems sound.  Where do 
you suggest that this kind of information reside?  Is there a precedent 
for doing this from other RFCs?  If so, then it seems a template for 
creating such an adjunct document exists.

I would submit that such a collection of deployment, caveats, etc., 
information discussed so far could logically rest somewhere within 
OpenSPF.Org

Alan M.

---

From: Scott Kitterman <[email-masked]>
Date: Mon, 26 Mar 2012 02:27:05 -0400

>Do most list servers these days replace the envelope sender?  I can't

AFAIK, they do.

Scott K

---

From: "John Levine" <[email-masked]>
Date: 26 Mar 2012 08:22:37 -0000

>Do most list servers these days replace the envelope sender?  I can't remember.

They all do.  They have to, if they're going to do bounce management.

MLM's haven't had any trouble with SPF other than in faulty client
implementations that attempt to correlate some bit of the message
headers with the bounce address.

R's,
John

---

From: S Moonesamy <[email-masked]>
Date: Mon, 26 Mar 2012 01:29:22 -0700

Hi Murray,
At 22:59 25-03-2012, Murray S. Kucherawy wrote:
>Given Scott's last message, I think there's been enough discussion 
>about deployment advice, caveats, etc., that we should decide 
>whether there's enough material to warrant its own document.  If the 
>volume of such pedagogy is significant, then packing it all into a 
>really big appendix of RFC4408bis might be considered poor form.
>
>It's obviously not one of our deliverables in the current charter, 
>but I suspect it wouldn't be a big deal to add it since it doesn't 
>affect the constraints on protocol changes.

Could you please bring this up during the WG session on Friday?  As 
the Responsible AD will be in the room, it should be easy to resolve 
the deliverable angle.

Could the WG please provide feedback about this so that we don't get 
into a long discussion on Friday?

Regards,
S. Moonesamy
SPFBIS WG co-chair

---

From: Scott Kitterman <[email-masked]>
Date: Mon, 26 Mar 2012 06:53:16 -0400

>into a long discussion on Friday?

I am not convinced of the need. I think it's generally better to keep it in one document.

Scott K

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 26 Mar 2012 11:22:50 +0000

> From: [email-masked] [mailto:[email-masked]] On Behalf Of Commerco WebMaster
> Sent: Sunday, March 25, 2012 11:10 PM
> To: [email-masked]
> Subject: Re: [spfbis] Deployment document?
> 
> Despite my not being a co-chair, I think the idea seems sound.  Where
> do you suggest that this kind of information reside?  Is there a
> precedent for doing this from other RFCs?  If so, then it seems a
> template for creating such an adjunct document exists.

DKIM has a deployment document that's separate from the protocol specification, for example.  They also did an overview document.

I'm sensitive to Scott's concern that it all be in one document.  There's some question about whether anyone reading RFC4408bis would even think to look for separate documents.  We could, of course, do them as a cluster, though that will have the side effect of delaying publication until both are ready.

I'd be fine with doing this in RFC4408bis itself unless the volume of it gets too big to fit practically in an appendix.  At that point I think it should be in its own Informational.

As SM requested, I'll bring it up Friday.

-MSK

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 26 Mar 2012 15:15:47 +0200

On Mon 26/Mar/2012 15:00:29 +0200 S Moonesamy wrote:
>> Given Scott's last message,

Was it the one about Received-SPF?

>> I think there's been enough discussion about deployment advice, caveats,
>> etc., that we should decide whether there's enough material to warrant
>> its own document.  If the volume of such pedagogy is significant, then
>> packing it all into a really big appendix of RFC4408bis might be
>> considered poor form.
>>
>> It's obviously not one of our deliverables in the current charter, but I
>> suspect it wouldn't be a big deal to add it since it doesn't affect the
>> constraints on protocol changes.
> 
> Could you please bring this up during the WG session on Friday?  As the
> Responsible AD will be in the room, it should be easy to resolve the
> deliverable angle.
> 
> Could the WG please provide feedback about this so that we don't get into a
> long discussion on Friday?

It's not very clear what should go to the separate document.  Differently
than DKIM, SPF is a self contained protocol, or at least used to.  Some
difficulties originate from unclear logic (e.g. break forwarding being
predicated true and false at the same time) but if we are able to get them
straight, we might actually happen to shorten the original text.  That is to
say, I'd defer deciding about an overflow document to when we see that we
need to be verbose.  I hope it won't be necessary.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 26 Mar 2012 13:19:49 +0000

> Sent: Monday, March 26, 2012 6:16 AM
> To: [email-masked]

I was referring to Scott's message that included a PDF which showed possible places in the handling chain where SPF evaluation makes sense.  That by itself could be developed into its own informational document, along with other things we've talked about including as operational and/or deployment advice.

> It's not very clear what should go to the separate document.
> Differently than DKIM, SPF is a self contained protocol, or at least
> used to.  Some difficulties originate from unclear logic (e.g. break
> forwarding being predicated true and false at the same time) but if we
> are able to get them straight, we might actually happen to shorten the
> original text.  That is to say, I'd defer deciding about an overflow
> document to when we see that we need to be verbose.  I hope it won't be
> necessary.

We certainly should inventory what we want to put into such a document before starting.  But it feels to me as though there's quite a bit to write already.

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 26 Mar 2012 15:40:26 +0200

On Mon 26/Mar/2012 15:25:54 +0200 Murray S. Kucherawy wrote:
> I was referring to Scott's message that included a PDF which showed
> possible places in the handling chain where SPF evaluation makes sense.

I see.  Those figures will have to be manually converted to ascii-art, but
aren't they part of the main protocol?  I'm not an artist, but I volunteer to
do that conversion, if nobody else wants to.  The description of those
figures might be short and clear or lengthy and fuzzy, that's up to how
clever we are.

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 27 Mar 2012 10:17:05 +0200

On Tue 27/Mar/2012 09:51:51 +0200 Murray S. Kucherawy wrote:
>> IMHO, it is the smallest change that is compatible with plain reject on
>> fail.
> 
> Still, you're proposing adding a new requirement onto the entirety of a
> decades-old infrastructure.  I imagine for that trick to work, the benefit
> would have to be pretty enormous, and a huge majority would have to be on
> board with adopting it.

I'd rather ask the opposite question.  That is, what functionality requires
to preserve the envelope unaltered?  Severe changes for avoiding backscatter
have been carried out on almost every system, in recent years.  Dropping the
envelope sender on external "alias expansion" would probably pass unnoticed,
except for an unprecedented simplification of the SPF receiver-behavior spec.