Subject: [spfbis] SPF and empty mail from envelope spec is unclear.

From: Franck Martin <[email-masked]>
Date: Fri, 27 Jul 2012 00:14:27 +0000

Hi,

This may be something, you have discussed before, but I'll jump in, instead of reading the archives…

I'm looking at the current spec, and I'm not sure it is clear to use the domain in the HELO/EHLO string when the mail from: is empty (bounce).

I think this part should be made clearer, and may be help define what is the domain inside the HELO/EHLO string. Is this defined as localhost.domain? Or is it left to the SPF implementation to figure it out and look for an SPF record for each portion of the string: a.b.c.d then b.c.d then c.d …

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 27 Jul 2012 01:53:42 -0400

On Friday, July 27, 2012 12:14:27 AM Franck Martin wrote:
> I'm looking at the current spec, and I'm not sure it is clear to use the
> domain in the HELO/EHLO string when the mail from: is empty (bounce).
> 
> I think this part should be made clearer, and may be help define what is the
> domain inside the HELO/EHLO string. Is this defined as localhost.domain? Or
> is it left to the SPF implementation to figure it out and look for an SPF
> record for each portion of the string: a.b.c.d then b.c.d then c.d …

It's the domain used by the connection SMTP client as their HELO/EHLO name.  
Isn't that sufficiently defined in RFC 5321?  There is no walk up the domain 
tree.  That was tried pre-RFC 4408 and it turned out to have a number of 
operational problems and was abandoned.

I'm not sure what's unclear, so perhaps you could propose text?

Scott K

---

From: Hector Santos <[email-masked]>
Date: Fri, 27 Jul 2012 23:47:00 -0400

Maybe some clarification is required for Section 2.1 in the form of 
what is the HELO identity in relationship to the IP and perhaps why 
SPF may be applicable.  Of course, it can point to RFC5321 but maybe a 
sentence can save the reader some time since SPF is a protocol that 
attempts to raise the bar for the domain::IP association.

Something as simple:

    For SMTP compliant clients, the HELO identity MUST be the public 
domain
    name of the computer (machine) sending starting the SMTP session. 
  This
    domain MUST match the IP address of the connecting machine. While 
RFC5321
    has a protocol requirement for a valid public domain name, and if the
    client can not provide a valid public domain, it MUST use squared 
bracketed
    IP literal. Unfortunately,  there is a history of invalid values. 
  It could be a
    valid private domain name or even a NETBIOS computer name 
resolvable locally,
    however, it is invalid public identity hence the unreliability of 
performing
    reverse DNS lookups (PTR) and domain::IP matching attempts.

    SPF attempts to raise the bar by providing a policy to associate 
the IP with
    the HELO identity verifying the machine sending the machine. 
However, since
    of the unreliability for a proper public value, performing an SPF 
check on
    the HELO identity SHOULD remain optional
    at the deployment level.

You get the idea.

Note: I wonder when a SPF(HELO) check is a valid, whether a rDNS is 
also valid. I don't know off hand if that would be always the case, 
but if the argument is made a PTR is not enforced, then SPF may be 
viewed as a (unenforced) replacement rule.   So this could be a 
discussion on an implementation "suggestion" for the advocation of PTR 
deprecation:

      SMTP receivers may wish to consider not enforcing PTR 
requirement and instead
      check the SPF HELO identity policy.

And what makes this a technical consideration on its merit is the 
suggestion the TXT lookup is less expensive than a PTR.

PS: RFC1001 has this for a reference (no official RFC for NETBIOS) and 
footnote:

    NetBIOS defines a software interface not a protocol.  There is no
    "official" NetBIOS service standard.  In practice, however, the
    IBM PC-Network version is used as a reference.  That version is
    described in the IBM document 6322916, "Technical Reference PC
    Network"[2].

   [2]  IBM Corp., "IBM PC Network Technical Reference Manual", No.
        6322916, First Edition, September 1984.

Scott Kitterman wrote:
> _______________________________________________
> spfbis mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/spfbis

-- 
HLS

---

From: Franck Martin <[email-masked]>
Date: Sat, 28 Jul 2012 07:17:35 +0000

I see...

from experience when it is not broken what I have seen is that the helo does not contain a domain but the fully qualified hostname of the client: hostname.domain

It could then difficult to find the domain from that.

I'm not sure, any SMTP implementation respect this RFC. May be someone as some data?

---

From: Scott Kitterman <[email-masked]>
Date: Sat, 28 Jul 2012 03:20:05 -0400

I guess I was speaking imprecisely.  fully qualified hostname of the client: 
hostname.domain is exactly what you use.  

Scott K

On Saturday, July 28, 2012 07:17:35 AM Franck Martin wrote:
> I see...
> 
> from experience when it is not broken what I have seen is that the helo does
> not contain a domain but the fully qualified hostname of the client:
> hostname.domain
> 
> >> This may be something, you have discussed before, but I'll jump in,
> >> instead
> >> of reading the archives…
> >> 
> >> I think this part should be made clearer, and may be help define what is
> >> the domain inside the HELO/EHLO string. Is this defined as
> >> localhost.domain? Or is it left to the SPF implementation to figure it
> >> out and look for an SPF record for each portion of the string: a.b.c.d
> >> then b.c.d then c.d …> 
> > Isn't that sufficiently defined in RFC 5321?  There is no walk up the
> > domain tree.  That was tried pre-RFC 4408 and it turned out to have a
> > number of operational problems and was abandoned.
> >

---

From: Hector Santos <[email-masked]>
Date: Sat, 28 Jul 2012 08:30:09 -0400

And more to the precision - a public FQHN. FQDN is also a common 
acronym.  Makes no use if its private.

-- 
HLS

---

From: Stuart Gathman <[email-masked]>
Date: Sun, 29 Jul 2012 22:39:47 -0400

Long ago, Nostradamus foresaw that on 07/27/2012 11:47 PM, Hector Santos
would write:
>
>      SMTP receivers may wish to consider not enforcing PTR requirement
> and instead
For checking the identity of an SMTP connection with valid HELO that
matches the IP, there is no need for SPF or PTR.  The matching HELO
establishes that the owner of the HELO domain is responsible for the
sending MTA. 

Where checking SPF HELO comes in is rejecting *unauthorized*
connections, and in authorizing connections where an SPF record is
simpler than assigning a unique HELO to each IP used by MTAs (e.g. large
MTA  farms).

---

From: Franck Martin <[email-masked]>
Date: Mon, 30 Jul 2012 19:53:10 +0000

Digging more in this issue, I'm keen on proposing some text, but also
looking at my anecdotal operational experience, isn't it something that
should go on the deprecated list, due to lack of correct implementation?

On 7/28/12 12:20 AM, "Scott Kitterman" <[email-masked]> wrote:

>> I'm not sure, any SMTP implementation respect this RFC. May be someone
>>as
>> some data?
>> >> of reading the archivesŠ
>> >> then b.c.d then c.d Š>

---

From: Scott Kitterman <[email-masked]>
Date: Mon, 30 Jul 2012 16:25:01 -0400

I'm not sure what you mean.  At least all open source libraries of which I'm 
aware implement this correctly.  It's certainly not unused.

Scott K

On Monday, July 30, 2012 07:53:10 PM Franck Martin wrote:
> >> from experience when it is not broken what I have seen is that the helo
> >>
> >>does
> >>
> >> >> I'm looking at the current spec, and I'm not sure it is clear to use
> >>
> >>the
> >>
> >> >> I think this part should be made clearer, and may be help define
> >>
> >>what is
> >>
> >> >> localhost.domain? Or is it left to the SPF implementation to figure
> >>
> >>it
> >>

---

From: Franck Martin <[email-masked]>
Date: Mon, 30 Jul 2012 20:46:46 +0000

What I mean, is who present a correct helo string associated with a SPF
record, so the SPF validation can work for a bounce? Is that part widely
implemented?

---

From: Scott Kitterman <[email-masked]>
Date: Mon, 30 Jul 2012 16:58:23 -0400

Yes.

Scott K

---

From: Franck Martin <[email-masked]>
Date: Mon, 30 Jul 2012 22:16:05 +0000

Thanks. 

I'm confused because I see often host.domain where domain is the one with
the SPF record.

Based on http://datatracker.ietf.org/doc/draft-ietf-spfbis-4408bis/

An added section under 9.1 could be useful

9.1.3 Bounces
As explained in 2.2, [RFC5321] allows the reverse-path to be null, which
is typical of some Delivery Status Notification [RFC3464], commonly called
email bounces. In this case the only entity available for performing an
SPF check is the HELO identity defined in 1.1. Administrators
MUST(SHOULD?) ensure that this identity is set correctly and has an
appropriate SPF record. It is common to have the HELO identity set to
hostname instead of domain. Administrators SHOULD consider publishing an
SPF for *.domain (wildcard domains) in that case.

Feel free to rewrite, which I'm sure you will.

On 7/30/12 1:58 PM, "Scott Kitterman" <[email-masked]> wrote:

>> >I'm not sure what you mean.  At least all open source libraries of
>>which
>> >I'm

---

From: Roland Turner <[email-masked]>
Date: Tue, 31 Jul 2012 13:28:43 +0800

Franck,

I've read the thread, and don't understand what you're getting at. 4408 
2.2 is clear, when MAIL FROM is empty (DSN/MDN/bounce), the identifier 
that's being assessed is postmaster@$HeloIdentity. Questions of matching 
organisational suffixes are not addressed by SPF (and may not make much 
sense; if the machine generating the bounce is a gateway for a thousand 
domains, which is the "correct" one for you to look at?).

Perhaps another way to tackle this is to ask: why do you care about the 
difference between "HELO a.example.com" and "HELO example.com"? What use 
are you trying to make of an authenticated HELO string?

- Roland

---

From: Franck Martin <[email-masked]>
Date: Tue, 31 Jul 2012 15:13:16 +0000

The first helo will not work because the SPF record is on the second one.

If you take postfix by default the helo contains the hostname http://www.postfix.org/postconf.5.html#smtp_helo_name but the spf record is usually on the domain name, not the hostname. So the document needs to make a bit more of advocacy regarding this issue. This is why I suggest to insert 9.1.3 to discuss this case in detail and draw attention to it.

On Jul 30, 2012, at 10:28 PM, Roland Turner wrote:

> Franck,
> 
> I've read the thread, and don't understand what you're getting at. 4408 2.2 is clear, when MAIL FROM is empty (DSN/MDN/bounce), the identifier that's being assessed is postmaster@$HeloIdentity. Questions of matching organisational suffixes are not addressed by SPF (and may not make much sense; if the machine generating the bounce is a gateway for a thousand domains, which is the "correct" one for you to look at?).
> 
> Perhaps another way to tackle this is to ask: why do you care about the difference between "HELO a.example.com" and "HELO example.com"? What use are you trying to make of an authenticated HELO string?
> 
> - Roland
> 
> 
>

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 31 Jul 2012 14:53:49 -0400

It's expected that one publishes SPF records for the hostname as well.  This 
is not news.  It's been this way since 2005.

Scott K

On Tuesday, July 31, 2012 03:13:16 PM Franck Martin wrote:
> If you take postfix by default the helo contains the hostname
> http://www.postfix.org/postconf.5.html#smtp_helo_name but the spf record is
> usually on the domain name, not the hostname. So the document needs to make
> a bit more of advocacy regarding this issue. This is why I suggest to
> insert 9.1.3 to discuss this case in detail and draw attention to it.
> >>>>>> looking at my anecdotal operational experience, isn't it something
> >>>> 
> >>>> that
> >>>> 
> >>>>>> should go on the deprecated list, due to lack of correct
> >>>> 
> >>>> implementation?
> >>>>

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 31 Jul 2012 13:03:08 -0700

> is not news.  It's been this way since 2005.
It doesn't take much thinking to realize that if one wants to rely on HELO
validation, this is a necessary step.

That said, does anyone feel strongly that RFC4408bis should say so?

-MSK

---

From: Derek Diget <[email-masked]>
Date: Tue, 31 Jul 2012 18:59:16 -0400 (EDT)

On Jul 31, 2012 at 13:03 -0700, Murray S. Kucherawy wrote:
=>On Tue, Jul 31, 2012 at 11:53 AM, Scott Kitterman <[email-masked]>wrote:
=>
=>> It's expected that one publishes SPF records for the hostname as well.
=>>  This is not news.  It's been this way since 2005.
=>>
=>It doesn't take much thinking to realize that if one wants to rely on HELO
=>validation, this is a necessary step.
=>
=>That said, does anyone feel strongly that RFC4408bis should say so?

No it shouldn't.  It is already mentioned at 
<http://www.openspf.org/FAQ/Common_mistakes#helo> and would go in the 
BCP/deployment guide (if there is one).

-- 
***********************************************************************
Derek Diget                            Office of Information Technology
Western Michigan University - Kalamazoo  Michigan  USA - www.wmich.edu/
***********************************************************************

---

From: Franck Martin <[email-masked]>
Date: Wed, 1 Aug 2012 01:48:07 +0000

>BCP/deployment guide (if there is one).

So why is it such a common mistake?

$ dig +short txt ietf.org
"v=spf1 ip4:12.22.58.0/26 ip4:64.170.98.0/26 ip4:208.66.40.224/27
ip6:2001:1890:123a::0/56 ip6:2001:1890:1112:1::0/64 -all"
$ dig +short txt mail.ietf.org
<no result>

Received: from [65.55.34.216] ([65.55.34.216:9424]
 helo=col0-omc4-s14.col0.hotmail.com)

$ dig +short txt col0-omc4-s14.col0.hotmail.com
<no result>
$ dig +short txt hotmail.com
"v=spf1 include:spf-a.hotmail.com include:spf-b.hotmail.com
include:spf-c.hotmail.com include:spf-d.hotmail.com ~all"

May I suggest because it is not mentioned clearly in the RFC? :P

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 31 Jul 2012 22:33:52 -0400

> May I suggest because it is not mentioned clearly in the RFC? :P

I think it's worth mentioning in section 9.  I've got your proposed words and 
when I dive through the document next, I'll see how I can fit them in.

HELO/EHLO definitely gets a lot less attention than Mail From and so I think 
it's worth mentioning.

Scott K

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 31 Jul 2012 19:38:37 -0700

On Tue 31/Jul/2012 13:03:08 -0700 Murray S. Kucherawy wrote:
> On Tue, Jul 31, 2012 at 11:53 AM, Scott Kitterman wrote:
> 
>> It's expected that one publishes SPF records for the hostname as 
>> well.  This is not news.  It's been this way since 2005.
> 
> It doesn't take much thinking to realize that if one wants to rely on
> HELO validation, this is a necessary step.

It takes too much thinking.  Some of it is required to determine if
one needs to rely on HELO validation.  At this point the plot
thickens:  What difference does that make?

> That said, does anyone feel strongly that RFC4408bis should say so?

I, for one.  We have an open ticket that is (also) about this:
http://trac.tools.ietf.org/wg/spfbis/trac/ticket/12

---

From: Franck Martin <[email-masked]>
Date: Wed, 1 Aug 2012 02:49:05 +0000

I think the issue is nobody think about bounces when deploying SPF.

Thanks for looking into fitting the words.

Toute connaissance est une réponse à une question.