Subject: [marf] Message bodies in ARF reports

From: Scott Kitterman <[email-masked]>
Date: Mon, 13 Feb 2012 01:20:47 -0500

I think I've seen mention a few times on this list about doing certain things 
with ARFs depending on if they have the message bodies attached.

Now that I'm working on my implementation of draft-ietf-marf-spf-reporting, 
I'm questioning where the requirement for message bodies or headers in ARFs 
is.

As I read RFC 5965 2.d, either the message body or rfc822-headers must be 
included as the third part of the message.

For SPF auth-failure reports, I'm not sure what, if anything, it makes to 
include in this third MIME part of the message.  Since it's during the SMTP 
transaction, there are SMTP exchanges of identities, but no headers.

Since Mail From is already included as an explicit field in the second MIME 
part, I'm not sure what, if anything should be required for the last part.  In 
the case of the work I'm doing, I'm using the Postfix policy delegation 
interface (see http://www.postfix.org/SMTPD_POLICY_README.html if you're 
interested) and I don't have access to anything in the body of the message 
(nor is there a reason for me to have it).

Is this MUST in 5965 being ignored or am mis-interpreting something?

Scott K

---

From: "John Levine" <[email-masked]>
Date: 13 Feb 2012 15:41:32 -0000

>I'm questioning where the requirement for message bodies or headers in ARFs 

Interesting question.

ARF is a version of multipart/report as defined in RFC 3462.  In that
document, the message/rfc822 or message/headers is optional.  In RFC
5965, we made it mandatory in section 2.d.  We did that because
the usage scenario we were thinking of was spam button reports where
you always have a message in hand, not for any deeper reason.

You are of course correct that SPF failures sometimes don't accept the
message.  I'd be willing to file an erratum that 2.d. should have said
that the report MUST include the message if the reporting entity has
received it.

R's,
John

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 13 Feb 2012 17:28:33 +0100

> included as the third part of the message.

Yup.

> transaction, there are SMTP exchanges of identities, but no headers.

AFAICS, the third part is needed to let the sender understand what
message failed to authenticate.  For example, if a user of theirs sent
a message via an unauthorized server, the envelope is possibly not
enough to take action (e.g. tell the user to not do that.)

> (nor is there a reason for me to have it).

Could it be worked around designing a two-tier process?  That is,

1) generate non-ARF data (essentially, ARF w/o 3rd part), and then
2) finalize, possibly redact, and send.

The outcome of the first tier can still be useful for debugging or
internal auditing.  If the message gets accepted, the second tier has
to be able to retrieve the file.

> Is this MUST in 5965 being ignored or am mis-interpreting something?

2.d implies that only accepted or rejected-after-data messages can
actually be reported, while SPF rejections occur before data.  That
point could be overridden in spf-reporting or, if it's not too late,
in authailure-report.  A use case is for domains that need to collect
as many SPF *fail* reports as possible, e.g. to gather evidence on
which third parties habitually infringe on their registered name.

I'd be favorable to adding such possibility.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 13 Feb 2012 08:58:32 -0800

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of John Levine
> Sent: Monday, February 13, 2012 7:42 AM
> To: [email-masked]
> Subject: Re: [marf] Message bodies in ARF reports
> 
> Interesting question.
> 
> 5965, we made it mandatory in section 2.d.  We did that because the
> usage scenario we were thinking of was spam button reports where you
> always have a message in hand, not for any deeper reason.
> 

I'd support that.

I'd also support the idea that an SPF report can synthesize the text/rfc822-headers for a message whose content has not actually arrived.  All you legally need is a From: field as I recall, and that can be generated based on the rejected RFC5321.MailFrom.

---

From: Scott Kitterman <[email-masked]>
Date: Mon, 13 Feb 2012 14:26:47 -0500

On Monday, February 13, 2012 08:58:32 AM Murray S. Kucherawy wrote:
> > From: [email-masked] [mailto:[email-masked]] On Behalf Of
> > John Levine Sent: Monday, February 13, 2012 7:42 AM
> > To: [email-masked]

That's great.  Is MUST include if you have ... or SHOULD include (unless you 
do not have ...) a better construction?  I thought the trend in IETF documents 
was away from MUSTs if they weren't essential.

> I'd also support the idea that an SPF report can synthesize the
> text/rfc822-headers for a message whose content has not actually arrived. 
> All you legally need is a From: field as I recall, and that can be
> generated based on the rejected RFC5321.MailFrom.

If by 'synthesize' you mean 'assume the RFC5321.MailFrom and the FRC5322.From 
are the same', then yes, it could do that.  That's not the same as saying I 
think it's a good idea.  In cases where this heuristic is wrong (I've seen 
numbers between 5% and 20% of mail), it will probably be deeply confusing to 
receivers of such reports.  I don't think we should.

Scott K

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 14 Feb 2012 09:34:09 -0800

> From: [email-masked] [mailto:[email-masked]] On Behalf Of Scott Kitterman
> Sent: Monday, February 13, 2012 11:27 AM
> To: [email-masked]
> > > ARF is a version of multipart/report as defined in RFC 3462.  In
> > > that document, the message/rfc822 or message/headers is optional.
> > > In RFC 5965, we made it mandatory in section 2.d.  We did that
> > > because the usage scenario we were thinking of was spam button
> > > reports where you always have a message in hand, not for any deeper reason.
> > >
> > > You are of course correct that SPF failures sometimes don't accept
> > > the message.  I'd be willing to file an erratum that 2.d. should
> > > have said that the report MUST include the message if the reporting
> > > entity has received it.
> >
> > I'd support that.
> 
> That's great.  Is MUST include if you have ... or SHOULD include
> (unless you do not have ...) a better construction?  I thought the
> trend in IETF documents was away from MUSTs if they weren't essential.

I'd prefer "MUST ... unless" for this one.  I'm not sure I can explain why though.

This one's for Barry: Can a Standards Track document contradict another one on the basis of an accepted erratum?  Or should we do a quick update to RFC5965 (not a "bis", just a standards-track correction)?

> If by 'synthesize' you mean 'assume the RFC5321.MailFrom and the
> FRC5322.From are the same', then yes, it could do that.  That's not the
> same as saying I think it's a good idea.  In cases where this heuristic
> is wrong (I've seen numbers between 5% and 20% of mail), it will
> probably be deeply confusing to receivers of such reports.  I don't
> think we should.

Yeah, I realize it's not guaranteed to be right.  It's a question of what fits within the rules.  We could certainly say in the SPF reporting document that the synthesized third part is based on the envelope only and thus could be a misrepresentation of what was really in the (rejected) message.  Of course if the message failed SPF but was still delivered, the content is available anyway.

-MSK

---

From: Barry Leiba <[email-masked]>
Date: Tue, 14 Feb 2012 12:46:41 -0500

> This one's for Barry: Can a Standards Track document contradict another
> one on the basis of an accepted erratum?  Or should we do a quick update
> to RFC5965 (not a "bis", just a standards-track correction)?

The second (contradicting) S-T document can be the one to do the
"quick update" (you don't need a separate document to do it).  Just
make it "Updates 5965", include text in the introduction that says it
updates 5965 to change XYZ, and then in the normative text where you
make the contradiction, add a sentence that says "This updates the
requirement [or whatever] in RFC 5965."

b

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 14 Feb 2012 18:59:32 +0100

On 14/Feb/12 18:34, Murray S. Kucherawy wrote:
>> From: ietf.org On Behalf Of Scott Kitterman
>> 

We could get away with "... unless an extension specifies otherwise".

> This one's for Barry: Can a Standards Track document contradict
> another one on the basis of an accepted erratum?  Or should we do a
> quick update to RFC5965 (not a "bis", just a standards-track
> correction)?

IMHO we have to change authfailure-report anyway.  Section 3.1 says:

 The third MIME part of the message is either of type "message/rfc822"
 (as defined in [MIME-TYPES]) or "text/rfc822-headers" (as defined in
 [REPORT]) and contains a copy of the entire header block from the
 original message.  This part MUST be included (contrary to [REPORT],
 which makes it optional).

jm2c

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 14 Feb 2012 10:10:20 -0800

> From: [email-masked] [mailto:[email-masked]] On Behalf Of Alessandro Vesely
> Sent: Tuesday, February 14, 2012 10:00 AM
> To: [email-masked]

The synthesis option handles this, if we want to go that way.

---

From: "John Levine" <[email-masked]>
Date: 14 Feb 2012 18:32:46 -0000

> generated based on the rejected RFC5321.MailFrom.

I think that would cause far more confusion than enlightenment.  The
kind of analysis I do on ARFs looks for specific quirks of the headers
that my software adds.  If it doesn't find them, it'll discard the
whole report as bogus.  Better to send no headers than fake ones.

Since it's fine for any other multipart/report to omit the copy of the
message, seems to me a lot tidier just to fix 5965 to be consistent.

Based on my prior experience with errata, it's OK to use the corrected
version of the doc as a base for future work without recycling.  In
RFC 5518 we made a fairly bad mistake, saying that you use the i= rather
than d= in a DKIM result.  We fixed that with an erratum, seems to be OK.

R's,
John

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 14 Feb 2012 19:55:27 +0100

On 14/Feb/12 19:10, Murray S. Kucherawy wrote:
>> From: ietf.org On Behalf Of Alessandro Vesely
>

Which "synthesis option"?

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 14 Feb 2012 11:12:42 -0800

> Sent: Tuesday, February 14, 2012 10:55 AM
> To: [email-masked]

http://www.ietf.org/mail-archive/web/marf/current/msg02109.html

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 14 Feb 2012 11:14:09 -0800

> From: John Levine [mailto:[email-masked]]
> Sent: Tuesday, February 14, 2012 10:33 AM
> To: [email-masked]
> Cc: Murray S. Kucherawy
> version of the doc as a base for future work without recycling.  In RFC
> 5518 we made a fairly bad mistake, saying that you use the i= rather

So do we need to say "Updates RFC5965" in Scott's draft, perhaps pointing at the erratum in an appendix or something, just to tie it all together?

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 15 Feb 2012 13:39:38 +0100

> http://www.ietf.org/mail-archive/web/marf/current/msg02109.html

Ah, that.  I'd prefer to avoid it...

I foolishly though there was a possibility to apply an update (or
errata) recursively to all RFCs derived from an updated (corrected)
format.

Probably, it is an error of authfailure-report to have the paragraph
quoted above --almost equal to the 5965's text that John corrected
yesterday.  Is it necessary to have it published before correcting it?

---

From: Shmuel (Seymour J.) Metz <[email-masked]>
Date: Wed, 15 Feb 2012 12:15:59 -0500

In
<[email-masked]>,
on 02/14/2012
   at 11:14 AM, "Murray S. Kucherawy" <[email-masked]> said:

>So do we need to say "Updates RFC5965" in Scott's draft, perhaps
>pointing at the erratum in an appendix or something, just to tie it
>all together?

That's my take. Also, is failure to respond to an erratum taken as
concurrence or non-concurrence?

-- 
     Shmuel (Seymour J.) Metz, SysProg and JOAT
     Atid/2        <http://patriot.net/~shmuel>
We don't care. We don't have to care, we're Congress.
(S877: The Shut up and Eat Your spam act of 2003)

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 15 Feb 2012 10:44:57 -0800

> From: [email-masked] [mailto:[email-masked]] On Behalf Of Shmuel Metz
> Sent: Wednesday, February 15, 2012 9:16 AM
> To: [email-masked]

I think only a verified erratum has any effect, although the effect only matters if one knows to go looking for it.

The real way to get the erratum implemented is a -bis effort, but I'm not sure that would be wise for RFC5965 just now.

-MSK