Subject: [apps-discuss] I-D Action: draft-ietf-appsawg-greylisting-01.txt

From: [email-masked]
Date: Fri, 20 Jan 2012 01:21:32 -0800

A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the Applications Area Working Group Working Group of the IETF.

	Title           : Best Current Practices for Email Greylisting
	Author(s)       : Murray S. Kucherawy
	Filename        : draft-ietf-appsawg-greylisting-01.txt
	Pages           : 10
	Date            : 2012-01-20

   This memo describes best current practices for the art of email
   greylisting, the practice of providing temporarily degraded service
   to unknown email clients as an anti-abuse mechanism.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-appsawg-greylisting-01.txt

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

This Internet-Draft can be retrieved at:
ftp://ftp.ietf.org/internet-drafts/draft-ietf-appsawg-greylisting-01.txt

---

From: "Paul E. Jones" <[email-masked]>
Date: Fri, 20 Jan 2012 13:55:04 -0500

Mark,

I did seen this previously, but it is interesting to see it appear
nonetheless.

I am one of those who implement greylisting.  I have experienced the
negative aspects you describe in the document.  Perhaps the most frustrating
is that I have found two service providers that do not properly deal with
greylisting, or perhaps I don't deal with them properly.  These service
providers apparently have a fairly large array of mail servers and they
appear to select an IP address randomly when they re-try.  On my server,
that just imposes more delay. In some cases, messages do not get through
until I whitelist their server address ranges.

Quickly grabbing some statistic on my server for this week, the different
approaches have blocked the following percentages of mail:

Greylisting 16%
SPF         38%
Spamhaus    40%
RDNS         4%
Own RBL      2%

These numbers are not exact. I would estimate a margin of error of not more
than 1% on greylisting, SPF, and Spamhaus.

Our own RBL blocks very little right now, as I changed the algorithm to be a
bit more forgiving.  It now only blocks those who tend to be serious "repeat
offenders".

I suspect if I turn off greylisting, Spamhaus and SPF will likely catch the
bulk of the spam blocked by greylisting.  What these two services do is
catch senders who are "out of policy" and who are known spammers.  I hate
being forced to blacklist domains, IP addresses, etc., but it is extremely
effective and has proven to be necessary.

I think it's useful to document greylisting for information purposes, but do
we want to make it a BCP?  The reason I ask is that longer-term, I would
think we'd want to recommend policy-oriented solutions (e.g., SPF or DKIM).
If policies were strictly enforced and properly implemented, machines
controlled by bots would not get past the policy enforcement.

I also question whether greylisting would be effective long-term.  I
actually have seen some machines re-send email.  You are right that many
just "fire and forget", but that would change if required.  So, elevating
greylisting to BCP might just force a change in spammer tactics, thus
rendering greylisting completely ineffective.

The reason I am supportive of documenting as informational is that servers
should consider such implementations and that it has utility today.  But,
calling it a "best" practice seems to be a bit much.  It's a practice driven
out of necessity, mainly because we do not have all of the kinks worked out
of policy-based solutions.

Paul

> -----Original Message-----
> From: [email-masked] [mailto:apps-discuss-
> [email-masked]] On Behalf Of [email-masked]
> Sent: Friday, January 20, 2012 4:22 AM
> To: [email-masked]
> Cc: [email-masked]
> Subject: [apps-discuss] I-D Action: draft-ietf-appsawg-greylisting-
> 01.txt
> 
> 
> A New Internet-Draft is available from the on-line Internet-Drafts
> directories. This draft is a work item of the Applications Area Working
> Group Working Group of the IETF.
> 
> _______________________________________________
> apps-discuss mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/apps-discuss

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 20 Jan 2012 11:16:06 -0800

Hi Paul,

> From: Paul E. Jones [mailto:[email-masked]]
> Sent: Friday, January 20, 2012 10:55 AM
> To: Murray S. Kucherawy
> Subject: RE: [apps-discuss] I-D Action: draft-ietf-appsawg-greylisting-01.txt
> 
> Mark,

Who?  ;-)

> I think it's useful to document greylisting for information purposes,
> but do we want to make it a BCP?  The reason I ask is that longer-term,
> I would think we'd want to recommend policy-oriented solutions (e.g., SPF or DKIM).
> actually have seen some machines re-send email.  You are right that
> many just "fire and forget", but that would change if required.  So,
> elevating greylisting to BCP might just force a change in spammer
> tactics, thus rendering greylisting completely ineffective.
> 
> The reason I am supportive of documenting as informational is that
> servers should consider such implementations and that it has utility
> today.  But, calling it a "best" practice seems to be a bit much.  It's
> a practice driven out of necessity, mainly because we do not have all
> of the kinks worked out of policy-based solutions.

The original (and current) plan is to include some consensus recommended practices in terms of greylisting.  If the Working Group decides it wants to stop short of making any recommendations, then changing to Informational is entirely appropriate.  I don't have a particular preference.

The -01 version of this draft fills out the definitions of each kind of greylisting we know of and presents some background (thanks to John Levine for the text there), but I didn't fill out the recommendations section because we haven't talked about what we would like to say in there yet.

There is a small team of people within MAAWG that want to make some recommendations about greylisting, so the output of that work might be useful here.  They meet in about a month, so I'll report back after that, and we can figure out what to do next.

-MSK

---

From: "John Levine" <[email-masked]>
Date: 21 Jan 2012 04:44:37 -0000

Based on my experience, here's some advice for how to make greylisting
work.  My goal is to use it to recognize broken mail clients that
don't retry.  Once a client has shown that it knows how to retry,
there is no further benefit from greylisting it.  (It may be sending
spam, but you'll have to recognize it other ways.)

* Greylist on the triple [IP, MAIL FROM, RCPT TO].  After a successful
retry, whitelist all further mail from the IP.  I don't feel strongly
whether you should put just one RCPT TO in the triple or wait for DATA
and put all of them.

* The greylister should have a time window within which retries will
be accepted.  On mine, the minimum is one minute, the maximum is 24
hours.  I think those are reasonable.  You want a minimum because some
spamware retries immediately, you want a maximum both to keep the
retry table from filling up with junk and so two spams sent a week
apart aren't interpreted as a retry.  Some people set their minimum
much longer, several hours, but I don't see any benefit to that unless
you believe in the blacklist theory.

* Whitelist entries should time out eventually.  I delete them when
there's been no mail from the IP for a week.

* If you have multiple incoming servers at the same MX distance, they
should share a single greylist database.  This handles the case where
the retry goes to a different server, and lets all the servers share
the IP whitelist.  I do this by making the greylister a standalone
daemon, and use a very simple UDP protocol to query it.  If you're
Google or Yahoo and have multiple server pools all over the world,
I suppose a database per pool would do.

* Since some senders have multiple outgoing servers, allow approximate
matches on the retry IP.  I take anything in the same /24 which seems
to work OK.  I only whitelist the IP that retries, but I should
probably whitelist both.

* The greylister MUST have a manual whitelist.  Some legit senders
just don't work with greylisting for a variety of reasons, most of
which are not in conflict with RFC 5321.

R's,
John

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 26 Jan 2012 16:59:44 -0800

Thanks for this, John.  I'm working them into the next version.

Among the notes I made for possible topics, I have one outstanding question: Should a greylister return a 421 reply and drop the connection when enforcing the rule, or some other 4yz code?  Or does it matter?

-MSK