From asrg-bounces@ietf.org Sat Apr 01 05:25:17 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FPdIB-0004rI-8B; Sat, 01 Apr 2006 05:25:15 -0500
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FPdH9-0002is-0L
	for asrg@ietf.org; Sat, 01 Apr 2006 05:24:11 -0500
Received: from mpls-qmqp-03.inet.qwest.net ([63.231.195.114])
	by ietf-mx.ietf.org with smtp (Exim 4.43) id 1FPdH7-0001EY-KP
	for asrg@ietf.org; Sat, 01 Apr 2006 05:24:10 -0500
Received: (qmail 3753 invoked by uid 0); 1 Apr 2006 10:24:09 -0000
Received: from unknown (63.231.195.8)
	by mpls-qmqp-03.inet.qwest.net with QMQP; 1 Apr 2006 10:24:09 -0000
Received: from 71-212-183-171.hlrn.qwest.net (HELO ?192.168.0.3?)
	(71.212.183.171)
	by mpls-pop-08.inet.qwest.net with SMTP; 1 Apr 2006 10:24:09 -0000
Date: Sat, 1 Apr 2006 03:24:00 -0700
Message-Id: <4D0E9449-AD81-4D29-BB6B-CA8B301B06C2@qwest.net>
From: "Dan Oetting" <dan_oetting@qwest.net>
To: "Anti Spam Research Group" <asrg@ietf.org>
Mime-Version: 1.0 (Apple Message framework v746.3)
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed
X-Mailer: Apple Mail (2.746.3)
X-Spam-Score: 0.0 (/)
X-Scan-Signature: 93238566e09e6e262849b4f805833007
Subject: [Asrg] Building a better blacklist
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

Why are the performance figures for blacklists so low? I saw someone  
post a figure of 80% blocking. While less than 50% of the spam I get  
in my unfiltered accounts would have been blocked by SBL+XBL. Why  
can't the blocking rate be in the high 90's? I realize that it's not  
possible to hit 100% blocking unless every mailbox were a spamtrap  
feeding the blacklist. But with a significantly large trap farm it  
should be possible to detect almost all spam sources within a  
reasonable time.

The other big issue is why don't more ISPs use blacklists? The main  
concern I suppose is that they can't afford to loose mail sent to  
their customers. To address this, a blacklist systems could be  
designed to recover automatically when the spam stops. If the mail is  
rejected with a 4xx response code the non-spam mail from legitimate  
ISPs would be delivered (only slightly delayed) once the spam is  
cleaned up.

That leads to the question of how to clean up the spam in real time.  
Since the spam traps have already captured samples of the spam  
emanating from the blocked ISP, it should be easy enough to construct  
a profile or signature of the spam that the source ISP could use to  
quarantine the remaining spam in the queue.

-- Dan Oetting

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Sat Apr 01 11:33:37 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FPj2C-0006gm-79; Sat, 01 Apr 2006 11:33:08 -0500
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FPj2A-0006gh-B8
	for asrg@ietf.org; Sat, 01 Apr 2006 11:33:06 -0500
Received: from zcars04f.nortel.com ([47.129.242.57])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FPj29-0006gM-0X
	for asrg@ietf.org; Sat, 01 Apr 2006 11:33:06 -0500
Received: from zcars2ky.ca.nortel.com (zcars2ky.ca.nortel.com [47.129.242.221])
	by zcars04f.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id
	k31GX2B20331
	for <asrg@ietf.org>; Sat, 1 Apr 2006 11:33:02 -0500 (EST)
Received: from [47.130.24.162] (acart17a.ca.nortel.com [47.130.24.162])
	by zcars2ky.ca.nortel.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id
	k31Gb4w10948
	for <asrg@ietf.org>; Sat, 1 Apr 2006 11:37:05 -0500 (EST)
Message-ID: <442EAB38.6060906@americasm01.nt.com>
Date: Sat, 01 Apr 2006 11:32:56 -0500
From: "Chris Lewis" <clewis@nortel.com>
User-Agent: Mozilla Thunderbird 1.0.6 (Windows/20050716)
X-Accept-Language: en-us, en
MIME-Version: 1.0
CC: Anti Spam Research Group <asrg@ietf.org>
Subject: Re: [Asrg] Building a better blacklist
References: <4D0E9449-AD81-4D29-BB6B-CA8B301B06C2@qwest.net>
In-Reply-To: <4D0E9449-AD81-4D29-BB6B-CA8B301B06C2@qwest.net>
X-Enigmail-Version: 0.93.0.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-Spam-Score: 0.0 (/)
X-Scan-Signature: c3a18ef96977fc9bcc21a621cbf1174b
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Dan Oetting wrote:
> Why are the performance figures for blacklists so low? I saw someone 
> post a figure of 80% blocking. While less than 50% of the spam I get  in
> my unfiltered accounts would have been blocked by SBL+XBL. Why  can't
> the blocking rate be in the high 90's?

Because, quite simply, a significant amount of spam comes from "mixed
sources", such as ISP mail servers, so, unless you're willing to put up
with very high FPs, DNSBLs are simply not suitable for _that_ segment of
spam.

Effective spam fighting requires a hybrid of techniques.  DNSBLs are for
when the IP clearly shows that it's compromised or "malicious" in some
fashion, and accepting _anything_ from it is a bad idea, hence blocking
everything from it results in zero false positives (from that source).
Other filters are suitable when that's too blunt an instrument.

Hotmail's webmail servers were one of the biggest individual sources of
spam on the Internet (95%+ advance fee fraud at one point).  Unless
you're willing to nuke all of their email, DNSBLs are the wrong instrument.

[I'll point out that until a few weeks ago, that's exactly what we had
to do - block about 1/3 of all of hotmail's active webmail servers...
We're still 100% blocking tin.it's mail servers.]

> I realize that it's not  possible
> to hit 100% blocking unless every mailbox were a spamtrap  feeding the
> blacklist. But with a significantly large trap farm it  should be
> possible to detect almost all spam sources within a  reasonable time.

I work with VERY large spamtraps.  The one I personally operate (6-7
million emails/day) is perhaps less than 1/100th of that.

The intelligence this derives is _exceedlingly_ useful in catching the
majority of spam that is amenible to DNSBLs. But the nature of the spam
beast is that _some_ of it isn't amenable to DNSBLs.  Unless you have a
far higher tolerance for FPs than most.

> The other big issue is why don't more ISPs use blacklists?

A lot more ISPs and anti-spam vendors are using DNSBLs than you think.
Most of them simply don't tell you that they are.  In many cases, the
DNSBLs aren't being used for definitive yes/no blocking, they're used in
conjunction with scoring systems.

> The main 
> concern I suppose is that they can't afford to loose mail sent to  their
> customers. To address this, a blacklist systems could be  designed to
> recover automatically when the spam stops.

Many DNSBLs already do this, virtually all of the reputable ones do.

> If the mail is  rejected with
> a 4xx response code the non-spam mail from legitimate  ISPs would be
> delivered (only slightly delayed) once the spam is  cleaned up.

That's presuming that "spam is cleaned up" in a sufficiently timely
fashion to fall within retry limits.  It seldom is.  Indeed, most spam
simply doesn't retry, so, issuing a 4xx response the first time  you see
a particular spam is a highly useful technique.  It's called greylisting.

> That leads to the question of how to clean up the spam in real time. 
> Since the spam traps have already captured samples of the spam 
> emanating from the blocked ISP, it should be easy enough to construct  a
> profile or signature of the spam that the source ISP could use to 
> quarantine the remaining spam in the queue.

Some ISPs have done this in the past, however, it's seldom an effective
or generalized enough solution.  Furthermore, most spam _never_ sits in
queues, because it's not coming from mail servers.

The key to blocking a significant amount of spam is identifying whether
the thing sending the email to you really is a mail server.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3-nr1 (Windows 2000)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iQCVAwUBRC6rOJ3FmCyJjHfhAQJAvgP7Bnqsx5yluJ+7NBFbghD/T5VB3O9xXDVv
lqYrbzKpgfbT5gnybOOhVHOMw4LB6umB8/yUclfDNse2jeBjzEtgq/9Wsdbw+uz+
1qzCe1b3tE/j6uMwAreAomuXAcoAJPz1TW5lTXw9a84uXdGa2o3bvE06HCBU1tRc
4kO5AnDS5nE=
=x7L+
-----END PGP SIGNATURE-----

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Sat Apr 01 15:53:20 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FPn5p-0005Vt-1H; Sat, 01 Apr 2006 15:53:09 -0500
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FPn5n-0005Vo-ST
	for asrg@ietf.org; Sat, 01 Apr 2006 15:53:07 -0500
Received: from zcars04f.nortel.com ([47.129.242.57])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FPn5m-00080J-GL
	for asrg@ietf.org; Sat, 01 Apr 2006 15:53:07 -0500
Received: from zcars2ky.ca.nortel.com (zcars2ky.ca.nortel.com [47.129.242.221])
	by zcars04f.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id
	k31Kr4B10696
	for <asrg@ietf.org>; Sat, 1 Apr 2006 15:53:04 -0500 (EST)
Received: from [47.130.24.162] (acart17a.ca.nortel.com [47.130.24.162])
	by zcars2ky.ca.nortel.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id
	k31Kuaw25628
	for <asrg@ietf.org>; Sat, 1 Apr 2006 15:56:36 -0500 (EST)
Message-ID: <442EE80C.5080500@americasm01.nt.com>
Date: Sat, 01 Apr 2006 15:52:28 -0500
From: "Chris Lewis" <clewis@nortel.com>
User-Agent: Mozilla Thunderbird 1.0.6 (Windows/20050716)
X-Accept-Language: en-us, en
MIME-Version: 1.0
CC: asrg@ietf.org
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
References: <88885AD16AAA9F4899BB9FB751D4F5A5B70919@phost016.habeas.intermedia.net>
	<E1FP1vs-0008WV-00@chiark.greenend.org.uk>
	<20060331005839.GB6958@ender> <442D3368.7040202@americasm01.nt.com>
	<20060331234310.GA9179@ender> <442DCB75.7050409@americasm01.nt.com>
	<20060401021616.GA9651@ender>
In-Reply-To: <20060401021616.GA9651@ender>
X-Enigmail-Version: 0.93.0.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-Spam-Score: 0.0 (/)
X-Scan-Signature: 2086112c730e13d5955355df27e3074b
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Laird Breyer wrote:
> On Mar 31 2006, Chris Lewis wrote:
> 
> 
>>>whatever the incentive. So claiming incentive as an argument for the
>>>success of dnsbls won't (and really shouldn't) sway standards committees.
>>
>>You're missing the intent.  The intent is to show that our FP statistics
>>have pretty strong validity, full stop.  In that, it should sway those
>>wanting such statistics as proof/disproof of the success of DNSBLs.
> 
> 
> But in terms of arguments, a generic success is canceled out by a generic
> failure.  You need to qualify your success claim with a domain of applicability

I'm a lot closer to showing a generic success, than Church is of even a
limited scope failure.

He's reporting for a spam filter responsible for handling at most a
handful of users.  I have 65,000, and handle 500 times as much email per
_day_ as his (I believe) multiple day dataset.  Sites handling 1000
times as much email as I do are ALSO using DNSBLs.

I also have far more stringent requirements regarding FPs than most,
because a single lost email could cost us millions (of my employer's) money.

We do this for a living.  I've been living in this dataset for over 10
years.  As have some of the bigger environments, who've also found
DNSBLs to be successful.

The fact that we even have to answer to such nonsense, instead of
summarily throwing that in the trash is an insult to professionals.

But we do it anyway, because we're professionals.

> which covers your users but is narrow enough to not cover cases where
> others might legitimately claim failure.

I don't presuppose to cover vanity or hobby domains.  Which is pretty
clearly what Church is talking about.

> The author of the draft isn't the only one who claims to have been
> bitten by dnsbls, there are plenty of rants on the net, e.g.
> http://paulgraham.com/spamhausblacklist.html

Paul chose to deliberately misrepresent the situation for political
ends.  As did Moveon.  As did Gilmore.  And at least _two_ of those
listings were legitimate spam blocks. There are similarly plenty of
issues about other types of filtering goofing up.

Come guys, _no_ filter is perfect.  You can't simply pick the horror
story about the filtering technique you don't like, and ignore the rest.

> While that doesn't make their claims of failure statistically
> significant, it means that your counterclaim of strong success of
> dnsbls will necessarily be treated very skeptically, unless you can
> explain where in the email usage pattern your results apply, ie how
> can your claim be true and also the church claim be true simultaneously?

Church has a personal axe to grind, his usage patterns/user community
are unknown, his "preferred" filter is only described in terms of
unverifiable handwaving - no numbers, and is of a size that can best be
described as statistically irrelevant.

That should be enough.

> What's a plausible distinguishing feature? Quality of blacklist, 
> type of user, ... ?

Yes.  And others.

>>It simply isn't possible to generate statistical accuracy in this field.
>> Spam/Ham collections large enough to be useful can't be generated
>>timely enough to give accurate measures of real-time reputation systems.

> I'm not convinced it can't be. 

Okay, suggest how a spam/ham collection can be used to measure the
effectiveness of the following techniques that are, or can be, used in
an anti-spam solution:

1) grey listing
2) sender/sender domain verification
3) Challenge/response
4) SPF and DKIM
5) PKI
6) CSV
7) Non-existant users
8) DCC or other distributed checksumming methodologies.

Similarly DNSBLs.

> The NIST TREC people
> (http://plg.uwaterloo.ca/~gvcormac/spam/) are interested in this very
> kind of problem, so it's at least "on the radar", whether it will
> ultimately be successful or not.

They've been at that for rather a long time.  They should be doing a
reasonably good job for a specific class of filtering technology.

But it's not _spam_ filtering technology.

_Anything_ that uses "adjudicators" to determine whether an email is
spam or ham is clearly missing an important clue: spam isn't about
content, it's about _consent_.  It is a behaviour (sending without
consent), NOT what is in the email.

DNSBLs that can, for example, detect compromised machines or illegal
methods being used to send the email have a better handle on intent,
than an "adjudicator" could possibly have.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3-nr1 (Windows 2000)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iQCVAwUBRC7oDJ3FmCyJjHfhAQIb5wP/aknRrWADXG0eC27TfP2Z2Mz/+Fxy5GLV
y21XYvu6hQoSSPY3vdKfDmumcLHMRKCG+xMT/VmJgb9jy1WtES9M3LDmQMws+jcz
8sdM9Ef/cUC3ZrtIplBV2ugtfNEbWvq5ZzMbjDwR7R+aSlJxKqInrTeYvlniP7CE
BM7/6XQjJ2Q=
=FdiU
-----END PGP SIGNATURE-----

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Sat Apr 01 17:21:23 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FPoT5-0007ty-Oa; Sat, 01 Apr 2006 17:21:15 -0500
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FPoT4-0007tt-SP
	for asrg@ietf.org; Sat, 01 Apr 2006 17:21:14 -0500
Received: from vms044pub.verizon.net ([206.46.252.44])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FPoT0-0003EH-Io
	for asrg@ietf.org; Sat, 01 Apr 2006 17:21:14 -0500
Received: from torch.brasslantern.com ([71.116.81.225])
	by vms044.mailsrvcs.net (Sun Java System Messaging Server 6.2-4.02
	(built Sep
	9 2005)) with ESMTPA id <0IX200FWRE35BI80@vms044.mailsrvcs.net> for
	asrg@ietf.org; Sat, 01 Apr 2006 16:21:06 -0600 (CST)
Received: from torch.brasslantern.com (localhost.localdomain [127.0.0.1])
	by torch.brasslantern.com (8.13.1/8.13.1) with ESMTP id
	k31ML4qO009349	for <asrg@ietf.org>; Sat, 01 Apr 2006 14:21:05 -0800
Received: (from schaefer@localhost)	by torch.brasslantern.com
	(8.13.1/8.13.1/Submit) id k31ML4Jp009348	for asrg@ietf.org; Sat,
	01 Apr 2006 14:21:04 -0800
Date: Sat, 01 Apr 2006 14:21:04 -0800
From: Bart Schaefer <schaefer@brasslantern.com>
Subject: Re: [Asrg] Walter's comments on draft-church-dnsbl-harmful-01.txt
In-reply-to: <20060401015927.GA27452@waltdnes.org>
To: ASRG list <asrg@ietf.org>
Message-id: <060401142104.ZM9347@torch.brasslantern.com>
MIME-version: 1.0
X-Mailer: OpenZMail Classic (0.9.2 24April2005)
Content-type: text/plain; charset=us-ascii
References: <20060401015927.GA27452@waltdnes.org>
Comments: In reply to "Walter Dnes" <waltdnes@waltdnes.org>
	"[Asrg] Walter's comments on draft-church-dnsbl-harmful-01.txt" (Mar 31,
	8:59pm)
X-Spam-Score: 0.1 (/)
X-Scan-Signature: 39bd8f8cbb76cae18b7e23f7cf6b2b9f
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

On Mar 31,  8:59pm, Walter Dnes wrote:
}
}   2) DNSBLs don't block anything, just as restaurant reviewers don't
} pay goons to stand at the entrances of certain restaurants and attack
} would-be patrons with basball bats.  Just like restaurant reviewers,
} DNSBLs publish opinions about certain establishments.  System
} administrators are free to accept or not accept those opinions.

I've never liked this particular flavor of analogy for DNSBLs.  For one
thing, it's backwards.  Independent of who hired the goon, his job is
not to keep people out of the establishment that was reviewed -- it's to
identify those associated with that establishment in order to keep them
out of of *another* establishment.

Some DNSBLs publish opinions ("based on Y, we believe X is a source of
spam"); some publish verifiable facts ("X is an open relay"; "X is in an
IP range allocated to Z"); some publish unverifiable claims of fact ("a
spamtrap received mail from X").  Some publish specific information, and
some publish broad generalities.

Most complaints are about DNSBLs that publish generalities, and involve
individuals who claim either that they don't fit the profile, or that
the profiling itself is invalid; not about "certain establishments" who
claim they've specifically been given a "bad review."

Here's an example of what I consider a more accurate analogy:  It's a
common practice for auto insurance companies to charge different rates
depending on the ZIP code where you live.  If a DNSBL publishing lists
of dynamic IPs is like a ZIP code directory, then the sysadmins who use
it are like auto insurers.

If you want a restaurant analogy:  It's like a restaurant hiring a goon
and giving him a yearbook so he can turn away all local high-schoolers,
because a couple of them once vandalized a restroom.

It's not inherently harmful to publish a ZIP code directory or a high
school yearbook.  It's all a matter of what's done with the information.

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Sat Apr 01 18:15:23 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FPpJI-0005O9-UF; Sat, 01 Apr 2006 18:15:12 -0500
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FPpJI-0005O4-9H
	for asrg@ietf.org; Sat, 01 Apr 2006 18:15:12 -0500
Received: from mail2.panix.com ([166.84.1.73])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FPpJH-00058Z-3Z
	for asrg@ietf.org; Sat, 01 Apr 2006 18:15:12 -0500
Received: from panix1.panix.com (panix1.panix.com [166.84.1.1])
	by mail2.panix.com (Postfix) with ESMTP id DB2789F1F3
	for <asrg@ietf.org>; Sat,  1 Apr 2006 18:15:04 -0500 (EST)
Received: (from sethb@localhost)
	by panix1.panix.com (8.11.6p3/8.8.8/PanixN1.1) id k31NF4V19502;
	Sat, 1 Apr 2006 18:15:04 -0500 (EST)
Date: Sat, 1 Apr 2006 18:15:04 -0500 (EST)
Message-Id: <200604012315.k31NF4V19502@panix1.panix.com>
From: Seth Breidbart <sethb@panix.com>
To: asrg@ietf.org
In-reply-to: <060401142104.ZM9347@torch.brasslantern.com> (message from Bart
	Schaefer on Sat, 01 Apr 2006 14:21:04 -0800)
Subject: Re: [Asrg] Walter's comments on draft-church-dnsbl-harmful-01.txt
References: <20060401015927.GA27452@waltdnes.org>
	<060401142104.ZM9347@torch.brasslantern.com>
X-Spam-Score: 0.0 (/)
X-Scan-Signature: 856eb5f76e7a34990d1d457d8e8e5b7f
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

Bart Schaefer <schaefer@brasslantern.com> wrote:

> If you want a restaurant analogy: It's like a restaurant hiring a
> goon and giving him a yearbook so he can turn away all local
> high-schoolers, because a couple of them once vandalized a restroom.

What is the "it" you're referring to?

The DNSBL publisher is like the yearbook publisher in that example; it
just publishes data.

The restaurant owner is like the mailswerver owner.

One key difference in the US is that the mailserver owners is
_specifically_ permitted by law to take whatever action _he_ thinks is
appropriate to block unwanted email.

Seth

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Sat Apr 01 19:33:44 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FPqXA-0002zq-NL; Sat, 01 Apr 2006 19:33:36 -0500
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FPqXA-0002zJ-4n
	for asrg@ietf.org; Sat, 01 Apr 2006 19:33:36 -0500
Received: from zcars04f.nortel.com ([47.129.242.57])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FPqX8-0008VI-Rc
	for asrg@ietf.org; Sat, 01 Apr 2006 19:33:36 -0500
Received: from zcars2ky.ca.nortel.com (zcars2ky.ca.nortel.com [47.129.242.221])
	by zcars04f.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id
	k320XWg28226
	for <asrg@ietf.org>; Sat, 1 Apr 2006 19:33:32 -0500 (EST)
Received: from [47.130.16.140] (acart04g.ca.nortel.com [47.130.16.140])
	by zcars2ky.ca.nortel.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id
	k320bJC01692
	for <asrg@ietf.org>; Sat, 1 Apr 2006 19:37:20 -0500 (EST)
Message-ID: <442F1BC7.9010904@americasm01.nt.com>
Date: Sat, 01 Apr 2006 19:33:11 -0500
From: "Chris Lewis" <clewis@nortel.com>
User-Agent: Mozilla Thunderbird 1.0.6 (Windows/20050716)
X-Accept-Language: en-us, en
MIME-Version: 1.0
CC: ASRG list <asrg@ietf.org>
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
References: <88885AD16AAA9F4899BB9FB751D4F5A5B70919@phost016.habeas.intermedia.net>	
	<E1FP1vs-0008WV-00@chiark.greenend.org.uk>	
	<20060331005839.GB6958@ender>
	<442D3368.7040202@americasm01.nt.com>	
	<20060331234310.GA9179@ender>
	<442DCB75.7050409@americasm01.nt.com>	
	<20060401021616.GA9651@ender> <442EE80C.5080500@americasm01.nt.com>
	<6c399e450604011354ub614dcbu109f5ff6c172e39e@mail.gmail.com>
In-Reply-To: <6c399e450604011354ub614dcbu109f5ff6c172e39e@mail.gmail.com>
X-Enigmail-Version: 0.93.0.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-Spam-Score: 0.0 (/)
X-Scan-Signature: 82c9bddb247d9ba4471160a9a865a5f3
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Justin Mason wrote:
> Chris said:
> 
>>Okay, suggest how a spam/ham collection can be used to measure the
>>effectiveness of the following techniques that are, or can be, used in
>>an anti-spam solution:
>>
>>1) grey listing
>>2) sender/sender domain verification
>>3) Challenge/response
>>4) SPF and DKIM
>>5) PKI
>>6) CSV
>>7) Non-existant users
>>8) DCC or other distributed checksumming methodologies.
>>
>>Similarly DNSBLs.
> 
> 
> The effectiveness of all of these, except possibly for greylisting and
> C/R, can be measured accurately. We are doing it in SpamAssassin ;) --
> here's how.
> 
> First, you accept every message, and record the "real-time" data
> points regarding how the message was listed against those services,
> and/or how it *would* have been rejected at SMTP transaction time (if
> at all). However, you don't reject, you accept everything.
> 
> Then, later, provide a way for hand-sorting to take place, and compare
> the results of the hand-sorting with what the various other techniques
> would have done with those messages.
> 
> Hey presto, you've now got a way to compare accuracy and effectiveness
> of those techniques!  Simple as that.

Do you see any provision for that with _any_ of the spam/ham
collections?  Can you do that with a pre-existing spam/ham collection if
the technique you were trying to test _wasn't_ being collected at the
time the spam/ham collection was being made?

This points up my very point: in order to do a proper sampling of the
effectiveness of a technique is that you do it in real time, at the time
the emails were sent.  In other words, on real mail streams.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3-nr1 (Windows 2000)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iQCVAwUBRC8bx53FmCyJjHfhAQL1cgP/eFRHoOrKLE4QKfHIDAmyRMpx3+CDEzKD
J3wBSECeOy2NdgIEO42ivKiATmWCLP1sqLLCV5bCeEiZ2vLVyua1at3n28ProgUE
f8kUxawXbk8phRFbklBGk3YxXskrQkKsCb/bj+OzV840j3eVeR8xrpS3iAP2+tYz
jqaLUvcnoo0=
=hlXg
-----END PGP SIGNATURE-----

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Sun Apr 02 00:14:36 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FPuv0-0002OQ-Fs; Sun, 02 Apr 2006 00:14:30 -0500
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FPuuz-0002OL-H7
	for asrg@ietf.org; Sun, 02 Apr 2006 00:14:29 -0500
Received: from vms048pub.verizon.net ([206.46.252.48])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FPuux-0004CZ-8f
	for asrg@ietf.org; Sun, 02 Apr 2006 00:14:29 -0500
Received: from torch.brasslantern.com ([71.116.81.225])
	by vms048.mailsrvcs.net (Sun Java System Messaging Server 6.2-4.02
	(built Sep
	9 2005)) with ESMTPA id <0IX2000FGX81YYM7@vms048.mailsrvcs.net> for
	asrg@ietf.org; Sat, 01 Apr 2006 23:14:26 -0600 (CST)
Received: from torch.brasslantern.com (localhost.localdomain [127.0.0.1])
	by torch.brasslantern.com (8.13.1/8.13.1) with ESMTP id
	k325EGuv009547	for <asrg@ietf.org>; Sat, 01 Apr 2006 21:14:25 -0800
Received: (from schaefer@localhost)	by torch.brasslantern.com
	(8.13.1/8.13.1/Submit) id k325EGMY009546	for asrg@ietf.org; Sat,
	01 Apr 2006 21:14:16 -0800
Date: Sat, 01 Apr 2006 21:14:16 -0800
From: Bart Schaefer <schaefer@brasslantern.com>
Subject: Re: [Asrg] Walter's comments on draft-church-dnsbl-harmful-01.txt
In-reply-to: <200604012315.k31NF4V19502@panix1.panix.com>
To: asrg@ietf.org
Message-id: <060401211416.ZM9545@torch.brasslantern.com>
MIME-version: 1.0
X-Mailer: OpenZMail Classic (0.9.2 24April2005)
Content-type: text/plain; charset=us-ascii
References: <20060401015927.GA27452@waltdnes.org>
	<060401142104.ZM9347@torch.brasslantern.com>
	<200604012315.k31NF4V19502@panix1.panix.com>
Comments: In reply to Seth Breidbart <sethb@panix.com>
	"Re: [Asrg] Walter's comments on draft-church-dnsbl-harmful-01.txt"
	(Apr 1, 6:15pm)
X-Spam-Score: 0.1 (/)
X-Scan-Signature: c1c65599517f9ac32519d043c37c5336
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

On Apr 1,  6:15pm, Seth Breidbart wrote:
} Subject: Re: [Asrg] Walter's comments on draft-church-dnsbl-harmful-01.txt
}
} Bart Schaefer <schaefer@brasslantern.com> wrote:
} 
} > If you want a restaurant analogy: It's like a restaurant hiring a
} > goon and giving him a yearbook so he can turn away all local
} > high-schoolers, because a couple of them once vandalized a restroom.
} 
} What is the "it" you're referring to?

Blocking mail by checking a DNSBL that lists IP ranges wherein some of
the IPs have been spam sources.

} The DNSBL publisher is like the yearbook publisher in that example;

Yes.

} it just publishes data.

Not precisely; in contrast to most DNSBLs, the yearbook publisher
doesn't advertise the book with a description that reads, "Pictures of
people you might want to keep out of your place of business."

However, I'm not making value judgements here, just analogies.

} The restaurant owner is like the mailswerver owner.

Right.  The restaurant owner could do a lot better with, say, a police
mugbook of known street gang members, but he doesn't have one, or maybe
his other patrons just don't like dining with teenagers anyway.

To carry the analogy further, requiring DSL customers to send mail via
their ISP server is like allowing the kids into the restaurant only if
accompanied by their parents.  Some of the kids, and maybe even some of
the parents, are going to complain.  The Church draft, in effect, argues
that those complaints are enough reason to disparage the whole practice.

I believe that's a misconception, but I don't believe that the counter-
arguments should be based on *different* misconceptions.

} One key difference in the US is that the mailserver owners is
} _specifically_ permitted by law to take whatever action _he_ thinks
} is appropriate to block unwanted email.

True, and not a bad thing either, but not really relevant to whether
some of those actions make DNSBLs in general "harmful."

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Sun Apr 02 04:43:18 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FPyAt-0003zl-5d; Sun, 02 Apr 2006 04:43:07 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FPyAr-0003zg-Sh
	for asrg@ietf.org; Sun, 02 Apr 2006 04:43:05 -0400
Received: from stsc1260-eth-s1-s1p1-vip.va.neustar.com ([156.154.16.129]
	helo=chiedprmail1.ietf.org)
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FPvao-0005Qo-KY
	for asrg@ietf.org; Sun, 02 Apr 2006 00:57:42 -0500
Received: from zcars04f.nortel.com ([47.129.242.57])
	by chiedprmail1.ietf.org with esmtp (Exim 4.43) id 1FPvNo-0000tm-Mf
	for asrg@ietf.org; Sun, 02 Apr 2006 00:44:18 -0500
Received: from zcars2ky.ca.nortel.com (zcars2ky.ca.nortel.com [47.129.242.221])
	by zcars04f.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id
	k325iCO27797
	for <asrg@ietf.org>; Sun, 2 Apr 2006 00:44:13 -0500 (EST)
Received: from [47.130.16.140] (acart04g.ca.nortel.com [47.130.16.140])
	by zcars2ky.ca.nortel.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id
	k325m1x23687
	for <asrg@ietf.org>; Sun, 2 Apr 2006 00:48:01 -0500 (EST)
Message-ID: <442F6497.4000400@americasm01.nt.com>
Date: Sun, 02 Apr 2006 00:43:51 -0500
From: "Chris Lewis" <clewis@nortel.com>
User-Agent: Mozilla Thunderbird 1.0.6 (Windows/20050716)
X-Accept-Language: en-us, en
MIME-Version: 1.0
CC: asrg@ietf.org
Subject: Re: [Asrg] Walter's comments on draft-church-dnsbl-harmful-01.txt
References: <20060401015927.GA27452@waltdnes.org>	<060401142104.ZM9347@torch.brasslantern.com>	<200604012315.k31NF4V19502@panix1.panix.com>
	<060401211416.ZM9545@torch.brasslantern.com>
In-Reply-To: <060401211416.ZM9545@torch.brasslantern.com>
X-Enigmail-Version: 0.93.0.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-Spam-Score: -2.5 (--)
X-Scan-Signature: e1e48a527f609d1be2bc8d8a70eb76cb
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Bart Schaefer wrote:

> Right.  The restaurant owner could do a lot better with, say, a police
> mugbook of known street gang members, but he doesn't have one, or maybe
> his other patrons just don't like dining with teenagers anyway.

The restaurant gets to choose from police mugbooks of convicted bathroom
vandals, books of security cam pictures of kids vandalizing bathrooms,
books of pictures of kids carrying sledgehammers into bathrooms, kids
living in the same house as bathroom vandals, kids in the same gang as
those carrying sledgehammers, kids in homes that have sledgehammers,
kids that look "funny", kids that look like other kids, pictures of
random kids, pictures of kids who don't vandalize bathrooms, pictures of
all kids, pictures of kids whose first name starts with a letter in the
first half of the alphabet etc.  Some of these books aren't labeled very
well or accurately published.  Some of them are quite accurate.

The restaurant needs to choose which books are appropriate, which ones
keep out the most likely to vandalize, without killing their business by
keeping too many out. If they don't, they're going to lose.

> To carry the analogy further, requiring DSL customers to send mail via
> their ISP server is like allowing the kids into the restaurant only if
> accompanied by their parents.  Some of the kids, and maybe even some of
> the parents, are going to complain.  The Church draft, in effect, argues
> that those complaints are enough reason to disparage the whole practice.

It does?  Ouch.  Flies in the face of a number of existing published BCPs.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3-nr1 (Windows 2000)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iQCVAwUBRC9kl53FmCyJjHfhAQI6+AP9F7jKAX+d3i3oIaEqztqmV5kmE2QbsZ2C
6rk/6fAQDYKojs7B2zUCQ/X3PK54zqQaRqxasHMTjFORurLucZmqVeK+dmqD805M
TXpsANpm+3vQcKjEzgr9mGFj8fAFZh33qqxLtCY7bF7ffWLkRjLpkT+4aMc4UJix
GIPQemn4bO8=
=xrn3
-----END PGP SIGNATURE-----

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Sun Apr 02 11:17:52 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQ4Kj-0007H4-R3; Sun, 02 Apr 2006 11:17:41 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQ4Kh-0007Fd-VL
	for asrg@ietf.org; Sun, 02 Apr 2006 11:17:39 -0400
Received: from mpls-qmqp-01.inet.qwest.net ([63.231.195.112])
	by ietf-mx.ietf.org with smtp (Exim 4.43) id 1FQ4Kd-0003Ed-Ir
	for asrg@ietf.org; Sun, 02 Apr 2006 11:17:39 -0400
Received: (qmail 45537 invoked by uid 0); 2 Apr 2006 15:17:34 -0000
Received: from unknown (63.231.195.11)
	by mpls-qmqp-01.inet.qwest.net with QMQP; 2 Apr 2006 15:17:34 -0000
Received: from 71-212-183-171.hlrn.qwest.net (HELO ?192.168.0.3?)
	(71.212.183.171)
	by mpls-pop-11.inet.qwest.net with SMTP; 2 Apr 2006 15:17:34 -0000
Date: Sun, 2 Apr 2006 09:17:32 -0600
Message-Id: <83D7F122-74E7-427A-8FDF-55D9C66AF885@qwest.net>
From: "Dan Oetting" <dan_oetting@qwest.net>
To: "Anti Spam Research Group" <asrg@ietf.org>
Mime-Version: 1.0 (Apple Message framework v746.3)
In-Reply-To: <442F6497.4000400@americasm01.nt.com>
References: <20060401015927.GA27452@waltdnes.org>	<060401142104.ZM9347@torch.brasslantern.com>	<200604012315.k31NF4V19502@panix1.panix.com>
	<060401211416.ZM9545@torch.brasslantern.com>
	<442F6497.4000400@americasm01.nt.com>
Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed
Content-Transfer-Encoding: 7bit
Subject: Re: [Asrg] Walter's comments on draft-church-dnsbl-harmful-01.txt
X-Mailer: Apple Mail (2.746.3)
X-Spam-Score: 0.0 (/)
X-Scan-Signature: bb8f917bb6b8da28fc948aeffb74aa17
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org


On Apr 1, 2006, at 10:43 PM, Chris Lewis wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Bart Schaefer wrote:
>
>> Right.  The restaurant owner could do a lot better with, say, a  
>> police
>> mugbook of known street gang members, but he doesn't have one, or  
>> maybe
>> his other patrons just don't like dining with teenagers anyway.
>
> The restaurant gets to choose from police mugbooks of convicted  
> bathroom
> vandals, books of security cam pictures of kids vandalizing bathrooms,
> books of pictures of kids carrying sledgehammers into bathrooms, kids
> living in the same house as bathroom vandals, kids in the same gang as
> those carrying sledgehammers, kids in homes that have sledgehammers,
> kids that look "funny", kids that look like other kids, pictures of
> random kids, pictures of kids who don't vandalize bathrooms,  
> pictures of
> all kids, pictures of kids whose first name starts with a letter in  
> the
> first half of the alphabet etc.  Some of these books aren't labeled  
> very
> well or accurately published.  Some of them are quite accurate.

What we don't do enough of is pinning a note to the back of the kids  
shirt to tell the responsible parents that their kid is vandalizing  
restrooms and need to be monitored until it grows up.




_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Sun Apr 02 15:21:42 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQ88d-0006Uk-8k; Sun, 02 Apr 2006 15:21:27 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQ88b-0006Uf-Of
	for asrg@ietf.org; Sun, 02 Apr 2006 15:21:25 -0400
Received: from dogma.boxhost.net ([195.218.96.101])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FQ88a-0005Ya-FE
	for asrg@ietf.org; Sun, 02 Apr 2006 15:21:25 -0400
Received: from radish.jmason.org (localhost [127.0.0.1])
	by dogma.boxhost.net (Postfix) with ESMTP id 24BEF310013;
	Sun,  2 Apr 2006 20:21:02 +0100 (IST)
Received: from jmason.org (localhost [127.0.0.1])
	by radish.jmason.org (Postfix) with ESMTP id E4225590021;
	Sun,  2 Apr 2006 20:20:50 +0100 (IST)
To: "Chris Lewis" <clewis@nortel.com>
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt 
In-Reply-To: <442F1BC7.9010904@americasm01.nt.com> 
From: jm@jmason.org (Justin Mason)
X-GPG-Key-Fingerprint: 1368 71CE 3627 9CD3 FA1B  0B63 3091 7972 298B C7D0
Date: Sun, 02 Apr 2006 20:20:50 +0100
Message-Id: <20060402192050.E4225590021@radish.jmason.org>
X-Spam-Score: 0.0 (/)
X-Scan-Signature: 0ddefe323dd869ab027dbfff7eff0465
Cc: ASRG list <asrg@ietf.org>
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org


Chris Lewis writes:
> Justin Mason wrote:
> > Chris said:
> >>Okay, suggest how a spam/ham collection can be used to measure the
> >>effectiveness of the following techniques that are, or can be, used in
> >>an anti-spam solution:
> >>
> >>1) grey listing
> >>2) sender/sender domain verification
> >>3) Challenge/response
> >>4) SPF and DKIM
> >>5) PKI
> >>6) CSV
> >>7) Non-existant users
> >>8) DCC or other distributed checksumming methodologies.
> >>
> >>Similarly DNSBLs.
> > 
> > The effectiveness of all of these, except possibly for greylisting and
> > C/R, can be measured accurately. We are doing it in SpamAssassin ;) --
> > here's how.
> > 
> > First, you accept every message, and record the "real-time" data
> > points regarding how the message was listed against those services,
> > and/or how it *would* have been rejected at SMTP transaction time (if
> > at all). However, you don't reject, you accept everything.
> > 
> > Then, later, provide a way for hand-sorting to take place, and compare
> > the results of the hand-sorting with what the various other techniques
> > would have done with those messages.
> > 
> > Hey presto, you've now got a way to compare accuracy and effectiveness
> > of those techniques!  Simple as that.
> 
> Do you see any provision for that with _any_ of the spam/ham
> collections?  Can you do that with a pre-existing spam/ham collection if
> the technique you were trying to test _wasn't_ being collected at the
> time the spam/ham collection was being made?
> 
> This points up my very point: in order to do a proper sampling of the
> effectiveness of a technique is that you do it in real time, at the time
> the emails were sent.  In other words, on real mail streams.

Ah, if you're talking specifically about *pre-existing* spam/ham
collections, then it's correct to say that they won't help, no.   To
measure network test efficacy, you need to create the spam/ham collection
in real-time, during the test.

--j.

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Sun Apr 02 16:42:21 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQ9On-0005UR-2Y; Sun, 02 Apr 2006 16:42:13 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQ9Ol-0005UM-FG
	for asrg@ietf.org; Sun, 02 Apr 2006 16:42:11 -0400
Received: from goliath.word-to-the-wise.com ([208.187.80.130]
	helo=m.wordtothewise.com) by ietf-mx.ietf.org with esmtp (Exim 4.43)
	id 1FQ9Oi-0000qF-2C
	for asrg@ietf.org; Sun, 02 Apr 2006 16:42:11 -0400
Received: from [172.16.31.11] (adsl-69-229-153-9.dsl.pltn13.pacbell.net
	[69.229.153.9]) (using TLSv1 with cipher RC4-SHA (128/128 bits))
	(No client certificate requested)
	by m.wordtothewise.com (Postfix) with ESMTP id DCBA5106B3
	for <asrg@ietf.org>; Sun,  2 Apr 2006 13:42:06 -0700 (PDT)
Mime-Version: 1.0 (Apple Message framework v746.3)
In-Reply-To: <20060402192050.E4225590021@radish.jmason.org>
References: <20060402192050.E4225590021@radish.jmason.org>
Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed
Message-Id: <2BC9207C-3E51-4B88-97A8-7445E9A1313B@blighty.com>
Content-Transfer-Encoding: 7bit
From: Steve Atkins <steve@blighty.com>
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt 
Date: Sun, 2 Apr 2006 13:42:04 -0700
To: ASRG list <asrg@ietf.org>
X-Mailer: Apple Mail (2.746.3)
X-Spam-Score: 0.1 (/)
X-Scan-Signature: b7b9551d71acde901886cc48bfc088a6
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org


On Apr 2, 2006, at 12:20 PM, Justin Mason wrote:

>
> Chris Lewis writes:
>> Justin Mason wrote:
>>> Chris said:
>>>> Okay, suggest how a spam/ham collection can be used to measure the
>>>> effectiveness of the following techniques that are, or can be,  
>>>> used in
>>>> an anti-spam solution:
>>>>
>>>> 1) grey listing
>>>> 2) sender/sender domain verification
>>>> 3) Challenge/response
>>>> 4) SPF and DKIM
>>>> 5) PKI
>>>> 6) CSV
>>>> 7) Non-existant users
>>>> 8) DCC or other distributed checksumming methodologies.
>>>>
>>>> Similarly DNSBLs.
>>>
>>> The effectiveness of all of these, except possibly for  
>>> greylisting and
>>> C/R, can be measured accurately. We are doing it in  
>>> SpamAssassin ;) --
>>> here's how.
>>>
>>> First, you accept every message, and record the "real-time" data
>>> points regarding how the message was listed against those services,
>>> and/or how it *would* have been rejected at SMTP transaction time  
>>> (if
>>> at all). However, you don't reject, you accept everything.

Not rejecting the mail alters the behavior of the sender. That skews
the data from this approach. Doesn't mean it's not a useful metric, but
it's not a real measurement, and can't be used to give good estimates
of real-world performance (at least, not without quite a lot of  
additional
data, anyway).

It's still a good approach, as long as you remember that different
categories of email delivery will respond differently to changes in
recipient behaviour. Often dramatically so.

>>>
>>> Then, later, provide a way for hand-sorting to take place, and  
>>> compare
>>> the results of the hand-sorting with what the various other  
>>> techniques
>>> would have done with those messages.
>>>
>>> Hey presto, you've now got a way to compare accuracy and  
>>> effectiveness
>>> of those techniques!  Simple as that.
>>
>> Do you see any provision for that with _any_ of the spam/ham
>> collections?  Can you do that with a pre-existing spam/ham  
>> collection if
>> the technique you were trying to test _wasn't_ being collected at the
>> time the spam/ham collection was being made?
>>
>> This points up my very point: in order to do a proper sampling of the
>> effectiveness of a technique is that you do it in real time, at  
>> the time
>> the emails were sent.  In other words, on real mail streams.
>
> Ah, if you're talking specifically about *pre-existing* spam/ham
> collections, then it's correct to say that they won't help, no.   To
> measure network test efficacy, you need to create the spam/ham  
> collection
> in real-time, during the test.

And to measure actual efficacy you need to actually reject mail,
if what you're measuring is behaviour if you reject mail. (If the
behaviour you're trying to model is "what happens if we bulk folder
or devnull mail detected by a filter?" then that's not a problem,
but one of the usually mentioned advantages of DNSBLs is that
you don't need to accept the mail).

Cheers,
   Steve


_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Sun Apr 02 19:48:47 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQCJ9-0000gk-Te; Sun, 02 Apr 2006 19:48:35 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQCJ7-0000en-Bq
	for asrg@ietf.org; Sun, 02 Apr 2006 19:48:34 -0400
Received: from omta04ps.mx.bigpond.com ([144.140.83.156])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FQCJ6-0001KW-4i
	for asrg@ietf.org; Sun, 02 Apr 2006 19:48:33 -0400
Received: from localhost.localdomain ([60.226.77.107])
	by omta04ps.mx.bigpond.com with ESMTP id
	<20060402234829.CRPV17345.omta04ps.mx.bigpond.com@localhost.localdomain>;
	Sun, 2 Apr 2006 23:48:29 +0000
Received: from ender (ender.scoobynet [192.168.0.3])
	by localhost.localdomain (Postfix) with ESMTP id 17929C0;
	Mon,  3 Apr 2006 09:48:06 +1000 (EST)
Received: by ender (Postfix, from userid 1000)
	id BD3474DD61; Mon,  3 Apr 2006 09:46:35 +1000 (EST)
Date: Mon, 3 Apr 2006 09:46:35 +1000
From: Laird Breyer <laird@lbreyer.com>
To: Steve Atkins <steve@blighty.com>
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
Message-ID: <20060402234635.GA11416@ender>
Mail-Followup-To: Steve Atkins <steve@blighty.com>, ASRG list <asrg@ietf.org>
References: <20060402192050.E4225590021@radish.jmason.org>
	<2BC9207C-3E51-4B88-97A8-7445E9A1313B@blighty.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <2BC9207C-3E51-4B88-97A8-7445E9A1313B@blighty.com>
User-Agent: Mutt/1.5.11+cvs20060126
X-Spam-Score: 0.1 (/)
X-Scan-Signature: 50a516d93fd399dc60588708fd9a3002
Cc: ASRG list <asrg@ietf.org>
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
Reply-To: laird@lbreyer.com
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

On Apr 02 2006, Steve Atkins wrote:

> > Chris Lewis writes:
> >> Justin Mason wrote:

> >>> First, you accept every message, and record the "real-time" data
> >>> points regarding how the message was listed against those services,
> >>> and/or how it *would* have been rejected at SMTP transaction time  
> >>> (if
> >>> at all). However, you don't reject, you accept everything.
> 
> Not rejecting the mail alters the behavior of the sender. That skews
> the data from this approach. Doesn't mean it's not a useful metric, but
> it's not a real measurement, and can't be used to give good estimates
> of real-world performance (at least, not without quite a lot of  
> additional
> data, anyway).

During the SMTP transaction, the server might reject the data from the
client, and would do so on _some_ grounds (eg IP address). Strictly
speaking, _these_ grounds are what needs to be logged and verified by
a human somewhere down the line.

As such, it's not necessary to accept the message at all and no sender
skewing takes place. In a formal test, a human adjudicator may well
decide that a single logged IP address + dnsbl response and without an
accepted message body are insufficient grounds for summary rejection,
and count it as an FP. That's still a valid test though, because the
human observer takes the final responsibility, and can explain his
reasoning if necessary.

For example, the criterion might be consent (as in CAN-SPAM) and the 
adjudicator might be a lawyer, and a single IP listed on a dnsbl might
not constitute proof of consent or dissent under the regulations. 

> 
> And to measure actual efficacy you need to actually reject mail,
> if what you're measuring is behaviour if you reject mail. (If the
> behaviour you're trying to model is "what happens if we bulk folder
> or devnull mail detected by a filter?" then that's not a problem,
> but one of the usually mentioned advantages of DNSBLs is that
> you don't need to accept the mail).

True, but in that case it is doubly important to not amalgamate
accuracy testing with efficacy testing.  These aspects should be
clearly separated in arguments intended for non-experts.

-- 
Laird Breyer.

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Sun Apr 02 20:53:38 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQDJz-0003p5-6d; Sun, 02 Apr 2006 20:53:31 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQDJy-0003n4-2a
	for asrg@ietf.org; Sun, 02 Apr 2006 20:53:30 -0400
Received: from omta02ps.mx.bigpond.com ([144.140.83.154])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FQDJw-0003J1-A5
	for asrg@ietf.org; Sun, 02 Apr 2006 20:53:30 -0400
Received: from localhost.localdomain ([60.226.77.107])
	by omta02ps.mx.bigpond.com with ESMTP id
	<20060403005320.BVGT3168.omta02ps.mx.bigpond.com@localhost.localdomain>;
	Mon, 3 Apr 2006 00:53:20 +0000
Received: from ender (ender.scoobynet [192.168.0.3])
	by localhost.localdomain (Postfix) with ESMTP id 2F9BAC0;
	Mon,  3 Apr 2006 10:52:54 +1000 (EST)
Received: by ender (Postfix, from userid 1000)
	id 17EFD4DD61; Mon,  3 Apr 2006 10:51:24 +1000 (EST)
Date: Mon, 3 Apr 2006 10:51:24 +1000
From: Laird Breyer <laird@lbreyer.com>
To: Chris Lewis <clewis@nortel.com>
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
Message-ID: <20060403005123.GB11416@ender>
Mail-Followup-To: Chris Lewis <clewis@nortel.com>, asrg@ietf.org
References: <88885AD16AAA9F4899BB9FB751D4F5A5B70919@phost016.habeas.intermedia.net>
	<E1FP1vs-0008WV-00@chiark.greenend.org.uk>
	<20060331005839.GB6958@ender> <442D3368.7040202@americasm01.nt.com>
	<20060331234310.GA9179@ender> <442DCB75.7050409@americasm01.nt.com>
	<20060401021616.GA9651@ender> <442EE80C.5080500@americasm01.nt.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <442EE80C.5080500@americasm01.nt.com>
User-Agent: Mutt/1.5.11+cvs20060126
X-Spam-Score: 0.1 (/)
X-Scan-Signature: b22590c27682ace61775ee7b453b40d3
Cc: asrg@ietf.org
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
Reply-To: laird@lbreyer.com
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

On Apr 01 2006, Chris Lewis wrote:
> 
> He's reporting for a spam filter responsible for handling at most a
> handful of users.  I have 65,000, and handle 500 times as much email per
> _day_ as his (I believe) multiple day dataset.  Sites handling 1000
> times as much email as I do are ALSO using DNSBLs.

Right, this is a useful point against the draft.

> The fact that we even have to answer to such nonsense, instead of
> summarily throwing that in the trash is an insult to professionals.
> 
> But we do it anyway, because we're professionals.

Yes, and you're appreciated for it. But if the answer is easy enough
to dismiss it won't stop the next person from deeming dnsbls harmful
yet again. There's a lot of folklore and crap on the net, and that's
the environment in which opinions are formed. The ASRG doesn't have
the luxury of dismissing ideas from the (valuable) experience of its
members only.

> I don't presuppose to cover vanity or hobby domains.  Which is pretty
> clearly what Church is talking about.

Right, this is a useful point against the draft.

> 
> > The author of the draft isn't the only one who claims to have been
> > bitten by dnsbls, there are plenty of rants on the net, e.g.
> > http://paulgraham.com/spamhausblacklist.html
> 
> Paul chose to deliberately misrepresent the situation for political
> ends.  As did Moveon.  As did Gilmore.  And at least _two_ of those
> listings were legitimate spam blocks. There are similarly plenty of
> issues about other types of filtering goofing up.

Which doesn't help in rejecting the draft. I know I brought it up,
but it's best to stay clear of this kind of argument.

> his usage patterns/user community
> are unknown, his "preferred" filter is only described in terms of
> unverifiable handwaving - no numbers, and is of a size that can best be
> described as statistically irrelevant.
> 
> That should be enough.

Yes, I suggest backing that point up with the numbers you mentioned
at the top.

> 
> Okay, suggest how a spam/ham collection can be used to measure the
> effectiveness of the following techniques that are, or can be, used in
> an anti-spam solution:

I'll address a point you make later first, as it shows where we disagree:

> _Anything_ that uses "adjudicators" to determine whether an email is
> spam or ham is clearly missing an important clue: spam isn't about
> content, it's about _consent_.  It is a behaviour (sending without
> consent), NOT what is in the email.

It still needs a human adjudicator. In this case, a human who
interprets the CAN-SPAM regulations and decides whether (the legal
definition of) consent did apply.

> 
> DNSBLs that can, for example, detect compromised machines or illegal
> methods being used to send the email have a better handle on intent,
> than an "adjudicator" could possibly have.
> 

Being an automated method of filtering, it depends on a set of criteria
and conditions stored in the program logic and database. 
These criteria and conditions, if given in full to a human adjudicator,
could be judged by said human just as well with authority.  

Now on to your list, I answered it first but it's based on the above
paragraphs:

> 
> 1) grey listing
> 2) sender/sender domain verification
> 3) Challenge/response
> 4) SPF and DKIM
> 5) PKI
> 6) CSV
> 7) Non-existant users
> 8) DCC or other distributed checksumming methodologies.
> 
> Similarly DNSBLs.
> 

As Steve Atkins mentioned later in the thread, there's efficacy and 
accuracy, two different aspects. I don't think there's anybody who would
dispute efficacy - rejecting at the source is clearly more efficient.

For accuracy testing, any method has to involve a human adjudicator
somewhere downstream. Moreover, whatever the method being tested, all
relevant information (ie information used for that particular decision) 
must be logged. 

If a machine can take a decision, then a human who is given the same
information (no more no less) must be able to take a decision too. The
human has the last word, and that's the basis for accuracy testing.

This basis is approriate whether you take spam as "whatever I don't want"
or whether it's UCE under CAN-SPAM etc. In the latter case, the last word
would be interpretation of the law by humans in a court, in principle. 

Now if you have a record of each transaction, and if you build a
system which submits a sample of these records to intended recipients
for verification, (and your recipients are willing to help test the
system, etc) then I think you have all the elements you need for testing.

For example, when a message is rejected via dnsbl, your record
contains the dnsbl query and a statement of rejection. That record can
be shown to the intended recipient to be ok'd. If the recipient
disagrees with the machine, then that's a FP or FN.

For non-existant users, the decision is vacuous, ie anything the server 
does is appropriate. For PKI, there is no need to ask a human to verify
the decryption with pencil and paper, obviously. Etc.



-- 
Laird Breyer.

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Sun Apr 02 20:59:46 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQDQ1-0007Ts-1G; Sun, 02 Apr 2006 20:59:45 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQDQ0-0007TR-1G
	for asrg@ietf.org; Sun, 02 Apr 2006 20:59:44 -0400
Received: from goliath.word-to-the-wise.com ([208.187.80.130]
	helo=m.wordtothewise.com) by ietf-mx.ietf.org with esmtp (Exim 4.43)
	id 1FQDPz-0003f1-Kl
	for asrg@ietf.org; Sun, 02 Apr 2006 20:59:44 -0400
Received: from [10.3.2.16] (184.word-to-the-wise.com [208.187.80.184])
	(using TLSv1 with cipher RC4-SHA (128/128 bits))
	(No client certificate requested)
	by m.wordtothewise.com (Postfix) with ESMTP id 2330D106AE
	for <asrg@ietf.org>; Sun,  2 Apr 2006 17:59:43 -0700 (PDT)
Mime-Version: 1.0 (Apple Message framework v746.3)
In-Reply-To: <20060402234635.GA11416@ender>
References: <20060402192050.E4225590021@radish.jmason.org>
	<2BC9207C-3E51-4B88-97A8-7445E9A1313B@blighty.com>
	<20060402234635.GA11416@ender>
Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed
Message-Id: <A762C388-9E0F-4CB7-9862-7EA64F427BBD@blighty.com>
Content-Transfer-Encoding: 7bit
From: Steve Atkins <steve@blighty.com>
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
Date: Sun, 2 Apr 2006 17:59:41 -0700
To: ASRG list <asrg@ietf.org>
X-Mailer: Apple Mail (2.746.3)
X-Spam-Score: 0.1 (/)
X-Scan-Signature: 386e0819b1192672467565a524848168
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org


On Apr 2, 2006, at 4:46 PM, Laird Breyer wrote:

> On Apr 02 2006, Steve Atkins wrote:
>
>>> Chris Lewis writes:
>>>> Justin Mason wrote:
>
>>>>> First, you accept every message, and record the "real-time" data
>>>>> points regarding how the message was listed against those  
>>>>> services,
>>>>> and/or how it *would* have been rejected at SMTP transaction time
>>>>> (if
>>>>> at all). However, you don't reject, you accept everything.
>>
>> Not rejecting the mail alters the behavior of the sender. That skews
>> the data from this approach. Doesn't mean it's not a useful  
>> metric, but
>> it's not a real measurement, and can't be used to give good estimates
>> of real-world performance (at least, not without quite a lot of
>> additional
>> data, anyway).
>
> During the SMTP transaction, the server might reject the data from the
> client, and would do so on _some_ grounds (eg IP address). Strictly
> speaking, _these_ grounds are what needs to be logged and verified by
> a human somewhere down the line.
>
> As such, it's not necessary to accept the message at all and no sender
> skewing takes place. In a formal test, a human adjudicator may well
> decide that a single logged IP address + dnsbl response and without an
> accepted message body are insufficient grounds for summary rejection,
> and count it as an FP. That's still a valid test though, because the
> human observer takes the final responsibility, and can explain his
> reasoning if necessary.

If you reject the message during testing where you would have accepted
the message in operation then you may change the delivery policy of the
sender to that recipient for all future messages. That invalidates  
any data
that includes any future delivery attempts from the same sender (and  
that
sender might well be a spammer coming from any of thousands of open
proxies sending hundreds of different messages).

That means that any test where your rejection behavior differs from your
operational rejection behaviour will see different delivery policies,  
and
hence different numbers and origins of delivery attempts. That means  
that
you _cannot_ use this approach of testing multiple rejection criteria  
against
a single delivery stream and get entirely valid data (except for the  
case where
you accept all the mail, both in testing and operation).

That doesn't mean it can't provide useful data, but it will not give  
the same
answers as if you really rejected the mail in real time as you would in
practice.

>
> For example, the criterion might be consent (as in CAN-SPAM) and the
> adjudicator might be a lawyer, and a single IP listed on a dnsbl might
> not constitute proof of consent or dissent under the regulations.
>
>>
>> And to measure actual efficacy you need to actually reject mail,
>> if what you're measuring is behaviour if you reject mail. (If the
>> behaviour you're trying to model is "what happens if we bulk folder
>> or devnull mail detected by a filter?" then that's not a problem,
>> but one of the usually mentioned advantages of DNSBLs is that
>> you don't need to accept the mail).
>
> True, but in that case it is doubly important to not amalgamate
> accuracy testing with efficacy testing.  These aspects should be
> clearly separated in arguments intended for non-experts.

Neither accuracy nor efficacy are accurately measurable against
multiple different filtering policies in parallel in this sort of  
test. By
correcting for sender behaviour (which I've never seen anyone
claiming these sorts of numbers do[1]) you could certainly improve
the quality of the results, but they'd still be skewed relative to using
the same approach during real mail delivery.

It's a metric. It's not a bad metric, but don't mistake it for even an
accurate measure of results on the one set of mail tested.

Cheers,
   Steve


_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Sun Apr 02 22:03:18 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQEPM-0007Y1-W8; Sun, 02 Apr 2006 22:03:08 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQEPM-0007Xw-BP
	for asrg@ietf.org; Sun, 02 Apr 2006 22:03:08 -0400
Received: from omta02ps.mx.bigpond.com ([144.140.83.154])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FQEPK-0005xB-Rx
	for asrg@ietf.org; Sun, 02 Apr 2006 22:03:08 -0400
Received: from localhost.localdomain ([60.226.77.107])
	by omta02ps.mx.bigpond.com with ESMTP id
	<20060403020305.FGWZ3168.omta02ps.mx.bigpond.com@localhost.localdomain>;
	Mon, 3 Apr 2006 02:03:05 +0000
Received: from ender (ender.scoobynet [192.168.0.3])
	by localhost.localdomain (Postfix) with ESMTP id CC526C0;
	Mon,  3 Apr 2006 12:02:42 +1000 (EST)
Received: by ender (Postfix, from userid 1000)
	id 97D444DD61; Mon,  3 Apr 2006 12:01:12 +1000 (EST)
Date: Mon, 3 Apr 2006 12:01:12 +1000
From: Laird Breyer <laird@lbreyer.com>
To: Steve Atkins <steve@blighty.com>
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
Message-ID: <20060403020112.GA12073@ender>
Mail-Followup-To: Steve Atkins <steve@blighty.com>, ASRG list <asrg@ietf.org>
References: <20060402192050.E4225590021@radish.jmason.org>
	<2BC9207C-3E51-4B88-97A8-7445E9A1313B@blighty.com>
	<20060402234635.GA11416@ender>
	<A762C388-9E0F-4CB7-9862-7EA64F427BBD@blighty.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <A762C388-9E0F-4CB7-9862-7EA64F427BBD@blighty.com>
User-Agent: Mutt/1.5.11+cvs20060126
X-Spam-Score: 0.1 (/)
X-Scan-Signature: 4adaf050708fb13be3316a9eee889caa
Cc: ASRG list <asrg@ietf.org>
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
Reply-To: laird@lbreyer.com
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

On Apr 02 2006, Steve Atkins wrote:
> 
> If you reject the message during testing where you would have
> accepted the message in operation then [...]
> invalidates any data that includes any future delivery attempts 

Yes, we agree on that. Another way of saying this is that predictive
simulation isn't truly possible because the internal state of the
spammer's software is unknowable. Thus comparing two filtering methods
by _simulation_ , ie creating hypothetical worlds where each method is
used on its own, only makes sense in some cases.

It makes sense when the simulation is valid, eg for testing local
content filtering inside a mail client. It isn't valid for dnsbl
comparisons at the SMTP level because as you say the internal state of
the spammer software is unknown.

But predictive simulation as a way of comparing two filters (or one
filter, and the absence thereof) is not the only way to do accuracy
testing. I was saying that humans who adjudicate ex post facto the
machine decision rules (rather than the messages themselves, which may
or may not be available) can validly count misclassifications after
the fact.

This gives a valid measure of accuracy for the effect of a single
filter, that can be summarized in terms of FP and FN. In turn, FP and
FN can be compared at an abstract level.

However, you are exactly right in pointing out that multiple filters
cannot be therefore compared on the same datastream when predictive
simulation isn't possible.

But even that is only part of the story. If you take two spam datastreams
and wait long enough, you'll get some identical messages in both streams.
If you prune these datastreams to keep only those identical messages, you
can compare two filters on the same data, even though they operate on
separate streams. 

That's not much use in general, but suppose you're comparing two
dnsbls, and suppose these dnsbls identify spam messages by requiring
at least two identical message submissions. In that case, comparing
the intersection of two datastreams makes a lot more sense, because
those dnsbls only advise on multiply sent spams.

-- 
Laird Breyer.

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Sun Apr 02 22:24:33 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQEia-0003IR-AK; Sun, 02 Apr 2006 22:23:00 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQEiZ-0003IM-Qe
	for asrg@ietf.org; Sun, 02 Apr 2006 22:22:59 -0400
Received: from mail1.panix.com ([166.84.1.72])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FQEiY-0007Ta-KN
	for asrg@ietf.org; Sun, 02 Apr 2006 22:22:59 -0400
Received: from panix1.panix.com (panix1.panix.com [166.84.1.1])
	by mail1.panix.com (Postfix) with ESMTP id 267495B3CC
	for <asrg@ietf.org>; Sun,  2 Apr 2006 22:22:58 -0400 (EDT)
Received: (from sethb@localhost)
	by panix1.panix.com (8.11.6p3/8.8.8/PanixN1.1) id k332MvS14090;
	Sun, 2 Apr 2006 22:22:57 -0400 (EDT)
Date: Sun, 2 Apr 2006 22:22:57 -0400 (EDT)
Message-Id: <200604030222.k332MvS14090@panix1.panix.com>
From: Seth Breidbart <sethb@panix.com>
To: asrg@ietf.org
In-reply-to: <20060403005123.GB11416@ender> (message from Laird Breyer on Mon, 
	3 Apr 2006 10:51:24 +1000)
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
References: <88885AD16AAA9F4899BB9FB751D4F5A5B70919@phost016.habeas.intermedia.net>
	<E1FP1vs-0008WV-00@chiark.greenend.org.uk>
	<20060331005839.GB6958@ender> <442D3368.7040202@americasm01.nt.com>
	<20060331234310.GA9179@ender> <442DCB75.7050409@americasm01.nt.com>
	<20060401021616.GA9651@ender> <442EE80C.5080500@americasm01.nt.com>
	<20060403005123.GB11416@ender>
X-Spam-Score: 0.0 (/)
X-Scan-Signature: 2409bba43e9c8d580670fda8b695204a
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

Laird Breyer <laird@lbreyer.com> wrote:
> On Apr 01 2006, Chris Lewis wrote:

>> _Anything_ that uses "adjudicators" to determine whether an email is
>> spam or ham is clearly missing an important clue: spam isn't about
>> content, it's about _consent_.  It is a behaviour (sending without
>> consent), NOT what is in the email.
>
> It still needs a human adjudicator. In this case, a human who
> interprets the CAN-SPAM regulations and decides whether (the legal
> definition of) consent did apply.

If the adjudicator comes in after the fact, it is _impossible_ to
tell.

Example: sethbr@panix.com subscribed to a non-confirming newsletter,
and typoed the email address.  When he noticed he wasn't getting it,
he subscribed again, typing the address correctly this time.

Panix gets two copies.  One to me, one to him.

One of them is spam.

Unless you get Dave Hayes to write the adjudicator, it won't always be
right.

Seth

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Sun Apr 02 22:38:31 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQExX-0003Da-6F; Sun, 02 Apr 2006 22:38:27 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQExW-0003DP-3u
	for asrg@ietf.org; Sun, 02 Apr 2006 22:38:26 -0400
Received: from sparkle.rodents.montreal.qc.ca ([216.46.5.7])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FQExU-0007vh-Ou
	for asrg@ietf.org; Sun, 02 Apr 2006 22:38:26 -0400
Received: (from mouse@localhost)
	by Sparkle.Rodents.Montreal.QC.CA (8.8.8/8.8.8) id WAA01085;
	Sun, 2 Apr 2006 22:38:23 -0400 (EDT)
From: der Mouse <mouse@Rodents.Montreal.QC.CA>
Message-Id: <200604030238.WAA01085@Sparkle.Rodents.Montreal.QC.CA>
Mime-Version: 1.0
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: 8bit
X-Erik-Conspiracy: There is no Conspiracy - and if there were I wouldn't be
	part of it anyway.
X-Message-Flag: Microsoft: the company who gave us the botnet zombies.
Date: Sun, 2 Apr 2006 22:32:35 -0400 (EDT)
To: asrg@ietf.org
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
In-Reply-To: <20060403005123.GB11416@ender>
References: <88885AD16AAA9F4899BB9FB751D4F5A5B70919@phost016.habeas.intermedia.net>
	<E1FP1vs-0008WV-00@chiark.greenend.org.uk>
	<20060331005839.GB6958@ender> <442D3368.7040202@americasm01.nt.com>
	<20060331234310.GA9179@ender> <442DCB75.7050409@americasm01.nt.com>
	<20060401021616.GA9651@ender> <442EE80C.5080500@americasm01.nt.com>
	<20060403005123.GB11416@ender>
X-Spam-Score: 0.0 (/)
X-Scan-Signature: de4f315c9369b71d7dd5909b42224370
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

> As Steve Atkins mentioned later in the thread, there's efficacy and
> accuracy, two different aspects.  I don't think there's anybody who
> would dispute efficacy - rejecting at the source is clearly more
> efficient.

That last word makes it appear you are confusing efficacy
("effectiveness") with efficiency ("efficientness").

As for whether rejecting at the source is more efficient, that depends
on what you are measuring efficiency with respect to.  Network traffic?
CPU cycles?  Implementor time?  Maintainer time?  It makes a big
difference.

/~\ The ASCII				der Mouse
\ / Ribbon Campaign
 X  Against HTML	       mouse@rodents.montreal.qc.ca
/ \ Email!	     7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Sun Apr 02 22:44:11 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQF32-00079H-Iw; Sun, 02 Apr 2006 22:44:08 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQF31-00079B-FZ
	for asrg@ietf.org; Sun, 02 Apr 2006 22:44:07 -0400
Received: from zcars04f.nortelnetworks.com ([47.129.242.57]
	helo=zcars04f.nortel.com) by ietf-mx.ietf.org with esmtp (Exim 4.43)
	id 1FQF2y-0008FO-0L
	for asrg@ietf.org; Sun, 02 Apr 2006 22:44:07 -0400
Received: from zcars2ky.ca.nortel.com (zcars2ky.ca.nortel.com [47.129.242.221])
	by zcars04f.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id
	k332i1818372
	for <asrg@ietf.org>; Sun, 2 Apr 2006 22:44:01 -0400 (EDT)
Received: from [47.130.17.133] (acart264.ca.nortel.com [47.130.17.133])
	by zcars2ky.ca.nortel.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id
	k332lm515236
	for <asrg@ietf.org>; Sun, 2 Apr 2006 22:47:48 -0400 (EDT)
Message-ID: <44308BDB.8060306@americasm01.nt.com>
Date: Sun, 02 Apr 2006 22:43:39 -0400
From: "Chris Lewis" <clewis@nortel.com>
User-Agent: Mozilla Thunderbird 1.0.6 (Windows/20050716)
X-Accept-Language: en-us, en
MIME-Version: 1.0
CC: asrg@ietf.org
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
References: <88885AD16AAA9F4899BB9FB751D4F5A5B70919@phost016.habeas.intermedia.net>
	<E1FP1vs-0008WV-00@chiark.greenend.org.uk>
	<20060331005839.GB6958@ender> <442D3368.7040202@americasm01.nt.com>
	<20060331234310.GA9179@ender> <442DCB75.7050409@americasm01.nt.com>
	<20060401021616.GA9651@ender> <442EE80C.5080500@americasm01.nt.com>
	<20060403005123.GB11416@ender>
In-Reply-To: <20060403005123.GB11416@ender>
X-Enigmail-Version: 0.93.0.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-Spam-Score: 0.1 (/)
X-Scan-Signature: 33cc095b503da4365ce57c727e553cf1
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Laird Breyer wrote:

Thomas is composing a reply to the paper, and I've taken note of some of
your comments for inclusion.

>>_Anything_ that uses "adjudicators" to determine whether an email is
>>spam or ham is clearly missing an important clue: spam isn't about
>>content, it's about _consent_.  It is a behaviour (sending without
>>consent), NOT what is in the email.

> It still needs a human adjudicator. In this case, a human who
> interprets the CAN-SPAM regulations and decides whether (the legal
> definition of) consent did apply.

First of all, CAN-SPAM doesn't define what is spam, and what isn't. It
simply defines a subset of spam that is legally actionable under US law.
The definition of spam doesn't depend on local ordinances.

Analysis under CAN-SPAM determinations simply isn't relevant, for
example, in our jurisdiction.  Spam is spam, whether it's in Canada, or
the US or Australia.

Secondly, it simply isn't possible to determine to that level of
certainty "consent" in anything more than a tiny fraction of spam.
Spammers simply don't cooperate.  Indeed, many of those who appear to
cooperate lie.

[I'll omit the story about proving that all of the "consents" Richter
used to justify email to ~700 of our users were fraudulent. I hesitate
to suggest that adjudicators would know what questions to ask, and how
to determine that they were fake.  Let alone being able to do it at all
in a large or broad enough sample to be statistically relevant.]

>>DNSBLs that can, for example, detect compromised machines or illegal
>>methods being used to send the email have a better handle on intent,
>>than an "adjudicator" could possibly have.

> Being an automated method of filtering, it depends on a set of criteria
> and conditions stored in the program logic and database. 
> These criteria and conditions, if given in full to a human adjudicator,
> could be judged by said human just as well with authority.  

And these adjudicators would be able to make such judgements correctly?
Unlikely.  And furthermore, even if they were not 100% accurate, ONLY
the overall result matters.

An IP HELO'ing as your server is unlikely to be judged "proof positive"
of compromise.  It isn't.  But testing shows that the number of
desirable emails mistakenly blocked by such a heuristic are essentially
non-existant.

We don't judge testing of drugs on whether "adjudicators" understand how
the drugs work, we judge drug efficacy on _results_ in trials.

> Now on to your list, I answered it first but it's based on the above
> paragraphs:

>>1) grey listing
>>2) sender/sender domain verification
>>3) Challenge/response
>>4) SPF and DKIM
>>5) PKI
>>6) CSV
>>7) Non-existant users
>>8) DCC or other distributed checksumming methodologies.

>>Similarly DNSBLs.

> As Steve Atkins mentioned later in the thread, there's efficacy and 
> accuracy, two different aspects. I don't think there's anybody who would
> dispute efficacy - rejecting at the source is clearly more efficient.

Efficacy has nothing to do with the success/failure of DNSBLs in a
general sense.  Nor is rejecting at source a necessary consequence of
DNSBLs.  We use DNSBLs, do rejections, but still get to quarantine the
whole email.

More importantly, Steve (and I) pointed out that these techniques simply
can't be tested to the level you might want to see, because they have to
be done with real mailstreams in realtime, and with, for example, grey
listing, it's simply NOT possible to judge in your terms whether the
email was spam or not.  Applying the heuristic changes the sender
behavior, and you don't have the email to judge.

Similarly, with DNSBLs, we actually see spammers noticing that one
sender was blocked, and trying a chain of source IPs until one gets through.

The situation is simple: there's a variety of techniques that simply
cannot be tested to this level of academic certainty with "recipient" or
"recipient's judge" techniques, because applying them changes the
behaviour, and secondly, the recipient _cannot_ have anything to judge
it with.

> For accuracy testing, any method has to involve a human adjudicator
> somewhere downstream. Moreover, whatever the method being tested, all
> relevant information (ie information used for that particular decision) 
> must be logged.

We already have the best possible scenario: the human _upstream_ who
noticed his email didn't get through, and then contacts the
administrators to get the problem fixed.

Letting the email get through in some sense (whether by quarantine or by
tagging mechanisms) simply doesn't work in the large scale. Given the
volumes that people get and the quantities they'd have to wade through,
they're simply not going to give an accurate picture - FPs will be missed.

Instead, we know that for the most part, only valid senders get notified
of rejections.  They had a reason to send it in the first place.  The
user at the other end doesn't know that there was something valid to
look for.

> If a machine can take a decision, then a human who is given the same
> information (no more no less) must be able to take a decision too. The
> human has the last word, and that's the basis for accuracy testing.

This particular line of reasoning is to do with the notion that the
machine may be mistaken that the source IP is listed in the DNSBL under
evaluation.  This is the least of our concerns.

> This basis is approriate whether you take spam as "whatever I don't want"
> or whether it's UCE under CAN-SPAM etc. In the latter case, the last word
> would be interpretation of the law by humans in a court, in principle. 

I could just imagine the industry being crippled in that fashion - you
can only tell how well your filters work by getting all the spammers in
court.  If that's the criteria for success, that criteria will be ignored.

> Now if you have a record of each transaction, and if you build a
> system which submits a sample of these records to intended recipients
> for verification, (and your recipients are willing to help test the
> system, etc) then I think you have all the elements you need for testing.

I contend that sender notification is a more accurate and practical
metric.  Which is what we do.

Still, we do have exactly what you suggest.  Our quarantine - each time
someone forwards a message to themselves out of the quarantine (hence
suggestive of a FP), is recorded and sent to us for evaluation, where we
can tell what the original blocking reason was.  I don't recall _ever_
seeing one about the XBL or SBL.

> For example, when a message is rejected via dnsbl, your record
> contains the dnsbl query and a statement of rejection. That record can
> be shown to the intended recipient to be ok'd. If the recipient
> disagrees with the machine, then that's a FP or FN.

> For non-existant users, the decision is vacuous, ie anything the server 
> does is appropriate. For PKI, there is no need to ask a human to verify
> the decryption with pencil and paper, obviously. Etc.

What does showing the exact reason to the end user have to do with
whether it's a FP of the dnsbl?  A valid email being blocked is a FP,
period.  Counting it as a FP against the DNSBL as appropriate, need not
be done by the user.

Secondly, there's no reason to ask a human or adjudicator to verify that
a rejection based on a given DNSBL is really listed in the DNSBL.  Given
the volumes we're doing, errors of that nature would be immediately
catastrophically obvious.

Thirdly, even I'm not that sanguine about "non-existant users". Which is
why we simply don't count emails to non-existant users.  Otherwise, I'd
include the CBL against our spamtrap - our spamtrap does something like
3-6 million emails/day.  The CBL kills 80% of it, and in at least some
sense, that has no FPs by definition.  The numbers are useful, but you
cannot reliably determine _real_ FPs from it.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3-nr1 (Windows 2000)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iQCVAwUBRDCL253FmCyJjHfhAQKaWQP/cmxbSKfCtOk5+C+0FoaMTCsXlhUL4rA9
158k+MW/SniLHK1cM8QG2ohhjgQZNBliFT+KFHh8+ptrs9S07khVvZ6Q26zg2GxS
VltQkU3KLPbSBQ0vrt4yvF5S0LZlwP67zPU6tGxnOgIVbBKh/LZ9oZ/n3RnPWu3Y
wBCR7WIydV0=
=Pzqk
-----END PGP SIGNATURE-----

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Sun Apr 02 23:11:12 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQFT1-0002HA-82; Sun, 02 Apr 2006 23:10:59 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQFT0-0002H2-Kd
	for asrg@ietf.org; Sun, 02 Apr 2006 23:10:58 -0400
Received: from zcars04f.nortelnetworks.com ([47.129.242.57]
	helo=zcars04f.nortel.com) by ietf-mx.ietf.org with esmtp (Exim 4.43)
	id 1FQFSz-0000is-BZ
	for asrg@ietf.org; Sun, 02 Apr 2006 23:10:58 -0400
Received: from zcars2ky.ca.nortel.com (zcars2ky.ca.nortel.com [47.129.242.221])
	by zcars04f.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id
	k333As824662
	for <asrg@ietf.org>; Sun, 2 Apr 2006 23:10:54 -0400 (EDT)
Received: from [47.130.17.133] (acart264.ca.nortel.com [47.130.17.133])
	by zcars2ky.ca.nortel.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id
	k333EX500071
	for <asrg@ietf.org>; Sun, 2 Apr 2006 23:14:37 -0400 (EDT)
Message-ID: <44309214.4040607@americasm01.nt.com>
Date: Sun, 02 Apr 2006 23:10:12 -0400
From: "Chris Lewis" <clewis@nortel.com>
User-Agent: Mozilla Thunderbird 1.0.6 (Windows/20050716)
X-Accept-Language: en-us, en
MIME-Version: 1.0
CC: asrg@ietf.org
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
References: <88885AD16AAA9F4899BB9FB751D4F5A5B70919@phost016.habeas.intermedia.net>	<E1FP1vs-0008WV-00@chiark.greenend.org.uk>	<20060331005839.GB6958@ender>
	<442D3368.7040202@americasm01.nt.com>	<20060331234310.GA9179@ender>
	<442DCB75.7050409@americasm01.nt.com>	<20060401021616.GA9651@ender>
	<442EE80C.5080500@americasm01.nt.com>	<20060403005123.GB11416@ender>
	<200604030238.WAA01085@Sparkle.Rodents.Montreal.QC.CA>
In-Reply-To: <200604030238.WAA01085@Sparkle.Rodents.Montreal.QC.CA>
X-Enigmail-Version: 0.93.0.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-Spam-Score: 0.1 (/)
X-Scan-Signature: 8b30eb7682a596edff707698f4a80f7d
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

der Mouse wrote:
>>As Steve Atkins mentioned later in the thread, there's efficacy and
>>accuracy, two different aspects.  I don't think there's anybody who
>>would dispute efficacy - rejecting at the source is clearly more
>>efficient.
> 
> 
> That last word makes it appear you are confusing efficacy
> ("effectiveness") with efficiency ("efficientness").

Obviously, I blindly followed that misuse, to whit:
- -------------------------------
>As Steve Atkins mentioned later in the thread, there's efficacy and
>> accuracy, two different aspects. I don't think there's anybody who would
>> dispute efficacy - rejecting at the source is clearly more efficient.


Efficacy has nothing to do with the success/failure of DNSBLs in a
general sense.  Nor is rejecting at source a necessary consequence of
DNSBLs.  We use DNSBLs, do rejections, but still get to quarantine the
whole email.
- -----------------------------
What I intended to say is that the DNSBL's efficiency (if that) isn't
relevant to success/failure _unless_ it impairs its useability.  And as
such, that's a per-instance issue, not generic to the technique.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3-nr1 (Windows 2000)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iQCVAwUBRDCSE53FmCyJjHfhAQJblgQA1CUQ8zDR3bOrkrBNK+jtOI5YSPYYgXWS
yeuVuHDh+Oc+poX9Mi4ijEGNXe4VEKCg5dY7UfJ1AuETGtJPrH0p/riIRLoMXcIZ
WEAzcZb+qidZYV586guhnRPtJgwkSF+RvmQp7oS1A2eTBHeLYn05lu082aMsRomh
wKhJKU3h9kU=
=yzxg
-----END PGP SIGNATURE-----

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Sun Apr 02 23:38:09 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQFtF-0007UX-Si; Sun, 02 Apr 2006 23:38:05 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQFtD-0007UM-Lv
	for asrg@ietf.org; Sun, 02 Apr 2006 23:38:03 -0400
Received: from omta02ps.mx.bigpond.com ([144.140.83.154])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FQFtC-0001mJ-7g
	for asrg@ietf.org; Sun, 02 Apr 2006 23:38:03 -0400
Received: from localhost.localdomain ([60.226.77.107])
	by omta02ps.mx.bigpond.com with ESMTP id
	<20060403033759.JLFG3168.omta02ps.mx.bigpond.com@localhost.localdomain>
	for <asrg@ietf.org>; Mon, 3 Apr 2006 03:37:59 +0000
Received: from ender (ender.scoobynet [192.168.0.3])
	by localhost.localdomain (Postfix) with ESMTP id 7D381C0
	for <asrg@ietf.org>; Mon,  3 Apr 2006 13:37:37 +1000 (EST)
Received: by ender (Postfix, from userid 1000)
	id 50C724DD61; Mon,  3 Apr 2006 13:36:07 +1000 (EST)
Date: Mon, 3 Apr 2006 13:36:07 +1000
From: Laird Breyer <laird@lbreyer.com>
To: asrg@ietf.org
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
Message-ID: <20060403033607.GA12316@ender>
Mail-Followup-To: asrg@ietf.org
References: <88885AD16AAA9F4899BB9FB751D4F5A5B70919@phost016.habeas.intermedia.net>
	<E1FP1vs-0008WV-00@chiark.greenend.org.uk>
	<20060331005839.GB6958@ender> <442D3368.7040202@americasm01.nt.com>
	<20060331234310.GA9179@ender> <442DCB75.7050409@americasm01.nt.com>
	<20060401021616.GA9651@ender> <442EE80C.5080500@americasm01.nt.com>
	<20060403005123.GB11416@ender>
	<200604030222.k332MvS14090@panix1.panix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <200604030222.k332MvS14090@panix1.panix.com>
User-Agent: Mutt/1.5.11+cvs20060126
X-Spam-Score: 0.1 (/)
X-Scan-Signature: 39bd8f8cbb76cae18b7e23f7cf6b2b9f
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
Reply-To: laird@lbreyer.com
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

On Apr 02 2006, Seth Breidbart wrote:
> Laird Breyer <laird@lbreyer.com> wrote:
> > On Apr 01 2006, Chris Lewis wrote:
> 
> >> _Anything_ that uses "adjudicators" to determine whether an email is
> >> spam or ham is clearly missing an important clue: spam isn't about
> >> content, it's about _consent_.  It is a behaviour (sending without
> >> consent), NOT what is in the email.
> >
> > It still needs a human adjudicator. In this case, a human who
> > interprets the CAN-SPAM regulations and decides whether (the legal
> > definition of) consent did apply.
> 
> If the adjudicator comes in after the fact, it is _impossible_ to
> tell.
> 
> Example: sethbr@panix.com subscribed to a non-confirming newsletter,
> and typoed the email address.  When he noticed he wasn't getting it,
> he subscribed again, typing the address correctly this time.
> 
> Panix gets two copies.  One to me, one to him.
> 
> One of them is spam.

Correct. The formal test must specify the adjudicator. It doesn't
matter what the cause of the spam is (ie a benign typo here), only the
measurable result. If you're the adjudicator, then it's spam. If
sethbr is the adjudicator, then it's not spam. If you both are, then
the formal test must specify how to deal with ties. If the law is the
test then there are two deliveries, and one of them increments the
spam count.

We can always argue about testing criteria, but as long as they are
clearly spelled out to the experts, there is value in such tests.

-- 
Laird Breyer.

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Sun Apr 02 23:47:09 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQG1z-0001Ub-Jw; Sun, 02 Apr 2006 23:47:07 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQG1z-0001UW-0s
	for asrg@ietf.org; Sun, 02 Apr 2006 23:47:07 -0400
Received: from omta01ps.mx.bigpond.com ([144.140.82.153])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FQG1w-0002BL-Jj
	for asrg@ietf.org; Sun, 02 Apr 2006 23:47:07 -0400
Received: from localhost.localdomain ([60.226.77.107])
	by omta01ps.mx.bigpond.com with ESMTP id
	<20060403034656.HKMN19070.omta01ps.mx.bigpond.com@localhost.localdomain>;
	Mon, 3 Apr 2006 03:46:56 +0000
Received: from ender (ender.scoobynet [192.168.0.3])
	by localhost.localdomain (Postfix) with ESMTP id DC341C0;
	Mon,  3 Apr 2006 13:46:33 +1000 (EST)
Received: by ender (Postfix, from userid 1000)
	id AED704DD61; Mon,  3 Apr 2006 13:45:03 +1000 (EST)
Date: Mon, 3 Apr 2006 13:45:03 +1000
From: Laird Breyer <laird@lbreyer.com>
To: der Mouse <mouse@Rodents.Montreal.QC.CA>
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
Message-ID: <20060403034503.GB12316@ender>
Mail-Followup-To: der Mouse <mouse@Rodents.Montreal.QC.CA>,
	asrg@ietf.org
References: <88885AD16AAA9F4899BB9FB751D4F5A5B70919@phost016.habeas.intermedia.net>
	<E1FP1vs-0008WV-00@chiark.greenend.org.uk>
	<20060331005839.GB6958@ender> <442D3368.7040202@americasm01.nt.com>
	<20060331234310.GA9179@ender> <442DCB75.7050409@americasm01.nt.com>
	<20060401021616.GA9651@ender> <442EE80C.5080500@americasm01.nt.com>
	<20060403005123.GB11416@ender>
	<200604030238.WAA01085@Sparkle.Rodents.Montreal.QC.CA>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <200604030238.WAA01085@Sparkle.Rodents.Montreal.QC.CA>
User-Agent: Mutt/1.5.11+cvs20060126
X-Spam-Score: 0.1 (/)
X-Scan-Signature: 7a6398bf8aaeabc7a7bb696b6b0a2aad
Cc: asrg@ietf.org
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
Reply-To: laird@lbreyer.com
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

On Apr 02 2006, der Mouse wrote:
> > As Steve Atkins mentioned later in the thread, there's efficacy and
> > accuracy, two different aspects.  I don't think there's anybody who
> > would dispute efficacy - rejecting at the source is clearly more
> > efficient.
> 
> That last word makes it appear you are confusing efficacy
> ("effectiveness") with efficiency ("efficientness").

Ah, you're right, I am using the wrong word :(, I meant efficiency. Thanks.

-- 
Laird Breyer.

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Mon Apr 03 01:19:01 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQHS8-0004C7-SF; Mon, 03 Apr 2006 01:18:12 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQHS6-0004B0-OC
	for asrg@ietf.org; Mon, 03 Apr 2006 01:18:10 -0400
Received: from omta01sl.mx.bigpond.com ([144.140.92.153])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FQHS6-0005UW-05
	for asrg@ietf.org; Mon, 03 Apr 2006 01:18:10 -0400
Received: from localhost.localdomain ([60.226.77.107])
	by omta01sl.mx.bigpond.com with ESMTP id
	<20060403051802.CSBO25377.omta01sl.mx.bigpond.com@localhost.localdomain>;
	Mon, 3 Apr 2006 05:18:02 +0000
Received: from ender (ender.scoobynet [192.168.0.3])
	by localhost.localdomain (Postfix) with ESMTP id 31475C0;
	Mon,  3 Apr 2006 15:17:39 +1000 (EST)
Received: by ender (Postfix, from userid 1000)
	id E902D4DD61; Mon,  3 Apr 2006 15:16:08 +1000 (EST)
Date: Mon, 3 Apr 2006 15:16:08 +1000
From: Laird Breyer <laird@lbreyer.com>
To: Chris Lewis <clewis@nortel.com>
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
Message-ID: <20060403051608.GC12316@ender>
Mail-Followup-To: Chris Lewis <clewis@nortel.com>, asrg@ietf.org
References: <88885AD16AAA9F4899BB9FB751D4F5A5B70919@phost016.habeas.intermedia.net>
	<E1FP1vs-0008WV-00@chiark.greenend.org.uk>
	<20060331005839.GB6958@ender> <442D3368.7040202@americasm01.nt.com>
	<20060331234310.GA9179@ender> <442DCB75.7050409@americasm01.nt.com>
	<20060401021616.GA9651@ender> <442EE80C.5080500@americasm01.nt.com>
	<20060403005123.GB11416@ender>
	<44308BDB.8060306@americasm01.nt.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <44308BDB.8060306@americasm01.nt.com>
User-Agent: Mutt/1.5.11+cvs20060126
X-Spam-Score: 0.1 (/)
X-Scan-Signature: 7fa173a723009a6ca8ce575a65a5d813
Cc: asrg@ietf.org
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
Reply-To: laird@lbreyer.com
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

On Apr 02 2006, Chris Lewis wrote:

> First of all, CAN-SPAM doesn't define what is spam, and what isn't. It
> simply defines a subset of spam that is legally actionable under US law.
> The definition of spam doesn't depend on local ordinances.
> 
> Analysis under CAN-SPAM determinations simply isn't relevant, for
> example, in our jurisdiction.  Spam is spam, whether it's in Canada, or
> the US or Australia.

Only when using your definition as anything that is sent without the
receiver's consent. The merits of various definitions have been
discussed to death on this list in the past, there's no need to rehash
them.  I mentioned CAN-SPAM as I got the impression you used consent
in a more than informal way.

To stay on topic, do you accept that with your definition, the only
authority which can reliably decide consent (and therefore spamminess)
is the receiver?  Moreover, insofar as the automated filtering systems
that are in place are concerned, these act as proxies for the
recipient only.
 
I ask this because it is relevant to the next bit:

> > Being an automated method of filtering, it depends on a set of criteria
> > and conditions stored in the program logic and database. 
> > These criteria and conditions, if given in full to a human adjudicator,
> > could be judged by said human just as well with authority.  
> 
> And these adjudicators would be able to make such judgements correctly?
> Unlikely.  And furthermore, even if they were not 100% accurate, ONLY
> the overall result matters.

How do you define the overall result if you don't take
recipients authority as final. I should say that the adjudicators I mention
are assumed to be actual recipients in this case.

> 
> We don't judge testing of drugs on whether "adjudicators" understand how
> the drugs work, we judge drug efficacy on _results_ in trials.

Correct, but _results_ are well defined in terms of observable
biological effects. If spam is defined in terms of recipient consent,
then observation requires asking the recipients if their consent is
given in each test case.


> > As Steve Atkins mentioned later in the thread, there's efficacy and 
> > accuracy, two different aspects. I don't think there's anybody who would
> > dispute efficacy - rejecting at the source is clearly more efficient.
> 
> Efficacy has nothing to do with the success/failure of DNSBLs in a
> general sense.  Nor is rejecting at source a necessary consequence of
> DNSBLs.  We use DNSBLs, do rejections, but still get to quarantine the
> whole email.

I apologize for using efficacy incorrectly, I meant efficiency in the
sense of machine workload. 

However, if you accept the email in quarantine and silently reject
after the fact (correct me if I'm wrong), then I do not see that
Steve's point applies. You can test the effectiveness of your choice
of dnsbls without interacting with the spammer who sends the
message. In particular, you can do what Justin Mason outlined.

> 
> More importantly, Steve (and I) pointed out that these techniques simply
> can't be tested to the level you might want to see, because they have to
> be done with real mailstreams in realtime, and with, for example, grey
> listing, it's simply NOT possible to judge in your terms whether the
> email was spam or not.  Applying the heuristic changes the sender
> behavior, and you don't have the email to judge.

What you do have is all the information that the mail server has when
it makes the rejection choice. Namely X number of attempts from the
same host in Y time. If consent can only be authoritatively judged by
the recipient, then this recipient should be asked to judge based on
precisely that information (during a QA test, for verification). But
see the next paragraph. 

> The situation is simple: there's a variety of techniques that simply
> cannot be tested to this level of academic certainty with "recipient" or
> "recipient's judge" techniques, because applying them changes the
> behaviour, and secondly, the recipient _cannot_ have anything to judge
> it with.

No behaviour is changed by auditing the accuracy of a system after the
fact. But it doesn't come for free, logging relevant information is
necessary.

The recipient may well find it difficult to judge based on so little
information and without seeing the never accepted mail, whereas you
have much more experience with mail systems in general and for you the
greylisting record is quite sufficient, without the need to see the
mail at all.

I think that points to the fact that it makes more practical sense for
_you_ to judge consent than for the intended recipients, in these
cases.  But that requires changing the meaning of spam slightly to
include you as an authoritative proxy for the recipient in difficult
cases. If so, then your comments (notifying the sender etc)
makes a lot more sense to me.

I don't want to make this reply too long, so I won't do a point by
point reply. I think we agree that your points apply to regular
frequent business communication, so there's no need to criticize them
again from a different perspective.


-- 
Laird Breyer.

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Mon Apr 03 01:51:15 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQHxI-0003az-HB; Mon, 03 Apr 2006 01:50:24 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQHxG-0003ZA-Qu
	for asrg@ietf.org; Mon, 03 Apr 2006 01:50:22 -0400
Received: from tom.iecc.com ([208.31.42.38])
	by ietf-mx.ietf.org with smtp (Exim 4.43) id 1FQHxF-0006tF-Fa
	for asrg@ietf.org; Mon, 03 Apr 2006 01:50:22 -0400
Received: (qmail 18761 invoked from network); 3 Apr 2006 05:50:21 -0000
Received: from mail.iecc.com (208.31.42.99)
	by mail.iecc.com with QMQP; 3 Apr 2006 05:50:21 -0000
Date: 3 Apr 2006 05:50:20 -0000
Message-ID: <20060403055020.19752.qmail@simone.iecc.com>
From: John Levine <asrg@johnlevine.com>
To: asrg@ietf.org
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
In-Reply-To: <20060403051608.GC12316@ender>
Organization: 
Mime-Version: 1.0
Content-type: text/plain; charset=iso-8859-1
Content-transfer-encoding: 7bit
X-Spam-Score: -4.3 (----)
X-Scan-Signature: 798b2e660f1819ae38035ac1d8d5e3ab
Cc: laird@lbreyer.com
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

>To stay on topic, do you accept that with your definition, the only
>authority which can reliably decide consent (and therefore spamminess)
>is the receiver?

Not really.  I get spam complaints all the time for mail from lists
that I know perfectly well that they signed up for and confirmed.
"Oh, I don't want that any more."  For the ones that aren't totally
redacted, my setup turns them into unsubs so they don't get any more
mail for that particular list, but I don't think it's fair to count
mail as spam if it depends on reading the recipient's mind in
real-time.

Testing spam filters is a really, really hard problem, for all the
reasons that people have mentioned, basically that you can't test
without perturbing the system you're testing, and you can't capture
enough state to rerun the same test more than once.  So you have to
estimate based on complaint rates from bounces and the like.

R's,
John



_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Mon Apr 03 05:42:37 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQLZs-0005VX-Bv; Mon, 03 Apr 2006 05:42:28 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQLZr-0005VS-ED
	for asrg@ietf.org; Mon, 03 Apr 2006 05:42:27 -0400
Received: from mailout1.igs.net ([216.58.97.34])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FQLZo-0000ki-5w
	for asrg@ietf.org; Mon, 03 Apr 2006 05:42:27 -0400
Received: from waltdnes.org (i216-58-60-248.cybersurf.com [216.58.60.248])
	by mailout1.igs.net (Postfix) with SMTP id 872845A3A
	for <asrg@ietf.org>; Mon,  3 Apr 2006 05:42:23 -0400 (EDT)
Received: by waltdnes.org (sSMTP sendmail emulation);
	Mon,  3 Apr 2006 05:42:23 -0400
From: "Walter Dnes" <waltdnes@waltdnes.org>
Date: Mon, 3 Apr 2006 05:42:23 -0400
To: ASRG list <asrg@ietf.org>
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
Message-ID: <20060403094223.GA18195@waltdnes.org>
References: <88885AD16AAA9F4899BB9FB751D4F5A5B70919@phost016.habeas.intermedia.net>
	<E1FP1vs-0008WV-00@chiark.greenend.org.uk>
	<20060331005839.GB6958@ender> <442D3368.7040202@americasm01.nt.com>
	<20060331234310.GA9179@ender> <442DCB75.7050409@americasm01.nt.com>
	<20060401021616.GA9651@ender> <442EE80C.5080500@americasm01.nt.com>
	<20060403005123.GB11416@ender>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20060403005123.GB11416@ender>
User-Agent: Mutt/1.5.11
X-Spam-Score: 0.1 (/)
X-Scan-Signature: 50a516d93fd399dc60588708fd9a3002
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

On Mon, Apr 03, 2006 at 10:51:24AM +1000, Laird Breyer wrote

> Yes, and you're appreciated for it. But if the answer is easy enough
> to dismiss it won't stop the next person from deeming dnsbls harmful
> yet again. There's a lot of folklore and crap on the net, and that's
> the environment in which opinions are formed. The ASRG doesn't have
> the luxury of dismissing ideas from the (valuable) experience of its
> members only.

  Maybe the only permanent answer is to...

  - *not* deny that there may be *SOME* harm from DNSBLs

  - but show that there is *MORE* harm in *NOT* using them


  In Chris' case...

  - how many messages do his sytems reject per day?

  - how much additional DASD farm would be required to store the
    messages that wouldn't be rejected?

  - how much additional CPU power would be required to crunch those
    no-longer-rejected messages through Church's pet filters?

  - how much additional air-conditioned and fully-powered computer rack
    space would Chris' employer have to rent or buy+maintain to house
    the additional storage+servers required in the 2 previous points?

  - how much would it cost to pay for additional salary and benefits,
    and office-cubicle space, and desktop PCs that would be required for
    the extra employees to maintain+operate the extra storage+servers
    identified above?

  And furthermore, remember that in the case of a false-positive...

  - a DNSBL will send a 5XX reject to a legitimate sender's MTA, which
    will notify the sender of the reject.

  - a content-filter will bury the email in a "spam folder" with
    thousands of real spam, where it'll probably never be found.  The
    sender will believe that the intended recipient has received the
    message and ignored it, while the intended recipient will believe
    that the sender hasn't sent the message.

-- 
Walter Dnes <waltdnes@waltdnes.org> In linux /sbin/init is Job #1
My musings on technology and security at http://tech_sec.blog.ca

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Mon Apr 03 08:00:03 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQNiq-0006Yd-Nh; Mon, 03 Apr 2006 07:59:52 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQNip-0006YY-6C
	for asrg@ietf.org; Mon, 03 Apr 2006 07:59:51 -0400
Received: from dogma.boxhost.net ([195.218.96.101])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FQNin-0006kq-UA
	for asrg@ietf.org; Mon, 03 Apr 2006 07:59:51 -0400
Received: from radish.jmason.org (localhost [127.0.0.1])
	by dogma.boxhost.net (Postfix) with ESMTP id E59843100B1;
	Mon,  3 Apr 2006 12:59:38 +0100 (IST)
Received: from jmason.org (localhost [127.0.0.1])
	by radish.jmason.org (Postfix) with ESMTP id 4A93A590021;
	Mon,  3 Apr 2006 12:33:04 +0100 (IST)
To: John Levine <asrg@johnlevine.com>
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt 
In-Reply-To: <20060403055020.19752.qmail@simone.iecc.com> 
From: jm@jmason.org (Justin Mason)
X-GPG-Key-Fingerprint: 1368 71CE 3627 9CD3 FA1B  0B63 3091 7972 298B C7D0
Date: Mon, 03 Apr 2006 12:33:04 +0100
Message-Id: <20060403113304.4A93A590021@radish.jmason.org>
X-Spam-Score: 0.0 (/)
X-Scan-Signature: cf4fa59384e76e63313391b70cd0dd25
Cc: asrg@ietf.org, laird@lbreyer.com
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org


John Levine writes:
> Testing spam filters is a really, really hard problem, for all the
> reasons that people have mentioned, basically that you can't test
> without perturbing the system you're testing, and you can't capture
> enough state to rerun the same test more than once.  So you have to
> estimate based on complaint rates from bounces and the like.

The Spam Heisenberg Uncertainty Principle. ;)

by the way this is one reason why I, personally, prefer to accept spam in
full, instead of rejecting during the SMTP transaction, greylisting, or
tarpitting.   It's more practical to test and instrument that way, and
thereby get a solid idea of FP rates, instead of guesstimation.

--j.

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Mon Apr 03 11:08:21 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQQf6-0007eW-EZ; Mon, 03 Apr 2006 11:08:12 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQQf4-0007dX-Fj
	for asrg@ietf.org; Mon, 03 Apr 2006 11:08:10 -0400
Received: from adsl-216-102-208-10.dsl.snfc21.pacbell.net ([216.102.208.10]
	helo=fasolt.mtcc.com) by ietf-mx.ietf.org with esmtp (Exim 4.43)
	id 1FQQf3-0006Qo-2l
	for asrg@ietf.org; Mon, 03 Apr 2006 11:08:10 -0400
DKIM-Signature: a=rsa-sha1; q=dns; l=1272; t=1144076886; x=1144940886;
	c=relaxed/simple; s=dicks.drop.kirkwood;
	h=Content-Type:From:Subject:Content-Transfer-Encoding:MIME-Version;
	d=mtcc.com; i=mike@mtcc.com; z=From:Michael=20Thomas=20<mike@mtcc.com>
	|Subject:Re=3A=20[Asrg]=20Comments=20on=20draft-church-dnsbl-harmful-01.txt
	|To:Walter=20Dnes=20<waltdnes@waltdnes.org>
	|Cc:ASRG=20list=20<asrg@ietf.org>;
	X=v=3Dcisco.com=3B=20h=3DDupfz3d845EH0fvEYDWwrRm3m/0=3D;
	b=JtP3/YIe1C3He1VWTBk39c9skWeUDsQSoITGBid8DqKZyjod2k1zMEPGymk26r5AYqWWWKBA
	bPsEnsUUhIlJ8vlgIZPJZ+CL68CL3kiNKbEBTOlPKJAXGk7HuVVPsJ/F;
Received: from [216.102.208.10]
	(IDENT:U2FsdGVkX1+pJJXOSIZmZF+Scd5iv/uxEl2+HkyDt6c@fasolt.mtcc.com
	[216.102.208.10] (may be forged))
	by fasolt.mtcc.com (8.13.1/8.13.1) with ESMTP id k33F86lJ016623;
	Mon, 3 Apr 2006 08:08:06 -0700
Message-ID: <44313A55.3050207@mtcc.com>
Date: Mon, 03 Apr 2006 08:08:05 -0700
From: Michael Thomas <mike@mtcc.com>
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
	rv:1.7.10) Gecko/20050720 Thunderbird/1.0.6 Fedora/1.0.6-1.1.fc3
	Mnenhy/0.7.2.0
X-Accept-Language: en-us, en
MIME-Version: 1.0
To: Walter Dnes <waltdnes@waltdnes.org>
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
References: <88885AD16AAA9F4899BB9FB751D4F5A5B70919@phost016.habeas.intermedia.net>	<E1FP1vs-0008WV-00@chiark.greenend.org.uk>	<20060331005839.GB6958@ender>
	<442D3368.7040202@americasm01.nt.com>	<20060331234310.GA9179@ender>
	<442DCB75.7050409@americasm01.nt.com>	<20060401021616.GA9651@ender>
	<442EE80C.5080500@americasm01.nt.com>	<20060403005123.GB11416@ender>
	<20060403094223.GA18195@waltdnes.org>
In-Reply-To: <20060403094223.GA18195@waltdnes.org>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Authentication-Results: fasolt.mtcc.com; header.From=mike@mtcc.com; dkim=pass (
	sig from mtcc.com verified; ); 
X-Spam-Score: 0.1 (/)
X-Scan-Signature: 9466e0365fc95844abaf7c3f15a05c7d
Cc: ASRG list <asrg@ietf.org>
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

Walter Dnes wrote:
>   Maybe the only permanent answer is to...
> 
>   - *not* deny that there may be *SOME* harm from DNSBLs
> 
>   - but show that there is *MORE* harm in *NOT* using them

It doesn't seem to me that boosterism is really the right answer
though. DNSBL's are what they are. What would be nice to know as
an outsider is _what_ they are, how they roughly work, what their
upside is, and what their downside is. That way you can make the
risk/reward tradeoff which is invariably context sensitive.

>   - a DNSBL will send a 5XX reject to a legitimate sender's MTA, which
>     will notify the sender of the reject.
> 
>   - a content-filter will bury the email in a "spam folder" with
>     thousands of real spam, where it'll probably never be found.  The
>     sender will believe that the intended recipient has received the
>     message and ignored it, while the intended recipient will believe
>     that the sender hasn't sent the message.

That depends a whole lot on the receiving end's policy, doesn't it?
Spamassassin for example can use DNSBL's, the result of which is
effectively silent discard. And it would be a fairly trivial exercise
to make Spamassassin cause a 5xx to be generated for > 5.0 SA scores.

		Mike


_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Mon Apr 03 11:32:03 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQR28-0002Uq-Mk; Mon, 03 Apr 2006 11:32:00 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQR26-0002Uk-Gk
	for asrg@ietf.org; Mon, 03 Apr 2006 11:31:58 -0400
Received: from mpls-qmqp-02.inet.qwest.net ([63.231.195.113])
	by ietf-mx.ietf.org with smtp (Exim 4.43) id 1FQR25-0007Pw-6c
	for asrg@ietf.org; Mon, 03 Apr 2006 11:31:58 -0400
Received: (qmail 64526 invoked by uid 0); 3 Apr 2006 15:31:56 -0000
Received: from unknown (63.231.195.7)
	by mpls-qmqp-02.inet.qwest.net with QMQP; 3 Apr 2006 15:31:56 -0000
Received: from 71-212-183-171.hlrn.qwest.net (HELO ?192.168.0.3?)
	(71.212.183.171)
	by mpls-pop-07.inet.qwest.net with SMTP; 3 Apr 2006 15:31:56 -0000
Date: Mon, 3 Apr 2006 09:31:53 -0600
Message-Id: <4C092F54-E522-4E1D-9D97-A658B4846318@qwest.net>
From: "Dan Oetting" <dan_oetting@qwest.net>
To: "Anti Spam Research Group" <asrg@ietf.org>
Mime-Version: 1.0 (Apple Message framework v746.3)
In-Reply-To: <Pine.GSO.4.64.0603301605540.23037@nber6.nber.org>
References: <88885AD16AAA9F4899BB9FB751D4F5A5B708FA@phost016.habeas.intermedia.net>
	<Pine.GSO.4.64.0603301230330.6299@nber6.nber.org>
	<442C1D5F.1040407@mtcc.com>
	<Pine.GSO.4.64.0603301605540.23037@nber6.nber.org>
Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed
Content-Transfer-Encoding: 7bit
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
X-Mailer: Apple Mail (2.746.3)
X-Spam-Score: 0.0 (/)
X-Scan-Signature: 97adf591118a232206bdb5a27b217034
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org


On Mar 30, 2006, at 2:27 PM, Daniel Feenberg wrote:

> A better study of false positives would require a large corpus of  
> known good mail for a diverse set of destinations, with connecting  
> MTA IP addresses. One could query the DNSBLs for those IP  
> addresses, and calculate the probability that a legitimate message  
> would be blocked. But I haven't found a corpus of known good mail.  
> One source would be email confirmations of mailing-list signups, if  
> anyone would like to share that with me. The saved mail file of an  
> individual isn't very representative even if it is large.

Looking only at mailing list signups won't necessarily be a  
representative sample because mailing lists will tend to be clustered  
and possibly run on different servers from general email.

Tracking replies based on references headers and linking those to the  
record of the received mail should give a better overall picture of  
your users good mail sources. You would of corse need to filter out  
replies to abuse desks, vacation auto responders and forwarding  
accounts. There may be other anomalies in you mail pattern so some  
random sampling should be done to look for anything that might skew  
the results.

The overlap between the good mail sources and blacklisted sources is  
where most of the false positives are going to be. The magnitude of  
this overlap will be good enough for the first order estimate of the  
false positive potential.

The set of mail sources that are not identified as good in the above  
tracking and not identified as bad in the blacklists would be  
something to investigate further.



_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Mon Apr 03 11:46:01 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQRFd-0001YU-6E; Mon, 03 Apr 2006 11:45:57 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQRFb-0001YO-TZ
	for asrg@ietf.org; Mon, 03 Apr 2006 11:45:55 -0400
Received: from xproxy.gmail.com ([66.249.82.204])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FQRFa-0007xX-Mo
	for asrg@ietf.org; Mon, 03 Apr 2006 11:45:55 -0400
Received: by xproxy.gmail.com with SMTP id s12so910787wxc
	for <asrg@ietf.org>; Mon, 03 Apr 2006 08:45:54 -0700 (PDT)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com;
	h=received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition;
	b=ZTvqmN/IepNHK1f6LZL/agyqEKkrbgf5sNv0HDRNpGGO8g0kT0vu1ku68V6CWeZIWMdzd3vuJub+9SiljjTWbsjmOX6yfRF0Z54ZSLipCQTnzId6a9qFTNS+Wo8EwNAmqmye6iV1P+QrO4yg/mqTUUPKkjaUuLkSIXOL2IjsW54=
Received: by 10.70.96.15 with SMTP id t15mr1159257wxb;
	Mon, 03 Apr 2006 08:45:53 -0700 (PDT)
Received: by 10.70.95.3 with HTTP; Mon, 3 Apr 2006 08:45:53 -0700 (PDT)
Message-ID: <934f64a20604030845h768a55b9l3656867537ecba51@mail.gmail.com>
Date: Mon, 3 Apr 2006 10:45:53 -0500
From: "David Nicol" <davidnicol@gmail.com>
To: "anti-spam research group" <asrg@ietf.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline
X-Spam-Score: 0.0 (/)
X-Scan-Signature: 7bac9cb154eb5790ae3b2913587a40de
Subject: [Asrg] entirely balanced infoworld article on blacklisting
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

log-in required.  If you have the free subscriptiobn in paper form, it
starts on page 35.
http://www.infoworld.com/pdf/special_report/2006/13SRblacklist.pdf

--
David L Nicol
"the opportunity of making good with a hard undertaking - one that had
been held impossible of realization - would be a strong asset to the
firm's reputation" -- William Russell, 1860
http://cronos.advenge.com/pc/americana/PonyExpress/p15.html

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Mon Apr 03 12:12:03 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQRep-0008Rc-8Z; Mon, 03 Apr 2006 12:11:59 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQRen-0008RE-Rv
	for asrg@ietf.org; Mon, 03 Apr 2006 12:11:57 -0400
Received: from zcars04f.nortelnetworks.com ([47.129.242.57]
	helo=zcars04f.nortel.com) by ietf-mx.ietf.org with esmtp (Exim 4.43)
	id 1FQRen-0000bF-Es
	for asrg@ietf.org; Mon, 03 Apr 2006 12:11:57 -0400
Received: from zrtphxs1.corp.nortel.com
	(zrtphxs1.corp.nort...s0.corp.nortel.com [47.140.202.46])
	by zcars04f.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id
	k33GBst07338
	for <asrg@ietf.org>; Mon, 3 Apr 2006 12:11:54 -0400 (EDT)
Received: from zrtps0m6.us.nortel.com ([47.140.192.58]) by
	zrtphxs1.corp.nortel.com with Microsoft SMTPSVC(6.0.3790.211); 
	Mon, 3 Apr 2006 12:10:47 -0400
Received: from [47.129.150.195] (clewis-2.ca.nortel.com [47.129.150.195])
	by zrtps0m6.us.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id
	k33GAXg06476
	for <asrg@ietf.org>; Mon, 3 Apr 2006 12:10:33 -0400 (EDT)
Message-ID: <443148F9.3040400@americasm01.nt.com>
Date: Mon, 03 Apr 2006 12:10:33 -0400
From: "Chris Lewis" <clewis@nortel.com>
User-Agent: Mozilla Thunderbird 1.0.6 (Windows/20050716)
X-Accept-Language: en-us, en
MIME-Version: 1.0
CC: ASRG list <asrg@ietf.org>
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
References: <88885AD16AAA9F4899BB9FB751D4F5A5B70919@phost016.habeas.intermedia.net>	<E1FP1vs-0008WV-00@chiark.greenend.org.uk>	<20060331005839.GB6958@ender>	<442D3368.7040202@americasm01.nt.com>	<20060331234310.GA9179@ender>	<442DCB75.7050409@americasm01.nt.com>	<20060401021616.GA9651@ender>	<442EE80C.5080500@americasm01.nt.com>	<20060403005123.GB11416@ender>	<20060403094223.GA18195@waltdnes.org>
	<44313A55.3050207@mtcc.com>
In-Reply-To: <44313A55.3050207@mtcc.com>
X-Enigmail-Version: 0.93.0.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-OriginalArrivalTime: 03 Apr 2006 16:10:47.0870 (UTC)
	FILETIME=[2B59DDE0:01C65739]
X-Spam-Score: 0.1 (/)
X-Scan-Signature: 0fa76816851382eb71b0a882ccdc29ac
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Michael Thomas wrote:
> Walter Dnes wrote:
> 
>>   Maybe the only permanent answer is to...
>>
>>   - *not* deny that there may be *SOME* harm from DNSBLs
>>
>>   - but show that there is *MORE* harm in *NOT* using them
> 
> 
> It doesn't seem to me that boosterism is really the right answer
> though. DNSBL's are what they are. What would be nice to know as
> an outsider is _what_ they are, how they roughly work, what their
> upside is, and what their downside is. That way you can make the
> risk/reward tradeoff which is invariably context sensitive.

I think as a nutshell, from the IETF perspective, Church's draft would
have to:

   a) become a DNSBLs _may_ be harmful if you don't...
   b) _know_ what the filters you're using actually do (at least
     in terms of results).

As such, it could be a kernel for a more general "filtering BCP" paper.
But I don't think Church is willing to write one of those ;-)

[I've written a sketch for one, but nothing beyond that.  There was a
mailing list created by Crocker (I think) to carry that onwards to
something useful, but that mailing list was DOA.]

>>   - a DNSBL will send a 5XX reject to a legitimate sender's MTA, which
>>     will notify the sender of the reject.

It doesn't have to.  But it can (and does in perhaps most implementations).

>>   - a content-filter will bury the email in a "spam folder" with
>>     thousands of real spam, where it'll probably never be found.  The
>>     sender will believe that the intended recipient has received the
>>     message and ignored it, while the intended recipient will believe
>>     that the sender hasn't sent the message.

It doesn't have to.  But it can (and does in many implementations).

> That depends a whole lot on the receiving end's policy, doesn't it?
> Spamassassin for example can use DNSBL's, the result of which is
> effectively silent discard. And it would be a fairly trivial exercise
> to make Spamassassin cause a 5xx to be generated for > 5.0 SA scores.

You have to be careful in your generalizations.  Virtually _no_
filtering (aside from things like grey-listing) forces you to adopt one
form of disposition or other.

In our installation: DNSBLs reject inline _AND_ quarantine[+] (which
means it still takes the session up to the end of DATA, and then 550's
that).

Similarly, our content filters, including SpamAssassin, ALSO reject
inline and quarantine (550's after DATA).  There is no "post-accept"
filtering of any consequence in our environment (except for the
infrastructure A-V to catch the viruses the front end filters miss, and
a few users using their own client-end filters).

[+] Quarantine is conceptually equivalent to spam folders, but
implemented differently.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3-nr1 (Windows 2000)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iQCVAwUBRDFI+Z3FmCyJjHfhAQKz3wP+J6vnFEdg2uwdAXG6Fo7UBMLViXQphZQg
IAqCBcKT34nzFZKKNWqtXvj5g9gyCfyH5corIqobymZ6gXuOyMe3KhYFmJzUd40K
+0V1kb120Z4YSPV/4vkGoFSyJFTiOD7OrQ50/MdCumSjL5sgFoyOEXjvfrQFF0ct
1ufHI3Hrevo=
=d8Ry
-----END PGP SIGNATURE-----

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Mon Apr 03 12:12:53 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQRfh-0000Gd-PW; Mon, 03 Apr 2006 12:12:53 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQRfh-0000G2-0E
	for asrg@ietf.org; Mon, 03 Apr 2006 12:12:53 -0400
Received: from zrtps0kn.nortelnetworks.com ([47.140.192.55]
	helo=zrtps0kn.nortel.com) by ietf-mx.ietf.org with esmtp (Exim 4.43)
	id 1FQRff-0000el-Lq
	for asrg@ietf.org; Mon, 03 Apr 2006 12:12:52 -0400
Received: from zrtphxs1.corp.nortel.com
	(zrtphxs1.corp.nort...s0.corp.nortel.com [47.140.202.46])
	by zrtps0kn.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id
	k33GCnD05060
	for <asrg@ietf.org>; Mon, 3 Apr 2006 12:12:49 -0400 (EDT)
Received: from zrtps0m6.us.nortel.com ([47.140.192.58]) by
	zrtphxs1.corp.nortel.com with Microsoft SMTPSVC(6.0.3790.211); 
	Mon, 3 Apr 2006 12:12:46 -0400
Received: from [47.129.150.195] (clewis-2.ca.nortel.com [47.129.150.195])
	by zrtps0m6.us.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id
	k33GCVg08349
	for <asrg@ietf.org>; Mon, 3 Apr 2006 12:12:31 -0400 (EDT)
Message-ID: <4431496F.9060507@americasm01.nt.com>
Date: Mon, 03 Apr 2006 12:12:31 -0400
From: "Chris Lewis" <clewis@nortel.com>
User-Agent: Mozilla Thunderbird 1.0.6 (Windows/20050716)
X-Accept-Language: en-us, en
MIME-Version: 1.0
CC: asrg@ietf.org
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
References: <20060403113304.4A93A590021@radish.jmason.org>
In-Reply-To: <20060403113304.4A93A590021@radish.jmason.org>
X-Enigmail-Version: 0.93.0.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-OriginalArrivalTime: 03 Apr 2006 16:12:46.0695 (UTC)
	FILETIME=[722D1F70:01C65739]
X-Spam-Score: 0.1 (/)
X-Scan-Signature: 0bc60ec82efc80c84b8d02f4b0e4de22
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Justin Mason wrote:

> The Spam Heisenberg Uncertainty Principle. ;)

Perhaps not surprisingly, I was thinking about Heisenberg too.

Or, at least, the "Cockroach Uncertainty Principle".  Spammers, like
cockroaches, behave differently when you shine a light on them.

> by the way this is one reason why I, personally, prefer to accept spam in
> full, instead of rejecting during the SMTP transaction, greylisting, or
> tarpitting.   It's more practical to test and instrument that way, and
> thereby get a solid idea of FP rates, instead of guesstimation.

Which is why we do that too ;-)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3-nr1 (Windows 2000)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iQCVAwUBRDFJb53FmCyJjHfhAQIRjwQAxeC1RCR9jEUcRJji29ZwAlMauF/FlPA0
xdt2nrdwI4pE/BVHvGHPhnFjMvQychIh6Cc8AnkkAzfSkpwurJAhq3X48mJpHVE+
L12kvzyGLm4mdlohLy+UBDeYOeQ8nUoqywuqLVNOGik62ZqNoSjM9DqNSdn70wKC
oXA8h7DcWeQ=
=cZu+
-----END PGP SIGNATURE-----

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Mon Apr 03 12:36:47 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQS2k-00027W-I6; Mon, 03 Apr 2006 12:36:42 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQS2j-00027R-06
	for asrg@ietf.org; Mon, 03 Apr 2006 12:36:41 -0400
Received: from mail3.panix.com ([166.84.1.74])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FQS2h-00026U-QU
	for asrg@ietf.org; Mon, 03 Apr 2006 12:36:40 -0400
Received: from panix1.panix.com (panix1.panix.com [166.84.1.1])
	by mail3.panix.com (Postfix) with ESMTP id 3F3A913EAC7
	for <asrg@ietf.org>; Mon,  3 Apr 2006 12:21:44 -0400 (EDT)
Received: (from sethb@localhost)
	by panix1.panix.com (8.11.6p3/8.8.8/PanixN1.1) id k33GLiu25860;
	Mon, 3 Apr 2006 12:21:44 -0400 (EDT)
Date: Mon, 3 Apr 2006 12:21:44 -0400 (EDT)
Message-Id: <200604031621.k33GLiu25860@panix1.panix.com>
From: Seth Breidbart <sethb@panix.com>
To: asrg@ietf.org
In-reply-to: <20060403094223.GA18195@waltdnes.org>
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
References: <88885AD16AAA9F4899BB9FB751D4F5A5B70919@phost016.habeas.intermedia.net>
	<E1FP1vs-0008WV-00@chiark.greenend.org.uk>
	<20060331005839.GB6958@ender> <442D3368.7040202@americasm01.nt.com>
	<20060331234310.GA9179@ender> <442DCB75.7050409@americasm01.nt.com>
	<20060401021616.GA9651@ender> <442EE80C.5080500@americasm01.nt.com>
	<20060403005123.GB11416@ender> <20060403094223.GA18195@waltdnes.org>
X-Spam-Score: 0.0 (/)
X-Scan-Signature: 7655788c23eb79e336f5f8ba8bce7906
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

"Walter Dnes" <waltdnes@waltdnes.org> wrote:

>  And furthermore, remember that in the case of a false-positive...
>
>  - a DNSBL will send a 5XX reject to a legitimate sender's MTA, which
>    will notify the sender of the reject.
>
>  - a content-filter will bury the email in a "spam folder" with
>    thousands of real spam, where it'll probably never be found.  The
>    sender will believe that the intended recipient has received the
>    message and ignored it, while the intended recipient will believe
>    that the sender hasn't sent the message.

Actually, a content-filter COULD generate a 5XX; of course, that
requires even more CPU capacity, because the filter has to run in
almost real time.

But that would be a useful metric: "Not using XBL would cost $COMPANY
$BIGNUM per year, and $COMPANY receives X% the amount of email AOL
does . . ."  leading to an estimate that "Not using DNSBLs would cost
those who now use them $HUGENUM per year".

Seth

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Mon Apr 03 12:46:03 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQSBm-0001Bo-Tq; Mon, 03 Apr 2006 12:46:02 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQSBl-0001B3-Fl
	for asrg@ietf.org; Mon, 03 Apr 2006 12:46:01 -0400
Received: from zcars04f.nortelnetworks.com ([47.129.242.57]
	helo=zcars04f.nortel.com) by ietf-mx.ietf.org with esmtp (Exim 4.43)
	id 1FQSBk-0002Sk-2h
	for asrg@ietf.org; Mon, 03 Apr 2006 12:46:01 -0400
Received: from zrtphxs1.corp.nortel.com
	(zrtphxs1.corp.nort...s0.corp.nortel.com [47.140.202.46])
	by zcars04f.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id
	k33Gjvt18901
	for <asrg@ietf.org>; Mon, 3 Apr 2006 12:45:57 -0400 (EDT)
Received: from zrtps0m6.us.nortel.com ([47.140.192.58]) by
	zrtphxs1.corp.nortel.com with Microsoft SMTPSVC(6.0.3790.211); 
	Mon, 3 Apr 2006 12:45:54 -0400
Received: from [47.129.150.195] (clewis-2.ca.nortel.com [47.129.150.195])
	by zrtps0m6.us.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id
	k33GjXg06347
	for <asrg@ietf.org>; Mon, 3 Apr 2006 12:45:33 -0400 (EDT)
Message-ID: <4431512B.5040506@americasm01.nt.com>
Date: Mon, 03 Apr 2006 12:45:31 -0400
From: "Chris Lewis" <clewis@nortel.com>
User-Agent: Mozilla Thunderbird 1.0.6 (Windows/20050716)
X-Accept-Language: en-us, en
MIME-Version: 1.0
To: asrg@ietf.org
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
References: <20060403055020.19752.qmail@simone.iecc.com>
In-Reply-To: <20060403055020.19752.qmail@simone.iecc.com>
X-Enigmail-Version: 0.93.0.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-OriginalArrivalTime: 03 Apr 2006 16:45:54.0720 (UTC)
	FILETIME=[1321AA00:01C6573E]
X-Spam-Score: 0.1 (/)
X-Scan-Signature: 32b73d73e8047ed17386f9799119ce43
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

John Levine wrote:
>>To stay on topic, do you accept that with your definition, the only
>>authority which can reliably decide consent (and therefore spamminess)
>>is the receiver?

> Not really.  I get spam complaints all the time for mail from lists
> that I know perfectly well that they signed up for and confirmed.
> "Oh, I don't want that any more."  For the ones that aren't totally
> redacted, my setup turns them into unsubs so they don't get any more
> mail for that particular list, but I don't think it's fair to count
> mail as spam if it depends on reading the recipient's mind in
> real-time.

Similarly, we get complaints from receivers that we blocked certain
emails that we _know_ are spam.  Like the individual who insisted that
it was necessary for him to receive the "nortel.com abuse team" messages
(aka Bagle).  Or the people who complain about us blocking important
email from their bank... that are in reality phishes.  Or the people
who've fallen for Nigerian money laundering, "change for a fraudulent
money order" scams.

[As far as we know, we managed to catch the only one of the latter here
just before the victim wrote the cheque.]

In fact, receiver adjudication would likely _lower_ the FP rate. On
balance, I believe it's more likely that a user doesn't recognize
something he really did ask for (thru some twisty chain of branding,
lowering the FP rate) than "recognize" something he didn't ask for
(raising the FP rate).

In other words, receiver adjudication would bias the evaluation
_towards_ DNSBLs (lower FPs), not away (higher FPs).  Which, if you must
accept bias (and are evaluating in a conservative approach), is the
_wrong_ direction.

Thus, receiver adjudication has an error rate, further as I'll mention
below, likely _higher_ than the measures we're trying to test.

I'll point out that in most environments who're running multiple
techniques, FN rates of individual techniques is not relevant, except in
evaluating incremental implementation cost of that technique (cost vs.
benefits).  So what if SBL-XBL only catches, say, 80% of our spam flow?
Our other techniques fill in that gap.  Our other techniques put
together can't reach even 80%.

> Testing spam filters is a really, really hard problem, for all the
> reasons that people have mentioned, basically that you can't test
> without perturbing the system you're testing, and you can't capture
> enough state to rerun the same test more than once.  So you have to
> estimate based on complaint rates from bounces and the like.

Exactly.

Except in samples that are too small to be statistically significant,
it's not possible to use "receiver adjudication" in an effective
fashion.  Not because we can't do that, but because the error rate in
"receiver adjudication" is going to be at least an order of magnitude
higher than some of the methods we're trying to test, and in fact,
likely to be strongly biased in _favour_ of of the filtering technique
(for reasons outlined above).

Most of the time the receiver is correct, no question.  Which is why we
don't deny more than an infinitessimal fraction of FP/FN complaints.

But human nature being what it is, when faced with hundreds of emails in
a folder to assess, a miniscule error rate of 1% isn't unreasonable to
expect.

Yet, we already know that the CBL FP rate on our stream is approximately
2 orders of magnitude _smaller_ (and we whitelisted them, so there's no
ongoing issues), and the SBL FP rate is usually not measurable, because
we don't see any for days at a time (out of 300,000-600,000 filtered per
day).

You can't measure the accuracy of something to .01% if the instrument
you're measuring it with is only 1% accurate.  Further, we'd find a 1%
FP rate on our filtering _totally_ unacceptable (that's 3000+/day). I'd
be looking for another job in another field if it was remotely that bad.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3-nr1 (Windows 2000)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iQCVAwUBRDFRK53FmCyJjHfhAQKCCQQAnrWXIBYdAPO36b5myopoQfV0Tt8TYJSC
5cBoYE2mq4gfXimN+C1c2UwrcLZRwfGcgfrf5YuG9k/6PGXFhQ6aKRYdYA/DNapz
sijutUEyIEzoiioTs4Y0AVe40IvxvZwTEW2LLninl6iB8htSul2EPNLdurK2ijBL
z9Ul4e9P7iM=
=pBmi
-----END PGP SIGNATURE-----

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Mon Apr 03 20:01:04 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQYyO-0005Oe-PF; Mon, 03 Apr 2006 20:00:40 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQYyN-0005OZ-7U
	for asrg@ietf.org; Mon, 03 Apr 2006 20:00:39 -0400
Received: from omta03ps.mx.bigpond.com ([144.140.82.155])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FQYyM-0005C2-Ag
	for asrg@ietf.org; Mon, 03 Apr 2006 20:00:39 -0400
Received: from localhost.localdomain ([60.226.77.107])
	by omta03ps.mx.bigpond.com with ESMTP id
	<20060404000035.QEBP15112.omta03ps.mx.bigpond.com@localhost.localdomain>;
	Tue, 4 Apr 2006 00:00:35 +0000
Received: from ender (ender.scoobynet [192.168.0.3])
	by localhost.localdomain (Postfix) with ESMTP id D3BA1C0;
	Tue,  4 Apr 2006 10:00:12 +1000 (EST)
Received: by ender (Postfix, from userid 1000)
	id 478DC4DD61; Tue,  4 Apr 2006 09:58:41 +1000 (EST)
Date: Tue, 4 Apr 2006 09:58:41 +1000
From: Laird Breyer <laird@lbreyer.com>
To: John Levine <asrg@johnlevine.com>
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
Message-ID: <20060403235840.GA27140@ender>
Mail-Followup-To: John Levine <asrg@johnlevine.com>, asrg@ietf.org
References: <20060403051608.GC12316@ender>
	<20060403055020.19752.qmail@simone.iecc.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20060403055020.19752.qmail@simone.iecc.com>
User-Agent: Mutt/1.5.11+cvs20060126
X-Spam-Score: 0.1 (/)
X-Scan-Signature: 3e15cc4fdc61d7bce84032741d11c8e5
Cc: asrg@ietf.org
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
Reply-To: laird@lbreyer.com
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

On Apr 03 2006, John Levine wrote:
> >To stay on topic, do you accept that with your definition, the only
> >authority which can reliably decide consent (and therefore spamminess)
> >is the receiver?
> 
> Not really.  I get spam complaints all the time for mail from lists
> that I know perfectly well that they signed up for and confirmed.
> "Oh, I don't want that any more."  For the ones that aren't totally
> redacted, my setup turns them into unsubs so they don't get any more
> mail for that particular list, but I don't think it's fair to count
> mail as spam if it depends on reading the recipient's mind in
> real-time.

Isn't that perilously close to saying that if they decided something
in the past, they're not allowed to change their mind?

My position is that there's absolutely nothing wrong with the "spam is
what I say is spam" definition. It's workable, provided you keep spam
filtering at the ends of the network, ie the desktop. 

The issue is feedback. You don't want the spam complaints because you
don't consider yourself these people's spam filter. Chris actually 
is in some sense his customers' spamfilter, and he goes out of his way
to obtain feedback. A trainable desktop filter, whatever the way it 
works internally, is designed to allow quick feedback to make the "spam is
what I say is spam" definition workable.

I also have no problem admitting that for group filtering during SMTP
transport, the "spam is what I say is spam" definition isn't great.
For one, it's hard to reconcile two users who respectively want and
not want a particular type of message. Another definition such as 
"did you consent?" is possibly better. 

We know there are plenty of definitions. It's important to agree on one
in any one discussion.

> 
> Testing spam filters is a really, really hard problem, for all the
> reasons that people have mentioned, basically that you can't test
> without perturbing the system you're testing, and you can't capture
> enough state to rerun the same test more than once.  So you have to
> estimate based on complaint rates from bounces and the like.

Sure is, or nobody would make a big deal about it. I don't agree that
perturbing the system is necessary, however. In the end, testing is
about making the most of the information that's actually available. If
what you have is a dnsbl verdict rather than a message body (say),
then you use that and don't throw up your arms and say "why can't I
have a message body?".

But again, it's important that all discussants use the same
definition, etc.  The devil is in the details.

-- 
Laird Breyer.

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Mon Apr 03 20:23:43 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQZKd-0003rI-7o; Mon, 03 Apr 2006 20:23:39 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQZKb-0003rD-Lw
	for asrg@ietf.org; Mon, 03 Apr 2006 20:23:37 -0400
Received: from mail2.panix.com ([166.84.1.73])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FQZKa-0006ST-FK
	for asrg@ietf.org; Mon, 03 Apr 2006 20:23:37 -0400
Received: from panix5.panix.com (panix5.panix.com [166.84.1.5])
	by mail2.panix.com (Postfix) with ESMTP id CBD9F9D85B
	for <asrg@ietf.org>; Mon,  3 Apr 2006 20:23:35 -0400 (EDT)
Received: (from sethb@localhost)
	by panix5.panix.com (8.11.6p3/8.8.8/PanixN1.1) id k340NaN23848;
	Mon, 3 Apr 2006 20:23:36 -0400 (EDT)
Date: Mon, 3 Apr 2006 20:23:36 -0400 (EDT)
Message-Id: <200604040023.k340NaN23848@panix5.panix.com>
From: Seth Breidbart <sethb@panix.com>
To: asrg@ietf.org
In-reply-to: <20060403235840.GA27140@ender> (message from Laird Breyer on Tue, 
	4 Apr 2006 09:58:41 +1000)
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
References: <20060403051608.GC12316@ender>
	<20060403055020.19752.qmail@simone.iecc.com>
	<20060403235840.GA27140@ender>
X-Spam-Score: 0.0 (/)
X-Scan-Signature: 69a74e02bbee44ab4f8eafdbcedd94a1
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

Laird Breyer <laird@lbreyer.com> wrote:
> On Apr 03 2006, John Levine wrote:

>> Not really.  I get spam complaints all the time for mail from lists
>> that I know perfectly well that they signed up for and confirmed.
>> "Oh, I don't want that any more."  For the ones that aren't totally
>> redacted, my setup turns them into unsubs so they don't get any more
>> mail for that particular list, but I don't think it's fair to count
>> mail as spam if it depends on reading the recipient's mind in
>> real-time.
>
> Isn't that perilously close to saying that if they decided something
> in the past, they're not allowed to change their mind?

No.  If someone asked for something, they're certainly entitled to
change their mind and request (and enforce) not getting it.  They're
not entitled to change their mind and decide they never asked for it.

> My position is that there's absolutely nothing wrong with the "spam
> is what I say is spam" definition. It's workable, provided you keep
> spam filtering at the ends of the network, ie the desktop.

You can't actually _filter_ according to that rule.

> The issue is feedback. You don't want the spam complaints because
> you don't consider yourself these people's spam filter.

It has nothing to do with spamming.  Unsubscribes and spam complaints
are not the same thing.  Spam complaints about solicited email are
bogus, incorrect, and wrong.

[testing]

> I don't agree that perturbing the system is necessary, however.

Describe how you'd test greylisting without perturbing the system.

Seth

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Mon Apr 03 20:27:32 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQZON-0005mX-2Q; Mon, 03 Apr 2006 20:27:31 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQZOM-0005kR-Fd
	for asrg@ietf.org; Mon, 03 Apr 2006 20:27:30 -0400
Received: from a.mail.sonic.net ([64.142.16.245])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FQZOL-0006gD-4d
	for asrg@ietf.org; Mon, 03 Apr 2006 20:27:30 -0400
Received: from [168.61.10.151] (SJC-Office-DHCP-151.Mail-Abuse.ORG
	[168.61.10.151]) (authenticated bits=0)
	by a.mail.sonic.net (8.13.6/8.13.3) with ESMTP id k340RR7u015473
	(version=TLSv1/SSLv3 cipher=RC4-SHA bits=128 verify=NO);
	Mon, 3 Apr 2006 17:27:28 -0700
In-Reply-To: <20060403235840.GA27140@ender>
References: <20060403051608.GC12316@ender>
	<20060403055020.19752.qmail@simone.iecc.com>
	<20060403235840.GA27140@ender>
Mime-Version: 1.0 (Apple Message framework v746.3)
Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed
Message-Id: <6465869A-D88F-4134-B615-AE504DF17347@mail-abuse.org>
Content-Transfer-Encoding: 7bit
From: Douglas Otis <dotis@mail-abuse.org>
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
Date: Mon, 3 Apr 2006 17:27:41 -0700
To: laird@lbreyer.com
X-Mailer: Apple Mail (2.746.3)
X-Spam-Score: 0.5 (/)
X-Scan-Signature: a7d6aff76b15f3f56fcb94490e1052e4
Cc: Anti-Spam Research Group Group <asrg@ietf.org>
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org


On Apr 3, 2006, at 4:58 PM, Laird Breyer wrote:

> On Apr 03 2006, John Levine wrote:
>>> To stay on topic, do you accept that with your definition, the  
>>> only authority which can reliably decide consent (and therefore  
>>> spamminess) is the receiver?
>>
>> Not really.  I get spam complaints all the time for mail from  
>> lists that I know perfectly well that they signed up for and  
>> confirmed. "Oh, I don't want that any more."  For the ones that  
>> aren't totally redacted, my setup turns them into unsubs so they  
>> don't get any more mail for that particular list, but I don't  
>> think it's fair to count mail as spam if it depends on reading the  
>> recipient's mind in real-time.
>
> Isn't that perilously close to saying that if they decided  
> something in the past, they're not allowed to change their mind?

This could be a case where the subscriber declares the list's email  
as spam even though they failed to take steps to unsubscribe.  There  
might also be a problem created by lists that fail to confirm a  
subscription as well.  The lack of a confirmation step would be a bad  
practice.  Some have subscribed victims to lists and then send their  
spam through the list when this confirmation step is missing.


> My position is that there's absolutely nothing wrong with the "spam  
> is what I say is spam" definition. It's workable, provided you keep  
> spam filtering at the ends of the network, ie the desktop.

There is no reason networks must endure abuse.  At least refusal at  
the SMTP session provides valuable feedback to the sender.  Nothing  
should force providers to adopt dangerous and unworkable "opt-out"  
methods either.  Fair rules can be established that _always_ consider  
bulk unsolicited messages as abuse.  This is a _simpler_ definition  
that can be implemented in a fashion that protects resources.  There  
is no reason to waste the time of resources of billions of people to  
achieve some nonsensical ideal.

-Doug


_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Mon Apr 03 20:44:44 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQZdY-0006W9-0j; Mon, 03 Apr 2006 20:43:12 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQZdW-0006Uh-8x
	for asrg@ietf.org; Mon, 03 Apr 2006 20:43:10 -0400
Received: from omta03ps.mx.bigpond.com ([144.140.82.155])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FQZdU-0007E4-QW
	for asrg@ietf.org; Mon, 03 Apr 2006 20:43:10 -0400
Received: from localhost.localdomain ([60.226.77.107])
	by omta03ps.mx.bigpond.com with ESMTP id
	<20060404004306.SFSD15112.omta03ps.mx.bigpond.com@localhost.localdomain>
	for <asrg@ietf.org>; Tue, 4 Apr 2006 00:43:06 +0000
Received: from ender (ender.scoobynet [192.168.0.3])
	by localhost.localdomain (Postfix) with ESMTP id EF124C0
	for <asrg@ietf.org>; Tue,  4 Apr 2006 10:42:44 +1000 (EST)
Received: by ender (Postfix, from userid 1000)
	id E29884DD61; Tue,  4 Apr 2006 10:41:13 +1000 (EST)
Date: Tue, 4 Apr 2006 10:41:13 +1000
From: Laird Breyer <laird@lbreyer.com>
To: asrg@ietf.org
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
Message-ID: <20060404004112.GB27140@ender>
Mail-Followup-To: asrg@ietf.org
References: <20060403055020.19752.qmail@simone.iecc.com>
	<4431512B.5040506@americasm01.nt.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <4431512B.5040506@americasm01.nt.com>
User-Agent: Mutt/1.5.11+cvs20060126
X-Spam-Score: 0.1 (/)
X-Scan-Signature: 4d87d2aa806f79fed918a62e834505ca
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
Reply-To: laird@lbreyer.com
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

On Apr 03 2006, Chris Lewis wrote:

> In fact, receiver adjudication would likely _lower_ the FP rate. On
> balance, I believe it's more likely that a user doesn't recognize
> something he really did ask for (thru some twisty chain of branding,
> lowering the FP rate) than "recognize" something he didn't ask for
> (raising the FP rate).

That's your judgement talking, not the user. You're saying that you
can't let users full reign because that's like herding cats, it would
mess with your aggregate server statistics and thereby impact the
business. It's a very reasonable position from your point of view.

A guy like Church doesn't accept your point of view because he has
different priorities. I find it hard to agree with you because my
"natural" definition puts the user first. I can see that a clueless
user is well advised to let you make decisions for him, but not
someone who has used mail for a long time.

> 
> Thus, receiver adjudication has an error rate, further as I'll mention
> below, likely _higher_ than the measures we're trying to test.
> 

Only from your point of view. From the point of view of the user, the
errors you mention aren't FPs, they are one instance of FN (he doesn't
recognize something he asked for, ie that your system delivers to his
inbox) and one instance of FP (he does recognize something he didn't
ask for, ie that your system doesn't deliver).

Again, it's a matter of definition. Just one last point:

> You can't measure the accuracy of something to .01% if the instrument
> you're measuring it with is only 1% accurate.  Further, we'd find a 1%
> FP rate on our filtering _totally_ unacceptable (that's 3000+/day). I'd
> be looking for another job in another field if it was remotely that bad.

It's not difficult to set a fixed FP rate on a classifier and trade
off FN to achieve it. Every filter can achieve < 0.01% FP. In the
trivial extreme, no filtering at all achieves 0% FP and 100% FN.

What I just want to point out is that your filtering costs (ie the
estimated cost of an FP and the estimated cost of an FN) are not the
same as for an individual user. They make sense for you, no question,
but it's another potential source of valid disagreement worth laying
on the table. 

One way to route around this problem is to use ROC curves, which give a
global picture that doesn't depend on any one person's cost estimates.

-- 
Laird Breyer.

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Mon Apr 03 20:55:01 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQZos-00038L-Lf; Mon, 03 Apr 2006 20:54:54 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQZoq-00038G-RG
	for asrg@ietf.org; Mon, 03 Apr 2006 20:54:52 -0400
Received: from omta02sl.mx.bigpond.com ([144.140.93.154])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FQZop-0007fn-DS
	for asrg@ietf.org; Mon, 03 Apr 2006 20:54:52 -0400
Received: from localhost.localdomain ([60.226.77.107])
	by omta02sl.mx.bigpond.com with ESMTP id
	<20060404005443.ZGTG24931.omta02sl.mx.bigpond.com@localhost.localdomain>
	for <asrg@ietf.org>; Tue, 4 Apr 2006 00:54:43 +0000
Received: from ender (ender.scoobynet [192.168.0.3])
	by localhost.localdomain (Postfix) with ESMTP id 92FDDC0
	for <asrg@ietf.org>; Tue,  4 Apr 2006 10:54:21 +1000 (EST)
Received: by ender (Postfix, from userid 1000)
	id 959DA4DD61; Tue,  4 Apr 2006 10:52:50 +1000 (EST)
Date: Tue, 4 Apr 2006 10:52:50 +1000
From: Laird Breyer <laird@lbreyer.com>
To: asrg@ietf.org
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
Message-ID: <20060404005250.GC27140@ender>
Mail-Followup-To: asrg@ietf.org
References: <88885AD16AAA9F4899BB9FB751D4F5A5B70919@phost016.habeas.intermedia.net>
	<E1FP1vs-0008WV-00@chiark.greenend.org.uk>
	<20060331005839.GB6958@ender> <442D3368.7040202@americasm01.nt.com>
	<20060331234310.GA9179@ender> <442DCB75.7050409@americasm01.nt.com>
	<20060401021616.GA9651@ender> <442EE80C.5080500@americasm01.nt.com>
	<20060403005123.GB11416@ender>
	<20060403094223.GA18195@waltdnes.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20060403094223.GA18195@waltdnes.org>
User-Agent: Mutt/1.5.11+cvs20060126
X-Spam-Score: 0.1 (/)
X-Scan-Signature: 7655788c23eb79e336f5f8ba8bce7906
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
Reply-To: laird@lbreyer.com
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

On Apr 03 2006, Walter Dnes wrote:
> 
>   - a content-filter will bury the email in a "spam folder" with
>     thousands of real spam, where it'll probably never be found.  The
>     sender will believe that the intended recipient has received the
>     message and ignored it, while the intended recipient will believe
>     that the sender hasn't sent the message.

That's merely an artifact of current implementations. The key to this
problem is ranking.

Do you remember the internet before Google? The top search engine
around 97 was Altavista, and it was at the top because it had a large
index and other features, but it didn't rank pages so it was necessary
to find sites by wading through many hits, like wading through an
unordered spam folder.

Google was a success because it was the first to offer usable ranking
of search hits, turning a deep search list into a shallow search list.
The same applies to the problem of finding false positives in a spam
folder.  

-- Laird Breyer.

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Mon Apr 03 21:25:31 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQaIM-0008Bn-UQ; Mon, 03 Apr 2006 21:25:22 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQaIL-0008AM-Bm
	for asrg@ietf.org; Mon, 03 Apr 2006 21:25:21 -0400
Received: from omta02ps.mx.bigpond.com ([144.140.83.154])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FQaIK-0000ZG-So
	for asrg@ietf.org; Mon, 03 Apr 2006 21:25:21 -0400
Received: from localhost.localdomain ([60.226.77.107])
	by omta02ps.mx.bigpond.com with ESMTP id
	<20060404012517.SFEF3168.omta02ps.mx.bigpond.com@localhost.localdomain>;
	Tue, 4 Apr 2006 01:25:17 +0000
Received: from ender (ender.scoobynet [192.168.0.3])
	by localhost.localdomain (Postfix) with ESMTP id AAEACC0;
	Tue,  4 Apr 2006 11:24:55 +1000 (EST)
Received: by ender (Postfix, from userid 1000)
	id A992F4DD61; Tue,  4 Apr 2006 11:23:24 +1000 (EST)
Date: Tue, 4 Apr 2006 11:23:24 +1000
From: Laird Breyer <laird@lbreyer.com>
To: Seth Breidbart <sethb@panix.com>
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
Message-ID: <20060404012324.GD27140@ender>
Mail-Followup-To: Seth Breidbart <sethb@panix.com>, asrg@ietf.org
References: <20060403051608.GC12316@ender>
	<20060403055020.19752.qmail@simone.iecc.com>
	<20060403235840.GA27140@ender>
	<200604040023.k340NaN23848@panix5.panix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <200604040023.k340NaN23848@panix5.panix.com>
User-Agent: Mutt/1.5.11+cvs20060126
X-Spam-Score: 0.1 (/)
X-Scan-Signature: f60d0f7806b0c40781eee6b9cd0b2135
Cc: asrg@ietf.org
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
Reply-To: laird@lbreyer.com
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

On Apr 03 2006, Seth Breidbart wrote:
> Laird Breyer <laird@lbreyer.com> wrote:
> > On Apr 03 2006, John Levine wrote:
> 
> >> Not really.  I get spam complaints all the time for mail from lists
> >> that I know perfectly well that they signed up for and confirmed.
> >> "Oh, I don't want that any more."  For the ones that aren't totally
> >> redacted, my setup turns them into unsubs so they don't get any more
> >> mail for that particular list, but I don't think it's fair to count
> >> mail as spam if it depends on reading the recipient's mind in
> >> real-time.
> >
> > Isn't that perilously close to saying that if they decided something
> > in the past, they're not allowed to change their mind?
> 
> No.  If someone asked for something, they're certainly entitled to
> change their mind and request (and enforce) not getting it.  They're
> not entitled to change their mind and decide they never asked for it.
>

That's because you're using the "consent" version of the spam definition.
You're not using the "spam is what I don't want" definition. Reread 
John's paragraph, he wrote "Oh, I don't want that any more".

 
> > My position is that there's absolutely nothing wrong with the "spam
> > is what I say is spam" definition. It's workable, provided you keep
> > spam filtering at the ends of the network, ie the desktop.
> 
> You can't actually _filter_ according to that rule.

As I said, it's a matter of obtaining timely feedback.  But you are
entirely right in saying that some systems, especially large
multi-user heavy traffic systems today can't possibly do this well.

I'm not pushing the definition on anybody, I'm defending it as valid and
freely admit that it's best applied to filtering at the MUA level.


> Describe how you'd test greylisting without perturbing the system.

I already have outlined it several times now. The greylisting system
logs all the events which enter into the final decision. When a mail
is rejected, the record of evidence which triggered that rejection 
(call it R) is kept. 

Later, during QA, some volunteer users are sent a fraction of
rejection records R for mail messages that were intended for them, to
be approved. When the definition is "spam is what I say is spam",
these volunteers are the final arbiters of mail intended for them,
whether other people think they could do a better job or not.

Finally, you measure the number of records R which the volunteers say
are incorrectly rejected by the greylisting system. In this way, no
heisenberg effect occurs.

If you like, you can also implement a tool which automatically derives
other information from the record R, such as reverse IP ->country
mapping, or inferring the sender domain, to make it easier for the
volunteers to arbitrate. In principle, a volunteer could implement
such a tool himself, or just go on WHOIS etc. Since the recipient is
the final arbiter, he can do anything he likes.

Again, if you're secretly (or not so secretly) using a different definition
of what spam is, then you'll find this bogus.

-- 
Laird Breyer.

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Mon Apr 03 22:03:35 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQarj-0004g7-Ex; Mon, 03 Apr 2006 22:01:55 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQarh-0004fz-KI
	for asrg@ietf.org; Mon, 03 Apr 2006 22:01:53 -0400
Received: from zcars04f.nortelnetworks.com ([47.129.242.57]
	helo=zcars04f.nortel.com) by ietf-mx.ietf.org with esmtp (Exim 4.43)
	id 1FQarg-0002EP-9H
	for asrg@ietf.org; Mon, 03 Apr 2006 22:01:53 -0400
Received: from zrtphxs1.corp.nortel.com
	(zrtphxs1.corp.nort...s0.corp.nortel.com [47.140.202.46])
	by zcars04f.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id
	k3421mN24560
	for <asrg@ietf.org>; Mon, 3 Apr 2006 22:01:49 -0400 (EDT)
Received: from zrtps0m6.us.nortel.com ([47.140.192.58]) by
	zrtphxs1.corp.nortel.com with Microsoft SMTPSVC(6.0.3790.211); 
	Mon, 3 Apr 2006 22:01:18 -0400
Received: from [47.130.18.134] (acart0cn.ca.nortel.com [47.130.18.134])
	by zrtps0m6.us.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id
	k3421En15717
	for <asrg@ietf.org>; Mon, 3 Apr 2006 22:01:15 -0400 (EDT)
Message-ID: <4431D369.2060505@americasm01.nt.com>
Date: Mon, 03 Apr 2006 22:01:13 -0400
From: "Chris Lewis" <clewis@nortel.com>
User-Agent: Thunderbird 1.5 (Windows/20051201)
MIME-Version: 1.0
To: asrg@ietf.org
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
References: <20060403051608.GC12316@ender>	<20060403055020.19752.qmail@simone.iecc.com>	<20060403235840.GA27140@ender>	<200604040023.k340NaN23848@panix5.panix.com>
	<20060404012324.GD27140@ender>
In-Reply-To: <20060404012324.GD27140@ender>
X-Enigmail-Version: 0.94.0.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-OriginalArrivalTime: 04 Apr 2006 02:01:18.0355 (UTC)
	FILETIME=[A990D630:01C6578B]
X-Spam-Score: 0.1 (/)
X-Scan-Signature: 41c17b4b16d1eedaa8395c26e9a251c4
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Laird Breyer wrote:
> On Apr 03 2006, Seth Breidbart wrote:
>> Laird Breyer <laird@lbreyer.com> wrote:
>>> On Apr 03 2006, John Levine wrote:
>>>> Not really.  I get spam complaints all the time for mail from lists
>>>> that I know perfectly well that they signed up for and confirmed.
>>>> "Oh, I don't want that any more."  For the ones that aren't totally
>>>> redacted, my setup turns them into unsubs so they don't get any more
>>>> mail for that particular list, but I don't think it's fair to count
>>>> mail as spam if it depends on reading the recipient's mind in
>>>> real-time.
>>> Isn't that perilously close to saying that if they decided something
>>> in the past, they're not allowed to change their mind?
>> No.  If someone asked for something, they're certainly entitled to
>> change their mind and request (and enforce) not getting it.  They're
>> not entitled to change their mind and decide they never asked for it.

> That's because you're using the "consent" version of the spam definition.
> You're not using the "spam is what I don't want" definition. Reread 
> John's paragraph, he wrote "Oh, I don't want that any more".

I don't think John is calling that spam, is he?  I don't either.

I won't block on that basis.  I'll just help the user stop getting more.

>> Describe how you'd test greylisting without perturbing the system.
> 
> I already have outlined it several times now. The greylisting system
> logs all the events which enter into the final decision. When a mail
> is rejected, the record of evidence which triggered that rejection 
> (call it R) is kept. 
> 
> Later, during QA, some volunteer users are sent a fraction of
> rejection records R for mail messages that were intended for them, to
> be approved. When the definition is "spam is what I say is spam",
> these volunteers are the final arbiters of mail intended for them,
> whether other people think they could do a better job or not.
> 
> Finally, you measure the number of records R which the volunteers say
> are incorrectly rejected by the greylisting system.

Sorry, that notion is nonsensical with greylisting.  Are you aware of
how greylisting works?

greylisting works by tempfailing the connection of _every_ "new" unique
sender/recipient/source IP triple (or some variant), then letting the
email through if the sending server retries.  The notion being that most
spam senders cannot retry.

[There's various timeout values involved w.r.t. minimum retry interval,
how long you remember a successful passthru etc.]

It never permanently rejects.  The only notion of "incorrectly rejected"
there could be is if the implementation is broken and not implementing
greylisting.  In which case, you're not testing greylisting, you're
testing something else.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3-nr1 (Windows 2000)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBRDHTaJ3FmCyJjHfhAQKPgAP6A2VH7+za0kG16Ud/92Ue5Q/hwWZN7mYZ
5iPc8P0FAMSTpafl2sexhjl8VaNm3kT3yPJamSj5ubrE0rp0DqFXH4sRi/uLEdAu
SkdI57SgZZY/i7r1CeOlPG9fambi9J2Pz4hfVATkuXKbtmtsmUyDLv82s3PPN+MO
SHQWdAtqMtU=
=r4Yh
-----END PGP SIGNATURE-----

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Mon Apr 03 22:17:12 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQb6R-0000U2-Bk; Mon, 03 Apr 2006 22:17:07 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQb6Q-0000SN-R7
	for asrg@ietf.org; Mon, 03 Apr 2006 22:17:06 -0400
Received: from zrtps0kp.nortelnetworks.com ([47.140.192.56]
	helo=zrtps0kp.nortel.com) by ietf-mx.ietf.org with esmtp (Exim 4.43)
	id 1FQb6P-0003Yn-Gt
	for asrg@ietf.org; Mon, 03 Apr 2006 22:17:06 -0400
Received: from zrtphxs1.corp.nortel.com
	(zrtphxs1.corp.nort...s0.corp.nortel.com [47.140.202.46])
	by zrtps0kp.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id
	k342H2w22725
	for <asrg@ietf.org>; Mon, 3 Apr 2006 22:17:03 -0400 (EDT)
Received: from zrtps0m6.us.nortel.com ([47.140.192.58]) by
	zrtphxs1.corp.nortel.com with Microsoft SMTPSVC(6.0.3790.211); 
	Mon, 3 Apr 2006 22:17:02 -0400
Received: from [47.130.18.134] (acart0cn.ca.nortel.com [47.130.18.134])
	by zrtps0m6.us.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id
	k342Gxn08660
	for <asrg@ietf.org>; Mon, 3 Apr 2006 22:16:59 -0400 (EDT)
Message-ID: <4431D719.6060703@americasm01.nt.com>
Date: Mon, 03 Apr 2006 22:16:57 -0400
From: "Chris Lewis" <clewis@nortel.com>
User-Agent: Thunderbird 1.5 (Windows/20051201)
MIME-Version: 1.0
To: asrg@ietf.org
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
References: <20060403055020.19752.qmail@simone.iecc.com>	<4431512B.5040506@americasm01.nt.com>
	<20060404004112.GB27140@ender>
In-Reply-To: <20060404004112.GB27140@ender>
X-Enigmail-Version: 0.94.0.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-OriginalArrivalTime: 04 Apr 2006 02:17:02.0508 (UTC)
	FILETIME=[DC5326C0:01C6578D]
X-Spam-Score: 0.1 (/)
X-Scan-Signature: 3e15cc4fdc61d7bce84032741d11c8e5
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Laird Breyer wrote:
> On Apr 03 2006, Chris Lewis wrote:
> 
>> In fact, receiver adjudication would likely _lower_ the FP rate. On
>> balance, I believe it's more likely that a user doesn't recognize
>> something he really did ask for (thru some twisty chain of branding,
>> lowering the FP rate) than "recognize" something he didn't ask for
>> (raising the FP rate).

> That's your judgement talking, not the user. You're saying that you
> can't let users full reign because that's like herding cats, it would
> mess with your aggregate server statistics and thereby impact the
> business. It's a very reasonable position from your point of view.

> A guy like Church doesn't accept your point of view because he has
> different priorities. I find it hard to agree with you because my
> "natural" definition puts the user first. I can see that a clueless
> user is well advised to let you make decisions for him, but not
> someone who has used mail for a long time.

The user is first.  I simply recognize that a user's accuracy rate on
determining FPs is _lower_ than the automated systems are.

You can't measure a voltage to 6 digits of accuracy if your equipment
only does three.

>> You can't measure the accuracy of something to .01% if the instrument
>> you're measuring it with is only 1% accurate.  Further, we'd find a 1%
>> FP rate on our filtering _totally_ unacceptable (that's 3000+/day). I'd
>> be looking for another job in another field if it was remotely that bad.
> 
> It's not difficult to set a fixed FP rate on a classifier and trade
> off FN to achieve it. Every filter can achieve < 0.01% FP. In the
> trivial extreme, no filtering at all achieves 0% FP and 100% FN.

But you're no longer testing what you were trying to test, are you?

> One way to route around this problem is to use ROC curves, which give a
> global picture that doesn't depend on any one person's cost estimates.

ROC - "Return on cost" curves?  Whassat?

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3-nr1 (Windows 2000)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBRDHXGZ3FmCyJjHfhAQJKaQP/RJEkc6E8TEvmhTME6AZMksVy28RTMp9g
gkx8hf6b/xo0lIvG98AFFMlrNGyZ6h7SKWS6m0SXIl+xfjcRWaB7hM4eiFhcHQI5
FWssmw+2K51QkRHN0UcfOAnyl+1VlJR5Ds8f5AXw9u//Eo0IAOfsz2cgXoh0E3Rj
zNAXWosVIgU=
=/guW
-----END PGP SIGNATURE-----

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Mon Apr 03 22:34:40 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQbNF-0004ux-0J; Mon, 03 Apr 2006 22:34:29 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQbND-0004us-Ol
	for asrg@ietf.org; Mon, 03 Apr 2006 22:34:27 -0400
Received: from mail2.panix.com ([166.84.1.73])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FQbNC-0004Ia-Iu
	for asrg@ietf.org; Mon, 03 Apr 2006 22:34:27 -0400
Received: from panix5.panix.com (panix5.panix.com [166.84.1.5])
	by mail2.panix.com (Postfix) with ESMTP id 3728C9D8F2
	for <asrg@ietf.org>; Mon,  3 Apr 2006 22:34:26 -0400 (EDT)
Received: (from sethb@localhost)
	by panix5.panix.com (8.11.6p3/8.8.8/PanixN1.1) id k342YQN21555;
	Mon, 3 Apr 2006 22:34:26 -0400 (EDT)
Date: Mon, 3 Apr 2006 22:34:26 -0400 (EDT)
Message-Id: <200604040234.k342YQN21555@panix5.panix.com>
From: Seth Breidbart <sethb@panix.com>
To: asrg@ietf.org
In-reply-to: <20060404012324.GD27140@ender> (message from Laird Breyer on Tue, 
	4 Apr 2006 11:23:24 +1000)
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
References: <20060403051608.GC12316@ender>
	<20060403055020.19752.qmail@simone.iecc.com>
	<20060403235840.GA27140@ender>
	<200604040023.k340NaN23848@panix5.panix.com>
	<20060404012324.GD27140@ender>
X-Spam-Score: 0.0 (/)
X-Scan-Signature: 93238566e09e6e262849b4f805833007
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

Laird Breyer <laird@lbreyer.com> wrote:
> On Apr 03 2006, Seth Breidbart wrote:

>> Describe how you'd test greylisting without perturbing the system.

That was a trick question.  You provably can't.

> I already have outlined it several times now. The greylisting system
> logs all the events which enter into the final decision. When a mail
> is rejected, the record of evidence which triggered that rejection
> (call it R) is kept.

Consider some spamware: based on whether it gets 4XX or 5XX, early or
late in the transaction, it decides how to continue (4XX resends the
same thing later, 5XX early sends from another IP address, 5XX late
changes hashbusters and sends from the same IP address, a second fail
changes hashbusters again and sends from a different IP address, where
to send other spams going to the same mailserver but to different
addresses from, . . .)

Now, whatever you do affects what you see later.  Therefore you
_cannot_ do a test without perturbing the system.

Seth

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Tue Apr 04 01:54:11 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQeTm-0005jP-Ny; Tue, 04 Apr 2006 01:53:26 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQeTc-0005TH-38
	for asrg@ietf.org; Tue, 04 Apr 2006 01:53:16 -0400
Received: from xproxy.gmail.com ([66.249.82.202])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FQeTT-000321-RJ
	for asrg@ietf.org; Tue, 04 Apr 2006 01:53:11 -0400
Received: by xproxy.gmail.com with SMTP id s12so1028302wxc
	for <asrg@ietf.org>; Mon, 03 Apr 2006 22:53:07 -0700 (PDT)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com;
	h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references;
	b=rRK/18w7ACF0RnRAtfjOqsDMEprEpd+4fK7bAaWgW4hJEzq05myMsh9/6Yj6IE+Cs/D8Z/kMvXIIxayDnyZsZoN90hKK5oO7UOFXOcRHZtrm/y1O+D86KgNFqs1fYCmTerCpvg2dAkFa3KPzcRwuRHTGnp9RZBIno710asR/osY=
Received: by 10.70.60.15 with SMTP id i15mr987758wxa;
	Mon, 03 Apr 2006 22:53:07 -0700 (PDT)
Received: by 10.70.95.3 with HTTP; Mon, 3 Apr 2006 22:53:07 -0700 (PDT)
Message-ID: <934f64a20604032253q10c52269g653418fafc8987fe@mail.gmail.com>
Date: Tue, 4 Apr 2006 00:53:07 -0500
From: "David Nicol" <davidnicol@gmail.com>
To: asrg@ietf.org
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
In-Reply-To: <20060403235840.GA27140@ender>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline
References: <20060403051608.GC12316@ender>
	<20060403055020.19752.qmail@simone.iecc.com>
	<20060403235840.GA27140@ender>
X-Spam-Score: 0.0 (/)
X-Scan-Signature: 79899194edc4f33a41f49410777972f8
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

On 4/3/06, Laird Breyer <laird@lbreyer.com> wrote:
> On Apr 03 2006, John Levine wrote:
> > Not really.  I get spam complaints all the time for mail from lists
> > that I know perfectly well that they signed up for and confirmed.
> > "Oh, I don't want that any more."  For the ones that aren't totally
> > redacted, my setup turns them into unsubs so they don't get any more
> > mail for that particular list, but I don't think it's fair to count
> > mail as spam if it depends on reading the recipient's mind in
> > real-time.

that's brilliant.  Imagine a shared-training filter (like dspam) that will
respect and follow header-listed unsubscribe instructions instead of
touching the beysian weights when someone marks a well-behaved
list posting as spam.

Hmm.  No, it would be very bad, as if a spam were to appear on a mailing
list everyone who reported it would get unsubscribed.

Maybe the MUA should have a "unsubscribe" button when it recognizes that
I am reading a mailing list message.

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Tue Apr 04 04:10:29 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQgcB-0004Mz-1Y; Tue, 04 Apr 2006 04:10:15 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQgc9-0004M3-Nc
	for asrg@ietf.org; Tue, 04 Apr 2006 04:10:13 -0400
Received: from zrtps0kp.nortelnetworks.com ([47.140.192.56]
	helo=zrtps0kp.nortel.com) by ietf-mx.ietf.org with esmtp (Exim 4.43)
	id 1FQgc9-0001w5-8Y
	for asrg@ietf.org; Tue, 04 Apr 2006 04:10:13 -0400
Received: from zcarhxm0.corp.nortel.com (zcarhxm0.corp.nortel.com
	[47.129.230.95])
	by zrtps0kp.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id
	k348AAE04087
	for <asrg@ietf.org>; Tue, 4 Apr 2006 04:10:11 -0400 (EDT)
Received: from [47.130.16.238] ([47.130.16.238] RDNS failed) by
	zcarhxm0.corp.nortel.com with Microsoft SMTPSVC(6.0.3790.211); 
	Tue, 4 Apr 2006 04:10:10 -0400
Message-ID: <44322933.7060103@nortel.com>
Date: Tue, 04 Apr 2006 04:07:15 -0400
From: "Thomas Choi" <tchoi@nortel.com>
User-Agent: Thunderbird 1.5 (Windows/20051201)
MIME-Version: 1.0
To: asrg@ietf.org
Subject: [Asrg] Tchoi Reply on draft-church-dnsbl-harmful-01.txt 
References: <20060403051608.GC12316@ender>	<20060403055020.19752.qmail@simone.iecc.com>	<20060403235840.GA27140@ender>
	<934f64a20604032253q10c52269g653418fafc8987fe@mail.gmail.com>
In-Reply-To: <934f64a20604032253q10c52269g653418fafc8987fe@mail.gmail.com>
Content-Type: text/plain; charset=windows-1252; format=flowed
X-OriginalArrivalTime: 04 Apr 2006 08:10:10.0330 (UTC)
	FILETIME=[314037A0:01C657BF]
Content-Transfer-Encoding: quoted-printable
X-MIME-Autoconverted: from 8bit to quoted-printable by zrtps0kp.nortel.com id
	k348AAE04087
X-Spam-Score: 0.1 (/)
X-Scan-Signature: 6e922792024732fb1bb6f346e63517e4
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

Here's my reply to Church's paper. :)

**************************

Church=92s paper entitled "DNS Blacklists Considered Harmful" discusses=20
some legitimate concerns with using DNS-blacklists. One such concern=20
with DNS-blacklists is the lack of control that an e-mail administrator=20
has over such lists. In particular, the author mentions that "it may=20
take a significant amount of time for the server operator's request for=20
removal from the blacklist to be answered.". With e-mail being a=20
business-critical means for communicating, it's essential that the=20
e-mail administrator have the ability to make immediate changes to their=20
filters when needed. Since DNS-blacklists are operated and maintained by=20
third party sources, and not the administrator him/herself, the author's=20
concern does seem to be legitimate. In such a case, a question becomes,=20
what can the administrator do when a third party blacklist is blocking=20
legitimate mail that needs to get through right away? To answer such a=20
question, one should note that it's possible for a spam filter using a=20
DNS-blacklist to not block messages just because a sender=92s source IP=20
address is listed in said list. For example, by using the spamassassin=20
tool, a score is associated to each DNS-blacklist rule and by lowering=20
said score, one could limit the disruption caused by false positives=20
from that list. Alternatively, one could simply setup their own=20
'whitelist' of IP addresses that they can use to ensure that mail from=20
specific sources will always get through.

Another concern that the author raised in his paper was with regards to=20
the actual practices of the operators of such blacklists. Specifically,=20
the author argues that "a far more serious problem is that of arbitrary=20
policies instituted and decisions made by blacklist operators.". In=20
response to the authors=92s concern, it should be noted that each list ha=
s=20
their own policies and as a result, some lists may have more=20
questionable policies than others. So before adding any DNS-blacklists=20
to their anti-spam solution, it is highly recommended that e-mail=20
administrators verify the reliability and integrity of the policies of=20
the lists they are interested in and select the one(s) that works best=20
for them. That being said, it should be noted that there are in fact=20
some high quality lists that are being widely used today by corporations=20
and Internet service providers as part of their filtering solution.

A final concern was raised after the author conducted an analysis of the=20
mail received and filtered on his mail server. In this analysis, the=20
author compared the effectiveness of a content-based filter against five=20
different blacklists on a corpus of 1,374 messages. When presenting the=20
results, the author noted that "only one of the blacklists (SBL-XBL)=20
managed to correctly flag more than 50% of incoming spam and all=20
blacklists rejected some of the incoming nonspam messages.". In=20
comparison, the content-based filter "correctly filtered 97% of the=20
spam, without any false positives.". Although the test results clearly=20
showed that the content-filters performed better than the=20
DNS-blacklists, it should be noted that sample size was quite small and=20
that there was no mention of the number of users nor the variety of spam=20
that was used in the experiment. With such a small sample size, it=92s=20
unlikely that the experiment was tested with the many different variants=20
of spams (i.e. scam spam, junk mail, non-commercial, phishes, viruses=20
etc.) that larger entities would likely see. For example, one such=20
entity is Nortel which has over 65 000 distinct user accounts and=20
receives an average of 600k e-mails per day. Nortel uses a hybrid of=20
filtering mechanisms comprising content rules and the CBL DNS-blacklist,=20
which blocks approximately 300k e-mails per day and includes all of the=20
variants described above. It should be noted that the blacklist alone=20
accounts for approximately 80% of all of their blocked e-mails with a=20
false positive rate of 0.01%. With other large entities reporting=20
similar DNS-blacklist effectiveness results, it=92s not surprising that=20
many large-scale environments are currently using blacklists as part of=20
their own anti-spam solution.

Towards the end of the paper, the author suggests that technological=20
advances will likely reduce the need for blocking mail at the server=20
level. Such advances would include mail client spam filtering software,=20
which can learn from previous spam. The author goes on to say that such=20
software is "far more effective than simple server-based blacklists,=20
particularly since they can adjust for the types of spam received by=20
individual users.". In response, it should be mentioned that in order to=20
effectively use such a filter, the end user is required to train the=20
filter so that it can distinguish between both good and bad mail. With=20
the limited amount of spare time in people's lives, it wouldn't be=20
surprising if most users would balk at the idea of having to train their=20
mail filter software to detect spam and non-spam. Instead, they would=20
probably prefer that spam be blocked at the gateway instead of being=20
filtered at the client. Furthermore, it should be noted that the author=20
acknowledges in the problem section of his paper that approximately 70%=20
of all e-mail is spam. With such an incredibly high volume of spam, does=20
it really makes sense to waste processing power of the user's desktop so=20
that its anti-virus scanner to scan extra message for viruses and does=20
it really make sense to waste extra bandwidth needed to deliver all mail=20
to the end user. Since e-mail virus infections are costly to any=20
corporate entity, most administrators would probably agree that it's=20
highly desirable to have the gateway filters block bad messages before=20
they get to the end user.

In conclusion, Church=92s paper brings up some legitimate concerns with=20
the use of DNS-blacklists. Each of these concerns were analyzed and=20
addressed in this report to show that DNS-blacklists can in fact be=20
useful when the administrator selects the right one for their particular=20
environment. In other words, when an e-mail administrator verifies the=20
reliability and integrity of any list they wish to use and selects the=20
one that works best for them, DNS-blacklists can be useful.



_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Tue Apr 04 07:49:56 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQk2f-0007OU-N5; Tue, 04 Apr 2006 07:49:49 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQk2e-0007OP-Et
	for asrg@ietf.org; Tue, 04 Apr 2006 07:49:48 -0400
Received: from mail1.nber.org ([66.251.72.7])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FQk2d-0001V8-5L
	for asrg@ietf.org; Tue, 04 Apr 2006 07:49:48 -0400
Received: from mail1 (root@localhost)
	by mail1.nber.org (8.12.11.20060308/8.12.8) with SMTP id k34BnkgQ006823
	for <asrg@ietf.org>; Tue, 4 Apr 2006 07:49:46 -0400
Received: from nber5.nber.org (nber5.nber.org [66.251.72.75])
	by mail1.nber.org (8.12.11.20060308/8.12.8) with ESMTP id
	k34Bnk2n006798; Tue, 4 Apr 2006 07:49:46 -0400
Received: from nber5.nber.org (localhost [127.0.0.1])
	by nber5.nber.org (8.12.10+Sun/8.12.10) with ESMTP id k34Bnkog024752;
	Tue, 4 Apr 2006 07:49:46 -0400 (EDT)
Received: from localhost (feenberg@localhost)
	by nber5.nber.org (8.12.10+Sun/8.12.9/Submit) with ESMTP id
	k34BnjIT024749; Tue, 4 Apr 2006 07:49:45 -0400 (EDT)
Date: Tue, 4 Apr 2006 07:49:45 -0400 (EDT)
From: Daniel Feenberg <feenberg@nber.org>
To: David Nicol <davidnicol@gmail.com>
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
In-Reply-To: <934f64a20604032253q10c52269g653418fafc8987fe@mail.gmail.com>
Message-ID: <Pine.GSO.4.64.0604040742020.20096@nber5.nber.org>
References: <20060403051608.GC12316@ender>
	<20060403055020.19752.qmail@simone.iecc.com>
	<20060403235840.GA27140@ender>
	<934f64a20604032253q10c52269g653418fafc8987fe@mail.gmail.com>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
X-Spam-Score: 0.0 (/)
X-Scan-Signature: 2409bba43e9c8d580670fda8b695204a
Cc: asrg@ietf.org
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org



On Tue, 4 Apr 2006, David Nicol wrote:


...>
> Maybe the MUA should have a "unsubscribe" button when it recognizes that
> I am reading a mailing list message.
>

There is already a standard "List-Unsubscribe" header described by 
RFC2369, so an MUA could do this. It would be a great advance. The syntax
of the field is not rigid (it could be a mailto or a URL, but the MUA 
could treat it as html and whatever it was, it would be.

I do get a lot of list mail with this header - it isn't DOA.

Daniel Feenberg
feenberg isat nber dotte org

> _______________________________________________
> Asrg mailing list
> Asrg@ietf.org
> https://www1.ietf.org/mailman/listinfo/asrg
>
>


_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Tue Apr 04 12:38:17 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQoXb-0006LV-CR; Tue, 04 Apr 2006 12:38:03 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQoXa-0006LI-6A
	for asrg@ietf.org; Tue, 04 Apr 2006 12:38:02 -0400
Received: from teal.hjp.at ([62.99.236.98])
	by ietf-mx.ietf.org with smtp (Exim 4.43) id 1FQoXY-0006Nc-L9
	for asrg@ietf.org; Tue, 04 Apr 2006 12:38:02 -0400
Received: (qmail 8789 invoked by uid 500); 4 Apr 2006 16:37:59 -0000
Date: Tue, 4 Apr 2006 18:37:59 +0200
From: "Peter J. Holzer" <hjp-asrg@hjp.at>
To: asrg@ietf.org
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
Message-ID: <20060404163759.GD8054@teal.hjp.at>
Mail-Followup-To: asrg@ietf.org
References: <20060403051608.GC12316@ender>
	<20060403055020.19752.qmail@simone.iecc.com>
	<20060403235840.GA27140@ender>
	<200604040023.k340NaN23848@panix5.panix.com>
	<20060404012324.GD27140@ender>
	<4431D369.2060505@americasm01.nt.com>
Mime-Version: 1.0
In-Reply-To: <4431D369.2060505@americasm01.nt.com>
User-Agent: Mutt/1.4.1i
X-Spam-Score: 0.0 (/)
X-Scan-Signature: 6e922792024732fb1bb6f346e63517e4
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Content-Type: multipart/mixed; boundary="===============1545559042=="
Errors-To: asrg-bounces@ietf.org


--===============1545559042==
Content-Type: multipart/signed; micalg=pgp-sha1;
	protocol="application/pgp-signature"; boundary="sgneBHv3152wZ8jf"
Content-Disposition: inline


--sgneBHv3152wZ8jf
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On 2006-04-03 22:01:13 -0400, Chris Lewis wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>=20
> Laird Breyer wrote:
> > On Apr 03 2006, Seth Breidbart wrote:
> >> Describe how you'd test greylisting without perturbing the system.
> >=20
> > I already have outlined it several times now. The greylisting system
> > logs all the events which enter into the final decision. When a mail
> > is rejected, the record of evidence which triggered that rejection=20
> > (call it R) is kept.=20
> >=20
> > Later, during QA, some volunteer users are sent a fraction of
> > rejection records R for mail messages that were intended for them, to
> > be approved. When the definition is "spam is what I say is spam",
> > these volunteers are the final arbiters of mail intended for them,
> > whether other people think they could do a better job or not.
> >=20
> > Finally, you measure the number of records R which the volunteers say
> > are incorrectly rejected by the greylisting system.
>=20
> Sorry, that notion is nonsensical with greylisting.  Are you aware of
> how greylisting works?
[...]
> It never permanently rejects.  The only notion of "incorrectly rejected"
> there could be is if the implementation is broken and not implementing
> greylisting.  In which case, you're not testing greylisting, you're
> testing something else.

I am testing whether greylisting has the intended effect. Sure, the MTA
implementing greylisting never issues as 5xx response. But the effect of
one or more 4xx responses may be the same - either because the client
MTA is broken (doesn't retry) or because of the client's queuing setup
and the servers greylisting parameters don't match.=20

So if I have records from the greylisting MTA, I can in theory look at
every delivery attempt and classify it as:

1) Delivery was accepted and message was not spam (TN)

2) Delivery was accepted and message was spam (FN)

3) Delivery was temporarily rejected, but a later delivery of the same
   message was accepted. This puts it into one of the two categories
   above, but adds extra info (number of delivery attempts, delay from
   first delivery attempt to acceptance).

4) Delivery was temporily rejected and no later delivery attempt was
   successful. Message was spam (TP).

5) Delivery was temporily rejected and no later delivery attempt was
   successful. Message was not spam (FP).

In practice this is difficult. Normally, greylisting is done at the RCPT
command, so there isn't much information available (EHLO, MAIL, RCPT,
IP-address of client, maybe size of the message). So it is even
difficult to determine whether two delivery attempts are for the same
message, and even more difficult to determine whether the message was
spam. Still, for small samples, where a human can go over the records
and apply real-life knowledge, it should be possible to get a (somewhat
fuzzy) categorization which is better than that determined by
greylisting itself.

(As an aside, we provide our users with a webbased view of the log files
for their own mail addresses, so they can do similar checks - some of
them do and report FPs due to greylisting to us)

Greylisting could be delayed until after the message is received. This
would make provide more information for auditing, but it alters the
algorithm enough that it cannot be considered the same algorithm any
more (so you cannot do this "just for testing", you have to do it in
production, too).

	hp

--=20
   _  | Peter J. Holzer    | Ich sehe nun ein, dass Computer wenig
|_|_) | Sysadmin WSR       | geeignet sind, um sich was zu merken.
| |   | hjp@hjp.at         |
__/   | http://www.hjp.at/ |	-- Holger Lembke in dan-am

--sgneBHv3152wZ8jf
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)

iD8DBQFEMqDmfZ+RkG8quy0RAjJHAJ4tXYhwWo0B5PgHvlGoCm1PpH4iOgCgomqu
eKXxHS7T8YTT1ZiuFFPBtjw=
=cDP5
-----END PGP SIGNATURE-----

--sgneBHv3152wZ8jf--


--===============1545559042==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg

--===============1545559042==--




From asrg-bounces@ietf.org Tue Apr 04 13:28:36 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQpIv-0007cM-Ot; Tue, 04 Apr 2006 13:26:57 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQpIu-0007cH-D7
	for asrg@ietf.org; Tue, 04 Apr 2006 13:26:56 -0400
Received: from xproxy.gmail.com ([66.249.82.202])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FQpIu-0000vR-6D
	for asrg@ietf.org; Tue, 04 Apr 2006 13:26:56 -0400
Received: by xproxy.gmail.com with SMTP id s12so1132062wxc
	for <asrg@ietf.org>; Tue, 04 Apr 2006 10:26:55 -0700 (PDT)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com;
	h=received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition;
	b=XKkA2h6UyiAXGWJn1YbzomJSv+iJYmjWJvVSms6HYEidcJLJam5+S/LgvunkXCKpbtXdAzw6KmdEptzZdam2xkHBJq7uK0GNUhBN+gHIPpqhQ0qpLEc6hpz2pL9eISPCkEQkwy3ImHif35UIhHEavIV3tMYZ5AxwR678GMSiMgI=
Received: by 10.70.125.20 with SMTP id x20mr5585473wxc;
	Tue, 04 Apr 2006 10:26:55 -0700 (PDT)
Received: by 10.70.95.3 with HTTP; Tue, 4 Apr 2006 10:26:55 -0700 (PDT)
Message-ID: <934f64a20604041026t3e3ea86ene5c95fb2c8b0bbb5@mail.gmail.com>
Date: Tue, 4 Apr 2006 12:26:55 -0500
From: "David Nicol" <davidnicol@gmail.com>
To: asrg@ietf.org
Subject: greylisting ((was Re: [Asrg] Comments on
	draft-church-dnsbl-harmful-01.txt
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline
X-Spam-Score: 0.0 (/)
X-Scan-Signature: e1e48a527f609d1be2bc8d8a70eb76cb
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

On 4/4/06, Peter J. Holzer <hjp-asrg@hjp.at> wrote:

> So if I have records from the greylisting MTA, I can in theory look at
> every delivery attempt and classify it as:
>
> 1) Delivery was accepted and message was not spam (TN)
>
> 2) Delivery was accepted and message was spam (FN)
>
> 3) Delivery was temporarily rejected, but a later delivery of the same
>    message was accepted. This puts it into one of the two categories
>    above, but adds extra info (number of delivery attempts, delay from
>    first delivery attempt to acceptance).
>
> 4) Delivery was temporily rejected and no later delivery attempt was
>    successful. Message was spam (TP).
>
> 5) Delivery was temporily rejected and no later delivery attempt was
>    successful. Message was not spam (FP).
>
> In practice this is difficult. Normally, greylisting is done at the RCPT
> command, so there isn't much information available (EHLO, MAIL, RCPT,
> IP-address of client, maybe size of the message). So it is even
> difficult to determine whether two delivery attempts are for the same
> message, and even more difficult to determine whether the message was
> spam. Still, for small samples, where a human can go over the records
> and apply real-life knowledge, it should be possible to get a (somewhat
> fuzzy) categorization which is better than that determined by
> greylisting itself.

I have seen this subject has been beaten to death numerous times.  I
believe that the standard greylisting plugins (for instance for qpsmtpd)
have a list of the major services with broken MTAs that don't retry, just
like the standard SPF plugins have a list of major forwarding sites
that don't do SRS.

In practice the FPs are easy enough to find because they generate
complaints.  To do QA on a new greylisting setup, I agree, would be
most tricky as the FPs are lost.  There's no way to capture a message
into a "would have been greylisted" folder without forking the whole
damn universe.  Which is impossible given the current laws of physics.

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Tue Apr 04 19:36:39 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQv4S-0005cx-DO; Tue, 04 Apr 2006 19:36:24 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQv4Q-0005cM-S7
	for asrg@ietf.org; Tue, 04 Apr 2006 19:36:22 -0400
Received: from omta01sl.mx.bigpond.com ([144.140.92.153])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FQv4P-0007gl-D7
	for asrg@ietf.org; Tue, 04 Apr 2006 19:36:22 -0400
Received: from localhost.localdomain ([60.226.77.107])
	by omta01sl.mx.bigpond.com with ESMTP id
	<20060404233618.RGRY25377.omta01sl.mx.bigpond.com@localhost.localdomain>;
	Tue, 4 Apr 2006 23:36:18 +0000
Received: from ender (ender.scoobynet [192.168.0.3])
	by localhost.localdomain (Postfix) with ESMTP id 25DEEC0;
	Wed,  5 Apr 2006 09:35:57 +1000 (EST)
Received: by ender (Postfix, from userid 1000)
	id 207764DD61; Wed,  5 Apr 2006 09:34:25 +1000 (EST)
Date: Wed, 5 Apr 2006 09:34:24 +1000
From: Laird Breyer <laird@lbreyer.com>
To: Chris Lewis <clewis@nortel.com>
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
Message-ID: <20060404233424.GA32404@ender>
Mail-Followup-To: Chris Lewis <clewis@nortel.com>, asrg@ietf.org
References: <20060403055020.19752.qmail@simone.iecc.com>
	<4431512B.5040506@americasm01.nt.com>
	<20060404004112.GB27140@ender>
	<4431D719.6060703@americasm01.nt.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <4431D719.6060703@americasm01.nt.com>
User-Agent: Mutt/1.5.11+cvs20060126
X-Spam-Score: 0.1 (/)
X-Scan-Signature: 9ed51c9d1356100bce94f1ae4ec616a9
Cc: asrg@ietf.org
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
Reply-To: laird@lbreyer.com
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

On Apr 03 2006, Chris Lewis wrote:
> 
> The user is first.  I simply recognize that a user's accuracy rate on
> determining FPs is _lower_ than the automated systems are.

Again, I think it's due to a disagreement as to what spam is.  You
claim a user's accuracy is lower than an automated system's at
detecting consent-spam. I claim the user's accuracy is necessarily
higher than a third party at detecting what-I-want-now-spam. We could
go around in circles for a long time with this, but it makes more
sense to recognize that we're talking about slightly different ideas
of spam, which implicitly means your FPs aren't the same FPs I'm
talking about etc. 

> 
> You can't measure a voltage to 6 digits of accuracy if your equipment
> only does three.

I agree, and I'm not proposing that. A typical statistical hypothesis
test has the option of being inconclusive. But here we're not even 
measuring the same FPs so it's moot. 

A ROC curve is a "Receiver Operating Characteristic". Imagine you have
a spam filter sytem and a panoply of subsystems and parameter values
to choose from. For each possible setting, you measure the (FP%,FN%)
pair on some data.  Then you plot these points in a square of side
length 100%. This gives a picture of the overall quality of the spam
filter as you insert or remove the subsystems and parameters.

It allows you to compare different spam filters over various operating
ranges, and gives insight into the accuracy tradeoffs you make when
you replace one subsystem with another.

This tutorial may be a little technical, but the important things are
the pictures.
http://www.cs.bris.ac.uk/~flach/ICML04tutorial/ROCtutorialPartI.pdf

This is a readable introduction with a lot of detail.
http://home.comcast.net/~tom.fawcett/public_html/papers/ROC101.pdf


-- 
Laird Breyer.

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Tue Apr 04 19:46:20 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQvE3-0004In-5t; Tue, 04 Apr 2006 19:46:19 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQvE1-0004Ii-OS
	for asrg@ietf.org; Tue, 04 Apr 2006 19:46:17 -0400
Received: from omta03ps.mx.bigpond.com ([144.140.82.155])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FQvE0-00081e-AM
	for asrg@ietf.org; Tue, 04 Apr 2006 19:46:17 -0400
Received: from localhost.localdomain ([60.226.77.107])
	by omta03ps.mx.bigpond.com with ESMTP id
	<20060404234614.CKDK15112.omta03ps.mx.bigpond.com@localhost.localdomain>;
	Tue, 4 Apr 2006 23:46:14 +0000
Received: from ender (ender.scoobynet [192.168.0.3])
	by localhost.localdomain (Postfix) with ESMTP id D72DAC0;
	Wed,  5 Apr 2006 09:45:52 +1000 (EST)
Received: by ender (Postfix, from userid 1000)
	id 0E11A4DD61; Wed,  5 Apr 2006 09:44:21 +1000 (EST)
Date: Wed, 5 Apr 2006 09:44:20 +1000
From: Laird Breyer <laird@lbreyer.com>
To: Seth Breidbart <sethb@panix.com>
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
Message-ID: <20060404234420.GB32404@ender>
Mail-Followup-To: Seth Breidbart <sethb@panix.com>, asrg@ietf.org
References: <20060403051608.GC12316@ender>
	<20060403055020.19752.qmail@simone.iecc.com>
	<20060403235840.GA27140@ender>
	<200604040023.k340NaN23848@panix5.panix.com>
	<20060404012324.GD27140@ender>
	<200604040234.k342YQN21555@panix5.panix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <200604040234.k342YQN21555@panix5.panix.com>
User-Agent: Mutt/1.5.11+cvs20060126
X-Spam-Score: 0.1 (/)
X-Scan-Signature: ea4ac80f790299f943f0a53be7e1a21a
Cc: asrg@ietf.org
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
Reply-To: laird@lbreyer.com
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

On Apr 03 2006, Seth Breidbart wrote:
> 
> Consider some spamware: based on whether it gets 4XX or 5XX, early or
> late in the transaction, it decides how to continue (4XX resends the
> same thing later, 5XX early sends from another IP address, 5XX late
> changes hashbusters and sends from the same IP address, a second fail
> changes hashbusters again and sends from a different IP address, where
> to send other spams going to the same mailserver but to different
> addresses from, . . .)
> 
> Now, whatever you do affects what you see later.  Therefore you
> _cannot_ do a test without perturbing the system.

I don't think we're talking about the same thing. When I'm talking
about measuring, I'm not talking about intervening directly, making a
choice about 4XX or 5XX on the spot.

First, your greylisting system does what it normally does. The spammer
reacts and life goes on.  A week later (say), you go through the logs
and you verify that the system is not losing mail you think it should
accept. Think of it as debugging, QA, whatever. That's the accuracy
test.

You summarize this QA with a FP/FN rate, which counts the number of
mistakes/bugs of each type. There you are, an FP/FN rate. You can now
compare these numbers with FP/FNs obtained from another spamfilter on
comparable data.

-- 
Laird Breyer.

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Tue Apr 04 20:05:09 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQvW1-0007Tn-J0; Tue, 04 Apr 2006 20:04:53 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQvW0-0007Ti-2N
	for asrg@ietf.org; Tue, 04 Apr 2006 20:04:52 -0400
Received: from xproxy.gmail.com ([66.249.82.207])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FQvVx-0000Y5-RL
	for asrg@ietf.org; Tue, 04 Apr 2006 20:04:52 -0400
Received: by xproxy.gmail.com with SMTP id s12so1187218wxc
	for <asrg@ietf.org>; Tue, 04 Apr 2006 17:04:49 -0700 (PDT)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com;
	h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references;
	b=bWjWS24W5c64JOLfafZYrx26gum1PJ9Th1ID8gnqIE+AIlMlLznxQ4R1LS1mZsMdmOIaWIK1gc9BuHgvpYWOcoyEgNQNN1Mk6sBMcSo6WiXdfhFq4xSBp4jyuXi5TmGb+V61AUhoRevVisL5EWjnobWut5SmIaYnI+S29CC3dFQ=
Received: by 10.70.125.20 with SMTP id x20mr6041522wxc;
	Tue, 04 Apr 2006 17:04:49 -0700 (PDT)
Received: by 10.70.95.3 with HTTP; Tue, 4 Apr 2006 17:04:48 -0700 (PDT)
Message-ID: <934f64a20604041704x10889090j87e7a825a9b5b436@mail.gmail.com>
Date: Tue, 4 Apr 2006 19:04:48 -0500
From: "David Nicol" <davidnicol@gmail.com>
To: laird@lbreyer.com, "Chris Lewis" <clewis@nortel.com>, asrg@ietf.org
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
In-Reply-To: <20060404233424.GA32404@ender>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline
References: <20060403055020.19752.qmail@simone.iecc.com>
	<4431512B.5040506@americasm01.nt.com> <20060404004112.GB27140@ender>
	<4431D719.6060703@americasm01.nt.com> <20060404233424.GA32404@ender>
X-Spam-Score: 0.0 (/)
X-Scan-Signature: 8abaac9e10c826e8252866cbe6766464
Cc: 
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

On 4/4/06, Laird Breyer <laird@lbreyer.com> wrote:

> I agree, and I'm not proposing that. A typical statistical hypothesis
> test has the option of being inconclusive. But here we're not even
> measuring the same FPs so it's moot.
>
> A ROC curve is a "Receiver Operating Characteristic". Imagine you have
> a spam filter sytem and a panoply of subsystems and parameter values
> to choose from. For each possible setting, you measure the (FP%,FN%)
> pair on some data.  Then you plot these points in a square of side
> length 100%. This gives a picture of the overall quality of the spam
> filter as you insert or remove the subsystems and parameters.
>
> It allows you to compare different spam filters over various operating
> ranges, and gives insight into the accuracy tradeoffs you make when
> you replace one subsystem with another.
>
> This tutorial may be a little technical, but the important things are
> the pictures.
> http://www.cs.bris.ac.uk/~flach/ICML04tutorial/ROCtutorialPartI.pdf
>
> This is a readable introduction with a lot of detail.
> http://home.comcast.net/~tom.fawcett/public_html/papers/ROC101.pdf

Which brings us back to tagging.  In the tagging vision, the end-user gets
to adjust their own sliders and weights based on verifiable analyses done
by upstream expert systems.


--
David L Nicol
Should the bike shed have bunks?  Or maybe cots?

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Tue Apr 04 20:14:07 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQveu-0000ZD-Qv; Tue, 04 Apr 2006 20:14:04 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQves-0000YT-RE
	for asrg@ietf.org; Tue, 04 Apr 2006 20:14:02 -0400
Received: from omta05sl.mx.bigpond.com ([144.140.93.195])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FQver-0001Nf-VC
	for asrg@ietf.org; Tue, 04 Apr 2006 20:14:02 -0400
Received: from localhost.localdomain ([60.226.77.107])
	by omta05sl.mx.bigpond.com with ESMTP id
	<20060405001354.XRIL25409.omta05sl.mx.bigpond.com@localhost.localdomain>;
	Wed, 5 Apr 2006 00:13:54 +0000
Received: from ender (ender.scoobynet [192.168.0.3])
	by localhost.localdomain (Postfix) with ESMTP id 641D9C0;
	Wed,  5 Apr 2006 10:13:32 +1000 (EST)
Received: by ender (Postfix, from userid 1000)
	id 88B244DD61; Wed,  5 Apr 2006 10:12:00 +1000 (EST)
Date: Wed, 5 Apr 2006 10:12:00 +1000
From: Laird Breyer <laird@lbreyer.com>
To: Chris Lewis <clewis@nortel.com>
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
Message-ID: <20060405001200.GC32404@ender>
Mail-Followup-To: Chris Lewis <clewis@nortel.com>, asrg@ietf.org
References: <20060403051608.GC12316@ender>
	<20060403055020.19752.qmail@simone.iecc.com>
	<20060403235840.GA27140@ender>
	<200604040023.k340NaN23848@panix5.panix.com>
	<20060404012324.GD27140@ender>
	<4431D369.2060505@americasm01.nt.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <4431D369.2060505@americasm01.nt.com>
User-Agent: Mutt/1.5.11+cvs20060126
X-Spam-Score: 0.1 (/)
X-Scan-Signature: 25620135586de10c627e3628c432b04a
Cc: asrg@ietf.org
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
Reply-To: laird@lbreyer.com
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

On Apr 03 2006, Chris Lewis wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Laird Breyer wrote:
> > On Apr 03 2006, Seth Breidbart wrote:
> >> Laird Breyer <laird@lbreyer.com> wrote:
> >>> On Apr 03 2006, John Levine wrote:
> >>>> Not really.  I get spam complaints all the time for mail from lists
> >>>> that I know perfectly well that they signed up for and confirmed.
> >>>> "Oh, I don't want that any more."  For the ones that aren't totally
> >>>> redacted, my setup turns them into unsubs so they don't get any more
> >>>> mail for that particular list, but I don't think it's fair to count
> >>>> mail as spam if it depends on reading the recipient's mind in
> >>>> real-time.
> >>> Isn't that perilously close to saying that if they decided something
> >>> in the past, they're not allowed to change their mind?
> >> No.  If someone asked for something, they're certainly entitled to
> >> change their mind and request (and enforce) not getting it.  They're
> >> not entitled to change their mind and decide they never asked for it.
> 
> > That's because you're using the "consent" version of the spam definition.
> > You're not using the "spam is what I don't want" definition. Reread 
> > John's paragraph, he wrote "Oh, I don't want that any more".
> 
> I don't think John is calling that spam, is he?  I don't either.
> 
> I won't block on that basis.  I'll just help the user stop getting more.

He's not, neither are you. I'm using that definition, as do
other people who develop Bayesian content filters. John was giving an 
example why he thinks that definition is unworkable. However, it gets
used every day by lots of people and filters. Seth mentioned deciding
they never asked for it. 

> 
> >> Describe how you'd test greylisting without perturbing the system.
> > 
> > I already have outlined it several times now. The greylisting system
> > logs all the events which enter into the final decision. When a mail
> > is rejected, the record of evidence which triggered that rejection 
> > (call it R) is kept. 
> > 
> > Later, during QA, some volunteer users are sent a fraction of
> > rejection records R for mail messages that were intended for them, to
> > be approved. When the definition is "spam is what I say is spam",
> > these volunteers are the final arbiters of mail intended for them,
> > whether other people think they could do a better job or not.
> > 
> > Finally, you measure the number of records R which the volunteers say
> > are incorrectly rejected by the greylisting system.
> 
> Sorry, that notion is nonsensical with greylisting.  Are you aware of
> how greylisting works?
> 
> greylisting works by tempfailing the connection of _every_ "new" unique
> sender/recipient/source IP triple (or some variant), then letting the
> email through if the sending server retries.  The notion being that most
> spam senders cannot retry.

Yes, I am aware of that. I'm just trying (perhaps wrongly) to keep it
high level instead of discussing rejection modes at the SMTP
transaction level, or the socket connection level, etc. I think the
important aspect for testing is that the server doesn't always want to
relay the message body that the client wants to submit. So the mail is
effectively rejected. When (if) the client tries again, who's to say
that mail is the same one?

If you were developing a greylisting system and you were debugging it,
you'd log a lot of information. That's what I believe can be packaged
for QA testers in this scenario.

-- 
Laird Breyer.

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Tue Apr 04 20:23:13 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQvni-0001aX-V1; Tue, 04 Apr 2006 20:23:10 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQvnh-0001aS-Nn
	for asrg@ietf.org; Tue, 04 Apr 2006 20:23:09 -0400
Received: from omta05ps.mx.bigpond.com ([144.140.83.195])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FQvng-0001b1-Ak
	for asrg@ietf.org; Tue, 04 Apr 2006 20:23:09 -0400
Received: from localhost.localdomain ([60.226.77.107])
	by omta05ps.mx.bigpond.com with ESMTP id
	<20060405002301.VLBA14751.omta05ps.mx.bigpond.com@localhost.localdomain>
	for <asrg@ietf.org>; Wed, 5 Apr 2006 00:23:01 +0000
Received: from ender (ender.scoobynet [192.168.0.3])
	by localhost.localdomain (Postfix) with ESMTP id 5B63EC0
	for <asrg@ietf.org>; Wed,  5 Apr 2006 10:22:39 +1000 (EST)
Received: by ender (Postfix, from userid 1000)
	id 7FED14DD61; Wed,  5 Apr 2006 10:21:07 +1000 (EST)
Date: Wed, 5 Apr 2006 10:21:07 +1000
From: Laird Breyer <laird@lbreyer.com>
To: asrg@ietf.org
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
Message-ID: <20060405002107.GD32404@ender>
Mail-Followup-To: asrg@ietf.org
References: <20060403051608.GC12316@ender>
	<20060403055020.19752.qmail@simone.iecc.com>
	<20060403235840.GA27140@ender>
	<200604040023.k340NaN23848@panix5.panix.com>
	<20060404012324.GD27140@ender>
	<4431D369.2060505@americasm01.nt.com>
	<20060404163759.GD8054@teal.hjp.at>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20060404163759.GD8054@teal.hjp.at>
User-Agent: Mutt/1.5.11+cvs20060126
X-Spam-Score: 0.1 (/)
X-Scan-Signature: 30ac594df0e66ffa5a93eb4c48bcb014
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
Reply-To: laird@lbreyer.com
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

On Apr 04 2006, Peter J. Holzer wrote:

(greylisting testing)

You've explained the point I was trying to make much better than I just
have. 

-- 
Laird Breyer.

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Tue Apr 04 21:07:58 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQwUx-0008S9-6K; Tue, 04 Apr 2006 21:07:51 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQwUv-0008Rz-FS
	for asrg@ietf.org; Tue, 04 Apr 2006 21:07:49 -0400
Received: from tom.iecc.com ([208.31.42.38])
	by ietf-mx.ietf.org with smtp (Exim 4.43) id 1FQwUt-0003AF-L8
	for asrg@ietf.org; Tue, 04 Apr 2006 21:07:49 -0400
Received: (qmail 18357 invoked from network); 5 Apr 2006 01:07:45 -0000
Received: from mail.iecc.com (208.31.42.99)
	by mail.iecc.com with QMQP; 5 Apr 2006 01:07:45 -0000
Date: 5 Apr 2006 01:07:45 -0000
Message-ID: <20060405010745.98882.qmail@simone.iecc.com>
From: John Levine <asrg@johnlevine.com>
To: asrg@ietf.org
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
In-Reply-To: <20060403235840.GA27140@ender>
Organization: 
Mime-Version: 1.0
Content-type: text/plain; charset=iso-8859-1
Content-transfer-encoding: 7bit
X-Spam-Score: -4.3 (----)
X-Scan-Signature: 69a74e02bbee44ab4f8eafdbcedd94a1
Cc: laird@lbreyer.com
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

>> Not really.  I get spam complaints all the time for mail from lists
>> that I know perfectly well that they signed up for and confirmed.
>> "Oh, I don't want that any more."  For the ones that aren't totally
>> redacted, my setup turns them into unsubs so they don't get any more
>> mail for that particular list, but I don't think it's fair to count
>> mail as spam if it depends on reading the recipient's mind in
>> real-time.
>
>Isn't that perilously close to saying that if they decided something
>in the past, they're not allowed to change their mind?

Of course not.  You can change your mind all you want, but you can't
expect me to know that you have done so until you tell me.  So if the
user unsubs and the mail keeps coming, then it's spam, but if he asked
for it and hasn't asked for it to stop, it's legit.

>> Testing spam filters is a really, really hard problem, for all the
>> reasons that people have mentioned, basically that you can't test
>> without perturbing the system you're testing, and you can't capture
>> enough state to rerun the same test more than once.  So you have to
>> estimate based on complaint rates from bounces and the like.
>
>Sure is, or nobody would make a big deal about it. I don't agree that
>perturbing the system is necessary, however.

Then I have to say that you're not thinking very clearly about the way
that spam filters work.  As soon as you do or don't deliver a message,
or delay it by greylisting, or query a DNSBL, or any of a zillion
other things that filters do, you've perturbed the system.

> In the end, testing is about making the most of the information
> that's actually available.

Well, sure, but it's also about knowing whether your synthetic tests
have any relevance to the real system you're trying to model.

R's,
John

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Tue Apr 04 21:18:01 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQwek-0001mY-Pe; Tue, 04 Apr 2006 21:17:58 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQwej-0001mT-Hr
	for asrg@ietf.org; Tue, 04 Apr 2006 21:17:57 -0400
Received: from tom.iecc.com ([208.31.42.38])
	by ietf-mx.ietf.org with smtp (Exim 4.43) id 1FQwei-0003Xm-2j
	for asrg@ietf.org; Tue, 04 Apr 2006 21:17:57 -0400
Received: (qmail 19533 invoked from network); 5 Apr 2006 01:17:54 -0000
Received: from mail.iecc.com (208.31.42.99)
	by mail.iecc.com with QMQP; 5 Apr 2006 01:17:54 -0000
Received: from localhost (sendmail-bs@127.0.0.1)
	by localhost with SMTP; 5 Apr 2006 01:17:54 -0000
Date: Tue, 4 Apr 2006 21:17:54 -0400 (EDT)
From: John L <johnl@iecc.com>
To: Anti Spam Research Group <asrg@ietf.org>
Message-ID: <20060404211653.U1211@simone.iecc.com>
Cleverness: None detected
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
X-Spam-Score: -4.3 (----)
X-Scan-Signature: 08e48e05374109708c00c6208b534009
Subject: [Asrg] About that Church draft
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

We sure have a lot of raw material.  Can I have a volunteer editor or two 
to moosh the good stuff into one reasonably coherent document explaining 
what we think would need to be changed for it to be good?

Regards,
John Levine, johnl@iecc.com, Primary Perpetrator of "The Internet for Dummies",
Information Superhighwayman wanna-be, http://iecc.com/johnl, Mayor
"I dropped the toothpaste", said Tom, crestfallenly.

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Tue Apr 04 22:11:25 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FQxUK-0007pv-JG; Tue, 04 Apr 2006 22:11:16 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FQxUJ-0007pQ-HW
	for asrg@ietf.org; Tue, 04 Apr 2006 22:11:15 -0400
Received: from omta02sl.mx.bigpond.com ([144.140.93.154])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FQxUF-0005Vi-0E
	for asrg@ietf.org; Tue, 04 Apr 2006 22:11:15 -0400
Received: from localhost.localdomain ([60.226.77.107])
	by omta02sl.mx.bigpond.com with ESMTP id
	<20060405021108.QIRI24931.omta02sl.mx.bigpond.com@localhost.localdomain>;
	Wed, 5 Apr 2006 02:11:08 +0000
Received: from ender (ender.scoobynet [192.168.0.3])
	by localhost.localdomain (Postfix) with ESMTP id B296BC0;
	Wed,  5 Apr 2006 12:10:46 +1000 (EST)
Received: by ender (Postfix, from userid 1000)
	id C750A4DD61; Wed,  5 Apr 2006 12:09:14 +1000 (EST)
Date: Wed, 5 Apr 2006 12:09:14 +1000
From: Laird Breyer <laird@lbreyer.com>
To: John Levine <asrg@johnlevine.com>
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
Message-ID: <20060405020914.GA700@ender>
Mail-Followup-To: John Levine <asrg@johnlevine.com>, asrg@ietf.org
References: <20060403235840.GA27140@ender>
	<20060405010745.98882.qmail@simone.iecc.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20060405010745.98882.qmail@simone.iecc.com>
User-Agent: Mutt/1.5.11+cvs20060126
X-Spam-Score: 0.1 (/)
X-Scan-Signature: 92df29fa99cf13e554b84c8374345c17
Cc: asrg@ietf.org
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
Reply-To: laird@lbreyer.com
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

On Apr 05 2006, John Levine wrote:
> >> Not really.  I get spam complaints all the time for mail from lists
> >> that I know perfectly well that they signed up for and confirmed.
> >> "Oh, I don't want that any more."  For the ones that aren't totally
> >> redacted, my setup turns them into unsubs so they don't get any more
> >> mail for that particular list, but I don't think it's fair to count
> >> mail as spam if it depends on reading the recipient's mind in
> >> real-time.
> >
> >Isn't that perilously close to saying that if they decided something
> >in the past, they're not allowed to change their mind?
> 
> Of course not.  You can change your mind all you want, but you can't
> expect me to know that you have done so until you tell me.  So if the
> user unsubs and the mail keeps coming, then it's spam, but if he asked
> for it and hasn't asked for it to stop, it's legit.

What if he likes to read the list messages on a single topic, but
doesn't want those messages that deal with another topic? 

Or take the ASRG list itself. For argument's sake, I might be only
interested in micropayment-type systems, and don't want to see any other
threads such as announcements on IETF drafts. When signing up, did I
know that I was going to get IETF announcements? Am I wrong to claim
the list sends me some spam?  Did the list description on the ASRG homepage
change at all from the time several years ago when I subscribed and now? 
Consent is not that simple either. If I unsubscribe, I lose the micropayment
threads, if I stay subscribed I get spam announcements :)

> 
> >> Testing spam filters is a really, really hard problem, for all the
> >> reasons that people have mentioned, basically that you can't test
> >> without perturbing the system you're testing, and you can't capture
> >> enough state to rerun the same test more than once.  So you have to
> >> estimate based on complaint rates from bounces and the like.
> >
> >Sure is, or nobody would make a big deal about it. I don't agree that
> >perturbing the system is necessary, however.
> 
> Then I have to say that you're not thinking very clearly about the way
> that spam filters work.  As soon as you do or don't deliver a message,
> or delay it by greylisting, or query a DNSBL, or any of a zillion
> other things that filters do, you've perturbed the system.

That's the spamfilter operating, not the QA test of how well the
spamfilter is judged to be working.

It is obvious that some spamfilters can change spammer behaviour
directly. We've been discussing (at least I think we have) whether the
act of measuring the operation of a spamfilter itself changes the
spammer behaviour. And my claim is that the act of measuring the
operation of the spamfilter need not affect spammer behaviour
directly.

There's also a subtlety with predicting the effect of a putative,
nonexistent spamfilter if its operation directly changes spammer
behaviour, but that's another thing again.

> 
> > In the end, testing is about making the most of the information
> > that's actually available.
> 
> Well, sure, but it's also about knowing whether your synthetic tests
> have any relevance to the real system you're trying to model.

If you can obtain FP and FN numbers, then I believe you have a
valuable summary of accuracy for that system, but it won't tell you
about other aspects such as cpu load etc. One thing at a time please.

-- 
Laird Breyer.

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Wed Apr 05 01:08:55 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FR0Ex-0006y0-0R; Wed, 05 Apr 2006 01:07:35 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FR0Ev-0006xv-2P
	for asrg@ietf.org; Wed, 05 Apr 2006 01:07:33 -0400
Received: from omta02sl.mx.bigpond.com ([144.140.93.154])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FR0Er-0003R6-Nk
	for asrg@ietf.org; Wed, 05 Apr 2006 01:07:33 -0400
Received: from localhost.localdomain ([60.226.77.107])
	by omta02sl.mx.bigpond.com with ESMTP id
	<20060405050726.YCTH24931.omta02sl.mx.bigpond.com@localhost.localdomain>;
	Wed, 5 Apr 2006 05:07:26 +0000
Received: from ender (ender.scoobynet [192.168.0.3])
	by localhost.localdomain (Postfix) with ESMTP id A6260C0;
	Wed,  5 Apr 2006 15:07:05 +1000 (EST)
Received: by ender (Postfix, from userid 1000)
	id 883AB4DD61; Wed,  5 Apr 2006 15:05:33 +1000 (EST)
Date: Wed, 5 Apr 2006 15:05:33 +1000
From: Laird Breyer <laird@lbreyer.com>
To: John L <johnl@iecc.com>
Subject: Re: [Asrg] About that Church draft
Message-ID: <20060405050533.GA1220@ender>
Mail-Followup-To: John L <johnl@iecc.com>,
	Anti Spam Research Group <asrg@ietf.org>
References: <20060404211653.U1211@simone.iecc.com>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="jRHKVT23PllUwdXP"
Content-Disposition: inline
In-Reply-To: <20060404211653.U1211@simone.iecc.com>
User-Agent: Mutt/1.5.11+cvs20060126
X-Spam-Score: 0.1 (/)
X-Scan-Signature: 29dc808194f5fb921c09d0040806d6eb
Cc: Anti Spam Research Group <asrg@ietf.org>
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
Reply-To: laird@lbreyer.com
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org


--jRHKVT23PllUwdXP
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

On Apr 04 2006, John L wrote:
> We sure have a lot of raw material.  Can I have a volunteer editor or two 
> to moosh the good stuff into one reasonably coherent document explaining 
> what we think would need to be changed for it to be good?

I've just read through the threads again and done a summary of all the
main relevant points everyone discussed on the list (see attachment). 

If someone with the time wants to rearrange the paragraphs and flesh
them out into a final document, please pipe up and do so. I tried to
relate every point explicitly to the Church draft. Apologies if I
missed someone's contribution.

-- 
Laird Breyer.
--jRHKVT23PllUwdXP
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="asrg-church-criticisms.txt"

Summary of the ASRG discussion on draft-church-dnsbl-harmful-01.txt as
of 04-05-2006.

This is a review of the draft-church-dnsbl-harmful-01.txt document as discussed
on the ASRG mailing list http://news.gmane.org/gmane.ietf.asrg/

Note: Some of the posts are missing from the archive, such as Dan Oetting's
message-id <84E9C57C-1CE6-4232-90FC-7BE5F9EEC422@qwest.net>. I was going
to refer to archive urls but ended up just giving Message-IDs.  

The church document has the following structure.

1) An informal description of IP DNS blacklists.
2) A criticism of the policies and goals of DNS blacklist operators.
3) A claim of damage to third party mail transport.
4) A claim that botnets cause DNS blacklists to be out of date immediately.
5) An experiment on the accuracy of filtering using DNS blacklists.
6) A suggestion for the improvement of IP recognition
7) A moral argument of responsibility by DNS blacklist operators.
8) A suggestion to deprecate parts of the internet mail system to newer technologies.

Taken together, the author comes to the conclusion that DNS blacklists should
not be used.

The ASRG discussion centered on several of these points.

Claims 2) and 3) are sensitive issues fraught with political aspects,
and are therefore not appropriate for technical criticism, although
they do represent some valid concerns that are often brought up.

Dan Oetting's thread deals with 4). There is agreement that botnets are
a problem, but there is some research into the problem.

Jim MacLeod <http://article.gmane.org/gmane.ietf.asrg/11516> thread makes
these points:

Regarding 4) IP assignments to particular computers tend to change
slowly enough (on the order of a month) to make IP lists
valuable. [reference?}

Regarding 2), diversity is a strength in anti-spam.

Regarding 5), the experiment is flawed due to its size.

Daniel Feenberg <Pine.GSO.4.64.0603290717090.4480@nber5.nber.org>
mentions the increased legitimacy of rejecting mail at the SMTP
transaction protocol level rather than silently discarding it for
filtering reasons. Unlike content filtering, DNSBLs can be used at the
transaction time.

Steve Atkins <4D2D1BA3-767A-47FC-BC9F-D54829C2E450@blighty.com>
suggests there are significant general problems with DNSBLs, such as being
misused as reputation systems, technical flaws in their usage,
etc. It may be (Chris Lewis
<442C2405.1040601@americasm01.nt.com>) that the real problem is that
people responsible for choosing DNSBLs aren't necessarily exercising
due diligence and knowing what they're getting.

Tony Finch disagrees with Steve Atkins, calling DNSBLs crucial. 
Nick Nicholas asked for solid evidence, thereby being relevant to 5).

There is a big thread on testing philosophy of DNSBLs. Everyone agrees
that 5) in Church's paper is flawed. The flaws are as follows:

Flaw: The Church sample size (1,374 msgs) is much too small to be
meaningful.  Chris Lewis <442EE80C.5080500@americasm01.nt.com>
disagrees with Church's conclusions based on 65,000 users and 600,000
emails per day at Nortel.

Flaw: the comparison of filters in Church's table is
inapplicable. Steve Atkins
<2BC9207C-3E51-4B88-97A8-7445E9A1313B@blighty.com> points out that
pretending that a DNSBL filter was used for comparison purposes, as is
done by Church, is meaningless when it actually wasn't, because filtering
during the SMTP transaction gives feedback (Daniel Feenberg's point)
and therefore changes the spammer's behaviour.

Flaw: Church doesn't explicitly define his terms, such as just what he
considers spam. In turn, this means derived definitions of FP, FN, TP
etc. are not fully mapped out. Laird Breyer and Chris Lewis discuss
the consequences of using different definitions, resulting in
contradictory claims.

Flaw: Church's experiment is applicable at best to one type of email user.
Chris Lewis's experience strongly contradicts the Church conclusion, and is
based on very different business user email patterns.

Flaw: Church's table data suggests rejecting DNSBLs individually on
accuracy grounds. However, this is flawed in that DNSBLs are often
used in conjunction with other filters. The table doesn't address
these other possibilities at all.

Several people point out that DNSBLs are meant to be used as part of a mix of
techniques, ie that the whole is bigger than the sum of its parts.
 
Peter J. Holzer <20060331191807.GA14686@teal.hjp.at> offers some statistics
on the most effective elements of his mix, others chime in on that thread.

Walter Dnes <20060401015927.GA27452@waltdnes.org> addresses 2) and 3),
which are out of scope of the technical criticism. He points out that
5) is flawed because it compares a DNSBL-only solution against an
unnamed, unknown custom method, and lists several other flaws
above. This is not realistic since DNSBLs are used in conjuction with
other methods. Walter also points out that 8) is ridiculous.

Thomas Choi <44322933.7060103@nortel.com> summarizes various issues
that appeared during the discussion. 

Regarding 2) and 3), it's important for users of DNSBLs to apply diligence
and caution. A single DNSBL verdict should never be used on its own, as
is implied in 5). 

Regarding 5), spam filtering statistics always depend on type of email
mix being received. This varies depending upon the type of user
(business or private individual) and the type of social interactions
(frequent mail with regular correspondents or regular short exchanges
with strangers), etc.

Regarding 8), Thomas points out that mail technologies arise out of
many different needs of users, and what is optimal for one person or
entity in terms of usability, time, expertise, processing power etc. is not
necessarily optimal for another.


--jRHKVT23PllUwdXP
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg

--jRHKVT23PllUwdXP--




From asrg-bounces@ietf.org Wed Apr 05 03:07:55 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FR274-0005yC-Ec; Wed, 05 Apr 2006 03:07:34 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FR272-0005x3-T3
	for asrg@ietf.org; Wed, 05 Apr 2006 03:07:32 -0400
Received: from zrtps0kp.nortel.com ([47.140.192.56])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FR271-0008AD-F0
	for asrg@ietf.org; Wed, 05 Apr 2006 03:07:32 -0400
Received: from zrtphxs1.corp.nortel.com
	(zrtphxs1.corp.nort...s0.corp.nortel.com [47.140.202.46])
	by zrtps0kp.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id
	k3577SF09718
	for <asrg@ietf.org>; Wed, 5 Apr 2006 03:07:29 -0400 (EDT)
Received: from zrtps0m6.us.nortel.com ([47.140.192.58]) by
	zrtphxs1.corp.nortel.com with Microsoft SMTPSVC(6.0.3790.211); 
	Wed, 5 Apr 2006 03:07:27 -0400
Received: from [47.130.24.98] (acart12z.ca.nortel.com [47.130.24.98])
	by zrtps0m6.us.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id
	k3577N401161
	for <asrg@ietf.org>; Wed, 5 Apr 2006 03:07:23 -0400 (EDT)
Message-ID: <44336CA8.5090804@americasm01.nt.com>
Date: Wed, 05 Apr 2006 03:07:20 -0400
From: "Chris Lewis" <clewis@nortel.com>
User-Agent: Thunderbird 1.5 (Windows/20051201)
MIME-Version: 1.0
To: Anti Spam Research Group <asrg@ietf.org>
Subject: Re: [Asrg] About that Church draft
References: <20060404211653.U1211@simone.iecc.com>
	<20060405050533.GA1220@ender>
In-Reply-To: <20060405050533.GA1220@ender>
X-Enigmail-Version: 0.94.0.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-OriginalArrivalTime: 05 Apr 2006 07:07:27.0519 (UTC)
	FILETIME=[98DA92F0:01C6587F]
X-Spam-Score: 0.1 (/)
X-Scan-Signature: 5011df3e2a27abcc044eaa15befcaa87
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Laird Breyer wrote:

> I've just read through the threads again and done a summary of all the
> main relevant points everyone discussed on the list (see attachment).

I think you did a rather excellent job, and summarized it all pretty well.

I'll discuss this with Thomas, and see if he wants to volunteer to do a
final draft, but, frankly, I think it's pretty good as-is.  I
particularly like how the various threads are summarized.

Almost ;-)  I'll add a couple of points:

> Flaw: Church's experiment is applicable at best to one type of email user.
> Chris Lewis's experience strongly contradicts the Church conclusion, and is
> based on very different business user email patterns.

Agreed, Church's environment and mine are _highly_ different.  However,
given our relatively permissive policy in terms of personal use of
email, we're probably not "extremely business pattern" either.  Much
"stranger to stranger" communications as well as communications with
non-corporate/non-business-related environments.

[You'd be amazed at how much business goes on with random
freemail/consumer ISP accounts for example.]

[This is more of an amplification of what you summarized - I added a
real world example of high variability]

Flaw: Church didn't describe his environment in terms of user
characteristics, nor population, and is likely to be a very small group.
Experience demonstrates that the "spam experience" varies wildly even
within a single domain.  In the Nortel environment (65,000 users),
roughly 50% get virtually no spam at all (<1/day), where as others are
more than 500/day.  Similarly, the types of spam, and which techniques
work best, vary wildly between users.

Hence: to get a broader more accurate general picture, you need a very
large user population and high volume of email.

> Regarding 2) and 3), it's important for users of DNSBLs to apply diligence
> and caution. A single DNSBL verdict should never be used on its own, as
> is implied in 5).

I disagree. I don't think you'll get general agreement on the last
sentence.  I'm going to back this up and make a more generalized comment
that I think should be reasonably acceptable to all:

We have no business dictating what individual site policy on email
acceptance policy should be.  What we assert is that site administrators
need to assess what's available (whether it be DNSBLs or any other
filtering technique) against their needs, and choose/implement
appropriately - due diligence.

If a site determines that it has no reason to communicate with, for
example, a given country, or sites based on specific listing criteria,
there is no reason why the site couldn't use a "single verdict" choice
based on an appropriate DNSBL or other filtering method.  On the other
hand, some DNSBLs are of a nature or the needs are such where scoring
methods will be more appropriate.

All filtering techniques have such trade offs. Just about all techniques
used "single verdict" can be just as prone, if not more so than many
DNSBLs, to miss-trigger[+].  We should not be singling out DNSBLs,
because other commonly used filters often have similar (or worse)
failure rates.

Site administrators must accept that it is _their_ responsibility to
_know_ what they're getting no matter what it is, know what their needs
are, apply the former appropriately to satisfy the latter, and continue
to monitor the system in terms of performance.  Success or failure of a
filtering system is always ultimately the administrator's responsibility.

[+] To raise an obvious one: "bad word" filters.  It's surprising how
many 4 and 5 letter "crude Anglo-Saxonisms" have found their way into
place names and common terminology.  We occasionally use "bad word"
filtering, but only the ones that we believe very unlikely to appear in
non-spam communications.  Unfortunately, on the last one I
miscalculated, and tripped over the name of a provincial park in
Newfoundland (almost caused some people to miss their reservations),
which also happens to be the "proper" term for part of a skydiver's
equipment...  We won't go into earlier experiences, for example, when a
very large ISP blocked the word "breast" - and entirely disrupted
several breast cancer support groups.  But, apparently "bad word"
filters have become a lot more common these days.  These things can be
hideously inaccurate.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3-nr1 (Windows 2000)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBRDNsp53FmCyJjHfhAQKduQP/eyHSwKX7H2h4jAFXL+rvSTO0dYFNAhBV
ck6+vFDTr4svNBGf141MWOJPih5HmPqoEClpR3dySwBHRYnOlzPcrfvEkPTDn4iH
G/g0rLFuQ5QuqOLVZoHcEVv2VZjkaq6CXktjJwknqW5eERSkcKtrU9eVQLkAFygm
8n6UL3McduI=
=0Pwz
-----END PGP SIGNATURE-----

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Wed Apr 05 07:33:08 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FR6Fs-0004Ys-VC; Wed, 05 Apr 2006 07:32:56 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FR6Fr-0004Yn-Gy
	for asrg@ietf.org; Wed, 05 Apr 2006 07:32:55 -0400
Received: from mail1.nber.org ([66.251.72.7])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FR6Fr-00027K-5L
	for asrg@ietf.org; Wed, 05 Apr 2006 07:32:55 -0400
Received: from mail1 (root@localhost)
	by mail1.nber.org (8.12.11.20060308/8.12.8) with SMTP id k35BWsHi010635
	for <asrg@ietf.org>; Wed, 5 Apr 2006 07:32:54 -0400
Received: from nber5.nber.org (nber5.nber.org [66.251.72.75])
	by mail1.nber.org (8.12.11.20060308/8.12.8) with ESMTP id
	k35BWsPx010617; Wed, 5 Apr 2006 07:32:54 -0400
Received: from nber5.nber.org (localhost [127.0.0.1])
	by nber5.nber.org (8.12.10+Sun/8.12.10) with ESMTP id k35BWsog010877;
	Wed, 5 Apr 2006 07:32:54 -0400 (EDT)
Received: from localhost (feenberg@localhost)
	by nber5.nber.org (8.12.10+Sun/8.12.9/Submit) with ESMTP id
	k35BWpVU010874; Wed, 5 Apr 2006 07:32:51 -0400 (EDT)
Date: Wed, 5 Apr 2006 07:32:51 -0400 (EDT)
From: Daniel Feenberg <feenberg@nber.org>
To: Laird Breyer <laird@lbreyer.com>
Subject: Re: [Asrg] About that Church draft
In-Reply-To: <20060405050533.GA1220@ender>
Message-ID: <Pine.GSO.4.64.0604050727140.8973@nber5.nber.org>
References: <20060404211653.U1211@simone.iecc.com>
	<20060405050533.GA1220@ender>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
X-Spam-Score: 0.0 (/)
X-Scan-Signature: 39bd8f8cbb76cae18b7e23f7cf6b2b9f
Cc: John L <johnl@iecc.com>, Anti Spam Research Group <asrg@ietf.org>
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org



On Wed, 5 Apr 2006, Laird Breyer wrote:

> On Apr 04 2006, John L wrote:
>> We sure have a lot of raw material.  Can I have a volunteer editor or two
>> to moosh the good stuff into one reasonably coherent document explaining
>> what we think would need to be changed for it to be good?
>
> I've just read through the threads again and done a summary of all the
> main relevant points everyone discussed on the list (see attachment).
>
> If someone with the time wants to rearrange the paragraphs and flesh
> them out into a final document, please pipe up and do so. I tried to
> relate every point explicitly to the Church draft. Apologies if I
> missed someone's contribution.

Should the draft be so compact? There isn't room to include a paragraph or 
two from each of the most coherent messages? Some of the posters were
quite eloquent.

The draft seems to accept Church's notion that if a spam message
comes from 123.123.123.123.dial-up.pool.example.com and that is added
to a DNSBL, the DNSBL operator has an obligation to detect the 
reassignment of that IP address to another customer next week, and remove
the address. Is that the sense of the group? I would argue that a 
perfectly legitimate DNSBL could block dynamic addresses, even when those
addresses were temporarily assigned to legitimate MTAs.



Daniel Feenberg

>
> -- 
> Laird Breyer.


_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Wed Apr 05 07:55:15 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FR6bQ-0003gB-SN; Wed, 05 Apr 2006 07:55:12 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FR6bQ-0003g6-G2
	for asrg@ietf.org; Wed, 05 Apr 2006 07:55:12 -0400
Received: from mar92-6-82-226-38-60.fbx.proxad.net ([82.226.38.60]
	helo=joe.j-chkmail.org) by ietf-mx.ietf.org with esmtp (Exim 4.43)
	id 1FR6bO-0002V5-RJ
	for asrg@ietf.org; Wed, 05 Apr 2006 07:55:12 -0400
Received: from [127.0.0.1] (localhost. [127.0.0.1])
	by joe.j-chkmail.org (sendmail X.1.0.PreAlpha3.0) with ESMTP
	id S00000000000004EF01; Wed,  5 Apr 2006 13:55:09 +0200
Message-ID: <4433B01C.8020303@ensmp.fr>
Date: Wed, 05 Apr 2006 13:55:08 +0200
From: Jose Marcio Martins da Cruz <Jose-Marcio.Martins@ensmp.fr>
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.12) Gecko/20050920
X-Accept-Language: en-us, en
MIME-Version: 1.0
To: Chris Lewis <clewis@nortel.com>
Subject: Re: [Asrg] About that Church draft
References: <20060404211653.U1211@simone.iecc.com>	<20060405050533.GA1220@ender>
	<44336CA8.5090804@americasm01.nt.com>
In-Reply-To: <44336CA8.5090804@americasm01.nt.com>
X-Spam-Score: 0.2 (/)
X-Scan-Signature: 0ff9c467ad7f19c2a6d058acd7faaec8
Cc: Anti Spam Research Group <asrg@ietf.org>
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
Reply-To: Jose-Marcio.Martins@ensmp.fr
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Content-Type: multipart/mixed; boundary="===============1931375954=="
Errors-To: asrg-bounces@ietf.org

This is a cryptographically signed message in MIME format.

--===============1931375954==
Content-Type: multipart/signed; protocol="application/x-pkcs7-signature";
	micalg=sha1; boundary="------------ms050503040805080809050307"

This is a cryptographically signed message in MIME format.

--------------ms050503040805080809050307
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit

Chris Lewis wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 

...

> 
> [You'd be amazed at how much business goes on with random
> freemail/consumer ISP accounts for example.]

Yess...


>>Regarding 2) and 3), it's important for users of DNSBLs to apply diligence
>>and caution. A single DNSBL verdict should never be used on its own, as
>>is implied in 5).
> 
> 
> I disagree. I don't think you'll get general agreement on the last
> sentence.  I'm going to back this up and make a more generalized comment
> that I think should be reasonably acceptable to all:
> 
> We have no business dictating what individual site policy on email
> acceptance policy should be.  What we assert is that site administrators
> need to assess what's available (whether it be DNSBLs or any other
> filtering technique) against their needs, and choose/implement
> appropriately - due diligence.
> 

Example :

This is the result of one day of mail filtering with mail-abuse. 
127.1.0.2 means the dial-up and dynamic addresses.

  -- 127.1.0.1        :   2301
  -- 127.1.0.2        :  30080
  -- 127.1.0.3        :    158
  -- 127.1.0.4        :     60
  -- 127.1.0.5        :      1
  -- 127.1.0.6        :      3
  -- 127.1.0.8        :    101
  -- 127.1.0.9        :      1
  -- 127.1.0.10       :     47

Data above is the equivalent of "If I use mail-abuse, I block everything 
from dial-up and dynamic IP addresses".

Some kind of organisations may say : "I have not to do with dial-up 
users - so I block them". On the other hand, a commercial organisation 
whose clients usually send their orders by mail coming from this kind of 
addresses will probably accept messages from this kind of addresses.

Not all blacklists results can be seen the same way, so saying what and 
how to do with blacklists is better placed in a BCP, not an RFC.

-- 
   ---------------------------------------------------------------
   Jose Marcio MARTINS DA CRUZ           Tel. :(33) 01.40.51.93.41
   Ecole des Mines de Paris              http://j-chkmail.ensmp.fr
   60, bd Saint Michel                http://www.ensmp.fr/~martins
   75272 - PARIS CEDEX 06      mailto:Jose-Marcio.Martins@ensmp.fr

--------------ms050503040805080809050307
Content-Type: application/x-pkcs7-signature; name="smime.p7s"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="smime.p7s"
Content-Description: S/MIME Cryptographic Signature

MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIKKDCC
BRAwggL4oAMCAQICAwHW9zANBgkqhkiG9w0BAQUFADB5MRAwDgYDVQQKEwdSb290IENBMR4w
HAYDVQQLExVodHRwOi8vd3d3LmNhY2VydC5vcmcxIjAgBgNVBAMTGUNBIENlcnQgU2lnbmlu
ZyBBdXRob3JpdHkxITAfBgkqhkiG9w0BCQEWEnN1cHBvcnRAY2FjZXJ0Lm9yZzAeFw0wNjAx
MjExNDQ4MjBaFw0wNzAxMjExNDQ4MjBaMFMxJDAiBgNVBAMTG0pvc2UtTWFyY2lvIE1hcnRp
bnMgZGEgQ3J1ejErMCkGCSqGSIb3DQEJARYcSm9zZS1NYXJjaW8uTWFydGluc0BlbnNtcC5m
cjCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMyu8K6aUJMX2ZJP69UGRaB/gZ2W
FzSXUpLmA+1lNGlc40X8D6N9mTwWX/IpI1Ppcxd9QYOBLm2M2Atc7IjV78MXj75dGYbOT0Dz
YUovwoTGhKOYg18VIuxQ4rinGEI8eIqX6gMJ/Ftzox3H/og04AiIZxyMClAjuV5QFN7FRRtI
uYGYgLaT+Hq5Q2LXizthF0ewXBfqH1GmJFyErhjkxWQTH4Dq8oFZuRnWA4V2Y30Ss6dn1mjx
ySA3AsMaMNQTAZssa6R0BOqAFui3vm58zghLL23Mp5De2bpM83Y2mA8yqMEsF0DWr8Hi40a8
6xCYZHqKtTVxjeXqVRv5OxF8J/sCAwEAAaOBxjCBwzAMBgNVHRMBAf8EAjAAMFYGCWCGSAGG
+EIBDQRJFkdUbyBnZXQgeW91ciBvd24gY2VydGlmaWNhdGUgZm9yIEZSRUUgaGVhZCBvdmVy
IHRvIGh0dHA6Ly93d3cuQ0FjZXJ0Lm9yZzAyBggrBgEFBQcBAQQmMCQwIgYIKwYBBQUHMAGG
Fmh0dHA6Ly9vY3NwLmNhY2VydC5vcmcwJwYDVR0RBCAwHoEcSm9zZS1NYXJjaW8uTWFydGlu
c0BlbnNtcC5mcjANBgkqhkiG9w0BAQUFAAOCAgEAh0j3DWg17BIm//AnhNDPDwTbDZUBtI3b
CPSXu3QZCvxSgkx980F8MxA2AJ0nW8BOH9siGYd/2KZX1N2juZqgz5H/kq8y5kSFAiMrqxWL
Oy8YdaJak+NRATe1JXl0V1VBSVTgi3/QlpcaHgQgHa+GZq5qDnnQvRqYjeajNyLLBXLWBPiL
w8rR/JwlMObaGEgaVggyNTtxBTHHf4bc0ErKdwV6y0P55kvvDqorYY5pQ1VZG2NQchfZHScs
I4RBNSRx1Dnm8c9sFxR2EJ972HsqTkLunz43NKDHOF15KXv4ePSoRbdMcHTCTEEvPuYMv8rS
i1bp0OlRdW1EWxT58MAI1nRGDbaAeAGRRVC6PfMx6QgGeyMQOL8ibK8NpXBIoMvggbxjF497
u1y5SnEfUCJdHDUpW0dzaTbh96tCfywdXxJTKfKivbrju3nnAqiMGMFX1+XJ2F444SD4eL76
Qbyme/vKsqvWsuKBB6+j1c8gyotKuke0HwQEOPc+zZ7YJUgLI/1ghrGdfB9h4qkvKjSyG4an
iXZyvod+iFlszEcF5Y6N9jEwN+7zrRNlvQfY0xHgJNmKOP8Y7s8iflr9UTKPAWgKaxcDDFAJ
07zM98jB7jiOpIS3MhFDj/njtNzeCFpgrojM/X490841ME0EYeoDt7ZhhWMsu1/FGk3Ov4mv
rNMwggUQMIIC+KADAgECAgMB1vcwDQYJKoZIhvcNAQEFBQAweTEQMA4GA1UEChMHUm9vdCBD
QTEeMBwGA1UECxMVaHR0cDovL3d3dy5jYWNlcnQub3JnMSIwIAYDVQQDExlDQSBDZXJ0IFNp
Z25pbmcgQXV0aG9yaXR5MSEwHwYJKoZIhvcNAQkBFhJzdXBwb3J0QGNhY2VydC5vcmcwHhcN
MDYwMTIxMTQ0ODIwWhcNMDcwMTIxMTQ0ODIwWjBTMSQwIgYDVQQDExtKb3NlLU1hcmNpbyBN
YXJ0aW5zIGRhIENydXoxKzApBgkqhkiG9w0BCQEWHEpvc2UtTWFyY2lvLk1hcnRpbnNAZW5z
bXAuZnIwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDMrvCumlCTF9mST+vVBkWg
f4Gdlhc0l1KS5gPtZTRpXONF/A+jfZk8Fl/yKSNT6XMXfUGDgS5tjNgLXOyI1e/DF4++XRmG
zk9A82FKL8KExoSjmINfFSLsUOK4pxhCPHiKl+oDCfxbc6Mdx/6INOAIiGccjApQI7leUBTe
xUUbSLmBmIC2k/h6uUNi14s7YRdHsFwX6h9RpiRchK4Y5MVkEx+A6vKBWbkZ1gOFdmN9ErOn
Z9Zo8ckgNwLDGjDUEwGbLGukdATqgBbot75ufM4ISy9tzKeQ3tm6TPN2NpgPMqjBLBdA1q/B
4uNGvOsQmGR6irU1cY3l6lUb+TsRfCf7AgMBAAGjgcYwgcMwDAYDVR0TAQH/BAIwADBWBglg
hkgBhvhCAQ0ESRZHVG8gZ2V0IHlvdXIgb3duIGNlcnRpZmljYXRlIGZvciBGUkVFIGhlYWQg
b3ZlciB0byBodHRwOi8vd3d3LkNBY2VydC5vcmcwMgYIKwYBBQUHAQEEJjAkMCIGCCsGAQUF
BzABhhZodHRwOi8vb2NzcC5jYWNlcnQub3JnMCcGA1UdEQQgMB6BHEpvc2UtTWFyY2lvLk1h
cnRpbnNAZW5zbXAuZnIwDQYJKoZIhvcNAQEFBQADggIBAIdI9w1oNewSJv/wJ4TQzw8E2w2V
AbSN2wj0l7t0GQr8UoJMffNBfDMQNgCdJ1vATh/bIhmHf9imV9Tdo7maoM+R/5KvMuZEhQIj
K6sVizsvGHWiWpPjUQE3tSV5dFdVQUlU4It/0JaXGh4EIB2vhmauag550L0amI3mozciywVy
1gT4i8PK0fycJTDm2hhIGlYIMjU7cQUxx3+G3NBKyncFestD+eZL7w6qK2GOaUNVWRtjUHIX
2R0nLCOEQTUkcdQ55vHPbBcUdhCfe9h7Kk5C7p8+NzSgxzhdeSl7+Hj0qEW3THB0wkxBLz7m
DL/K0otW6dDpUXVtRFsU+fDACNZ0Rg22gHgBkUVQuj3zMekIBnsjEDi/ImyvDaVwSKDL4IG8
YxePe7tcuUpxH1AiXRw1KVtHc2k24ferQn8sHV8SUynyor2647t55wKojBjBV9flydheOOEg
+Hi++kG8pnv7yrKr1rLigQevo9XPIMqLSrpHtB8EBDj3Ps2e2CVICyP9YIaxnXwfYeKpLyo0
shuGp4l2cr6HfohZbMxHBeWOjfYxMDfu860TZb0H2NMR4CTZijj/GO7PIn5a/VEyjwFoCmsX
AwxQCdO8zPfIwe44jqSEtzIRQ4/547Tc3ghaYK6IzP1+PdPONTBNBGHqA7e2YYVjLLtfxRpN
zr+Jr6zTMYIDhzCCA4MCAQEwgYAweTEQMA4GA1UEChMHUm9vdCBDQTEeMBwGA1UECxMVaHR0
cDovL3d3dy5jYWNlcnQub3JnMSIwIAYDVQQDExlDQSBDZXJ0IFNpZ25pbmcgQXV0aG9yaXR5
MSEwHwYJKoZIhvcNAQkBFhJzdXBwb3J0QGNhY2VydC5vcmcCAwHW9zAJBgUrDgMCGgUAoIIB
2zAYBgkqhkiG9w0BCQMxCwYJKoZIhvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0wNjA0MDUxMTU1
MDhaMCMGCSqGSIb3DQEJBDEWBBSR4qI5BUBz8X0KkoWTNnXKfGTamDBSBgkqhkiG9w0BCQ8x
RTBDMAoGCCqGSIb3DQMHMA4GCCqGSIb3DQMCAgIAgDANBggqhkiG9w0DAgIBQDAHBgUrDgMC
BzANBggqhkiG9w0DAgIBKDCBkQYJKwYBBAGCNxAEMYGDMIGAMHkxEDAOBgNVBAoTB1Jvb3Qg
Q0ExHjAcBgNVBAsTFWh0dHA6Ly93d3cuY2FjZXJ0Lm9yZzEiMCAGA1UEAxMZQ0EgQ2VydCBT
aWduaW5nIEF1dGhvcml0eTEhMB8GCSqGSIb3DQEJARYSc3VwcG9ydEBjYWNlcnQub3JnAgMB
1vcwgZMGCyqGSIb3DQEJEAILMYGDoIGAMHkxEDAOBgNVBAoTB1Jvb3QgQ0ExHjAcBgNVBAsT
FWh0dHA6Ly93d3cuY2FjZXJ0Lm9yZzEiMCAGA1UEAxMZQ0EgQ2VydCBTaWduaW5nIEF1dGhv
cml0eTEhMB8GCSqGSIb3DQEJARYSc3VwcG9ydEBjYWNlcnQub3JnAgMB1vcwDQYJKoZIhvcN
AQEBBQAEggEAAcKUeZK4zhcj37tPi16ywJdy7UY63N49uDiM3DE35XWvXoQX+QbqgUDjyRfp
N0Nx0REYGlJBbHgvLqdfA7lM+ZAIBAOyoCAjxhcRs7iXxw7VYTGPoiUoSb5REq9Ti3aNNC6a
jbR9cwytm9TxPsEFRDpzW3Zf6DQ5pmU2xittz7Z+HTdIfyhL8g98lIKsEd3/8oU/+m7Kjx2G
zvjXh7PhVHBm3AzP/M4BlLQX+er5loX1rYRdUQ9akTgbYXJoQ/ATYfKwigOHP5uIFugIAxD0
hfM59BweQUZ8QwPNQjAZb7JtIJoclq3s91Y8jFJvQZbqm+tGWuI14GHEXIXrkqOQxQAAAAAA
AA==
--------------ms050503040805080809050307--


--===============1931375954==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg

--===============1931375954==--




From asrg-bounces@ietf.org Wed Apr 05 11:03:21 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FR9XO-0002DR-Lg; Wed, 05 Apr 2006 11:03:14 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FR9XN-0002D1-TA
	for asrg@ietf.org; Wed, 05 Apr 2006 11:03:13 -0400
Received: from tom.iecc.com ([208.31.42.38])
	by ietf-mx.ietf.org with smtp (Exim 4.43) id 1FR9XM-00022p-Je
	for asrg@ietf.org; Wed, 05 Apr 2006 11:03:13 -0400
Received: (qmail 6464 invoked from network); 5 Apr 2006 15:03:12 -0000
Received: from mail.iecc.com (208.31.42.99)
	by mail.iecc.com with QMQP; 5 Apr 2006 15:03:12 -0000
Date: 5 Apr 2006 15:03:11 -0000
Message-ID: <20060405150311.1945.qmail@simone.iecc.com>
From: John Levine <asrg@johnlevine.com>
To: asrg@ietf.org
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
In-Reply-To: <20060405020914.GA700@ender>
Organization: 
Mime-Version: 1.0
Content-type: text/plain; charset=iso-8859-1
Content-transfer-encoding: 7bit
X-Spam-Score: -4.3 (----)
X-Scan-Signature: 8abaac9e10c826e8252866cbe6766464
Cc: laird@lbreyer.com
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

>>  if he asked
>> for it and hasn't asked for it to stop, it's legit.
>
>What if he likes to read the list messages on a single topic, but
>doesn't want those messages that deal with another topic? 

Then he should change his subscription, but that doesn't make the mail
spam.

>Or take the ASRG list itself. For argument's sake, ...

Same answer, deal with it any way you want, but mail you asked for
isn't spam.  I agree that if you signed up for one thing and they send
you something completely different, which sometimes happens, that's
bad, but splitting hairs isn't going to get anyone anywhere.

> And my claim is that the act of measuring the operation of the
> spamfilter need not affect spammer behaviour directly.

That's fine if you don't want to test filters that reject, greylist,
delay, or otherwise do anything that senders detect.  For the other
99% of spam filters, you've got a problem.

>If you can obtain FP and FN numbers, then I believe you have a
>valuable summary of accuracy for that system, but it won't tell you
>about other aspects such as cpu load etc.

Nor will it let you compare that system to others since the FP and FN
numbers you get on different inputs are not necessarily comparable.

R's,
John

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Wed Apr 05 20:01:20 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FRHvw-0003MU-Jt; Wed, 05 Apr 2006 20:01:08 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FRHvv-0003ML-JI
	for asrg@ietf.org; Wed, 05 Apr 2006 20:01:07 -0400
Received: from omta01ps.mx.bigpond.com ([144.140.82.153])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FRHvv-0003ec-4X
	for asrg@ietf.org; Wed, 05 Apr 2006 20:01:07 -0400
Received: from localhost.localdomain ([60.226.77.107])
	by omta01ps.mx.bigpond.com with ESMTP id
	<20060406000059.MJZL19070.omta01ps.mx.bigpond.com@localhost.localdomain>;
	Thu, 6 Apr 2006 00:00:59 +0000
Received: from ender (ender.scoobynet [192.168.0.3])
	by localhost.localdomain (Postfix) with ESMTP id 08DE5C0;
	Thu,  6 Apr 2006 10:00:34 +1000 (EST)
Received: by ender (Postfix, from userid 1000)
	id 1B7D74DD61; Thu,  6 Apr 2006 09:59:02 +1000 (EST)
Date: Thu, 6 Apr 2006 09:59:01 +1000
From: Laird Breyer <laird@lbreyer.com>
To: Daniel Feenberg <feenberg@nber.org>
Subject: Re: [Asrg] About that Church draft
Message-ID: <20060405235901.GA3981@ender>
Mail-Followup-To: Daniel Feenberg <feenberg@nber.org>,
	John L <johnl@iecc.com>, Anti Spam Research Group <asrg@ietf.org>
References: <20060404211653.U1211@simone.iecc.com>
	<20060405050533.GA1220@ender>
	<Pine.GSO.4.64.0604050727140.8973@nber5.nber.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <Pine.GSO.4.64.0604050727140.8973@nber5.nber.org>
User-Agent: Mutt/1.5.11+cvs20060126
X-Spam-Score: 0.1 (/)
X-Scan-Signature: 8b431ad66d60be2d47c7bfeb879db82c
Cc: John L <johnl@iecc.com>, Anti Spam Research Group <asrg@ietf.org>
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
Reply-To: laird@lbreyer.com
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

On Apr 05 2006, Daniel Feenberg wrote:
> 
> 
> On Wed, 5 Apr 2006, Laird Breyer wrote:
> 
> > On Apr 04 2006, John L wrote:
> >> We sure have a lot of raw material.  Can I have a volunteer editor or two
> >> to moosh the good stuff into one reasonably coherent document explaining
> >> what we think would need to be changed for it to be good?
> >
> > I've just read through the threads again and done a summary of all the
> > main relevant points everyone discussed on the list (see attachment).
> >
> > If someone with the time wants to rearrange the paragraphs and flesh
> > them out into a final document, please pipe up and do so. I tried to
> > relate every point explicitly to the Church draft. Apologies if I
> > missed someone's contribution.
> 
> Should the draft be so compact? There isn't room to include a paragraph or 
> two from each of the most coherent messages? Some of the posters were
> quite eloquent.

Yes! By all means expand on it. My intention was to bring up all the
main points equally so as to not lose people's comments, while skipping
the digressions and non technical parts.  We shouldn't lose the
momentum now.

Several people have given complete answers independently and this summary
is an aide-mémoire rather than a final document. When you read it, you
see that several paragraphs overlap, so can be rearranged to be more cohesive.

The web accessible archive doesn't carry all the messages, otherwise
I would have quoted URLs. Instead, you can search the Message-IDs in
your mailbox to obtain the relevant threads. 

> 
> The draft seems to accept Church's notion that if a spam message
> comes from 123.123.123.123.dial-up.pool.example.com and that is added
> to a DNSBL, the DNSBL operator has an obligation to detect the 
> reassignment of that IP address to another customer next week, and remove
> the address. Is that the sense of the group? I would argue that a 
> perfectly legitimate DNSBL could block dynamic addresses, even when those
> addresses were temporarily assigned to legitimate MTAs.

Please change this draft as appropriate. The attachment is there to be
edited and reposted (the ASRG policy is to show all working in the
mailing list I think). I am not the best person to edit the final
document, as my experience is in statistical filtering.

-- 
Laird Breyer.

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Wed Apr 05 20:25:58 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FRIJl-0003uE-3c; Wed, 05 Apr 2006 20:25:45 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FRIJj-0003u9-Ox
	for asrg@ietf.org; Wed, 05 Apr 2006 20:25:43 -0400
Received: from omta03sl.mx.bigpond.com ([144.140.92.155])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FRIJh-0005BW-Tx
	for asrg@ietf.org; Wed, 05 Apr 2006 20:25:43 -0400
Received: from localhost.localdomain ([60.226.77.107])
	by omta03sl.mx.bigpond.com with ESMTP id
	<20060406002539.DKYI1358.omta03sl.mx.bigpond.com@localhost.localdomain>
	for <asrg@ietf.org>; Thu, 6 Apr 2006 00:25:39 +0000
Received: from ender (ender.scoobynet [192.168.0.3])
	by localhost.localdomain (Postfix) with ESMTP id 18251C0
	for <asrg@ietf.org>; Thu,  6 Apr 2006 10:25:18 +1000 (EST)
Received: by ender (Postfix, from userid 1000)
	id 52DED4DD61; Thu,  6 Apr 2006 10:23:45 +1000 (EST)
Date: Thu, 6 Apr 2006 10:23:45 +1000
From: Laird Breyer <laird@lbreyer.com>
To: asrg@ietf.org
Subject: Re: [Asrg] About that Church draft
Message-ID: <20060406002345.GB3981@ender>
Mail-Followup-To: asrg@ietf.org
References: <20060404211653.U1211@simone.iecc.com>
	<20060405050533.GA1220@ender> <44336CA8.5090804@americasm01.nt.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <44336CA8.5090804@americasm01.nt.com>
User-Agent: Mutt/1.5.11+cvs20060126
X-Spam-Score: 0.1 (/)
X-Scan-Signature: 9466e0365fc95844abaf7c3f15a05c7d
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
Reply-To: laird@lbreyer.com
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

On Apr 05 2006, Chris Lewis wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Laird Breyer wrote:
> 
> > I've just read through the threads again and done a summary of all the
> > main relevant points everyone discussed on the list (see attachment).
> 
> I think you did a rather excellent job, and summarized it all pretty well.

Thanks. But do edit the document, I don't lay any claims to it. There
are details in the threads and in other people's comprehensive comments
which could be added.

> 
> If a site determines that it has no reason to communicate with, for
> example, a given country, or sites based on specific listing criteria,
> there is no reason why the site couldn't use a "single verdict" choice
> based on an appropriate DNSBL or other filtering method.  On the other
> hand, some DNSBLs are of a nature or the needs are such where scoring
> methods will be more appropriate.

It's a good general principle, but does it apply to big sites such as
Yahoo or Hotmail? I think that's one of the issues implied in the
Church draft we haven't explored yet. 

-- 
Laird Breyer.

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Wed Apr 05 20:47:11 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FRIeR-0006lh-RH; Wed, 05 Apr 2006 20:47:07 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FRIeQ-0006lL-Rg
	for asrg@ietf.org; Wed, 05 Apr 2006 20:47:06 -0400
Received: from omta05sl.mx.bigpond.com ([144.140.93.195])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FRIeQ-00067L-D9
	for asrg@ietf.org; Wed, 05 Apr 2006 20:47:06 -0400
Received: from localhost.localdomain ([60.226.77.107])
	by omta05sl.mx.bigpond.com with ESMTP id
	<20060406004703.MKIC25409.omta05sl.mx.bigpond.com@localhost.localdomain>;
	Thu, 6 Apr 2006 00:47:03 +0000
Received: from ender (ender.scoobynet [192.168.0.3])
	by localhost.localdomain (Postfix) with ESMTP id BF01EC0;
	Thu,  6 Apr 2006 10:46:42 +1000 (EST)
Received: by ender (Postfix, from userid 1000)
	id 0467E4DD61; Thu,  6 Apr 2006 10:45:09 +1000 (EST)
Date: Thu, 6 Apr 2006 10:45:09 +1000
From: Laird Breyer <laird@lbreyer.com>
To: John Levine <asrg@johnlevine.com>
Subject: Re: [Asrg] Comments on draft-church-dnsbl-harmful-01.txt
Message-ID: <20060406004509.GC3981@ender>
Mail-Followup-To: John Levine <asrg@johnlevine.com>, asrg@ietf.org
References: <20060405020914.GA700@ender>
	<20060405150311.1945.qmail@simone.iecc.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20060405150311.1945.qmail@simone.iecc.com>
User-Agent: Mutt/1.5.11+cvs20060126
X-Spam-Score: 0.1 (/)
X-Scan-Signature: 538aad3a3c4f01d8b6a6477ca4248793
Cc: asrg@ietf.org
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
Reply-To: laird@lbreyer.com
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

On Apr 05 2006, John Levine wrote:
> >>  if he asked
> >> for it and hasn't asked for it to stop, it's legit.
> >
> >What if he likes to read the list messages on a single topic, but
> >doesn't want those messages that deal with another topic? 
> 
> Then he should change his subscription, but that doesn't make the mail
> spam.
> 
> >Or take the ASRG list itself. For argument's sake, ...
> 
> Same answer, deal with it any way you want, but mail you asked for
> isn't spam.  I agree that if you signed up for one thing and they send
> you something completely different, which sometimes happens, that's
> bad, but splitting hairs isn't going to get anyone anywhere.

Agreed, I'm not looking to impose a single definition of spam, just 
point out that consent isn't a panacea.

> 
> > And my claim is that the act of measuring the operation of the
> > spamfilter need not affect spammer behaviour directly.
> 
> That's fine if you don't want to test filters that reject, greylist,
> delay, or otherwise do anything that senders detect.  For the other
> 99% of spam filters, you've got a problem.

I still don't agree, but I'm happy to label the topic as "non-trivial".

> 
> >If you can obtain FP and FN numbers, then I believe you have a
> >valuable summary of accuracy for that system, but it won't tell you
> >about other aspects such as cpu load etc.
> 
> Nor will it let you compare that system to others since the FP and FN
> numbers you get on different inputs are not necessarily comparable.

That's a minor issue. The basis of filter evaluation is that you analyze
performance in the past to judge performance in the future. If you
truly believe that your own email stream in the near future is
incomparable to your email in the recent past, then you can just pick
all your filters by flipping a coin.

Email streams of different people can well be incomparable, of course,
as can mail of one individual at different periods in time, etc.  Even
that has a limited variation however. For example, the TREC 2005 tests 
showed a fair amount of consistency over several test corpora. The
variation is not so great as to make FP and FN comparisons on
different corpora meaningless. 

-- 
Laird Breyer.

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Sat Apr 08 02:28:30 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FS6uP-0003PW-SH; Sat, 08 Apr 2006 02:26:57 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FS6uK-0003NM-FQ
	for asrg@ietf.org; Sat, 08 Apr 2006 02:26:52 -0400
Received: from mpls-qmqp-03.inet.qwest.net ([63.231.195.114])
	by ietf-mx.ietf.org with smtp (Exim 4.43) id 1FS6uK-0007U8-0b
	for asrg@ietf.org; Sat, 08 Apr 2006 02:26:52 -0400
Received: (qmail 40586 invoked by uid 0); 8 Apr 2006 06:26:51 -0000
Received: from unknown (63.231.195.13)
	by mpls-qmqp-03.inet.qwest.net with QMQP; 8 Apr 2006 06:26:51 -0000
Received: from 71-212-183-171.hlrn.qwest.net (HELO ?192.168.0.3?)
	(71.212.183.171)
	by mpls-pop-13.inet.qwest.net with SMTP; 8 Apr 2006 06:26:51 -0000
Date: Sat, 8 Apr 2006 00:26:41 -0600
Message-Id: <EFAA9DE8-315B-4F49-AF95-E3A61027CA0C@qwest.net>
From: "Dan Oetting" <dan_oetting@qwest.net>
To: "Anti Spam Research Group" <asrg@ietf.org>
Mime-Version: 1.0 (Apple Message framework v749.3)
In-Reply-To: <442EAB38.6060906@americasm01.nt.com>
References: <4D0E9449-AD81-4D29-BB6B-CA8B301B06C2@qwest.net>
	<442EAB38.6060906@americasm01.nt.com>
Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed
Content-Transfer-Encoding: 7bit
Subject: Re: [Asrg] Building a better blacklist
X-Mailer: Apple Mail (2.749.3)
X-Spam-Score: 0.0 (/)
X-Scan-Signature: 093efd19b5f651b2707595638f6c4003
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org


On Apr 1, 2006, at 9:32 AM, Chris Lewis wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Dan Oetting wrote:
>> Why are the performance figures for blacklists so low? I saw someone
>> post a figure of 80% blocking. While less than 50% of the spam I  
>> get  in
>> my unfiltered accounts would have been blocked by SBL+XBL. Why  can't
>> the blocking rate be in the high 90's?
>
> Because, quite simply, a significant amount of spam comes from "mixed
> sources", such as ISP mail servers, so, unless you're willing to  
> put up
> with very high FPs, DNSBLs are simply not suitable for _that_  
> segment of
> spam.

I found a combination that would block nearly 100% of the spam I  
receive. As I said earlier, the SBL+XBL would block 50% of the spam I  
see. The DCC Reputation system would block the other 50%. An  
interesting observation is that I see no overlap between the two.  
Either every commercial DCC server is protected by the SBL+XBL or my  
ISP is blocking on the combined score. Only one spam source that hit  
my account had a mixed reputation on the DCC list and was not listed  
in the blacklists.

> We're still 100% blocking tin.it's mail servers.

That's the one.


>> The main
>> concern I suppose is that they can't afford to loose mail sent to   
>> their
>> customers. To address this, a blacklist systems could be  designed to
>> recover automatically when the spam stops.
>
> Many DNSBLs already do this, virtually all of the reputable ones do.

The DCC Reputation system is almost exactly what I had been seeking  
for the last four years. It's even got the distributed spamtraps and  
a flood distributed update network, Except that it's cycle time is on  
the order of a few days instead of a few hours. And it's a closed  
system, only available to the commercial licensed sites.

The DCC Reputation doesn't appear to use any weighting based on the  
reliability of the spamtraps. I feel that the field of traps needs to  
be very diverse which means that some of the traps are going to get  
more noise than others and this should be taken into account when  
deciding if a spam threshold has been reached. Also, releasing the  
exact hit counts for a source address is dangerous and could lead to  
simple search attacks to discover the trap addresses.


>> If the mail is  rejected with
>> a 4xx response code the non-spam mail from legitimate  ISPs would be
>> delivered (only slightly delayed) once the spam is  cleaned up.
>
> That's presuming that "spam is cleaned up" in a sufficiently timely
> fashion to fall within retry limits.  It seldom is.  Indeed, most spam
> simply doesn't retry, so, issuing a 4xx response the first time   
> you see
> a particular spam is a highly useful technique.  It's called  
> greylisting.
>
>> That leads to the question of how to clean up the spam in real time.
>> Since the spam traps have already captured samples of the spam
>> emanating from the blocked ISP, it should be easy enough to  
>> construct  a
>> profile or signature of the spam that the source ISP could use to
>> quarantine the remaining spam in the queue.
>
> Some ISPs have done this in the past, however, it's seldom an  
> effective
> or generalized enough solution.  Furthermore, most spam _never_  
> sits in
> queues, because it's not coming from mail servers.

Who is worried about the non-mail server sources of spam? They will  
be listed for as long as they continue to try and deliver spam.

It's the "significant amount of spam comes from ""mixed sources"",  
such as ISP mail servers" that needs to be considered.

What I am proposing is a fast response (greylist) advisory that  
allows recipients to delay the acceptance of mail from the listed  
ISPs while signaling the listed ISP's in real time that they have a  
problem and to let the listed ISPs clean up the problem at the  
source. Then releasing the advisory when spam is no longer detected  
so the mail will again flow freely from the now clean source. In  
cases where the source ISP doesn't clean up in a reasonable time the  
recipient ISPs would fall back on their own filtering.

Unlike the typical blacklist, this advisory list can afford to have a  
hair trigger because the penalty of a false listing is at most a  
short delay in mail delivery. There is still a tradeoff to balance  
this penalty with the advantage of stopping more spam. Whitelists  
based on sender and recipient could circumvent the delay and further  
minimize the negative impacts.


One issue is how to signal the sending ISP in real time to tell them  
exactly what their problem is. [If this was the single ultimate  
solution to the spam problem then every ISP would subscribe to the  
advisory and the problem would be solved. :^}  ]  Realistically, an  
ISP that is accepting (or quarantining) mail during the advisory  
could return a specific response when, for instance, the mail is  
detected as bulk by the DCC. The Sending ISP upon receiving a number  
of these responses for mail from the same local user would know that  
the user is probably misbehaving and could take appropriate  
corrective action.


What is primarily gained by this proposed  advisory list is time to  
make a better determination on wether to block the listed ISP or not.  
And time to build a better signature of the bulk mail. The spam that  
would have come out in this time interval can be kept out of the  
recipients mailboxes.


-- Dan Oetting


_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Thu Apr 13 17:42:10 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FU9ZJ-0000rS-Vt; Thu, 13 Apr 2006 17:41:37 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FU9ZI-0000TK-Ka
	for asrg@ietf.org; Thu, 13 Apr 2006 17:41:36 -0400
Received: from tom.iecc.com ([208.31.42.38])
	by ietf-mx.ietf.org with smtp (Exim 4.43) id 1FU9ZB-0004pN-EJ
	for asrg@ietf.org; Thu, 13 Apr 2006 17:41:30 -0400
Received: (qmail 21473 invoked from network); 13 Apr 2006 21:41:28 -0000
Received: from mail.iecc.com (208.31.42.99)
	by mail.iecc.com with QMQP; 13 Apr 2006 21:41:28 -0000
Date: 13 Apr 2006 21:41:28 -0000
Message-ID: <20060413214128.11245.qmail@simone.iecc.com>
From: John Levine <asrg@johnlevine.com>
To: asrg@ietf.org
In-Reply-To: <562B8158-B748-49D3-834C-86856A35DA4D@mail-abuse.org>
Organization: 
Mime-Version: 1.0
Content-type: text/plain; charset=iso-8859-1
Content-transfer-encoding: 7bit
X-Spam-Score: -4.3 (----)
X-Scan-Signature: 7aefe408d50e9c7c47615841cb314bed
Cc: 
Subject: [Asrg] Re: Are these drafts suitable for ASRG?
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

>I was wondering whether you received the two drafts that I had sent you?

I did.  Given the complete lack of interest everywhere else, I doubt the
answer in the ASRG is going to be any different.

R's,
John

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Fri Apr 14 12:33:26 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FUREK-0003Dl-GZ; Fri, 14 Apr 2006 12:33:08 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FUREI-0003Dg-Mf
	for asrg@ietf.org; Fri, 14 Apr 2006 12:33:06 -0400
Received: from zcars04f.nortel.com ([47.129.242.57])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FUREH-0007uv-An
	for asrg@ietf.org; Fri, 14 Apr 2006 12:33:06 -0400
Received: from zcarhxs1.corp.nortel.com
	(zcarhxs1.corp.nort...s0.corp.nortel.com [47.129.230.89])
	by zcars04f.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id
	k3EGX2E17046
	for <asrg@ietf.org>; Fri, 14 Apr 2006 12:33:02 -0400 (EDT)
Received: from [47.130.24.175] ([47.130.24.175] RDNS failed) by
	zcarhxs1.corp.nortel.com with Microsoft SMTPSVC(6.0.3790.1830); 
	Fri, 14 Apr 2006 12:32:46 -0400
Message-ID: <443FCEA9.5090602@americasm01.nt.com>
Date: Fri, 14 Apr 2006 12:32:41 -0400
From: "Chris Lewis" <clewis@nortel.com>
User-Agent: Thunderbird 1.5 (Windows/20051201)
MIME-Version: 1.0
CC: asrg@ietf.org
Subject: Re: [Asrg] Re: Are these drafts suitable for ASRG?
References: <20060413214128.11245.qmail@simone.iecc.com>
In-Reply-To: <20060413214128.11245.qmail@simone.iecc.com>
X-Enigmail-Version: 0.94.0.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-OriginalArrivalTime: 14 Apr 2006 16:32:47.0396 (UTC)
	FILETIME=[1064CE40:01C65FE1]
X-Spam-Score: 0.0 (/)
X-Scan-Signature: 79899194edc4f33a41f49410777972f8
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

John Levine wrote:
>> I was wondering whether you received the two drafts that I had sent you?
> 
> I did.  Given the complete lack of interest everywhere else, I doubt the
> answer in the ASRG is going to be any different.

Have I missed something?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3-nr1 (Windows 2000)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBRD/OqZ3FmCyJjHfhAQK2mQP/WNRMH6IMEmJ5WNzBUyUjFbcs+dzTNstq
ZVj9XGlSzB5/FQo5YEw4Nw4xqxSuiPkKI+4POwhju1SsmIKHKa/VwOjUy00L9gvh
PwhbecC3uutHI2G7K6+Njj4R6KvMBtYOcZEMoGStjqAC5QT+5Y7HBByoThxEf4no
85ZoGNpUMgo=
=E9pq
-----END PGP SIGNATURE-----

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Fri Apr 14 15:26:05 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FUTvc-0003S0-UP; Fri, 14 Apr 2006 15:26:00 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FUTvb-0003Rs-LL
	for asrg@ietf.org; Fri, 14 Apr 2006 15:25:59 -0400
Received: from exchange.habeas.com ([64.78.21.106])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FUTva-0005EO-BD
	for asrg@ietf.org; Fri, 14 Apr 2006 15:25:59 -0400
Accreditor: Habeas
X-Habeas-SWE-1: winter into spring
X-Habeas-SWE-8: Message (HCM) and not spam.  Please report use of this
X-Habeas-SWE-7: warrant mark warrants that this is a Habeas Compliant
X-Habeas-SWE-9: mark in spam to http://www.habeas.com/report/.
X-Habeas-SWE-5: Sender Warranted Email (SWE) (tm). The sender of this
X-Habeas-SWE-6: email in exchange for a license for this Habeas
X-Habeas-SWE-4: Copyright 2002 Habeas (tm)
X-Habeas-SWE-2: brightly anticipated
X-Habeas-SWE-3: like Habeas SWE (tm)
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.3790.1830
Content-class: urn:content-classes:message
Importance: normal
Priority: normal
MIME-Version: 1.0
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Subject: RE: [Asrg] Re: Are these drafts suitable for ASRG?
Date: Fri, 14 Apr 2006 12:25:54 -0700
Message-ID: <88885AD16AAA9F4899BB9FB751D4F5A5C7AD42@phost016.habeas.intermedia.net>
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
Thread-Topic: [Asrg] Re: Are these drafts suitable for ASRG?
thread-index: AcZf421IlUTkEJtSQfmGGy2kHTbRywAFXzzg
From: "Nick Nicholas" <Nick@habeas.com>
To: <asrg@ietf.org>
X-Spam-Score: 0.0 (/)
X-Scan-Signature: ffa9dfbbe7cc58b3fa6b8ae3e57b0aa3
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

On Friday, April 14, 2006 at 9:33 AM Chris Lewis wrote:

> John Levine wrote:
> >> I was wondering whether you received the two drafts that I=20
> had sent you?
> >=20
> > I did.  Given the complete lack of interest everywhere=20
> else, I doubt=20
> > the answer in the ASRG is going to be any different.
>=20
> Have I missed something?

I'm a bit in the dark, too.  Can you shed a little light on this, John?

Regards,

Nick


--
=20
Nick Nicholas
Knowledge Engineer
Habeas Inc.
650-694-3320
nick@habeas.com

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Fri Apr 14 16:15:17 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FUUh6-0002fo-P3; Fri, 14 Apr 2006 16:15:04 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FUUh5-0002fj-Hm
	for asrg@ietf.org; Fri, 14 Apr 2006 16:15:03 -0400
Received: from xuxa.iecc.com ([208.31.42.42])
	by ietf-mx.ietf.org with smtp (Exim 4.43) id 1FUUh3-0007HB-8f
	for asrg@ietf.org; Fri, 14 Apr 2006 16:15:03 -0400
Received: (qmail 13818 invoked from network); 14 Apr 2006 20:15:00 -0000
Received: from simone.iecc.com (208.31.42.47)
	by mail2.iecc.com with QMQP; 14 Apr 2006 20:15:00 -0000
Date: 14 Apr 2006 20:15:00 -0000
Message-ID: <20060414201500.60511.qmail@simone.iecc.com>
From: John Levine <asrg@johnlevine.com>
To: asrg@ietf.org
Subject: Re: [Asrg] Re: Are these drafts suitable for ASRG?
In-Reply-To: <88885AD16AAA9F4899BB9FB751D4F5A5C7AD42@phost016.habeas.intermedia.net>
Organization: 
Mime-Version: 1.0
Content-type: text/plain; charset=iso-8859-1
Content-transfer-encoding: 7bit
X-Spam-Score: -4.3 (----)
X-Scan-Signature: cf4fa59384e76e63313391b70cd0dd25
Cc: Nick@habeas.com
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

>> >> I was wondering whether you received the two drafts that I 
>> had sent you?
>> > 
>> > I did.  Given the complete lack of interest everywhere 
>> else, I doubt the answer in the ASRG is going to be any different.
>> 
>> Have I missed something?

It was Doug Otis' draft on the risks of SPF due to all of the DNS
queries that an SPF lookup can provoke.  I don't see any thing researchy
about it, and the issue is well known to everyone who cares about SPF,
so like I said to Doug, I don't see what we would do with it.

R's,
John


_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Fri Apr 14 18:01:04 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FUWLS-0001YN-SN; Fri, 14 Apr 2006 18:00:50 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FUWLR-0001YD-Op
	for asrg@ietf.org; Fri, 14 Apr 2006 18:00:49 -0400
Received: from b.mail.sonic.net ([64.142.19.5])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FUWLQ-0002BN-Dh
	for asrg@ietf.org; Fri, 14 Apr 2006 18:00:49 -0400
Received: from [168.61.10.151] (SJC-Office-DHCP-151.Mail-Abuse.ORG
	[168.61.10.151]) (authenticated bits=0)
	by b.mail.sonic.net (8.13.6/8.13.3) with ESMTP id k3EM0l1M026441
	(version=TLSv1/SSLv3 cipher=RC4-SHA bits=128 verify=NO);
	Fri, 14 Apr 2006 15:00:47 -0700
In-Reply-To: <20060414201500.60511.qmail@simone.iecc.com>
References: <20060414201500.60511.qmail@simone.iecc.com>
Mime-Version: 1.0 (Apple Message framework v749.3)
Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed
Message-Id: <E8EE6062-EF0B-4C85-992A-F24511E9F38A@mail-abuse.org>
Content-Transfer-Encoding: 7bit
From: Douglas Otis <dotis@mail-abuse.org>
Subject: Re: [Asrg] Re: Are these drafts suitable for ASRG?
Date: Fri, 14 Apr 2006 15:00:51 -0700
To: John Levine <asrg@johnlevine.com>
X-Mailer: Apple Mail (2.749.3)
X-Spam-Score: 0.0 (/)
X-Scan-Signature: b19722fc8d3865b147c75ae2495625f2
Cc: Anti-Spam Research Group Group <asrg@ietf.org>
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org


On Apr 14, 2006, at 1:15 PM, John Levine wrote:

>>>>> I was wondering whether you received the two drafts that I
>>> had sent you?
>>>>
>>>> I did.  Given the complete lack of interest everywhere
>>> else, I doubt the answer in the ASRG is going to be any different.
>>>
>>> Have I missed something?
>
> It was Doug Otis' draft on the risks of SPF due to all of the DNS
> queries that an SPF lookup can provoke.  I don't see any thing  
> researchy
> about it, and the issue is well known to everyone who cares about SPF,
> so like I said to Doug, I don't see what we would do with it.

I have posted these drafts to the ietf.  There will be an attempt to  
demonstrate the threat and to make better measurements of these  
estimates.

A little researhy...  I agree with John, this should not be needed,  
but some want to see this.

http://www.sonic.net/~dougotis/id/draft-otis-spf-dos-exploit-00.html


An alternative approach to that of SPF has been postulated for  
comparison.

http://www.sonic.net/~dougotis/id/draft-otis-smtp-name-path-00.html

-Doug

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Fri Apr 21 08:37:36 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FWusS-00047P-OK; Fri, 21 Apr 2006 08:36:48 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FWusQ-00047K-QP
	for asrg@ietf.org; Fri, 21 Apr 2006 08:36:46 -0400
Received: from tom.iecc.com ([208.31.42.38])
	by ietf-mx.ietf.org with smtp (Exim 4.43) id 1FWusP-00041b-Jh
	for asrg@ietf.org; Fri, 21 Apr 2006 08:36:46 -0400
Received: (qmail 3811 invoked from network); 21 Apr 2006 12:36:45 -0000
Received: (ofmipd 127.0.0.1); 21 Apr 2006 12:36:23 -0000
Date: 21 Apr 2006 08:36:45 -0400
Message-ID: <Pine.BSI.4.56.0604210835560.3697@tom.iecc.com>
From: "John R Levine" <johnl@iecc.com>
To: "Anti Spam Research Group" <asrg@ietf.org>
Cleverness: None detected
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
X-Spam-Score: -4.3 (----)
X-Scan-Signature: 08e48e05374109708c00c6208b534009
Subject: [Asrg] Work items from IETF 65
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

I just uploaded my notes from the IETF 65 meeting, and will post some
articles to (with any luck) start threads tracking the work items started
there.

Regards,
John Levine, johnl@iecc.com, Primary Perpetrator of "The Internet for Dummies",
Information Superhighwayman wanna-be, http://www.johnlevine.com, Mayor
"I shook hands with Senators Dole and Inouye," said Tom, disarmingly.

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Fri Apr 21 08:51:27 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FWv6R-0000df-Vh; Fri, 21 Apr 2006 08:51:15 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FWv6Q-0000dZ-NE
	for asrg@ietf.org; Fri, 21 Apr 2006 08:51:14 -0400
Received: from tom.iecc.com ([208.31.42.38])
	by ietf-mx.ietf.org with smtp (Exim 4.43) id 1FWv6P-0004zH-E8
	for asrg@ietf.org; Fri, 21 Apr 2006 08:51:14 -0400
Received: (qmail 4926 invoked from network); 21 Apr 2006 12:44:32 -0000
Received: (ofmipd 127.0.0.1); 21 Apr 2006 12:44:10 -0000
Date: 21 Apr 2006 08:44:32 -0400
Message-ID: <Pine.BSI.4.56.0604210836530.3697@tom.iecc.com>
From: "John R Levine" <johnl@iecc.com>
To: "Anti Spam Research Group" <asrg@ietf.org>
Cleverness: None detected
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
X-Spam-Score: -4.3 (----)
X-Scan-Signature: 1ac7cc0a4cd376402b85bc1961a86ac2
Subject: [Asrg] DNSBL BCP(-ish) document
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

We have an early draft of a best practices document for DNSBLs.  Things to
do:

1) Send around some versions so we can try to get to a consensus version.

2) Figure out whether we want to make this a formal IETF BCP, which
requires coordination with the IESG.  See RFC 2026, pages 15 and 16.

Nick Nicholas and Pat Peterson have expressed interest in this activity.

Regards,
John Levine, johnl@iecc.com, Primary Perpetrator of "The Internet for Dummies",
Information Superhighwayman wanna-be, http://www.johnlevine.com, Mayor
"I shook hands with Senators Dole and Inouye," said Tom, disarmingly.

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Fri Apr 21 08:57:58 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FWvCt-0001PO-Qz; Fri, 21 Apr 2006 08:57:55 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FWvCs-0001PJ-Pn
	for asrg@ietf.org; Fri, 21 Apr 2006 08:57:54 -0400
Received: from tom.iecc.com ([208.31.42.38])
	by ietf-mx.ietf.org with smtp (Exim 4.43) id 1FWvCr-0005Ey-Gp
	for asrg@ietf.org; Fri, 21 Apr 2006 08:57:54 -0400
Received: (qmail 6874 invoked from network); 21 Apr 2006 12:57:53 -0000
Received: (ofmipd 127.0.0.1); 21 Apr 2006 12:57:31 -0000
Date: 21 Apr 2006 08:57:53 -0400
Message-ID: <Pine.BSI.4.56.0604210854430.3697@tom.iecc.com>
From: "John R Levine" <johnl@iecc.com>
To: "Anti Spam Research Group" <asrg@ietf.org>
Cleverness: None detected
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
X-Spam-Score: -4.3 (----)
X-Scan-Signature: 08170828343bcf1325e4a0fb4584481c
Subject: [Asrg] Third party DKIM signatures
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

DKIM lets anyone sign any message, with no necessary connection between
the signing domain and the domain in any other header such as From: or
Sender:.  By third-party signatures we mean signatures that don't match
the From: or don't match the Sender:, or don't match something else.

The semantics as well as definition of third party signatures are, to put
it mildly, somewhat unclear.  Some thought or actual experiments with such
signatures could be helpful.

Pat Peterson and Doug Otis expressed interest in this item.

Regards,
John Levine, johnl@iecc.com, Primary Perpetrator of "The Internet for Dummies",
Information Superhighwayman wanna-be, http://www.johnlevine.com, Mayor
"I shook hands with Senators Dole and Inouye," said Tom, disarmingly.

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Fri Apr 21 09:12:09 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FWvQY-0002i0-1c; Fri, 21 Apr 2006 09:12:02 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FWvQW-0002hl-CI
	for asrg@ietf.org; Fri, 21 Apr 2006 09:12:00 -0400
Received: from tom.iecc.com ([208.31.42.38])
	by ietf-mx.ietf.org with smtp (Exim 4.43) id 1FWvQV-0006JB-5s
	for asrg@ietf.org; Fri, 21 Apr 2006 09:12:00 -0400
Received: (qmail 8474 invoked from network); 21 Apr 2006 13:11:58 -0000
Received: (ofmipd 127.0.0.1); 21 Apr 2006 13:11:36 -0000
Date: 21 Apr 2006 09:11:58 -0400
Message-ID: <Pine.BSI.4.56.0604210857540.3697@tom.iecc.com>
From: "John R Levine" <johnl@iecc.com>
To: "Anti Spam Research Group" <asrg@ietf.org>
Cleverness: None detected
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
X-Spam-Score: -4.3 (----)
X-Scan-Signature: d17f825e43c9aed4fd65b7edddddec89
Subject: [Asrg] Characterizing message categories
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

It may be useful to categorize messages, which would need taxonomies of
categories.  The appropriate set of categories would probably depend on
what one planned to do with them, e.g., spam filtering vs. end user
filing.

Barry Leiba and Mark Delany expressed interest in this item.

Regards,
John Levine, johnl@iecc.com, Primary Perpetrator of "The Internet for Dummies",
Information Superhighwayman wanna-be, http://www.johnlevine.com, Mayor
"I shook hands with Senators Dole and Inouye," said Tom, disarmingly.

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Fri Apr 21 09:12:52 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FWvRM-0003Hl-8z; Fri, 21 Apr 2006 09:12:52 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FWvRL-0003HZ-2q
	for asrg@ietf.org; Fri, 21 Apr 2006 09:12:51 -0400
Received: from tom.iecc.com ([208.31.42.38])
	by ietf-mx.ietf.org with smtp (Exim 4.43) id 1FWvRK-0006O9-T3
	for asrg@ietf.org; Fri, 21 Apr 2006 09:12:51 -0400
Received: (qmail 8583 invoked from network); 21 Apr 2006 13:12:50 -0000
Received: (ofmipd 127.0.0.1); 21 Apr 2006 13:12:28 -0000
Date: 21 Apr 2006 09:12:50 -0400
Message-ID: <Pine.BSI.4.56.0604210912030.3697@tom.iecc.com>
From: "John R Levine" <johnl@iecc.com>
To: "Anti Spam Research Group" <asrg@ietf.org>
Cleverness: None detected
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
X-Spam-Score: -4.3 (----)
X-Scan-Signature: 08e48e05374109708c00c6208b534009
Subject: [Asrg] DNSSEC and MTAs ?
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

My final note from IETF 65 was about DNSSEC but I don't have any more
details in the notes.  If anyone remembers what we were planning, please
let us know.

Regards,
John Levine, johnl@iecc.com, Primary Perpetrator of "The Internet for Dummies",
Information Superhighwayman wanna-be, http://www.johnlevine.com, Mayor
"I shook hands with Senators Dole and Inouye," said Tom, disarmingly.

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Fri Apr 21 11:39:16 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FWxik-0004qt-2S; Fri, 21 Apr 2006 11:38:58 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FWxij-0004qo-Ae
	for asrg@ietf.org; Fri, 21 Apr 2006 11:38:57 -0400
Received: from a.mail.sonic.net ([64.142.16.245])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FWxih-00078C-V8
	for asrg@ietf.org; Fri, 21 Apr 2006 11:38:57 -0400
Received: from [168.61.10.151] (SJC-Office-DHCP-151.Mail-Abuse.ORG
	[168.61.10.151]) (authenticated bits=0)
	by a.mail.sonic.net (8.13.6/8.13.3) with ESMTP id k3LFcoNx016695
	(version=TLSv1/SSLv3 cipher=RC4-SHA bits=128 verify=NO);
	Fri, 21 Apr 2006 08:38:51 -0700
In-Reply-To: <Pine.BSI.4.56.0604210854430.3697@tom.iecc.com>
References: <Pine.BSI.4.56.0604210854430.3697@tom.iecc.com>
Mime-Version: 1.0 (Apple Message framework v749.3)
Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed
Message-Id: <64E23B7A-646E-4E90-AC31-1292D68C0671@mail-abuse.org>
Content-Transfer-Encoding: 7bit
From: Douglas Otis <dotis@mail-abuse.org>
Subject: Re: [Asrg] Third party DKIM signatures
Date: Fri, 21 Apr 2006 08:39:01 -0700
To: John R Levine <johnl@iecc.com>
X-Mailer: Apple Mail (2.749.3)
X-Spam-Score: 0.0 (/)
X-Scan-Signature: 2409bba43e9c8d580670fda8b695204a
Cc: Anti Spam Research Group <asrg@ietf.org>
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org


On Apr 21, 2006, at 5:57 AM, John R Levine wrote:

> DKIM lets anyone sign any message, with no necessary connection  
> between
> the signing domain and the domain in any other header such as From: or
> Sender:.  By third-party signatures we mean signatures that don't  
> match
> the From: or don't match the Sender:, or don't match something else.
>
> The semantics as well as definition of third party signatures are,  
> to put
> it mildly, somewhat unclear.  Some thought or actual experiments  
> with such
> signatures could be helpful.
>
> Pat Peterson and Doug Otis expressed interest in this item.

Here is a description of an approach that could be used to associate
third-party signatures with signing-domains.  This draft describes the
technique.

http://www.ietf.org/internet-drafts/draft-otis-smtp-name-path-00.txt

This technique is intended to provide a comparison with other testing.

-Doug

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Fri Apr 21 12:40:28 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FWyg6-0001Ys-Ty; Fri, 21 Apr 2006 12:40:18 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FWyg5-0001Qs-Ij
	for asrg@ietf.org; Fri, 21 Apr 2006 12:40:17 -0400
Received: from xproxy.gmail.com ([66.249.82.203])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FWyg4-0001T1-CW
	for asrg@ietf.org; Fri, 21 Apr 2006 12:40:17 -0400
Received: by xproxy.gmail.com with SMTP id s12so354806wxc
	for <asrg@ietf.org>; Fri, 21 Apr 2006 09:40:16 -0700 (PDT)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com;
	h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references;
	b=qDJcNtVp6lj9HDW87CxQ4ShdqSYb2k8RTI2JPeE/AIknWgCCnfSOakZgCnCz7JBLGm+gklBW/4eF2okBCAhkIViJtW9t1BMyBvWgdUYepzH2+kPtTiR+MqUqvKmdr9fexZ+ckAbu0fetkztZO0SDgERTfgZJ34MV+HmlxLsYMO0=
Received: by 10.70.94.4 with SMTP id r4mr396909wxb;
	Fri, 21 Apr 2006 09:40:13 -0700 (PDT)
Received: by 10.70.95.3 with HTTP; Fri, 21 Apr 2006 09:40:13 -0700 (PDT)
Message-ID: <934f64a20604210940o4a04936aw5de1c7d6db845d57@mail.gmail.com>
Date: Fri, 21 Apr 2006 11:40:13 -0500
From: "David Nicol" <davidnicol@gmail.com>
To: "John R Levine" <johnl@iecc.com>
Subject: Re: [Asrg] Characterizing message categories
In-Reply-To: <Pine.BSI.4.56.0604210857540.3697@tom.iecc.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline
References: <Pine.BSI.4.56.0604210857540.3697@tom.iecc.com>
X-Spam-Score: 0.0 (/)
X-Scan-Signature: 7baded97d9887f7a0c7e8a33c2e3ea1b
Cc: Anti Spam Research Group <asrg@ietf.org>
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

The consensus of the tagging committee was that
categories should be left unspecified as part of the
technical specification of a tagging infrasturcture,
with a tag registry maintained externally, and a
standard for experimental tags within the standard tagging
system.  So when you are experimenting with a new
classification Foo, your tag would look like

Is-X-Foo: ...

instead of

X-Is-Foo: ...

and after the Foo classification is properly registered, software
would upgrade to

Is-Foo: ...

The points of contention in the tagging committee were over
agreement on one technique for discouraging forgery.

I refer the reader to my earlier draft of a tagging protocol
including a practical forgery discouragement method
sent to this mailing list.  Given a sudden shower of tuits I
may reformat it in RFC form but no such event is expected
in the next couple months.


On 21 Apr 2006 09:11:58 -0400, John R Levine <johnl@iecc.com> wrote:
> It may be useful to categorize messages, which would need taxonomies of
> categories.  The appropriate set of categories would probably depend on
> what one planned to do with them, e.g., spam filtering vs. end user
> filing.
>
> Barry Leiba and Mark Delany expressed interest in this item.

--
David L Nicol
Document what you do, then do what you documented

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Fri Apr 21 14:02:58 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FWzxr-0003Re-6F; Fri, 21 Apr 2006 14:02:43 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FWzxq-0003RZ-GH
	for asrg@ietf.org; Fri, 21 Apr 2006 14:02:42 -0400
Received: from a.mail.sonic.net ([64.142.16.245])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FWzxo-00076C-3J
	for asrg@ietf.org; Fri, 21 Apr 2006 14:02:42 -0400
Received: from [168.61.10.151] (SJC-Office-DHCP-151.Mail-Abuse.ORG
	[168.61.10.151]) (authenticated bits=0)
	by a.mail.sonic.net (8.13.6/8.13.3) with ESMTP id k3LI2cnS008055
	(version=TLSv1/SSLv3 cipher=RC4-SHA bits=128 verify=NO);
	Fri, 21 Apr 2006 11:02:38 -0700
In-Reply-To: <934f64a20604210940o4a04936aw5de1c7d6db845d57@mail.gmail.com>
References: <Pine.BSI.4.56.0604210857540.3697@tom.iecc.com>
	<934f64a20604210940o4a04936aw5de1c7d6db845d57@mail.gmail.com>
Mime-Version: 1.0 (Apple Message framework v749.3)
Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed
Message-Id: <F5E6FA5B-00B6-4E7D-967E-B3A9A7382E01@mail-abuse.org>
Content-Transfer-Encoding: 7bit
From: Douglas Otis <dotis@mail-abuse.org>
Subject: Re: [Asrg] Characterizing message categories
Date: Fri, 21 Apr 2006 11:02:49 -0700
To: David Nicol <davidnicol@gmail.com>
X-Mailer: Apple Mail (2.749.3)
X-Spam-Score: 0.0 (/)
X-Scan-Signature: bdc523f9a54890b8a30dd6fd53d5d024
Cc: Anti Spam Research Group <asrg@ietf.org>
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org


On Apr 21, 2006, at 9:40 AM, David Nicol wrote:

> The consensus of the tagging committee was that categories should  
> be left unspecified as part of the technical specification of a  
> tagging infrasturcture, with a tag registry maintained externally,  
> and a standard for experimental tags within the standard tagging  
> system.  So when you are experimenting with a new classification  
> Foo, your tag would look like
>
> Is-X-Foo: ...
>
> instead of
>
> X-Is-Foo: ...
>
> and after the Foo classification is properly registered, software  
> would upgrade to
>
> Is-Foo: ...
>
> The points of contention in the tagging committee were over  
> agreement on one technique for discouraging forgery.
>
> I refer the reader to my earlier draft of a tagging protocol  
> including a practical forgery discouragement method sent to this  
> mailing list.  Given a sudden shower of tuits I may reformat it in  
> RFC form but no such event is expected in the next couple months.

I think interests related to DKIM may pertain more to a sender  
tagging scheme, with an additional desire related to trust  
annotations provided recipients.  This can overlap with that of a  
receiver tagging scheme however.

Within a given domain, not all messages may be as trustworthy,  
although DKIM verifies the signing domain.  Rather than establishing  
a plethora of sub-domains to establish separate levels of trust (and  
that ironically may play into the hands of phishers which DKIM  
intends to thwart), a tag contained within the key and encompassed by  
the signature, or perhaps a convention for naming the key selector  
could convey these different levels of trust.

Use of the key selector offers a greater level of security with  
little disruption, as this information is already signed.  The s=  
selector is part of the DKIM signature and is protected by the  
signature.

A convention might be possible where either the right-most or left- 
most label of the s= selector could begin with an '_t-' where the  
information following this could be used to distinguish separate  
categories of email with different levels of trust.  Perhaps  
"s=jan-06._t-trn" would indicate transactional mail from that domain.

The size of the tags should be kept small, as this consumes DNS  
response payloads.  Perhaps the convention could limit these tags to  
3 characters for a 9 byte increase in DNS payload.

For example:

   _t-trn   Transactional
   _t-adm   Administrative
   _t-sol   Solicitation
   _t-mat   Mature content
   _t-usr   Domain User
   _t-gst   Domain Guest
   _t-mdn   Mailer-Daemon
   _t-sys   System
   _t-unk   Unknown status

A message annotation may decide to include a subset of these  
categories, where trust reputations of each could be treated within  
subsets separately.

-Doug







_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Fri Apr 21 18:23:09 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FX412-0003mg-B3; Fri, 21 Apr 2006 18:22:16 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FX410-0003kw-VB
	for asrg@ietf.org; Fri, 21 Apr 2006 18:22:14 -0400
Received: from a.mail.sonic.net ([64.142.16.245])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FX410-0003pg-JX
	for asrg@ietf.org; Fri, 21 Apr 2006 18:22:14 -0400
Received: from [168.61.10.151] (SJC-Office-DHCP-151.Mail-Abuse.ORG
	[168.61.10.151]) (authenticated bits=0)
	by a.mail.sonic.net (8.13.6/8.13.3) with ESMTP id k3LMMBr8029169
	(version=TLSv1/SSLv3 cipher=RC4-SHA bits=128 verify=NO)
	for <asrg@ietf.org>; Fri, 21 Apr 2006 15:22:11 -0700
Mime-Version: 1.0 (Apple Message framework v749.3)
In-Reply-To: <F5E6FA5B-00B6-4E7D-967E-B3A9A7382E01@mail-abuse.org>
References: <Pine.BSI.4.56.0604210857540.3697@tom.iecc.com>
	<934f64a20604210940o4a04936aw5de1c7d6db845d57@mail.gmail.com>
	<F5E6FA5B-00B6-4E7D-967E-B3A9A7382E01@mail-abuse.org>
Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed
Message-Id: <49DA47FE-E90D-422B-A2EF-D87EF8DF11EA@mail-abuse.org>
Content-Transfer-Encoding: 7bit
From: Douglas Otis <dotis@mail-abuse.org>
Date: Fri, 21 Apr 2006 15:22:22 -0700
To: Anti Spam Research Group <asrg@ietf.org>
X-Mailer: Apple Mail (2.749.3)
X-Spam-Score: 0.0 (/)
X-Scan-Signature: 8b30eb7682a596edff707698f4a80f7d
Subject: [Asrg] Placement of message categories
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org


DKIM can list headers included within the signature, and  
architecturally this is a more extensible approach.  Adding a new  
header requires modifying the signing process to assure tagged source  
associations relate to this header, and to encompass that header  
within the signature.  A new header is one possibility.

Gleaning additional information from a key selection process should  
create a lower impact on the signing/verifying process.  For example,  
Key/Source association mechanisms may be in place to support the g=  
attributes that define the permitted local-parts, which already  
exists within the keys used by DKIM.  Being able to differentiate  
which tag belongs with a signature is also solved when incorporated  
at the key selector, rather than when using a new header.   
Establishing naming conventions for one of the labels within the key  
selector is a possibility beyond using a new header.

The down-side for using the key selector is a requirement to be  
frugal for key specific trust.  The up-side, from the aspect of  
building a database, is the requirement to be frugal where the key  
and tag can be evaluated together.  Perhaps only transactional and  
administrative tags would merit scrutiny, as such assessments are  
based upon content demanding additional resources.  The goal is to  
ignore a fair amount of the message traffic meriting little  
recognition.  Greater trust is possible when it is selective.

Messages annotation shown the recipient may increase their level of  
trust.  This trust may be desired when instructing recipients to  
click to some location, for example.  While an institution may want  
to assure their customers who know them by their domain name, this  
same domain name may also be used by poorly vetted users as well.   
Perhaps only those messages tagged as being administrative might  
receive elevated trust annotations, which would proactively prevent  
any mischief from those less vetted.  Only by way of these tags would  
a third-party application be able to discern which messages warrant  
elevated trust annotations. (Based upon assurance made by the signing- 
domain, and the reputation of that domain for this subset.)

-Doug 

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Mon Apr 24 15:01:19 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FY6HY-00078i-GJ; Mon, 24 Apr 2006 14:59:36 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FY6HW-00078d-U8
	for asrg@ietf.org; Mon, 24 Apr 2006 14:59:34 -0400
Received: from test-iport-3.cisco.com ([171.71.176.78])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FY6HV-0003d5-Lo
	for asrg@ietf.org; Mon, 24 Apr 2006 14:59:34 -0400
Received: from sj-core-2.cisco.com ([171.71.177.254])
	by test-iport-3.cisco.com with ESMTP; 24 Apr 2006 11:59:33 -0700
Received: from imail.cisco.com (sjc12-sbr-sw3-3f5.cisco.com [172.19.96.182])
	by sj-core-2.cisco.com (8.12.10/8.12.6) with ESMTP id k3OIxWh0016349;
	Mon, 24 Apr 2006 11:59:32 -0700 (PDT)
Received: from [128.107.163.57] (dhcp-128-107-163-57.cisco.com
	[128.107.163.57])
	by imail.cisco.com (8.12.11/8.12.10) with ESMTP id k3OIwSoc021386;
	Mon, 24 Apr 2006 11:58:28 -0700
Message-ID: <444D2014.7010104@cisco.com>
Date: Mon, 24 Apr 2006 11:59:32 -0700
From: Jim Fenton <fenton@cisco.com>
User-Agent: Thunderbird 1.5.0.2 (Windows/20060308)
MIME-Version: 1.0
To: John R Levine <johnl@iecc.com>
Subject: Re: [Asrg] Third party DKIM signatures
References: <Pine.BSI.4.56.0604210854430.3697@tom.iecc.com>
In-Reply-To: <Pine.BSI.4.56.0604210854430.3697@tom.iecc.com>
X-Enigmail-Version: 0.93.2.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
DKIM-Signature: a=rsa-sha1; q=dns; l=1001; t=1145905108; x=1146769108;
	c=relaxed/simple; s=oregon;
	h=To:Content-Type:From:Subject:Content-Transfer-Encoding:MIME-Version;
	d=cisco.com; i=fenton@cisco.com;
	z=From:Jim=20Fenton=20<fenton@cisco.com>
	|Subject:Re=3A=20[Asrg]=20Third=20party=20DKIM=20signatures
	|To:John=20R=20Levine=20<johnl@iecc.com>;
	X=v=3Dcisco.com=3B=20h=3DHLvh7v1Y/dDe6PQsbdqlB+i/R4Y=3D;
	b=XaLCvXUZ9ZTmhQHUTnftVzwzpNFMeR/tGyZiafvCe6SkqmV2AwnwrekXS26dCFuRCQXzQgO/
	tc4mT1JtSRh1qk1OZ1DNfTTXo6UT8y32nWiDE40nn2fZqMNIhHJMkpm+;
Authentication-Results: imail.cisco.com; header.From=fenton@cisco.com;
	dkim=pass ( sig from cisco.com verified; ); 
X-Spam-Score: 0.0 (/)
X-Scan-Signature: 79899194edc4f33a41f49410777972f8
Cc: Anti Spam Research Group <asrg@ietf.org>
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

John R Levine wrote:
> DKIM lets anyone sign any message, with no necessary connection between
> the signing domain and the domain in any other header such as From: or
> Sender:.  By third-party signatures we mean signatures that don't match
> the From: or don't match the Sender:, or don't match something else.
>   
The definition in the current SSP draft is that a third-party signature
is a signature that doesn't match the From:.  There is no reference to
Sender: or "something else".
> The semantics as well as definition of third party signatures are, to put
> it mildly, somewhat unclear.  Some thought or actual experiments with such
> signatures could be helpful.
>   
I have done a lot of thinking about this; other thought or experiments
welcome.  Given that SSP is the next document up for discussion in the
DKIM WG, I think this has progressed beyond "research", and I don't
think it's a good idea to fragment discussion between this mailing list
and that one.

-Jim

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Mon Apr 24 16:05:23 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FY7J4-0002mC-Sw; Mon, 24 Apr 2006 16:05:14 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FY7J3-0002m7-Go
	for asrg@ietf.org; Mon, 24 Apr 2006 16:05:13 -0400
Received: from zrtps0kp.nortel.com ([47.140.192.56])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FY7J1-0007Vu-Ue
	for asrg@ietf.org; Mon, 24 Apr 2006 16:05:13 -0400
Received: from zcarhxs1.corp.nortel.com
	(zcarhxs1.corp.nort...s0.corp.nortel.com [47.129.230.89])
	by zrtps0kp.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id
	k3OK59j19064
	for <asrg@ietf.org>; Mon, 24 Apr 2006 16:05:09 -0400 (EDT)
Received: from [47.130.17.101] ([47.130.17.101] RDNS failed) by
	zcarhxs1.corp.nortel.com with Microsoft SMTPSVC(6.0.3790.1830); 
	Mon, 24 Apr 2006 16:05:06 -0400
Message-ID: <444D2F6E.2050708@americasm01.nt.com>
Date: Mon, 24 Apr 2006 16:05:02 -0400
From: "Chris Lewis" <clewis@nortel.com>
User-Agent: Thunderbird 1.5.0.2 (Windows/20060308)
MIME-Version: 1.0
To: Anti Spam Research Group <asrg@ietf.org>
Subject: Re: [Asrg] About that Church draft
References: <20060404211653.U1211@simone.iecc.com>
	<20060405050533.GA1220@ender>
In-Reply-To: <20060405050533.GA1220@ender>
X-Enigmail-Version: 0.94.0.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-OriginalArrivalTime: 24 Apr 2006 20:05:07.0449 (UTC)
	FILETIME=[62301E90:01C667DA]
X-Spam-Score: 0.0 (/)
X-Scan-Signature: e472ca43d56132790a46d9eefd95f0a5
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Laird Breyer wrote:
> On Apr 04 2006, John L wrote:
>> We sure have a lot of raw material.  Can I have a volunteer editor or two 
>> to moosh the good stuff into one reasonably coherent document explaining 
>> what we think would need to be changed for it to be good?
> 
> I've just read through the threads again and done a summary of all the
> main relevant points everyone discussed on the list (see attachment).

Nobody else has stepped up to the plate it seems.  How do we go about
publishing this John?  Is what Laird (and my subsequent comments) wrote
good enough as is?

> 
> If someone with the time wants to rearrange the paragraphs and flesh
> them out into a final document, please pipe up and do so. I tried to
> relate every point explicitly to the Church draft. Apologies if I
> missed someone's contribution.
> 
> 
> 
> ------------------------------------------------------------------------
> 
> Summary of the ASRG discussion on draft-church-dnsbl-harmful-01.txt as
> of 04-05-2006.
> 
> This is a review of the draft-church-dnsbl-harmful-01.txt document as discussed
> on the ASRG mailing list http://news.gmane.org/gmane.ietf.asrg/
> 
> Note: Some of the posts are missing from the archive, such as Dan Oetting's
> message-id <84E9C57C-1CE6-4232-90FC-7BE5F9EEC422@qwest.net>. I was going
> to refer to archive urls but ended up just giving Message-IDs.  
> 
> The church document has the following structure.
> 
> 1) An informal description of IP DNS blacklists.
> 2) A criticism of the policies and goals of DNS blacklist operators.
> 3) A claim of damage to third party mail transport.
> 4) A claim that botnets cause DNS blacklists to be out of date immediately.
> 5) An experiment on the accuracy of filtering using DNS blacklists.
> 6) A suggestion for the improvement of IP recognition
> 7) A moral argument of responsibility by DNS blacklist operators.
> 8) A suggestion to deprecate parts of the internet mail system to newer technologies.
> 
> Taken together, the author comes to the conclusion that DNS blacklists should
> not be used.
> 
> The ASRG discussion centered on several of these points.
> 
> Claims 2) and 3) are sensitive issues fraught with political aspects,
> and are therefore not appropriate for technical criticism, although
> they do represent some valid concerns that are often brought up.
> 
> Dan Oetting's thread deals with 4). There is agreement that botnets are
> a problem, but there is some research into the problem.
> 
> Jim MacLeod <http://article.gmane.org/gmane.ietf.asrg/11516> thread makes
> these points:
> 
> Regarding 4) IP assignments to particular computers tend to change
> slowly enough (on the order of a month) to make IP lists
> valuable. [reference?}
> 
> Regarding 2), diversity is a strength in anti-spam.
> 
> Regarding 5), the experiment is flawed due to its size.
> 
> Daniel Feenberg <Pine.GSO.4.64.0603290717090.4480@nber5.nber.org>
> mentions the increased legitimacy of rejecting mail at the SMTP
> transaction protocol level rather than silently discarding it for
> filtering reasons. Unlike content filtering, DNSBLs can be used at the
> transaction time.
> 
> Steve Atkins <4D2D1BA3-767A-47FC-BC9F-D54829C2E450@blighty.com>
> suggests there are significant general problems with DNSBLs, such as being
> misused as reputation systems, technical flaws in their usage,
> etc. It may be (Chris Lewis
> <442C2405.1040601@americasm01.nt.com>) that the real problem is that
> people responsible for choosing DNSBLs aren't necessarily exercising
> due diligence and knowing what they're getting.
> 
> Tony Finch disagrees with Steve Atkins, calling DNSBLs crucial. 
> Nick Nicholas asked for solid evidence, thereby being relevant to 5).
> 
> There is a big thread on testing philosophy of DNSBLs. Everyone agrees
> that 5) in Church's paper is flawed. The flaws are as follows:
> 
> Flaw: The Church sample size (1,374 msgs) is much too small to be
> meaningful.  Chris Lewis <442EE80C.5080500@americasm01.nt.com>
> disagrees with Church's conclusions based on 65,000 users and 600,000
> emails per day at Nortel.
> 
> Flaw: the comparison of filters in Church's table is
> inapplicable. Steve Atkins
> <2BC9207C-3E51-4B88-97A8-7445E9A1313B@blighty.com> points out that
> pretending that a DNSBL filter was used for comparison purposes, as is
> done by Church, is meaningless when it actually wasn't, because filtering
> during the SMTP transaction gives feedback (Daniel Feenberg's point)
> and therefore changes the spammer's behaviour.
> 
> Flaw: Church doesn't explicitly define his terms, such as just what he
> considers spam. In turn, this means derived definitions of FP, FN, TP
> etc. are not fully mapped out. Laird Breyer and Chris Lewis discuss
> the consequences of using different definitions, resulting in
> contradictory claims.
> 
> Flaw: Church's experiment is applicable at best to one type of email user.
> Chris Lewis's experience strongly contradicts the Church conclusion, and is
> based on very different business user email patterns.
> 
> Flaw: Church's table data suggests rejecting DNSBLs individually on
> accuracy grounds. However, this is flawed in that DNSBLs are often
> used in conjunction with other filters. The table doesn't address
> these other possibilities at all.
> 
> Several people point out that DNSBLs are meant to be used as part of a mix of
> techniques, ie that the whole is bigger than the sum of its parts.
>  
> Peter J. Holzer <20060331191807.GA14686@teal.hjp.at> offers some statistics
> on the most effective elements of his mix, others chime in on that thread.
> 
> Walter Dnes <20060401015927.GA27452@waltdnes.org> addresses 2) and 3),
> which are out of scope of the technical criticism. He points out that
> 5) is flawed because it compares a DNSBL-only solution against an
> unnamed, unknown custom method, and lists several other flaws
> above. This is not realistic since DNSBLs are used in conjuction with
> other methods. Walter also points out that 8) is ridiculous.
> 
> Thomas Choi <44322933.7060103@nortel.com> summarizes various issues
> that appeared during the discussion. 
> 
> Regarding 2) and 3), it's important for users of DNSBLs to apply diligence
> and caution. A single DNSBL verdict should never be used on its own, as
> is implied in 5). 
> 
> Regarding 5), spam filtering statistics always depend on type of email
> mix being received. This varies depending upon the type of user
> (business or private individual) and the type of social interactions
> (frequent mail with regular correspondents or regular short exchanges
> with strangers), etc.
> 
> Regarding 8), Thomas points out that mail technologies arise out of
> many different needs of users, and what is optimal for one person or
> entity in terms of usability, time, expertise, processing power etc. is not
> necessarily optimal for another.
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Asrg mailing list
> Asrg@ietf.org
> https://www1.ietf.org/mailman/listinfo/asrg

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3-nr1 (Windows 2000)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBRE0vbp3FmCyJjHfhAQKUBAQA2yPLPs+GVPyFcK8JbvbbCWCGw8qZFfQh
IEfTTsuxXPGYQWM0lfs7bpYuh/xLae/Iex+1MjYELWlq9G98x8sa2R8CkbrKJctE
Bey0RQCAMEBCUCe+WdZ4svn8DACuiKe7Yw2k3VNH6FXi8ZAeJ/zxmF4ti2pInfQU
eu+ZMxMxd8Y=
=Pw80
-----END PGP SIGNATURE-----

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Mon Apr 24 16:07:28 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FY7L9-00034U-Lc; Mon, 24 Apr 2006 16:07:23 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FY7L8-00034P-MO
	for asrg@ietf.org; Mon, 24 Apr 2006 16:07:22 -0400
Received: from a.mail.sonic.net ([64.142.16.245])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FY7L8-0007bh-AF
	for asrg@ietf.org; Mon, 24 Apr 2006 16:07:22 -0400
Received: from [168.61.10.151] (SJC-Office-DHCP-151.Mail-Abuse.ORG
	[168.61.10.151]) (authenticated bits=0)
	by a.mail.sonic.net (8.13.6/8.13.3) with ESMTP id k3OK7JDT001228
	(version=TLSv1/SSLv3 cipher=RC4-SHA bits=128 verify=NO);
	Mon, 24 Apr 2006 13:07:19 -0700
In-Reply-To: <444D2014.7010104@cisco.com>
References: <Pine.BSI.4.56.0604210854430.3697@tom.iecc.com>
	<444D2014.7010104@cisco.com>
Mime-Version: 1.0 (Apple Message framework v749.3)
Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed
Message-Id: <7D30B8B0-48E3-4BC0-932B-DD7FD133015A@mail-abuse.org>
Content-Transfer-Encoding: 7bit
From: Douglas Otis <dotis@mail-abuse.org>
Subject: Re: [Asrg] Third party DKIM signatures
Date: Mon, 24 Apr 2006 13:07:37 -0700
To: Jim Fenton <fenton@cisco.com>
X-Mailer: Apple Mail (2.749.3)
X-Spam-Score: 0.0 (/)
X-Scan-Signature: f60d0f7806b0c40781eee6b9cd0b2135
Cc: Anti Spam Research Group <asrg@ietf.org>
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org


On Apr 24, 2006, at 11:59 AM, Jim Fenton wrote:

> John R Levine wrote:
>> DKIM lets anyone sign any message, with no necessary connection  
>> between the signing domain and the domain in any other header such  
>> as From: or Sender:.  By third-party signatures we mean signatures  
>> that don't match the From: or don't match the Sender:, or don't  
>> match something else.
>>
> The definition in the current SSP draft is that a third-party  
> signature is a signature that doesn't match the From:.  There is no  
> reference to Sender: or "something else".

This detail is still open, as SSP is at an early stage of development  
where details covered in the SSP raise concerns.

>> The semantics as well as definition of third party signatures are,  
>> to put it mildly, somewhat unclear.  Some thought or actual  
>> experiments with such signatures could be helpful.
>
> I have done a lot of thinking about this; other thought or  
> experiments welcome.  Given that SSP is the next document up for  
> discussion in the DKIM WG, I think this has progressed beyond  
> "research", and I don't think it's a good idea to fragment  
> discussion between this mailing list and that one.

The DKIM WG are dealing with the Base draft and there is a desire to  
postpone SSP related discussions.  Similar concepts discussed here  
should not cause any fragmentation at this time.

Concepts dealing with both DoS and message replay abuse concerns  
could be found with a strategy to make associations between the  
originating source (from/sender/resent-from/resent-sender) and the  
signing-domain.  Associations are practical for a substantial portion  
of the email traffic (including DKIM related traffic) without  
incurring a high level of DNS transactions while also providing a DoS  
protection strategy.  The current draft SSP draft neglects to offer  
such associations or protective strategies.

This draft offers a possible approach.
http://www.ietf.org/internet-drafts/draft-otis-smtp-name-path-00.txt


What is not covered in this example is a specific association between  
an originating source and a DKIM signature.  This association is  
assumed to provide both the third-party EHLO _and_ DKIM signature  
authorization list.  Purists may desire a specific PTR list such as  
_tps._smtp.<example.com> to define an explicit list of what domain is  
allowed to sign messages for a specific domain and not just transmit  
(if that really matters).  A special domain label was also not  
created to indicate no email is allowed.  This could be a domain of  
'-.' that indication no mail association, for example.  A list, in  
addition to making explicit associations, also defines the nature of  
list, negating a need for the current SSP syntax that does this sans  
any associations.

Rather than raising issues specific to just DKIM, this strategy  
relates to a broader class of emails, while also providing DKIM  
relevant protective strategies.

-Doug







_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Mon Apr 24 17:08:34 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FY8Hk-0007k5-71; Mon, 24 Apr 2006 17:07:56 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FY8Hj-0007jz-BC
	for asrg@ietf.org; Mon, 24 Apr 2006 17:07:55 -0400
Received: from xuxa.iecc.com ([208.31.42.42])
	by ietf-mx.ietf.org with smtp (Exim 4.43) id 1FY8Hi-0002Gl-4z
	for asrg@ietf.org; Mon, 24 Apr 2006 17:07:55 -0400
Received: (qmail 15103 invoked from network); 24 Apr 2006 21:07:48 -0000
Received: from simone.iecc.com (208.31.42.47)
	by mail2.iecc.com with QMQP; 24 Apr 2006 21:07:48 -0000
Date: 24 Apr 2006 21:07:48 -0000
Message-ID: <20060424210748.51580.qmail@simone.iecc.com>
From: John Levine <asrg@johnlevine.com>
To: asrg@ietf.org
Subject: Re: [Asrg] About that Church draft
In-Reply-To: <444D2F6E.2050708@americasm01.nt.com>
Organization: 
Mime-Version: 1.0
Content-type: text/plain; charset=iso-8859-1
Content-transfer-encoding: 7bit
X-Spam-Score: -4.3 (----)
X-Scan-Signature: 30ac594df0e66ffa5a93eb4c48bcb014
Cc: clewis@nortel.com
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

>Nobody else has stepped up to the plate it seems.  How do we go about
>publishing this John?  Is what Laird (and my subsequent comments) wrote
>good enough as is?

I sent Laird's summary to the RFC Editor.  They're smart, they know
what it means.

R's,
John

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Mon Apr 24 18:54:00 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FY9wK-0001tq-Al; Mon, 24 Apr 2006 18:53:56 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FY9wJ-0001tF-Od
	for asrg@ietf.org; Mon, 24 Apr 2006 18:53:55 -0400
Received: from zcars04f.nortel.com ([47.129.242.57])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FY9wI-0000Pj-G3
	for asrg@ietf.org; Mon, 24 Apr 2006 18:53:55 -0400
Received: from zcarhxs1.corp.nortel.com
	(zcarhxs1.corp.nort...s0.corp.nortel.com [47.129.230.89])
	by zcars04f.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id
	k3OMreH17615
	for <asrg@ietf.org>; Mon, 24 Apr 2006 18:53:40 -0400 (EDT)
Received: from [47.130.17.101] ([47.130.17.101] RDNS failed) by
	zcarhxs1.corp.nortel.com with Microsoft SMTPSVC(6.0.3790.1830); 
	Mon, 24 Apr 2006 18:53:09 -0400
Message-ID: <444D56D1.6040903@americasm01.nt.com>
Date: Mon, 24 Apr 2006 18:53:05 -0400
From: "Chris Lewis" <clewis@nortel.com>
User-Agent: Thunderbird 1.5.0.2 (Windows/20060308)
MIME-Version: 1.0
CC: asrg@ietf.org
Subject: Re: [Asrg] About that Church draft
References: <20060424210748.51580.qmail@simone.iecc.com>
In-Reply-To: <20060424210748.51580.qmail@simone.iecc.com>
X-Enigmail-Version: 0.94.0.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-OriginalArrivalTime: 24 Apr 2006 22:53:10.0414 (UTC)
	FILETIME=[DC1A7AE0:01C667F1]
X-Spam-Score: 0.0 (/)
X-Scan-Signature: ffa9dfbbe7cc58b3fa6b8ae3e57b0aa3
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

John Levine wrote:
>> Nobody else has stepped up to the plate it seems.  How do we go about
>> publishing this John?  Is what Laird (and my subsequent comments) wrote
>> good enough as is?
> 
> I sent Laird's summary to the RFC Editor.  They're smart, they know
> what it means.

Excellent.

With my followon comments? ;-)

[Hopefully they shouldn't be necessary.]
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3-nr1 (Windows 2000)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBRE1W0Z3FmCyJjHfhAQJrOAQA82QtBDRvdAUHkDJtskFdw/iTTj99tOou
4PuS9mvR8YZgFRecQXDG83/1KkhPFRj7ojAh5WB4c0j6vo2qztO11USA4VmrrSOt
3V8BntDQt59BEOO407YRuu7Z1v+1xKWjG/4vpYQIHK4aAnHQohTrzPkgo9vsYriq
r2tHVUMeCKM=
=3leK
-----END PGP SIGNATURE-----

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Thu Apr 27 11:46:04 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FZ8fz-0007NH-Lw; Thu, 27 Apr 2006 11:45:07 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FZ8fy-0007NB-N7
	for asrg@ietf.org; Thu, 27 Apr 2006 11:45:06 -0400
Received: from vms040pub.verizon.net ([206.46.252.40])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FZ8fw-0006Ej-Ea
	for asrg@ietf.org; Thu, 27 Apr 2006 11:45:06 -0400
Received: from torch.brasslantern.com ([71.116.76.26])
	by vms040.mailsrvcs.net (Sun Java System Messaging Server 6.2-4.02
	(built Sep
	9 2005)) with ESMTPA id <0IYE008UX12UOAP5@vms040.mailsrvcs.net> for
	asrg@ietf.org; Thu, 27 Apr 2006 10:44:55 -0500 (CDT)
Received: from torch.brasslantern.com (localhost.localdomain [127.0.0.1])
	by torch.brasslantern.com (8.13.1/8.13.1) with ESMTP id
	k3RFijKm005006	for <asrg@ietf.org>; Thu, 27 Apr 2006 08:44:53 -0700
Received: (from schaefer@localhost)	by torch.brasslantern.com
	(8.13.1/8.13.1/Submit) id k3RFiew0005005	for asrg@ietf.org; Thu,
	27 Apr 2006 08:44:40 -0700
Date: Thu, 27 Apr 2006 08:44:40 -0700
From: Bart Schaefer <schaefer@brasslantern.com>
To: asrg@ietf.org
Message-id: <060427084440.ZM5004@torch.brasslantern.com>
MIME-version: 1.0
X-Mailer: OpenZMail Classic (0.9.2 24April2005)
Content-type: text/plain; charset=us-ascii
X-Spam-Score: 0.1 (/)
X-Scan-Signature: 08e48e05374109708c00c6208b534009
Subject: [Asrg] "New Email Protocol Replaces SMTP"
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

Gotta love the FUD.

http://www.tmcnet.com/usubmit/2006/04/18/1581774.htm

I saw the above on Ed Falk's National Spam News blog:
http://thespamdiaries.blogspot.com/2006/04/national-spam-news_26.html

Ed asks "... but where's the RFC? Has the IETF even heard of it?"

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Thu Apr 27 12:22:41 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FZ9GI-0006LY-Mw; Thu, 27 Apr 2006 12:22:38 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FZ9GH-0006GL-Hn
	for asrg@ietf.org; Thu, 27 Apr 2006 12:22:37 -0400
Received: from chiark.greenend.org.uk ([193.201.200.170])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FZ9GD-0000cg-9E
	for asrg@ietf.org; Thu, 27 Apr 2006 12:22:37 -0400
Received: by chiark.greenend.org.uk (Debian Exim 3.36 #1) with local
	(return-path fanf@chiark.greenend.org.uk)
	id 1FZ9G8-00019y-00; Thu, 27 Apr 2006 17:22:28 +0100
To: schaefer@brasslantern.com
From: Tony Finch <dot@dotat.at>
Subject: Re: [Asrg] "New Email Protocol Replaces SMTP"
In-Reply-To: <060427084440.ZM5004@torch.brasslantern.com>
Message-Id: <E1FZ9G8-00019y-00@chiark.greenend.org.uk>
Date: Thu, 27 Apr 2006 17:22:28 +0100
X-Spam-Score: 0.2 (/)
X-Scan-Signature: 7a6398bf8aaeabc7a7bb696b6b0a2aad
Cc: asrg@ietf.org
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

Bart Schaefer <schaefer@brasslantern.com> wrote:
>
>Ed asks "... but where's the RFC? Has the IETF even heard of it?"

It's an intranet system, and isn't designed for open public deployment, 
which is how it can make its claims about spam, and which is why the 
IETF doesn't need to know or care.

Tony.
-- 
f.a.n.finch  <dot@dotat.at>  http://dotat.at/
NORTH FORELAND TO SELSEY BILL: VARIABLE BECOMING NORTH 3 WEATHER: FAIR. GOOD.
SMOOTH OR SLIGHT.

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Thu Apr 27 12:45:41 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FZ9cX-0008Nm-Hx; Thu, 27 Apr 2006 12:45:37 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FZ9cV-0008KE-Jh
	for asrg@ietf.org; Thu, 27 Apr 2006 12:45:35 -0400
Received: from adsl-216-102-208-10.dsl.snfc21.pacbell.net ([216.102.208.10]
	helo=fasolt.mtcc.com) by ietf-mx.ietf.org with esmtp (Exim 4.43)
	id 1FZ9cU-0001Vz-6R
	for asrg@ietf.org; Thu, 27 Apr 2006 12:45:35 -0400
DKIM-Signature: a=rsa-sha1; q=dns; l=1052; t=1146156324; x=1147020324;
	c=relaxed/simple; s=dicks.drop.kirkwood;
	h=Content-Type:From:Subject:Content-Transfer-Encoding:MIME-Version;
	d=mtcc.com; i=mike@mtcc.com; z=From:Michael=20Thomas=20<mike@mtcc.com>
	|Subject:Re=3A=20[Asrg]=20=22New=20Email=20Protocol=20Replaces=20SMTP=22
	|To:Tony=20Finch=20<dot@dotat.at>
	|Cc:=20schaefer@brasslantern.com,=20=20asrg@ietf.org;
	X=v=3Dcisco.com=3B=20h=3Dw15Im3dOihmCtLwai6vfZ2N0PMA=3D;
	b=NQwa7Jti1hpk5gA2hCubIwwNeMPKQ/dsW7C9E6dn3c9SSrocH1U9Idut8ESXMVhQxENyxis7
	IDiUZNsZi8iKaemPcaWqrvnVi6bUolIokfNzwGaTbJky2+zkMp9AQ9Dz;
Received: from [216.102.208.10]
	(IDENT:U2FsdGVkX19UvzwSlPo4q1qTPwEYM9q0CfjRPLGiUrM@fasolt.mtcc.com
	[216.102.208.10] (may be forged)) (authenticated bits=0)
	by fasolt.mtcc.com (8.13.1/8.13.1) with ESMTP id k3RGjM0O006248
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO);
	Thu, 27 Apr 2006 09:45:23 -0700
Message-ID: <4450F522.4080502@mtcc.com>
Date: Thu, 27 Apr 2006 09:45:22 -0700
From: Michael Thomas <mike@mtcc.com>
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
	rv:1.7.10) Gecko/20050720 Thunderbird/1.0.6 Fedora/1.0.6-1.1.fc3
	Mnenhy/0.7.2.0
X-Accept-Language: en-us, en
MIME-Version: 1.0
To: Tony Finch <dot@dotat.at>
Subject: Re: [Asrg] "New Email Protocol Replaces SMTP"
References: <E1FZ9G8-00019y-00@chiark.greenend.org.uk>
In-Reply-To: <E1FZ9G8-00019y-00@chiark.greenend.org.uk>
Authentication-Results: fasolt.mtcc.com; header.From=mike@mtcc.com; dkim=pass (
	16 extraneous bytes; sig from mtcc.com verified; ); 
X-XIPE-SCORES: dispose=pass; ACD=1.00; CLAM=0.00; COMPLY=0.00; URL=0.00;
	SA=2.30; HONEY=0.00; 
X-Spam-Score: 1.3 (+)
X-Scan-Signature: e5ba305d0e64821bf3d8bc5d3bb07228
Cc: asrg@ietf.org, schaefer@brasslantern.com
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Content-Type: multipart/mixed; boundary="===============1162468911=="
Errors-To: asrg-bounces@ietf.org

--===============1162468911==
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
  <meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
  <title></title>
</head>
<body bgcolor="#ffffff" text="#000000">
Tony Finch wrote:
<blockquote cite="midE1FZ9G8-00019y-00@chiark.greenend.org.uk"
 type="cite">
  <pre wrap="">Bart Schaefer <a class="moz-txt-link-rfc2396E" href="mailto:schaefer@brasslantern.com">&lt;schaefer@brasslantern.com&gt;</a> wrote:
  </pre>
  <blockquote type="cite">
    <pre wrap="">Ed asks "... but where's the RFC? Has the IETF even heard of it?"
    </pre>
  </blockquote>
  <pre wrap=""><!---->
It's an intranet system, and isn't designed for open public deployment, 
which is how it can make its claims about spam, and which is why the 
IETF doesn't need to know or care.

Tony.
  </pre>
</blockquote>
So if I packaged up sendmail, ran it on port 8025 behind my firewall,
could<br>
I have a revolutionary protocol that replaces SMTP too?<br>
<br>
&nbsp;&nbsp;&nbsp; &nbsp;&nbsp; Mike<br>
</body>
</html>


--===============1162468911==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg

--===============1162468911==--



From asrg-bounces@ietf.org Thu Apr 27 13:18:31 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FZA8G-00012y-Cl; Thu, 27 Apr 2006 13:18:24 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FZA8F-00012t-Go
	for asrg@ietf.org; Thu, 27 Apr 2006 13:18:23 -0400
Received: from wproxy.gmail.com ([64.233.184.236])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FZA8E-00034X-1m
	for asrg@ietf.org; Thu, 27 Apr 2006 13:18:23 -0400
Received: by wproxy.gmail.com with SMTP id i20so1503892wra
	for <asrg@ietf.org>; Thu, 27 Apr 2006 10:18:21 -0700 (PDT)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com;
	h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:references;
	b=OG6CaZYhnz7rTOZhJUcDapPN9VoEdrccugEyqRz7MZuXCG3l4KOHm2XfwSsUtulKAsnJxFNjO/ljJaTZ/GFL5DnHOOW5MNiJnSV6HkD3T7sDFv+V+K3gRuThOqJrDqbjsrCxplZygO3dretrI1/P09fCWqOsJE4sYbO6iE89trw=
Received: by 10.65.235.7 with SMTP id m7mr2607788qbr;
	Thu, 27 Apr 2006 10:18:21 -0700 (PDT)
Received: by 10.64.201.10 with HTTP; Thu, 27 Apr 2006 10:18:20 -0700 (PDT)
Message-ID: <90b025640604271018p73fdf24fi637e0ef3832b3804@mail.gmail.com>
Date: Thu, 27 Apr 2006 10:18:20 -0700
From: "Jim MacLeod" <jmacleod@gmail.com>
To: "Michael Thomas" <mike@mtcc.com>
Subject: Re: [Asrg] "New Email Protocol Replaces SMTP"
In-Reply-To: <4450F522.4080502@mtcc.com>
MIME-Version: 1.0
References: <E1FZ9G8-00019y-00@chiark.greenend.org.uk>
	<4450F522.4080502@mtcc.com>
X-Spam-Score: 0.1 (/)
X-Scan-Signature: 69a74e02bbee44ab4f8eafdbcedd94a1
Cc: asrg@ietf.org
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Content-Type: multipart/mixed; boundary="===============0821550705=="
Errors-To: asrg-bounces@ietf.org

--===============0821550705==
Content-Type: multipart/alternative; 
	boundary="----=_Part_6068_25049309.1146158300902"

------=_Part_6068_25049309.1146158300902
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

On 4/27/06, Michael Thomas <mike@mtcc.com> wrote:
>
> So if I packaged up sendmail, ran it on port 8025 behind my firewall,
> could
> I have a revolutionary protocol that replaces SMTP too?
>
>        Mike
>

Only if you secured venture capital and issued a Press Release.

-Jim

------=_Part_6068_25049309.1146158300902
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

<br><br><div><span class=3D"gmail_quote">On 4/27/06, <b class=3D"gmail_send=
ername">Michael Thomas</b> &lt;<a href=3D"mailto:mike@mtcc.com">mike@mtcc.c=
om</a>&gt; wrote:</span><blockquote class=3D"gmail_quote" style=3D"border-l=
eft: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left:=
 1ex;">
<div style=3D"direction: ltr;">So if I packaged up sendmail, ran it on port=
 8025 behind my firewall,
could<br>
I have a revolutionary protocol that replaces SMTP too?<br>
<br>
&nbsp;&nbsp;&nbsp; &nbsp;&nbsp; Mike</div></blockquote><div><br>Only if you=
 secured venture capital and issued a Press Release.<br><br>-Jim<br></div><=
br></div>

------=_Part_6068_25049309.1146158300902--


--===============0821550705==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg

--===============0821550705==--




From asrg-bounces@ietf.org Thu Apr 27 13:26:18 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FZAFt-0006sk-8I; Thu, 27 Apr 2006 13:26:17 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FZAFs-0006sf-8C
	for asrg@ietf.org; Thu, 27 Apr 2006 13:26:16 -0400
Received: from xproxy.gmail.com ([66.249.82.194])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FZAFr-0003Ww-0C
	for asrg@ietf.org; Thu, 27 Apr 2006 13:26:16 -0400
Received: by xproxy.gmail.com with SMTP id s12so1356851wxc
	for <asrg@ietf.org>; Thu, 27 Apr 2006 10:26:14 -0700 (PDT)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com;
	h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references;
	b=hAH1DKSbKOkXjKq/fMRt2w+KPfNIbDuHoN6Olk2nsHGjf8XAiRVGINrb/5xnl2l0mtzfC6vwYzFAkimyL6MCQGzWSWx96KXQMH3NRBtL9WcmUoWUhjqZoATYOq5HQVOFKHn3LYj9Nz94c2zMk+bTo2mK+BpeBk/BSCuV06yStz0=
Received: by 10.70.105.10 with SMTP id d10mr1088933wxc;
	Thu, 27 Apr 2006 10:26:14 -0700 (PDT)
Received: by 10.70.97.18 with HTTP; Thu, 27 Apr 2006 10:26:14 -0700 (PDT)
Message-ID: <934f64a20604271026w26c66629q793ffcd73673946d@mail.gmail.com>
Date: Thu, 27 Apr 2006 12:26:14 -0500
From: "David Nicol" <davidnicol@gmail.com>
To: "Bart Schaefer" <schaefer@brasslantern.com>
Subject: Re: [Asrg] "New Email Protocol Replaces SMTP"
In-Reply-To: <060427084440.ZM5004@torch.brasslantern.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline
References: <060427084440.ZM5004@torch.brasslantern.com>
X-Spam-Score: 0.0 (/)
X-Scan-Signature: ea4ac80f790299f943f0a53be7e1a21a
Cc: asrg@ietf.org
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

the problem is, SMTP _HAS_ authentication hooks in it.  The authentication
hook is the fact that there is an opportunity for policy refusal at various
points.  The absence of a universal consensus on the framework for making
those decisions (IMO, SPF is a very good start in that direction, as well a=
s
all the other proposals under debate here and elsewhere) is in no way a
"fundamental problem with SMTP."  Possibly a "fundamental problem
with human nature" though.  Eventually we're all going to be reduced to
stoking the generators to keep our robot overlords in kilowatts...

On 4/27/06, Bart Schaefer <schaefer@brasslantern.com> wrote:
> Gotta love the FUD.
>
> http://www.tmcnet.com/usubmit/2006/04/18/1581774.htm
>
> I saw the above on Ed Falk's National Spam News blog:
> http://thespamdiaries.blogspot.com/2006/04/national-spam-news_26.html
>
> Ed asks "... but where's the RFC? Has the IETF even heard of it?"
>
> _______________________________________________
> Asrg mailing list
> Asrg@ietf.org
> https://www1.ietf.org/mailman/listinfo/asrg
>


--
David L Nicol
Document what you do, then do what you documented

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Thu Apr 27 14:06:10 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FZAsN-0006Iz-Re; Thu, 27 Apr 2006 14:06:03 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FZAsM-0006Iu-BP
	for asrg@ietf.org; Thu, 27 Apr 2006 14:06:02 -0400
Received: from sb7.songbird.com ([208.184.79.137])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FZAsL-00066h-0T
	for asrg@ietf.org; Thu, 27 Apr 2006 14:06:02 -0400
Received: from [192.168.0.2] (adsl-71-131-51-71.dsl.sntc01.pacbell.net
	[71.131.51.71]) (authenticated bits=0)
	by sb7.songbird.com (8.12.11.20060308/8.12.11) with ESMTP id
	k3RI6TNa004869
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO)
	for <asrg@ietf.org>; Thu, 27 Apr 2006 11:06:30 -0700
Message-ID: <44510803.1030805@dcrocker.net>
Date: Thu, 27 Apr 2006 11:05:55 -0700
From: Dave Crocker <dhc@dcrocker.net>
Organization: Brandenburg InternetWorking
User-Agent: Thunderbird 1.5.0.2 (Windows/20060308)
MIME-Version: 1.0
To: asrg@ietf.org
Subject: Re: [Asrg] "New Email Protocol Replaces SMTP"
References: <060427084440.ZM5004@torch.brasslantern.com>
	<934f64a20604271026w26c66629q793ffcd73673946d@mail.gmail.com>
In-Reply-To: <934f64a20604271026w26c66629q793ffcd73673946d@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
X-SongbirdInformation: support@songbird.com for more information
X-Songbird: Clean
X-Songbird-From: dhc@dcrocker.net
X-Spam-Score: 0.0 (/)
X-Scan-Signature: ffa9dfbbe7cc58b3fa6b8ae3e57b0aa3
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
Reply-To: dcrocker@bbiw.net
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

David Nicol wrote:
> the problem is, SMTP _HAS_ authentication hooks in it.  The authentication
> hook is the fact that there is an opportunity for policy refusal at various
> points.  The absence of a universal consensus on...


My SMTP-replacement mantra:

When folks agree on the service (changes) needed, and
When that agreement does not break legitimate uses of email, and
When the community tries to retrofit those changes into the existing 
email infrastructure, and
When that effort fails,
Only then will an effort to replace SMTP make any sense at all.

To date, we have not even attempted the first 'when', nevermind 
succeeded at it (and the second 'when').

d/

-- 

Dave Crocker
Brandenburg InternetWorking
<http://bbiw.net>


_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Thu Apr 27 15:34:48 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FZCG3-00086b-5E; Thu, 27 Apr 2006 15:34:35 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FZCG2-00086W-Ie
	for asrg@ietf.org; Thu, 27 Apr 2006 15:34:34 -0400
Received: from zcars04f.nortel.com ([47.129.242.57])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FZCG1-0004LT-6b
	for asrg@ietf.org; Thu, 27 Apr 2006 15:34:34 -0400
Received: from zcarhxs1.corp.nortel.com
	(zcarhxs1.corp.nort...s0.corp.nortel.com [47.129.230.89])
	by zcars04f.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id
	k3RJYU001123
	for <asrg@ietf.org>; Thu, 27 Apr 2006 15:34:30 -0400 (EDT)
Received: from [47.129.150.195] ([47.129.150.195] RDNS failed) by
	zcarhxs1.corp.nortel.com with Microsoft SMTPSVC(6.0.3790.1830); 
	Thu, 27 Apr 2006 15:34:30 -0400
Message-ID: <44511CC4.2070507@americasm01.nt.com>
Date: Thu, 27 Apr 2006 15:34:28 -0400
From: "Chris Lewis" <clewis@nortel.com>
User-Agent: Thunderbird 1.5.0.2 (Windows/20060308)
MIME-Version: 1.0
CC: asrg@ietf.org
Subject: Re: [Asrg] "New Email Protocol Replaces SMTP"
References: <E1FZ9G8-00019y-00@chiark.greenend.org.uk>	<4450F522.4080502@mtcc.com>
	<90b025640604271018p73fdf24fi637e0ef3832b3804@mail.gmail.com>
In-Reply-To: <90b025640604271018p73fdf24fi637e0ef3832b3804@mail.gmail.com>
X-Enigmail-Version: 0.94.0.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-OriginalArrivalTime: 27 Apr 2006 19:34:30.0257 (UTC)
	FILETIME=[9A601E10:01C66A31]
X-Spam-Score: 0.0 (/)
X-Scan-Signature: 8b30eb7682a596edff707698f4a80f7d
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jim MacLeod wrote:
> 
> 
> On 4/27/06, *Michael Thomas* <mike@mtcc.com <mailto:mike@mtcc.com>> wrote:
> 
>     So if I packaged up sendmail, ran it on port 8025 behind my
>     firewall, could
>     I have a revolutionary protocol that replaces SMTP too?
> 
>            Mike
> 
> 
> Only if you secured venture capital and issued a Press Release.

Why bother with all the extraneous stuff like designing/implementing
anything, or having any funding?

Send out a Press Release, and everything else simply happens.  Money is
flung your way, consultants rave, sexual favours arrive at the most
unexpected times, and you make headlines.

It's magic, really.

It's even better than patenting something that the industry has already
been doing for decades - doesn't require you to find a scammy lawyer -
PR flacks are cheaper.  But it doesn't pay quite as well on average.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3-nr1 (Windows 2000)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBRFEcxJ3FmCyJjHfhAQKRJwP+NCU1/z6ZMdSdz6Y1Ivagdbm9uVqQ3JE7
CKEIKqjnGKXVfGdcr5evO8blzqkR4JSpltPo/XhVcRUF2+xUQXzuMgeJj/gYCoLy
jZVJeyrqbGOHEL9ctqaeiSRsRFBH5KJNgGYnlnBa6akwFSr/DrTbj/bdumluav+I
YcZiR14Rr14=
=zXnm
-----END PGP SIGNATURE-----

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Thu Apr 27 18:55:02 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FZFNh-0004Pi-NS; Thu, 27 Apr 2006 18:54:41 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FZFNg-0004MJ-1z
	for asrg@ietf.org; Thu, 27 Apr 2006 18:54:40 -0400
Received: from frank.servitor.co.uk ([83.142.230.145])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FZFNe-00014T-PS
	for asrg@ietf.org; Thu, 27 Apr 2006 18:54:40 -0400
Received: from cpc2-salf1-0-0-cust27.manc.cable.ntl.com ([82.6.72.28]
	helo=[192.168.1.101])
	by frank.servitor.co.uk with esmtpa (Exim 4.60 (FreeBSD))
	(envelope-from <paul@vagueware.com>)
	id 1FZFRK-0006d5-Ex; Thu, 27 Apr 2006 23:58:26 +0100
In-Reply-To: <44510803.1030805@dcrocker.net>
References: <060427084440.ZM5004@torch.brasslantern.com>
	<934f64a20604271026w26c66629q793ffcd73673946d@mail.gmail.com>
	<44510803.1030805@dcrocker.net>
Mime-Version: 1.0 (Apple Message framework v749.3)
Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed
Message-Id: <B4885FF5-F9C7-4927-88AD-DDA7785BBFB2@vagueware.com>
Content-Transfer-Encoding: 7bit
From: Paul Robinson <paul@vagueware.com>
Subject: Re: [Asrg] "New Email Protocol Replaces SMTP"
Date: Thu, 27 Apr 2006 23:54:35 +0100
To: dcrocker@bbiw.net
X-Mailer: Apple Mail (2.749.3)
X-Spam-Score: -4.2 (----)
X-Spam-Report: Spam detection software,
	running on the system "frank.servitor.co.uk", has
	identified this incoming email as possible spam. The original message
	has been attached to this so you can view it (if it isn't spam) or
	label similar future email.  If you have any questions, see
	the administrator of that system for details.
	Content preview:  On 27 Apr 2006, at 19:05, Dave Crocker wrote: > My
	SMTP-replacement mantra: > > When folks agree on the service (changes)
	needed, and > When that agreement does not break legitimate uses of
	email, and > When the community tries to retrofit those changes into the
	> existing email infrastructure, and > When that effort fails, > Only
	then will an effort to replace SMTP make any sense at all. > > To date, 
	we have not even attempted the first 'when', nevermind > succeeded at it
	(and the second 'when'). [...] 
	Content analysis details:   (-4.2 points, 5.0 required)
	pts rule name              description
	---- ----------------------
	--------------------------------------------------
	-1.8 ALL_TRUSTED Passed through trusted hosts only via SMTP
	-2.6 BAYES_00 BODY: Bayesian spam probability is 0 to 1%
	[score: 0.0000]
	0.2 AWL AWL: From: address is in the auto white-list
X-Spam-Score: 0.0 (/)
X-Scan-Signature: 52e1467c2184c31006318542db5614d5
Cc: asrg@ietf.org
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

On 27 Apr 2006, at 19:05, Dave Crocker wrote:

> My SMTP-replacement mantra:
>
> When folks agree on the service (changes) needed, and
> When that agreement does not break legitimate uses of email, and
> When the community tries to retrofit those changes into the  
> existing email infrastructure, and
> When that effort fails,
> Only then will an effort to replace SMTP make any sense at all.
>
> To date, we have not even attempted the first 'when', nevermind  
> succeeded at it (and the second 'when').

That's the most intelligent thing I've read on this list in a long  
time. For what it's worth, my less accurate and less worthy SMTP- 
replacement mantra:

Every 6 months somebody suggests replacing SMTP, which results in
Every possible solution imaginable for
Every problem we have all ever experienced with SMTP, all written by
Every IETF member who has ever received spam
And then it never goes anywhere sensible.

How about instead of trying to fix SMTP - a perfectly good protocol  
with lots of layers for encryption, authorisation, and decades of  
experience shared between us all - why don't we try fixing the people  
who respond to spam thereby making it an economically sound marketing  
proposition?

OK, unrealistic, I accept. But no more unrealistic than replacing  
SMTP any time soon.

--
Paul Robinson


_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Thu Apr 27 19:34:00 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FZFzU-0005Ob-Mq; Thu, 27 Apr 2006 19:33:44 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FZFzT-0005Jw-Np
	for asrg@ietf.org; Thu, 27 Apr 2006 19:33:43 -0400
Received: from sccrmhc13.comcast.net ([63.240.77.83])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FZFzS-0003t2-IA
	for asrg@ietf.org; Thu, 27 Apr 2006 19:33:43 -0400
Received: from raeburn.org (c-65-96-168-237.hsd1.ma.comcast.net[65.96.168.237])
	by comcast.net (sccrmhc13) with ESMTP
	id <20060427233341013001g1ime>; Thu, 27 Apr 2006 23:33:41 +0000
Received: from [18.18.1.160] (NOME-KING.MIT.EDU [18.18.1.160])
	by raeburn.org (8.12.11/8.12.11) with ESMTP id k3RNXd0Q009832;
	Thu, 27 Apr 2006 19:33:40 -0400 (EDT)
In-Reply-To: <B4885FF5-F9C7-4927-88AD-DDA7785BBFB2@vagueware.com>
References: <060427084440.ZM5004@torch.brasslantern.com>
	<934f64a20604271026w26c66629q793ffcd73673946d@mail.gmail.com>
	<44510803.1030805@dcrocker.net>
	<B4885FF5-F9C7-4927-88AD-DDA7785BBFB2@vagueware.com>
Mime-Version: 1.0 (Apple Message framework v749.3)
Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed
Message-Id: <158BE323-B171-48F6-B1CB-96C1D5FA0044@raeburn.org>
Content-Transfer-Encoding: 7bit
From: Ken Raeburn <raeburn@raeburn.org>
Subject: Re: [Asrg] "New Email Protocol Replaces SMTP"
Date: Thu, 27 Apr 2006 19:33:38 -0400
To: Paul Robinson <paul@vagueware.com>
X-Mailer: Apple Mail (2.749.3)
X-Spam-Score: 0.2 (/)
X-Scan-Signature: 7bac9cb154eb5790ae3b2913587a40de
Cc: asrg@ietf.org
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org

On Apr 27, 2006, at 18:54, Paul Robinson wrote:
> How about instead of trying to fix SMTP - a perfectly good protocol  
> with lots of layers for encryption, authorisation, and decades of  
> experience shared between us all - why don't we try fixing the  
> people who respond to spam thereby making it an economically sound  
> marketing proposition?

Sounds more like a job for the Social Engineering Task Force.

Ken

_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



From asrg-bounces@ietf.org Thu Apr 27 20:04:05 2006
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com)
	by megatron.ietf.org with esmtp (Exim 4.43)
	id 1FZGSf-0001tb-2j; Thu, 27 Apr 2006 20:03:53 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org)
	by megatron.ietf.org with esmtp (Exim 4.43) id 1FZGSe-0001tW-Lh
	for asrg@ietf.org; Thu, 27 Apr 2006 20:03:52 -0400
Received: from a.mail.sonic.net ([64.142.16.245])
	by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1FZGSc-0005C2-9f
	for asrg@ietf.org; Thu, 27 Apr 2006 20:03:52 -0400
Received: from [10.2.164.130] (SJC-Office-NAT-213.Mail-Abuse.ORG
	[168.61.10.213]) (authenticated bits=0)
	by a.mail.sonic.net (8.13.6/8.13.3) with ESMTP id k3S03mJB011067
	(version=TLSv1/SSLv3 cipher=RC4-SHA bits=128 verify=NO);
	Thu, 27 Apr 2006 17:03:49 -0700
In-Reply-To: <B4885FF5-F9C7-4927-88AD-DDA7785BBFB2@vagueware.com>
References: <060427084440.ZM5004@torch.brasslantern.com>
	<934f64a20604271026w26c66629q793ffcd73673946d@mail.gmail.com>
	<44510803.1030805@dcrocker.net>
	<B4885FF5-F9C7-4927-88AD-DDA7785BBFB2@vagueware.com>
Mime-Version: 1.0 (Apple Message framework v749.3)
Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed
Message-Id: <E8E03B91-84E7-4C5F-B642-EC5C81E70EAB@mail-abuse.org>
Content-Transfer-Encoding: 7bit
From: Douglas Otis <dotis@mail-abuse.org>
Subject: Re: [Asrg] "New Email Protocol Replaces SMTP"
Date: Thu, 27 Apr 2006 17:03:40 -0700
To: Paul Robinson <paul@vagueware.com>
X-Mailer: Apple Mail (2.749.3)
X-Spam-Score: 0.0 (/)
X-Scan-Signature: e1e48a527f609d1be2bc8d8a70eb76cb
Cc: Anti-Spam Research Group Group <asrg@ietf.org>
X-BeenThere: asrg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Anti-Spam Research Group - IRTF <asrg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/asrg>
List-Post: <mailto:asrg@ietf.org>
List-Help: <mailto:asrg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/asrg>,
	<mailto:asrg-request@ietf.org?subject=subscribe>
Errors-To: asrg-bounces@ietf.org


On Apr 27, 2006, at 3:54 PM, Paul Robinson wrote:

>
> How about instead of trying to fix SMTP - a perfectly good protocol  
> with lots of layers for encryption, authorisation, and decades of  
> experience shared between us all - why don't we try fixing the  
> people who respond to spam thereby making it an economically sound  
> marketing proposition?
>
> OK, unrealistic, I accept. But no more unrealistic than replacing  
> SMTP any time soon.

A non-functional aspect of SMTP, per the specification, prohibits  
assured verification of the sending host name.  This minor exception  
prevents an otherwise effective defense against many common abuses.   
When done at the EHLO, a verified host name offers a name-based DoS  
defense early in the SMTP session, and can describe a message path by  
name without risks of network amplification.  The name-based approach  
offers tell-tale information about the SMTP client based solely upon  
the history of their name server.  The name-based approach also  
reduces collateral blocking within shared address space.

When identifying clients solely by IP address, inordinate levels of  
network amplification may be realized when assembling global  
authorizations for a domain.  Construction of such IP address lists  
may require more than one hundred of DNS transactions for each right- 
hand name questioned within a message.  This IP address authorization  
technique will jeopardize DNS owing to the pervasive and distributed  
nature of email.  A noticeable characteristic of this technique is  
that it does not identify the controlling domain, nor does it afford  
any DoS protections.  The recent authentication techniques may invite  
more names to be questioned using this highly dangerous technique.

It is not unrealistic to fix this broken aspect of SMTP EHLO  
verification, and to discourage attempts at using IP address lists as  
a means of authorization.

-Doug



_______________________________________________
Asrg mailing list
Asrg@ietf.org
https://www1.ietf.org/mailman/listinfo/asrg



