From CLEWIS@nortel.com Tue Jan 4 04:00:24 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 5C10A3A69B7 for ; Tue, 4 Jan 2011 04:00:24 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.2 X-Spam-Level: X-Spam-Status: No, score=-6.2 tagged_above=-999 required=5 tests=[AWL=-0.400, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, SARE_SUB_RAND_LETTRS4=0.799] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XjrrcZMiQcWE for ; Tue, 4 Jan 2011 04:00:23 -0800 (PST) Received: from zcars04e.nortel.com (zcars04e.nortel.com [47.129.242.56]) by core3.amsl.com (Postfix) with ESMTP id 554383A6B5A for ; Tue, 4 Jan 2011 04:00:23 -0800 (PST) Received: from zrtphxs1.corp.nortel.com (zrtphxs1.corp.nortel.com [47.140.202.46]) by zcars04e.nortel.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id p04C2QK28085 for ; Tue, 4 Jan 2011 12:02:27 GMT Received: from zrtphx5h0.corp.nortel.com ([47.140.202.65]) by zrtphxs1.corp.nortel.com with Microsoft SMTPSVC(6.0.3790.4675); Tue, 4 Jan 2011 07:02:26 -0500 Received: from [47.130.80.33] (47.130.80.33) by zrtphx5h0.corp.nortel.com (47.140.202.65) with Microsoft SMTP Server (TLS) id 8.1.340.0; Tue, 4 Jan 2011 07:02:25 -0500 Message-ID: <4D230C50.8050508@nortel.com> Date: Tue, 4 Jan 2011 07:02:24 -0500 From: "Chris Lewis" Organization: Nortel User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.23) Gecko/20090812 Thunderbird/2.0.0.23 Mnenhy/0.7.6.666 MIME-Version: 1.0 To: ASRG Content-Type: multipart/mixed; boundary="------------010104070308090002010005" X-OriginalArrivalTime: 04 Jan 2011 12:02:26.0592 (UTC) FILETIME=[41070A00:01CBAC07] Subject: [Asrg] New Version Notification for draft-irtf-asrg-bcp-blacklists-07 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Jan 2011 12:00:24 -0000 --------------010104070308090002010005 Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit New and I hope final revision of the DNSBL BCP. https://datatracker.ietf.org/doc/draft-irtf-asrg-bcp-blacklists/ Section 2.2.3: more discussion on response times. new 2.2.5: conflict of interest 3.9: considerations for data corruption et. al. --------------010104070308090002010005 Content-Type: message/rfc822; name="New Version Notification for draft-irtf-asrg-bcp-blacklists-07.eml" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="New Version Notification for draft-irtf-asrg-bcp-blacklists-"; filename*1="07.eml" X-Account-Key: account1 X-Mozilla-Keys: Received: from zrtphxs1.corp.nortel.com (47.140.202.46) by zrtphx5h0.corp.nortel.com (47.140.202.65) with Microsoft SMTP Server id 8.1.340.0; Tue, 4 Jan 2011 06:39:18 -0500 Received: from zcarhxs1.corp.nortel.com ([47.129.230.89]) by zrtphxs1.corp.nortel.com with Microsoft SMTPSVC(6.0.3790.4675); Tue, 4 Jan 2011 06:39:18 -0500 Received: from zcars1ky.ca.nortel.com ([47.129.242.220]) by zcarhxs1.corp.nortel.com with Microsoft SMTPSVC(6.0.3790.4675); Tue, 4 Jan 2011 06:39:17 -0500 Received: from ecarh005.nortel.com (ecarh005.nortel.com [47.248.0.141]) by zcars1ky.ca.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id p04BdHK04950 for ; Tue, 4 Jan 2011 11:39:17 GMT Received: from mail.ietf.org (HELO mail.ietf.org) (64.170.98.32) by ecarh005.nortel.com (qpsmtpd/0.43rc1) with ESMTP; Tue, 04 Jan 2011 06:39:17 -0500 Received: by core3.amsl.com (Postfix, from userid 30) id DCEE33A69D1; Tue, 4 Jan 2011 03:37:08 -0800 (PST) From: IETF I-D Submission Tool To: "Lewis, Chris (CAR:W669)" CC: "msergeant@messagelabs.com" Date: Tue, 4 Jan 2011 06:37:08 -0500 Subject: New Version Notification for draft-irtf-asrg-bcp-blacklists-07 Thread-Topic: New Version Notification for draft-irtf-asrg-bcp-blacklists-07 Thread-Index: AcusBAWYvtmNoU1JQZ2Od73/7AS7ZA== Message-ID: <20110104113708.DCEE33A69D1@core3.amsl.com> Accept-Language: en-US Content-Language: en-US X-MS-Exchange-Organization-AuthAs: Internal X-MS-Exchange-Organization-AuthMechanism: 0b X-MS-Exchange-Organization-AuthSource: zrtphx5h0.corp.nortel.com X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originalarrivaltime: 04 Jan 2011 11:39:17.0702 (UTC) FILETIME=[052F3E60:01CBAC04] x-spam-status: No, hits=-2.3 required=7.0 tests=RCVD_IN_DNSWL_MED x-ntm-mail-from: wwwrun@core3.amsl.com x-ntm-helo: mail.ietf.org x-spam-check-by: ecarh005.nortel.com x-virus-checked: Checked by ClamAV on ecarh005.nortel.com x-ntm-peer-info: [64.170.98.32] x-ntm-id: 1294141156.144620000170@ecarh005 x-ntm-rcpt-to: x-spam-level: * Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 DQpBIG5ldyB2ZXJzaW9uIG9mIEktRCwgZHJhZnQtaXJ0Zi1hc3JnLWJjcC1ibGFja2xpc3RzLTA3 LnR4dCBoYXMgYmVlbiBzdWNjZXNzZnVsbHkgc3VibWl0dGVkIGJ5IENocmlzIExld2lzIGFuZCBw b3N0ZWQgdG8gdGhlIElFVEYgcmVwb3NpdG9yeS4NCg0KRmlsZW5hbWU6CSBkcmFmdC1pcnRmLWFz cmctYmNwLWJsYWNrbGlzdHMNClJldmlzaW9uOgkgMDcNClRpdGxlOgkJIE92ZXJ2aWV3IG9mIEVt YWlsIEROU0JMIEJlc3QgUHJhY3Rpc2UNCkNyZWF0aW9uX2RhdGU6CSAyMDExLTAxLTA0DQpXRyBJ RDoJCSBJbmRlcGVuZGVudCBTdWJtaXNzaW9uDQpOdW1iZXJfb2ZfcGFnZXM6IDIxDQoNCkFic3Ry YWN0Og0KVGhlIHJpc2Ugb2Ygc3BhbSBhbmQgb3RoZXIgYW50aS1zb2NpYWwgYmVoYXZpb3Igb24g dGhlIEludGVybmV0IGhhcw0KbGVkIHRvIHRoZSBjcmVhdGlvbiBvZiBzaGFyZWQgRE5TLWJhc2Vk IGxpc3RzICgiRE5TQkxzIikgb2YgSVANCmFkZHJlc3NlcyBvciBkb21haW4gbmFtZXMgaW50ZW5k ZWQgdG8gaGVscCBndWlkZSBlbWFpbCBmaWx0ZXJpbmcuDQpUaGlzIG1lbW8gc3VtbWFyaXplcyBn dWlkZWxpbmVzIG9mIGFjY2VwdGVkIGJlc3QgcHJhY3RpY2UgZm9yIHRoZQ0KbWFuYWdlbWVudCBv ZiBwdWJsaWMgRE5TQkxzIGJ5IHRoZWlyIG9wZXJhdG9ycyBhcyB3ZWxsIGFzIGZvciB0aGUNCnBy b3BlciB1c2Ugb2Ygc3VjaCBsaXN0cyBieSBtYWlsIHNlcnZlciBhZG1pbmlzdHJhdG9ycyAoRE5T QkwgdXNlcnMpLA0KYW5kIGl0IHByb3ZpZGVzIHVzZWZ1bCBiYWNrZ3JvdW5kIGZvciBib3RoIHBh cnRpZXMuICBJdCBpcyBub3QNCmludGVuZGVkIHRvIGFkdmlzZSBvbiB0aGUgdXRpbGl0eSBvciBl ZmZpY2FjeSBvZiBwYXJ0aWN1bGFyIEROU0JMcyBvcg0KdGhlIEROU0JMIGNvbmNlcHQgaW4gZ2Vu ZXJhbCwgbm9yIHRvIGFzc2lzdCBlbmQgdXNlcnMgd2l0aCBxdWVzdGlvbnMNCmFib3V0IHNwYW0u DQoNCkNvbW1lbnRzIGFuZCBkaXNjdXNzaW9uIG9mIHRoaXMgZG9jdW1lbnQgc2hvdWxkIGJlIGFk ZHJlc3NlZCB0byB0aGUNCmFzcmdAaWV0Zi5vcmcgbWFpbGluZyBsaXN0Lg0KICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgIA0KDQoNClRoZSBJRVRGIFNlY3JldGFyaWF0Lg0KDQoNCg== --------------010104070308090002010005-- From davidnicol@gmail.com Tue Jan 4 08:26:12 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id CB7793A6CE6 for ; Tue, 4 Jan 2011 08:26:12 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.876 X-Spam-Level: X-Spam-Status: No, score=-2.876 tagged_above=-999 required=5 tests=[AWL=-0.278, BAYES_00=-2.599, HTML_MESSAGE=0.001] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BeaAcjwk6UcL for ; Tue, 4 Jan 2011 08:26:11 -0800 (PST) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by core3.amsl.com (Postfix) with ESMTP id 9534A3A6CC0 for ; Tue, 4 Jan 2011 08:26:11 -0800 (PST) Received: by iyb26 with SMTP id 26so13683528iyb.13 for ; Tue, 04 Jan 2011 08:28:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:mime-version:received:in-reply-to :references:from:date:message-id:subject:to:content-type; bh=9F72d2Uk82oDMrjpHbLBh+vOgW2rLLSdcWp0gEP6Fys=; b=DxkGMAAQ5fgqzwk4QZ8NZbnqYDrsPbMGjT6MV6k3NiDft8o5VnZcPig2KnP8Zahf1U Oe3reLLBWZwx5fvdZW+XFkp8ulHlyi3Os8WxmEqkFSPKSsHSb+ZExOhz80Ff6WtaxgaF Ps5ODjFX6vMvdh19fewOGZbubVtd/kiZzdYUs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; b=BBJBbFlpZ8NsA50EDfiTEbVkt05SJtMMMjHyRHqHpdfqWJDGGVxsZXcuFgiHOn0Ku4 Di/wDN/rXbLnclv3+dthniRyb7Li1luMkKPpChaNvAVChWsBxzKRAfrTCr2wPWEiHvJB KdY8TPyxpigicZLr0JUkmApvc0L3SkaW6CW04= Received: by 10.231.35.3 with SMTP id n3mr10894786ibd.90.1294158498520; Tue, 04 Jan 2011 08:28:18 -0800 (PST) MIME-Version: 1.0 Received: by 10.231.206.148 with HTTP; Tue, 4 Jan 2011 08:27:58 -0800 (PST) In-Reply-To: References: From: David Nicol Date: Tue, 4 Jan 2011 10:27:58 -0600 Message-ID: To: Anti-Spam Research Group - IRTF Content-Type: multipart/alternative; boundary=0022152d63899660a7049907c159 Subject: Re: [Asrg] draft-levine-iprangepub-01 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Jan 2011 16:26:12 -0000 --0022152d63899660a7049907c159 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable I've been meaning to share a summary of December's discussions as they seemed to me, so here it is: The problem is extending listing protocol to support listing of a range as well as a point, hopefully without sacrificing too many data bits. The current DNSxL protocol SHOULD support ranges of no-data information, given properly implemented NXDOMAIN handling and presentation of the appropriate glue. Alternate listing protocols (such as the ones I presented, and Levine's b-tree proposal, which could easily extend to other information domains tha= t can be represented as a b-tree, which is, any mapping with sortable keys, such as (not to encourage black-hat behavior, but it's a good example of data as such) a rainbow table) I pursued a way of packing queries using natural bit flipping instead of explicit leaf ranges until it became clear that my end result was equivalen= t to rfc5782 representation of ipv6 address names, with the addition of supporting shorter names for ranges. This resulted in my envisioning what could be a protocol to compete with draft-levine-iprangepub-01 in this space, amounting to a couple of extensions to the 5782 protocol. In short, when there is a range at a.b.c.rangelister.example, all queries i= n subdomains of the range-representing entry, such as t.m.i.a.b.c.rangelister.example, get responded to with an NXDOMAIN error, while the range-representing entry holds the data for the range. This means that a list query library would start with the full query and work its way up until it gets a result, and the NXDOMAIN at the appropriate part of the glue determination process will prevent the too-long queries from reaching the server, as they will be handled by the cache server. As a policy enforcement measure, a server should identify query sources not respecting NXDOMAIN responses and block them. The above is intended as a "representing of ranges" extension to RFC 5782. On Wed, Dec 29, 2010 at 10:48 AM, John R. Levine wrote: > I've done another version of it, with mostly editorial changes. Comments > as always welcome. I guess now I should implement it. > > http://tools.ietf.org/html/draft-levine-iprangepub-01 > --=20 =E2=80=9CThe aeroplane is fatally defective. It is merely a toy=E2=80=94a s= porting play-thing. It can never become commercially practical." -- Nikola Tesla --0022152d63899660a7049907c159 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

I've been meaning to share a summary of December's d= iscussions as they seemed to me, so here it is:

The prob= lem is extending listing protocol to support listing of a range as well as = a point, hopefully without sacrificing too many data bits.

The current DNSxL protocol SHOULD support ranges of no-= data information, given properly implemented NXDOMAIN handling and presenta= tion of the appropriate glue.

Alternate listing pr= otocols (such as the ones I presented, and Levine's b-tree proposal, wh= ich could easily extend to other information domains that can be represente= d as a b-tree, which is, any mapping with sortable keys, such as (not to en= courage black-hat behavior, but it's a good example of data as such) a = rainbow table)=C2=A0

I pursued a way of packing queries using natural bit fl= ipping instead of explicit leaf ranges until it became clear that my end re= sult was equivalent to rfc5782 representation of ipv6 address names, with t= he addition of supporting shorter names for ranges.

This resulted in my envisioning what could be a protoco= l to compete with=C2=A0draft-levine-iprangepub-01=C2=A0in this= space, amounting to a couple of extensions to the 5782 protocol.

In short, when there is a range at a.b.c.rangelister.ex= ample, all queries in subdomains of the range-representing entry, such as t= .m.i.a.b.c.rangelister.example, get responded to with an NXDOMAIN error, wh= ile the range-representing entry holds the data for the range.

This means that a list query library would start with t= he full query and work its way up until it gets a result, and the NXDOMAIN = at the appropriate part of the glue determination process will prevent the = too-long queries from reaching the server, as they will be handled by the c= ache server.

As a policy enforcement measure, a server should identi= fy query sources not respecting NXDOMAIN responses and block them.

The above is intended as a "representing of ranges&qu= ot; extension to RFC 5782.



On Wed, Dec 29, 2010 at 10:48 AM, John R. Levine <johnl@iecc.com> wrote:
I've done another version of it, with mostly editorial changes. =C2=A0C= omments as always welcome. =C2=A0I guess now I should implement it.

http://tools.ietf.org/html/draft-levine-iprangepub-01
--
=E2=80=9CThe aeroplane is fatally defective. It is mer= ely a toy=E2=80=94a sporting play-thing. =C2=A0It can never become commerci= ally practical." -- Nikola Tesla
--0022152d63899660a7049907c159-- From johnl@iecc.com Tue Jan 4 13:01:20 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 6C99F3A6B8A for ; Tue, 4 Jan 2011 13:01:20 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -111.147 X-Spam-Level: X-Spam-Status: No, score=-111.147 tagged_above=-999 required=5 tests=[AWL=0.052, BAYES_00=-2.599, HABEAS_ACCREDITED_SOI=-4.3, RCVD_IN_BSP_TRUSTED=-4.3, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LZntYHBlUyu0 for ; Tue, 4 Jan 2011 13:01:19 -0800 (PST) Received: from gal.iecc.com (gal.iecc.com [64.57.183.53]) by core3.amsl.com (Postfix) with ESMTP id 23CCE3A6B86 for ; Tue, 4 Jan 2011 13:01:19 -0800 (PST) Received: (qmail 36819 invoked from network); 4 Jan 2011 21:03:24 -0000 Received: from mail1.iecc.com (64.57.183.56) by mail1.iecc.com with QMQP; 4 Jan 2011 21:03:24 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=iecc.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=5452.4d238b1c.k1101; i=johnl@user.iecc.com; bh=pJG+V69N9K7XXQZoSRJgt0do0S/N4y9YaQeyCXzXSWk=; b=voyMspI7x/+u508bjkvdPcDRi485LtEJdseomXd4UyAxjZ+6o+zVCEmKqIKQ2LQahhyazD6Qd+CD0GRLonz8Bk22io18uZSr7U0SuXCiO44gC6Gh0XjESy11E5xpM4KXKLrhKkWuWuZI1Qrr8WDU24aOpsyFy3JlSyD3Vby0kCI= DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=taugh.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=5452.4d238b1c.k1101; olt=johnl@user.iecc.com; bh=pJG+V69N9K7XXQZoSRJgt0do0S/N4y9YaQeyCXzXSWk=; b=dWc9eJ0+YQ0QRSygdb1EcwFYinQQS5qxp1O7pUhfi5QlBKK9jUE2mkdnFU/X4I4FORKCinUY2e1/mFdJINXiKCGpMeCHQKcJZ3jlUt7vP1rJLtSo9N0gOw6JRH7pibOYtBnksRRojOqAhAD6XDG9R1zJxsshkzrNaIsda2+pWT0= VBR-Info: md=iecc.com; mc=all; mv=dwl.spamhaus.org Date: 4 Jan 2011 21:03:24 -0000 Message-ID: <20110104210324.21585.qmail@joyce.lan> From: John Levine To: asrg@irtf.org In-Reply-To: Organization: Cc: X-Headerized: yes Mime-Version: 1.0 Content-type: text/plain; charset=utf-8 Content-transfer-encoding: 7bit Subject: Re: [Asrg] draft-levine-iprangepub-01 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Jan 2011 21:01:20 -0000 Thanks for the summary. >The current DNSxL protocol SHOULD support ranges of no-data >information, given properly implemented NXDOMAIN handling and >presentation of the appropriate glue. I don't think this is right, unless you're assuming that DNS caches will do stuff they don't do now. >This means that a list query library would start with the full query and >work its way up until it gets a result, and the NXDOMAIN at the appropriate >part of the glue determination process will prevent the too-long queries >from reaching the server, as they will be handled by the cache server. You might want to try this with a few caches. I'd be surprised if they synthesized answers for subdomains of NXDOMAIN entries. I just tried it with dnscache from djbdns, and it doesn't. R's, John From CLEWIS@nortel.com Tue Jan 4 14:11:03 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 1CC073A6C32 for ; Tue, 4 Jan 2011 14:11:00 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.399 X-Spam-Level: X-Spam-Status: No, score=-6.399 tagged_above=-999 required=5 tests=[AWL=0.200, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id a2rYQm1lCMWt for ; Tue, 4 Jan 2011 14:10:58 -0800 (PST) Received: from zcars04e.nortel.com (zcars04e.nortel.com [47.129.242.56]) by core3.amsl.com (Postfix) with ESMTP id D2A0E3A6BD0 for ; Tue, 4 Jan 2011 14:10:57 -0800 (PST) Received: from zrtphxs1.corp.nortel.com (casmtp.ca.nortel.com [47.140.202.46]) by zcars04e.nortel.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id p04MD1K27293 for ; Tue, 4 Jan 2011 22:13:01 GMT Received: from zrtphx5h0.corp.nortel.com ([47.140.202.65]) by zrtphxs1.corp.nortel.com with Microsoft SMTPSVC(6.0.3790.4675); Tue, 4 Jan 2011 17:13:01 -0500 Received: from [47.130.80.33] (47.130.80.33) by zrtphx5h0.corp.nortel.com (47.140.202.65) with Microsoft SMTP Server (TLS) id 8.1.340.0; Tue, 4 Jan 2011 17:13:00 -0500 Message-ID: <4D239B6B.2070304@nortel.com> Date: Tue, 4 Jan 2011 17:12:59 -0500 From: "Chris Lewis" Organization: Nortel User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.23) Gecko/20090812 Thunderbird/2.0.0.23 Mnenhy/0.7.6.666 MIME-Version: 1.0 To: References: <20110104210324.21585.qmail@joyce.lan> In-Reply-To: <20110104210324.21585.qmail@joyce.lan> Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 04 Jan 2011 22:13:01.0553 (UTC) FILETIME=[8D2A7610:01CBAC5C] Subject: Re: [Asrg] draft-levine-iprangepub-01 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Jan 2011 22:11:03 -0000 On 1/4/2011 4:03 PM, John Levine wrote: > You might want to try this with a few caches. I'd be surprised if > they synthesized answers for subdomains of NXDOMAIN entries. I just > tried it with dnscache from djbdns, and it doesn't. After having used dnscache for quite a while, with, er, not very satisfactory performance results[+], it might be worth while retrying the experiment with bind and/or, say, unbound (which is what I use now) which are better suited to heavy iron caching. While dnscache may indeed be quite popular at the smallish scale it don't do well at the medium to very large end. Which means that the problems it may represent for IPv6 caching may be common on a per-site level, but not common at a sheer volume level, and may not matter _that_ much overall. [+] The reasons as to why are OT, but I can elaborate if necessary. From johnl@iecc.com Tue Jan 4 17:52:58 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id E16CE3A677D for ; Tue, 4 Jan 2011 17:52:58 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -111.149 X-Spam-Level: X-Spam-Status: No, score=-111.149 tagged_above=-999 required=5 tests=[AWL=0.050, BAYES_00=-2.599, HABEAS_ACCREDITED_SOI=-4.3, RCVD_IN_BSP_TRUSTED=-4.3, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vXAqyjG2qZG7 for ; Tue, 4 Jan 2011 17:52:57 -0800 (PST) Received: from gal.iecc.com (gal.iecc.com [64.57.183.53]) by core3.amsl.com (Postfix) with ESMTP id 405F93A67D8 for ; Tue, 4 Jan 2011 17:52:57 -0800 (PST) Received: (qmail 85846 invoked from network); 5 Jan 2011 01:55:03 -0000 Received: from mail1.iecc.com (64.57.183.56) by mail1.iecc.com with QMQP; 5 Jan 2011 01:55:03 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=iecc.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=1bac.4d23cf77.k1101; i=johnl@user.iecc.com; bh=Qbde3orijnx4Uidup93sRrC+F2WrTRBPhFQLft/RvBo=; b=ABvoTZ7A6D3ohgSOKCTAhQELX/mX2BNTt53jVKxVBjM7+b99dmeFO7ORvi2bLX4LenXcBUgSjnORi3qgR0r46XuoOIbegH4NGXtfkwBA0p1AYFVGSw6uZVgaTD+qJ/AL2goaghj7BQD3X04vCcAyb9XVIg/0++JedVBdndvQnxQ= DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=taugh.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=1bac.4d23cf77.k1101; olt=johnl@user.iecc.com; bh=Qbde3orijnx4Uidup93sRrC+F2WrTRBPhFQLft/RvBo=; b=plExL1QVc5jbqE9BlQeo30Tx3epzKiewi18bZKj+pLnHTOjwxRuP814Nu2z57Dc8Fe8BHDSkaSOhGM+M64TK4ZlRsPrfQdCBb8ESFHgAbbqpmBjb0OkOHjZSv4SlV9G0fqtKNGsoYLW7V+hiMHSi4r/tJ1QRNjs7V4l8TvnOHro= VBR-Info: md=iecc.com; mc=all; mv=dwl.spamhaus.org Date: 5 Jan 2011 01:55:03 -0000 Message-ID: <20110105015503.7083.qmail@joyce.lan> From: John Levine To: asrg@irtf.org In-Reply-To: <4D239B6B.2070304@nortel.com> Organization: Cc: X-Headerized: yes Mime-Version: 1.0 Content-type: text/plain; charset=utf-8 Content-transfer-encoding: 7bit Subject: Re: [Asrg] NXDOMAIN cache behavior, was draft-levine-iprangepub-01 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 01:52:59 -0000 In article <4D239B6B.2070304@nortel.com> you write: >On 1/4/2011 4:03 PM, John Levine wrote: > >> You might want to try this with a few caches. I'd be surprised if >> they synthesized answers for subdomains of NXDOMAIN entries. I just >> tried it with dnscache from djbdns, and it doesn't. > >After having used dnscache for quite a while, with, er, not very >satisfactory performance results[+], it might be worth while retrying >the experiment with bind and/or, say, unbound (which is what I use now) >which are better suited to heavy iron caching. Grumble. I'll build unbound and see what it does, but I would be rather surprised if the results were any different. The DNS crowd has dislikes synthesizing answers, even in cases like this where you can argue that the synthesized answer should be correct. R's, John From johnl@taugh.com Tue Jan 4 18:13:27 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 6D2063A67E5 for ; Tue, 4 Jan 2011 18:13:27 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -11.15 X-Spam-Level: X-Spam-Status: No, score=-11.15 tagged_above=-999 required=5 tests=[AWL=0.049, BAYES_00=-2.599, HABEAS_ACCREDITED_SOI=-4.3, RCVD_IN_BSP_TRUSTED=-4.3] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UnSzMFJBnn+V for ; Tue, 4 Jan 2011 18:13:26 -0800 (PST) Received: from gal.iecc.com (gal.iecc.com [64.57.183.53]) by core3.amsl.com (Postfix) with ESMTP id 40CFD3A67D8 for ; Tue, 4 Jan 2011 18:13:26 -0800 (PST) Received: (qmail 88714 invoked from network); 5 Jan 2011 02:15:32 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=iecc.com; h=date:message-id:from:to:subject:in-reply-to:references:mime-version:content-type:vbr-info:user-agent:cleverness; s=15a89.4d23d444.k1101; i=johnl@submit.iecc.com; bh=BaiWExgjLAlAqMNFoPCtz5y12uotkuWjddYhwRmGXcs=; b=W2YEiXd7GSQ4s7ERMpCyYjOkOoUBdwQrnemiUHtE91MmK9sg0qfv1+KjdKDuXuypDNj9EqNXuV6LqOKwSkZCa3Fi5qteKa7QHc0IqBVpWX+ym/6jkE8YLRJlb+WqdyK7cWEP6UwoFMn20fiUnpaQwSl7HKWrKK+Nud5WpELK0wE= DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=taugh.com; h=date:message-id:from:to:subject:in-reply-to:references:mime-version:content-type:vbr-info:user-agent:cleverness; s=15a89.4d23d444.k1101; olt=johnl@submit.iecc.com; bh=BaiWExgjLAlAqMNFoPCtz5y12uotkuWjddYhwRmGXcs=; b=BJ/73VasBeBJbOxX1mlhKRHneE0+X3O/uUS80XYrHCppZjGmEz/nYulQ7P0dto76Yo0gpBbD9lDT6EI+c6EAVQ5lm3XcP7NoD0bTRK5XQ3oPUlyYbZ1HS+UN92as2fsyB0D9qX2/9Yv/EyIiZvkVpSG+bNaAU/oEAqwgUDhrShQ= VBR-Info: md=iecc.com; mc=all; mv=dwl.spamhaus.org Received: (ofmipd johnl@64.57.183.62) with (DHE-RSA-AES256-SHA encrypted) SMTP; 5 Jan 2011 02:15:10 -0000 Date: 4 Jan 2011 21:15:32 -0500 Message-ID: From: "John R Levine" To: "Anti Spam Research Group" In-Reply-To: <20110105015503.7083.qmail@joyce.lan> References: <20110105015503.7083.qmail@joyce.lan> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) Cleverness: None detected MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Subject: Re: [Asrg] NXDOMAIN cache behavior, was draft-levine-iprangepub-01 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 02:13:27 -0000 >>> You might want to try this with a few caches. I'd be surprised if >>> they synthesized answers for subdomains of NXDOMAIN entries. I just >>> tried it with dnscache from djbdns, and it doesn't. ... > Grumble. I'll build unbound and see what it does, but I would be > rather surprised if the results were any different. I wasn't surprised. I looked for foo.gurus.org, got an NXDOMAIN, then queried bar.foo.gurus.org, and it went back to the authoritative server to get the answer. Nice cache, though. Maybe I'll give it a try. Regards, John Levine, johnl@taugh.com, Taughannock Networks, Trumansburg NY "I dropped the toothpaste", said Tom, crestfallenly. From CLEWIS@nortel.com Tue Jan 4 18:22:33 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 1FC873A67E5 for ; Tue, 4 Jan 2011 18:22:33 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.466 X-Spam-Level: X-Spam-Status: No, score=-6.466 tagged_above=-999 required=5 tests=[AWL=0.133, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AI02NvOYQxKQ for ; Tue, 4 Jan 2011 18:22:32 -0800 (PST) Received: from zrtps0kp.nortel.com (zrtps0kp.nortel.com [47.140.192.56]) by core3.amsl.com (Postfix) with ESMTP id 0CE113A677D for ; Tue, 4 Jan 2011 18:22:31 -0800 (PST) Received: from zrtphxs1.corp.nortel.com (zrtphxs1.corp.nortel.com [47.140.202.46]) by zrtps0kp.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id p052OXt10298 for ; Wed, 5 Jan 2011 02:24:34 GMT Received: from zrtphx5h0.corp.nortel.com ([47.140.202.65]) by zrtphxs1.corp.nortel.com with Microsoft SMTPSVC(6.0.3790.4675); Tue, 4 Jan 2011 21:24:34 -0500 Received: from [47.130.80.33] (47.130.80.33) by zrtphx5h0.corp.nortel.com (47.140.202.65) with Microsoft SMTP Server (TLS) id 8.1.340.0; Tue, 4 Jan 2011 21:24:33 -0500 Message-ID: <4D23D65E.3060605@nortel.com> Date: Tue, 4 Jan 2011 21:24:30 -0500 From: "Chris Lewis" Organization: Nortel User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.23) Gecko/20090812 Thunderbird/2.0.0.23 Mnenhy/0.7.6.666 MIME-Version: 1.0 To: References: <20110105015503.7083.qmail@joyce.lan> In-Reply-To: <20110105015503.7083.qmail@joyce.lan> Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 05 Jan 2011 02:24:34.0033 (UTC) FILETIME=[B0FC2610:01CBAC7F] Subject: Re: [Asrg] NXDOMAIN cache behavior, was draft-levine-iprangepub-01 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 02:22:33 -0000 On 1/4/2011 8:55 PM, John Levine wrote: > Grumble. I'll build unbound and see what it does, but I would be > rather surprised if the results were any different. The DNS crowd > has dislikes synthesizing answers, even in cases like this where > you can argue that the synthesized answer should be correct. I get the impression that the unbound writers were a bit more pragmatic w.r.t. performance versus personal theories of "purity" than, say, DJB. [The dnscache performance issue is just plain dumb.] You may well be right, but I'd hesitate to use dnscache's behaviour as representative without actually testing something else. I have several unbound cache servers that might be easy to make visible on the net if you can't get your own unbound up quick. Unbound seems to be _really_ easy to set up tho. From johnl@iecc.com Tue Jan 4 19:54:56 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 4D3683A68DA for ; Tue, 4 Jan 2011 19:54:56 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -110.666 X-Spam-Level: X-Spam-Status: No, score=-110.666 tagged_above=-999 required=5 tests=[AWL=0.533, BAYES_00=-2.599, HABEAS_ACCREDITED_SOI=-4.3, RCVD_IN_BSP_TRUSTED=-4.3, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id L9aAb5G29SCn for ; Tue, 4 Jan 2011 19:54:53 -0800 (PST) Received: from gal.iecc.com (gal.iecc.com [64.57.183.53]) by core3.amsl.com (Postfix) with ESMTP id B01E33A6948 for ; Tue, 4 Jan 2011 19:54:52 -0800 (PST) Received: (qmail 2147 invoked from network); 5 Jan 2011 03:56:58 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=iecc.com; h=date:message-id:from:to:subject:mime-version:content-type:vbr-info:user-agent:cleverness; s=862.4d23ec0a.k1101; i=johnl@submit.iecc.com; bh=YkZdClBTuGNL6Q4034lFI8aKiyVS2rbplnCOQU6uGb0=; b=nb+2ditI43UEr3tHzQQSOrPFTdY/xVtL8JRrt9zowSKW+jomaasLmo/n9bZwV8D3jrENnR4uqnvWSUVVyKcbzuBDXr0r9ZO55rrEInt6P/cD3/2tMA/2WW4CIpMk3xGddR/FlcSCkpakvAoDRB5dBaqC+rbaEJZ0zaENshzz5KQ= VBR-Info: md=iecc.com; mc=all; mv=dwl.spamhaus.org Received: (ofmipd johnl@64.57.183.62) with (DHE-RSA-AES256-SHA encrypted) SMTP; 5 Jan 2011 03:56:36 -0000 Date: 4 Jan 2011 22:56:57 -0500 Message-ID: From: "John R. Levine" To: "Anti Spam Research Group" User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) Cleverness: None detected MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII Subject: [Asrg] How well do DNSxL lookups cache ? X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 03:54:56 -0000 Over on the Spamassassin users' list, we've been having a spirited debate about the design of DNSBLs and WLs for IPv6, starting with my draft. One of the key questions is how DNSxL lookups will play with DNS caches and servers, which of course depends on the pattern of IP addresses that send mail. David Skoll collected a few stats from his servers, which suggested that for low volume mail servers, only about 20% of lookups are answered from the cache, and for a largish server handling 1M connections/day, it was only about 2/3. That seemed hard to believe, so I took a week of logs from my mail server, did a similar analysis, and by golly, he's right. The analysis takes timestamps and IP addresses, and computes how many DNSxL queries would be answered from a DNS cache under various scenarios. Each group shows the number of cache hits and total queries by 24 hour period. (The first and last are short, since my logs didn't start and end at midnight.) Then it shows the number of unique IPs queried, and a list showing cache effectiveness, with the number of times each cache entry was used, e.g., in the first dataset, there was one cache entry used 514 times, down to 536,009 entries used only once. To see how sensitive these answers were to DNSxL design, I tried it with a 15 minute and 60 minute TTL, and I tried with /24 rather than /32 granularity. Neither made much difference: the cache effectiveness rarely got above 20% and the vast majority of cache entries were never reused. So now I'm scratching my head. If these numbers are typical, I don't understand why DNSBL servers don't all fall over now. My analyses use a small perl script which you can have if you have logs you'd like to try it on. All it needs for input is a series of IP addresses and timestamps. It extracts them from the peculiar log format that mailfront and daemontools produce, but adjusting the input format is easy. I'll see if I can get some data from larger mail systems and see how different the answers are. Regards, John Levine, johnl@iecc.com, Primary Perpetrator of "The Internet for Dummies", Please consider the environment before reading this e-mail. http://jl.ly --- 15 minute TTL --- 7078 of 56237 ( 12 %) Sat Dec 25 00:00:00 2010 6146 of 45213 ( 13 %) Sun Dec 26 00:00:00 2010 9398 of 67167 ( 13 %) Mon Dec 27 00:00:00 2010 12155 of 76827 ( 15 %) Tue Dec 28 00:00:00 2010 10845 of 83165 ( 13 %) Wed Dec 29 00:00:00 2010 8781 of 73894 ( 11 %) Thu Dec 30 00:00:00 2010 10240 of 59080 ( 17 %) Fri Dec 31 00:00:00 2010 7753 of 51736 ( 14 %) Sat Jan 1 00:00:00 2011 7918 of 53362 ( 14 %) Sun Jan 2 00:00:00 2011 8440 of 63016 ( 13 %) Mon Jan 3 00:00:00 2011 9022 of 56342 ( 16 %) Tue Jan 4 00:00:00 2011 Total IPs = 358782 hits count 514 1 423 1 420 1 415 1 403 1 378 1 373 1 369 1 368 1 366 2 309 1 283 1 280 1 269 1 267 1 266 1 265 1 257 2 251 1 247 1 246 1 244 1 239 1 223 1 220 1 218 1 208 1 203 1 199 1 197 1 195 1 193 1 181 2 175 1 173 1 168 1 166 1 162 1 160 1 159 2 157 1 156 1 149 1 148 1 145 1 141 1 135 1 134 1 133 2 132 1 131 1 127 1 124 3 122 1 120 1 119 1 118 1 115 1 113 1 111 1 110 1 106 1 105 1 104 1 103 1 100 12 99 4 98 4 97 2 95 3 94 1 93 2 92 2 91 1 90 1 89 1 87 1 86 2 81 1 79 2 77 1 75 1 73 1 72 1 68 4 67 1 66 2 65 1 64 1 62 2 61 1 59 5 58 4 57 2 56 5 54 2 53 3 52 2 51 3 50 7 48 1 46 1 45 3 43 1 42 3 41 3 40 1 39 1 38 2 37 2 36 3 32 1 31 4 30 9 29 9 28 8 27 6 26 2 25 5 24 10 23 6 22 7 21 5 20 53 19 9 18 5 17 10 16 17 15 19 14 35 13 26 12 49 11 60 10 69 9 113 8 227 7 185 6 411 5 762 4 1918 3 6599 2 41434 1 536009 --- 60 minute TTL --- 9652 of 56237 ( 17 %) Sat Dec 25 00:00:00 2010 8364 of 45213 ( 18 %) Sun Dec 26 00:00:00 2010 13141 of 67167 ( 19 %) Mon Dec 27 00:00:00 2010 15976 of 76827 ( 20 %) Tue Dec 28 00:00:00 2010 15708 of 83165 ( 18 %) Wed Dec 29 00:00:00 2010 12886 of 73894 ( 17 %) Thu Dec 30 00:00:00 2010 13266 of 59080 ( 22 %) Fri Dec 31 00:00:00 2010 10442 of 51736 ( 20 %) Sat Jan 1 00:00:00 2011 10391 of 53362 ( 19 %) Sun Jan 2 00:00:00 2011 11559 of 63016 ( 18 %) Mon Jan 3 00:00:00 2011 11876 of 56342 ( 21 %) Tue Jan 4 00:00:00 2011 Total IPs = 358782 hits count 514 1 423 1 420 1 415 1 411 1 403 1 378 1 369 1 368 1 366 2 309 1 283 1 280 1 270 1 269 2 267 1 266 1 265 1 259 1 257 2 251 1 248 1 246 1 244 1 239 1 223 1 220 1 219 1 208 1 203 1 199 1 197 1 195 1 193 1 186 1 181 2 175 1 173 1 168 1 162 1 160 1 159 2 157 1 154 1 149 1 148 1 145 1 141 1 135 1 134 1 133 2 132 1 131 1 124 3 122 1 120 1 119 1 117 1 115 1 111 1 110 1 106 1 104 1 103 1 100 12 99 4 98 4 97 2 95 2 94 1 93 1 92 2 91 1 90 1 89 1 87 1 86 2 85 1 83 1 81 1 79 2 77 1 75 1 74 1 73 1 72 1 69 1 68 3 67 1 66 2 65 2 64 1 62 2 61 2 59 2 58 4 57 2 56 6 54 2 53 2 52 2 51 3 50 7 48 1 47 1 46 2 45 4 43 1 42 3 41 2 40 2 39 2 38 1 37 3 36 4 35 2 34 1 33 2 32 2 31 2 30 8 29 10 28 8 27 3 26 3 25 6 24 9 23 8 22 9 21 12 20 60 19 8 18 13 17 19 16 20 15 31 14 42 13 37 12 58 11 65 10 89 9 149 8 302 7 311 6 656 5 1367 4 3209 3 10400 2 58038 1 477633 --- 15 minute TTL, combine /24s --- 7828 of 56237 ( 13 %) Sat Dec 25 00:00:00 2010 6837 of 45213 ( 15 %) Sun Dec 26 00:00:00 2010 10614 of 67167 ( 15 %) Mon Dec 27 00:00:00 2010 13518 of 76827 ( 17 %) Tue Dec 28 00:00:00 2010 12430 of 83165 ( 14 %) Wed Dec 29 00:00:00 2010 10142 of 73894 ( 13 %) Thu Dec 30 00:00:00 2010 11045 of 59080 ( 18 %) Fri Dec 31 00:00:00 2010 8433 of 51736 ( 16 %) Sat Jan 1 00:00:00 2011 8712 of 53362 ( 16 %) Sun Jan 2 00:00:00 2011 9342 of 63016 ( 14 %) Mon Jan 3 00:00:00 2011 9830 of 56342 ( 17 %) Tue Jan 4 00:00:00 2011 Total IPs = 186854 hits count 514 1 423 1 420 1 415 1 403 1 378 1 373 1 369 1 368 1 366 2 309 1 283 1 280 1 269 1 267 1 266 1 265 1 257 2 251 1 247 1 246 1 244 1 239 1 223 1 220 1 218 1 208 1 203 1 199 1 197 1 195 1 193 1 181 2 175 1 173 1 168 1 166 1 162 1 160 1 159 2 157 1 156 1 149 1 148 1 145 1 141 1 135 1 134 1 133 2 132 1 131 1 127 1 124 3 122 1 120 1 119 1 118 1 115 1 113 1 111 1 110 1 106 1 105 1 104 1 103 1 100 12 99 4 98 4 97 2 95 3 94 1 93 2 92 3 90 1 89 1 87 1 86 2 81 1 79 2 77 1 75 1 73 1 72 1 68 4 67 1 66 2 65 1 64 1 62 2 61 1 59 5 58 4 57 2 56 5 54 2 53 4 52 1 51 3 50 7 48 1 46 1 45 3 43 1 42 4 41 3 40 1 39 1 38 2 37 2 36 3 35 1 32 3 31 3 30 9 29 9 28 10 27 6 26 5 25 3 24 10 23 7 22 8 21 7 20 55 19 12 18 13 17 17 16 21 15 27 14 43 13 35 12 66 11 86 10 93 9 165 8 288 7 264 6 549 5 954 4 2253 3 7508 2 45131 1 519465 --- 60 minute TTL, combine /24s --- 11210 of 56237 ( 19 %) Sat Dec 25 00:00:00 2010 9595 of 45213 ( 21 %) Sun Dec 26 00:00:00 2010 15383 of 67167 ( 22 %) Mon Dec 27 00:00:00 2010 18675 of 76827 ( 24 %) Tue Dec 28 00:00:00 2010 18714 of 83165 ( 22 %) Wed Dec 29 00:00:00 2010 15380 of 73894 ( 20 %) Thu Dec 30 00:00:00 2010 14832 of 59080 ( 25 %) Fri Dec 31 00:00:00 2010 11827 of 51736 ( 22 %) Sat Jan 1 00:00:00 2011 11987 of 53362 ( 22 %) Sun Jan 2 00:00:00 2011 13420 of 63016 ( 21 %) Mon Jan 3 00:00:00 2011 13543 of 56342 ( 24 %) Tue Jan 4 00:00:00 2011 Total IPs = 186854 hits count 514 1 423 1 420 1 415 1 411 1 403 1 378 1 370 1 368 1 366 2 309 1 283 1 280 1 270 1 269 2 267 1 266 1 265 1 259 1 257 2 251 1 248 1 246 1 244 1 239 1 223 1 220 1 219 1 208 1 203 1 199 1 197 1 195 1 193 1 186 1 181 2 175 1 173 1 168 1 162 1 160 1 159 2 157 1 154 1 149 1 148 1 145 1 141 1 135 1 134 1 133 2 132 1 131 1 125 1 124 3 121 1 120 1 119 1 115 1 111 1 110 1 106 1 104 1 103 1 100 12 99 4 98 4 97 2 95 2 94 1 93 1 92 3 90 1 89 1 87 2 86 2 85 1 81 1 79 2 77 1 75 1 74 1 73 1 72 1 69 1 68 3 67 2 66 2 65 2 64 1 62 2 61 2 59 2 58 4 57 2 56 5 54 3 53 3 52 1 51 3 50 7 48 2 47 1 46 3 45 4 44 1 43 1 42 4 41 3 40 6 39 2 38 3 37 4 36 6 35 5 34 2 33 1 32 5 31 7 30 10 29 19 28 9 27 11 26 10 25 9 24 12 23 13 22 17 21 19 20 70 19 16 18 24 17 36 16 40 15 47 14 63 13 62 12 79 11 84 10 125 9 203 8 346 7 394 6 859 5 1666 4 4022 3 12676 2 64013 1 446298 From steve@blighty.com Tue Jan 4 20:04:11 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 057793A67E6 for ; Tue, 4 Jan 2011 20:04:11 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id q5MwQ9Ulwey8 for ; Tue, 4 Jan 2011 20:04:09 -0800 (PST) Received: from m.wordtothewise.com (b.wordtothewise.com [208.187.80.136]) by core3.amsl.com (Postfix) with ESMTP id E23C73A6A2C for ; Tue, 4 Jan 2011 20:04:09 -0800 (PST) Received: from platter.wordtothewise.com (184.wordtothewise.com [208.187.80.184]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: steve) by m.wordtothewise.com (Postfix) with ESMTPSA id 02D122DDD9 for ; Tue, 4 Jan 2011 20:06:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wordtothewise.com; s=1.wttw; t=1294200377; bh=SCFrfdYhLSDiM5G4uGKP1ILjSLzbqD60qnFDUX4qMOw=; h=Content-Type:Mime-Version:Subject:From:In-Reply-To:Date: Content-Transfer-Encoding:Message-Id:References:To; b=CIdj3QdCLzeRn3P9ahGbJmuicqmAeCaHLfkmAs9WaEA/A/ectr8CU79CjPtCK9eAi dTuRwsMmKaO2FtWFcHPXXgF7mU5UDEbpaW21rNEGaUgzvpXmUjqo4gQizctELIyVJ9 pmJL2ZIIBW/QRLdsiIc6161yRiFAAMW/kObOTzt4= Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Apple Message framework v1082) From: Steve Atkins In-Reply-To: Date: Tue, 4 Jan 2011 20:06:16 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <17BA27A9-2362-4DF5-A309-EF2C282B4538@blighty.com> References: To: Anti-Spam Research Group - IRTF X-Mailer: Apple Mail (2.1082) Subject: Re: [Asrg] How well do DNSxL lookups cache ? X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 04:04:11 -0000 On Jan 4, 2011, at 7:56 PM, John R. Levine wrote: > Over on the Spamassassin users' list, we've been having a spirited = debate about the design of DNSBLs and WLs for IPv6, starting with my = draft. >=20 > One of the key questions is how DNSxL lookups will play with DNS = caches and servers, which of course depends on the pattern of IP = addresses that send mail. David Skoll collected a few stats from his = servers, which suggested that for low volume mail servers, only about = 20% of lookups are answered from the cache, and for a largish server = handling 1M connections/day, it was only about 2/3. >=20 > That seemed hard to believe, so I took a week of logs from my mail = server, did a similar analysis, and by golly, he's right. The analysis = takes timestamps and IP addresses, and computes how many DNSxL queries = would be answered from a DNS cache under various scenarios. Each group = shows the number of cache hits and total queries by 24 hour period. (The = first and last are short, since my logs didn't start and end at = midnight.) Then it shows the number of unique IPs queried, and a list = showing cache effectiveness, with the number of times each cache entry = was used, e.g., in the first dataset, there was one cache entry used 514 = times, down to 536,009 entries used only once. >=20 > To see how sensitive these answers were to DNSxL design, I tried it = with a 15 minute and 60 minute TTL, and I tried with /24 rather than /32 = granularity. Neither made much difference: the cache effectiveness = rarely got above 20% and the vast majority of cache entries were never = reused. Assuming an infinite cache, that'll get much better as the traffic gets = higher, but it'll still be pretty ugly. > So now I'm scratching my head. If these numbers are typical, I don't = understand why DNSBL servers don't all fall over now. Because high-volume DNSBL consumers don't use them. They use a local copy of the database, pushed out to them (typically via = rsync, which is not ideal, but not *dreadful*) and they only use DNS as = a LAN protocol to communicate between their MTA and their local = authoritative DNS server, dedicated to serving DNSBL data. Moving away from running DNS for DNSBLs across the public internet has = been an obvious move for a long time. IPv6 does move us further in that = direction. DNS is still a perfectly good local protocol, though. Cheers, Steve From CLEWIS@nortel.com Tue Jan 4 20:37:36 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 828A23A67E6 for ; Tue, 4 Jan 2011 20:37:36 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.199 X-Spam-Level: X-Spam-Status: No, score=-6.199 tagged_above=-999 required=5 tests=[AWL=-0.200, BAYES_00=-2.599, J_CHICKENPOX_43=0.6, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6Rx6yxT-S68O for ; Tue, 4 Jan 2011 20:37:35 -0800 (PST) Received: from zrtps0kp.nortel.com (zrtps0kp.nortel.com [47.140.192.56]) by core3.amsl.com (Postfix) with ESMTP id 491103A67A2 for ; Tue, 4 Jan 2011 20:37:34 -0800 (PST) Received: from zrtphxs1.corp.nortel.com (zrtphxs1.corp.nortel.com [47.140.202.46]) by zrtps0kp.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id p054dcK18274 for ; Wed, 5 Jan 2011 04:39:38 GMT Received: from zrtphx5h0.corp.nortel.com ([47.140.202.65]) by zrtphxs1.corp.nortel.com with Microsoft SMTPSVC(6.0.3790.4675); Tue, 4 Jan 2011 23:39:37 -0500 Received: from [47.130.80.33] (47.130.80.33) by zrtphx5h0.corp.nortel.com (47.140.202.65) with Microsoft SMTP Server (TLS) id 8.1.340.0; Tue, 4 Jan 2011 23:39:37 -0500 Message-ID: <4D23F604.7070007@nortel.com> Date: Tue, 4 Jan 2011 23:39:32 -0500 From: "Chris Lewis" Organization: Nortel User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.23) Gecko/20090812 Thunderbird/2.0.0.23 Mnenhy/0.7.6.666 MIME-Version: 1.0 To: References: <20110105015503.7083.qmail@joyce.lan> In-Reply-To: Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 05 Jan 2011 04:39:37.0763 (UTC) FILETIME=[8F2F3730:01CBAC92] Subject: Re: [Asrg] NXDOMAIN cache behavior, was draft-levine-iprangepub-01 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 04:37:36 -0000 On 1/4/2011 9:15 PM, John R Levine wrote: > I wasn't surprised. Oh well. > I looked for foo.gurus.org, got an NXDOMAIN, then > queried bar.foo.gurus.org, and it went back to the authoritative server to > get the answer. I guess I must have misunderstood. If _that_ is how you test it, I can't imagine a DNS server doing otherwise. > Nice cache, though. Maybe I'll give it a try. Isn't it? _Highly_ recommended. dnscache has a fixed size outstanding query buffer (compile-time 100 queries IIRC), and throws queries away if you overflow it. If you do the obvious thing and recompile it with a larger buffer, as the match between query and response is a _linear search_ (!), it degrades atrociously. Especially if the admin naively recompiles for several thousand entries. [It's apparently even worse than that, but I disremember all the details.] Unbound's buffer is run-time allocated, and reply/query matching is hashed. I don't know anything about the guts of bind but it don't degrade like dnscache either. [I exiled myself off the Corporate DNS infrastructure (bind) for my mail servers/traps occasionally causing one of their servers to drop _other_ queries for minutes at a time. I was DDOS'ing it. Hence the initial foray into a server-coresident dnscache which behaved worse (but only affected me), and then finally to a server-coresident unbound (which has been 100% perfect so far).] From johnl@iecc.com Tue Jan 4 21:11:13 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 7651E3A69C1 for ; Tue, 4 Jan 2011 21:11:13 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -111.152 X-Spam-Level: X-Spam-Status: No, score=-111.152 tagged_above=-999 required=5 tests=[AWL=0.047, BAYES_00=-2.599, HABEAS_ACCREDITED_SOI=-4.3, RCVD_IN_BSP_TRUSTED=-4.3, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LaFYnaBXl+Lc for ; Tue, 4 Jan 2011 21:11:12 -0800 (PST) Received: from gal.iecc.com (gal.iecc.com [64.57.183.53]) by core3.amsl.com (Postfix) with ESMTP id DBA963A6805 for ; Tue, 4 Jan 2011 21:11:08 -0800 (PST) Received: (qmail 13824 invoked from network); 5 Jan 2011 05:13:15 -0000 Received: from mail1.iecc.com (64.57.183.56) by mail1.iecc.com with QMQP; 5 Jan 2011 05:13:15 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=iecc.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=e41c.4d23fdea.k1101; i=johnl@user.iecc.com; bh=jRF0ycsXc+udSnj/ulCmup4m4xtRXXMIQwXsRP76K1c=; b=xIXo5m7v45iUUmeqRmsgLYjuTvczNsroEbXeZXsXGoSUFscb5GMqqUMi8Lo4FT/OTqRuIRm4Sa3Db5IH0KLzj8yV+ob0ijKS4kEVbStELolyRtnNewBTn5/13j3DqzUtR+UVSAjxAU6g+RRJEB7GR+R60G3d//xOHfZ9o7R6KeI= DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=taugh.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=e41c.4d23fdea.k1101; olt=johnl@user.iecc.com; bh=jRF0ycsXc+udSnj/ulCmup4m4xtRXXMIQwXsRP76K1c=; b=p8+kXPfDqn+v/A78uF5BKQgrOgpMM0Au/6qaO32fpYfXHX4OYT85u/aJqg1pmO5kEzlMdqs+AO+2T8SJnWzuR9dvw0ju7ur8uxr45UCp7sIC9scAk12NYHQzAcxVkbe2HRZlHpULmXqxP3xLf2YJq5Rfhf196/+rzul0faJm3HI= VBR-Info: md=iecc.com; mc=all; mv=dwl.spamhaus.org Date: 5 Jan 2011 05:13:14 -0000 Message-ID: <20110105051314.58395.qmail@joyce.lan> From: John Levine To: asrg@irtf.org In-Reply-To: <4D23F604.7070007@nortel.com> Organization: Cc: X-Headerized: yes Mime-Version: 1.0 Content-type: text/plain; charset=utf-8 Content-transfer-encoding: 7bit Subject: Re: [Asrg] NXDOMAIN cache behavior, was draft-levine-iprangepub-01 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 05:11:13 -0000 >> I looked for foo.gurus.org, got an NXDOMAIN, then >> queried bar.foo.gurus.org, and it went back to the authoritative server to >> get the answer. > >I guess I must have misunderstood. If _that_ is how you test it, I >can't imagine a DNS server doing otherwise. It's not a totally ridiculous idea. There's a difference between NODATA, which means that the name exists but has no records, and NXDOMAIN which means the name doesn't exist. Assuming the data are consistent, if you see NXDOMAIN there should be no subdomains of the name with the NXDOMAIN. But the DNS crowd hates synthesizing answers for reasons that strike me as mostly religious, so nobody does. >dnscache has a fixed size outstanding query buffer (compile-time 100 >queries IIRC), and throws queries away if you overflow it. For the handful of computers that use my cache, it hasn't been a problem. R's, John From steve@blighty.com Tue Jan 4 21:47:52 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 49EBE3A6A2C for ; Tue, 4 Jan 2011 21:47:52 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.995 X-Spam-Level: X-Spam-Status: No, score=0.995 tagged_above=-999 required=5 tests=[AWL=-3.595, BAYES_00=-2.599, FB_WORD1_END_DOLLAR=3.294, FB_WORD2_END_DOLLAR=3.294, J_CHICKENPOX_74=0.6, NORMAL_HTTP_TO_IP=0.001] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0veqN2GGaRFF for ; Tue, 4 Jan 2011 21:47:50 -0800 (PST) Received: from m.wordtothewise.com (b.wordtothewise.com [208.187.80.136]) by core3.amsl.com (Postfix) with ESMTP id CA3C83A6A77 for ; Tue, 4 Jan 2011 21:47:50 -0800 (PST) Received: from platter.wordtothewise.com (184.wordtothewise.com [208.187.80.184]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: steve) by m.wordtothewise.com (Postfix) with ESMTPSA id DD8282DDD6 for ; Tue, 4 Jan 2011 21:49:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wordtothewise.com; s=1.wttw; t=1294206598; bh=/O1vITRxSO1uPS1cxAbvjaxDbRD9ixU3b7AASDMXohM=; h=Content-Type:Mime-Version:Subject:From:In-Reply-To:Date: Content-Transfer-Encoding:Message-Id:References:To; b=FloZmWbg8X9vvhnnc3t+94UJcexaxbJiAG1zfdPuDWz8cf0+/0ePlTPcF8fOXAf8T QC8nGl8ZBmtOX8aFb1f7T5TazB99xmpvc0DfdmWNuRQTOjmagHuM7LkEfyAzszoLHK yO4QyNNk84cYK2UVcFTQa4+Zvi6W9SYtLtCKet6o= Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Apple Message framework v1082) From: Steve Atkins In-Reply-To: <20110105051314.58395.qmail@joyce.lan> Date: Tue, 4 Jan 2011 21:49:57 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20110105051314.58395.qmail@joyce.lan> To: Anti-Spam Research Group - IRTF X-Mailer: Apple Mail (2.1082) Subject: Re: [Asrg] NXDOMAIN cache behavior, was draft-levine-iprangepub-01 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 05:47:52 -0000 On Jan 4, 2011, at 9:13 PM, John Levine wrote: >>> I looked for foo.gurus.org, got an NXDOMAIN, then >>> queried bar.foo.gurus.org, and it went back to the authoritative = server to >>> get the answer. >>=20 >> I guess I must have misunderstood. If _that_ is how you test it, I=20= >> can't imagine a DNS server doing otherwise. >=20 > It's not a totally ridiculous idea. =20 It is. > There's a difference between > NODATA, which means that the name exists but has no records, and > NXDOMAIN which means the name doesn't exist. Yes. > Assuming the data are > consistent, if you see NXDOMAIN there should be no subdomains of the > name with the NXDOMAIN. =20 No. > But the DNS crowd hates synthesizing answers > for reasons that strike me as mostly religious, so nobody does. They're generally not "religious" so much as they're the opinions of people who know DNS. NXDOMAIN for c.b.a simply means that there is no RR for c.b.a. It tells you absolutely nothing about the existence of an RR for = d.c.b.a, and any recursive resolver that synthesized results for such based on any RRset for c.b.a would be, simply, broken. platter:test steve$ host 0.0.127.sbl.spamhaus.org Host 0.0.127.sbl.spamhaus.org not found: 3(NXDOMAIN) platter:test steve$ host 2.0.0.127.sbl.spamhaus.org 2.0.0.127.sbl.spamhaus.org has address 127.0.0.2 Or a little more complex: platter:test steve$ host c.b.a.tupid.org Host c.b.a.tupid.org not found: 3(NXDOMAIN) platter:test steve$ host d.c.b.a.tupid.org d.c.b.a.tupid.org has address 127.0.0.1 Cheers, Steve From CLEWIS@nortel.com Tue Jan 4 22:27:20 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 5C46A3A6B6A for ; Tue, 4 Jan 2011 22:27:20 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.459 X-Spam-Level: X-Spam-Status: No, score=-6.459 tagged_above=-999 required=5 tests=[AWL=0.140, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aCx+XgxP2E2U for ; Tue, 4 Jan 2011 22:27:17 -0800 (PST) Received: from zcars04e.nortel.com (zcars04e.nortel.com [47.129.242.56]) by core3.amsl.com (Postfix) with ESMTP id F3F693A6B20 for ; Tue, 4 Jan 2011 22:27:16 -0800 (PST) Received: from zrtphxs1.corp.nortel.com (zrtphxs1.corp.nortel.com [47.140.202.46]) by zcars04e.nortel.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id p056TJA26846 for ; Wed, 5 Jan 2011 06:29:19 GMT Received: from zrtphx5h0.corp.nortel.com ([47.140.202.65]) by zrtphxs1.corp.nortel.com with Microsoft SMTPSVC(6.0.3790.4675); Wed, 5 Jan 2011 01:29:19 -0500 Received: from [47.130.64.29] (47.130.64.29) by zrtphx5h0.corp.nortel.com (47.140.202.65) with Microsoft SMTP Server (TLS) id 8.1.340.0; Wed, 5 Jan 2011 01:29:18 -0500 Message-ID: <4D240FBD.5020104@nortel.com> Date: Wed, 5 Jan 2011 01:29:17 -0500 From: "Chris Lewis" Organization: Nortel User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.23) Gecko/20090812 Thunderbird/2.0.0.23 Mnenhy/0.7.6.666 MIME-Version: 1.0 To: References: <20110105051314.58395.qmail@joyce.lan> In-Reply-To: <20110105051314.58395.qmail@joyce.lan> Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 05 Jan 2011 06:29:19.0180 (UTC) FILETIME=[E203C0C0:01CBACA1] Subject: Re: [Asrg] NXDOMAIN cache behavior, was draft-levine-iprangepub-01 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 06:27:20 -0000 On 1/5/2011 12:13 AM, John Levine wrote: >>> I looked for foo.gurus.org, got an NXDOMAIN, then >>> queried bar.foo.gurus.org, and it went back to the authoritative server to >>> get the answer. >> >> I guess I must have misunderstood. If _that_ is how you test it, I >> can't imagine a DNS server doing otherwise. > It's not a totally ridiculous idea. There's a difference between > NODATA, which means that the name exists but has no records, and > NXDOMAIN which means the name doesn't exist. Assuming the data are > consistent, if you see NXDOMAIN there should be no subdomains of the > name with the NXDOMAIN. But the DNS crowd hates synthesizing answers > for reasons that strike me as mostly religious, so nobody does. Er. But isn't this (a) whether you interpret NODATA (aka No Error) as NXDOMAIN or otherwise a DNS client library and/or DNSBL query function decision, and (b) does DNS really return "No Error" for b.a if there's no RR for b.a, but there is one for c.b.a? All my DNSBL query code is written well "below" the gethostbyname() level of abstraction, and I always have to make a conscious decision (so far) to treat NOERROR as NXDOMAIN on DNSBLs. On DNSWLs I treat anything but an affirmative NXDOMAIN (ie: server fail, timeout, no error) as listed. [Most of my DNSBLs and all of my DNSWLs are served locally via a single combined A query, so a DDOS on an external DNSWL doesn't cause me to accept everything, and I get 10 DNSxL queries for (almost as little) as the price of one.] >> dnscache has a fixed size outstanding query buffer (compile-time 100 >> queries IIRC), and throws queries away if you overflow it. > For the handful of computers that use my cache, it hasn't been a problem. Note "handful". I'd also suggest relatively low DNS-consumptive. In my case, I'm talking about a single mail server doing 4-8 DNS queries per email, and 20-100 emails/second blowing a dnscache instance's brains out. dnscache is fine for the average small DNS user. But if you're doing lots, it leaves something to be desired. When I crank up the process to ASNify the past week's worth of seem IPs (6+ million) via a local copy of the routeviews zone for global metrics, it matters if you want it done in a reasonable time (I'm doing 10 minutes now). Udns or ParaDNS is a great way to break nearby caches (and consumer level routers as some people have found out to their dismay) if you're not careful. From johnl@iecc.com Tue Jan 4 22:42:21 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 91EE63A6B73 for ; Tue, 4 Jan 2011 22:42:21 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.259 X-Spam-Level: X-Spam-Status: No, score=-106.259 tagged_above=-999 required=5 tests=[AWL=-4.849, BAYES_50=0.001, FB_WORD1_END_DOLLAR=3.294, FB_WORD2_END_DOLLAR=3.294, HABEAS_ACCREDITED_SOI=-4.3, J_CHICKENPOX_74=0.6, NORMAL_HTTP_TO_IP=0.001, RCVD_IN_BSP_TRUSTED=-4.3, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WD4Qdk2Vl0pl for ; Tue, 4 Jan 2011 22:42:18 -0800 (PST) Received: from gal.iecc.com (gal.iecc.com [64.57.183.53]) by core3.amsl.com (Postfix) with ESMTP id 479BB3A6806 for ; Tue, 4 Jan 2011 22:42:18 -0800 (PST) Received: (qmail 27306 invoked from network); 5 Jan 2011 06:44:24 -0000 Received: from mail1.iecc.com (64.57.183.56) by mail1.iecc.com with QMQP; 5 Jan 2011 06:44:24 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=iecc.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=13e16.4d241348.k1101; i=johnl@user.iecc.com; bh=lpilaMg6YkEavi2LqPPYW/g8nzvp3iuaOn+aaRbkv+M=; b=ZGpDhbjFw/uZf+lu0IuRTIjA/rhzQxHNKWPJPTriAtzYj9xCvdyAT9iRUK3m+WmBjEsIBvVwkYoN5/Kp1oPpgpdgw9TrZBu0zmQPMutbf2+FT0F3LT1r2whQS30Y00KCc6+kU4mVyHTWJZRaLUUVQa0IJUOYIj92Bg1kedS/pBA= DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=taugh.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=13e16.4d241348.k1101; olt=johnl@user.iecc.com; bh=lpilaMg6YkEavi2LqPPYW/g8nzvp3iuaOn+aaRbkv+M=; b=KvuDiqQtagWdnMxMKzbZsA2IBQxWGvX4gVzg+yK13iPz3/pfEYjG/YYXwwetQVGnP+Wr/RoDOGpR11WArcmrKmSINiRJT8BFGwQQf1kdFbsoYX776jYz3DJ1zyPeRRUu7eflMZ2yUBDENuzbH+kKH5zIcXO+EAafExev6ITHBec= VBR-Info: md=iecc.com; mc=all; mv=dwl.spamhaus.org Date: 5 Jan 2011 06:44:24 -0000 Message-ID: <20110105064424.81429.qmail@joyce.lan> From: John Levine To: asrg@irtf.org In-Reply-To: Organization: Cc: X-Headerized: yes Mime-Version: 1.0 Content-type: text/plain; charset=utf-8 Content-transfer-encoding: 7bit Subject: Re: [Asrg] NXDOMAIN cache behavior, was draft-levine-iprangepub-01 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 06:42:22 -0000 >NXDOMAIN for c.b.a simply means that there is no RR for c.b.a. >It tells you absolutely nothing about the existence of an RR for >d.c.b.a, and any recursive resolver that synthesized results for such >based on any RRset for c.b.a would be, simply, broken. You might want to look at RFC 4592, particularly section 2.2.2, which explains this corner of DNS arcana and the difference between NOERROR and NXDOMAIN. But you're in good company, since a lot of people who write alleged DNS software haven't read it either. >platter:test steve$ host 0.0.127.sbl.spamhaus.org >Host 0.0.127.sbl.spamhaus.org not found: 3(NXDOMAIN) >platter:test steve$ host 2.0.0.127.sbl.spamhaus.org >2.0.0.127.sbl.spamhaus.org has address 127.0.0.2 The guy who wrote rbldnsd didn't read RFC 4592. >platter:test steve$ host c.b.a.tupid.org >Host c.b.a.tupid.org not found: 3(NXDOMAIN) >platter:test steve$ host d.c.b.a.tupid.org >d.c.b.a.tupid.org has address 127.0.0.1 Nor did the guy who wrote whatever free01.editdns.net is running. I can't tell what it is, since the usual chaos version.bind query produces a bogus result that suggests he didn't read section 3.2 of RFC 1035, either. I agree that synthesizing results would be risky, since there is a substantial amount of DNS software that doesn't properly report the difference between NOERROR and NXDOMAIN. If DNS servers followed the specs, it should work, perhaps in a more perfect world than this one. With that said, I still like my b-tree hack, which makes queries that shouldn't get either NOERROR or NXDOMAIN, a lot better as a way to publish ranges of addresses in a DNSxL. R's, John From johnl@iecc.com Tue Jan 4 22:53:32 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 637063A6B6A for ; Tue, 4 Jan 2011 22:52:45 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -111.011 X-Spam-Level: X-Spam-Status: No, score=-111.011 tagged_above=-999 required=5 tests=[AWL=0.188, BAYES_00=-2.599, HABEAS_ACCREDITED_SOI=-4.3, RCVD_IN_BSP_TRUSTED=-4.3, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6p7IEgAtcO34 for ; Tue, 4 Jan 2011 22:52:43 -0800 (PST) Received: from gal.iecc.com (gal.iecc.com [64.57.183.53]) by core3.amsl.com (Postfix) with ESMTP id CE9C93A6BEC for ; Tue, 4 Jan 2011 22:52:27 -0800 (PST) Received: (qmail 28927 invoked from network); 5 Jan 2011 06:54:27 -0000 Received: from mail1.iecc.com (64.57.183.56) by mail1.iecc.com with QMQP; 5 Jan 2011 06:54:27 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=iecc.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=147eb.4d2415a3.k1101; i=johnl@user.iecc.com; bh=57tIgQ/3tKO5mqzfBZhYhWHAPdoYlWwo3wJcDFMy13M=; b=g4z7qZHnAl67Y4DJY3RtptpVO5VagwxwdWoYe8vtydVGyipnGX0j8EI71IOtQzsafjz2tiDclqOm9II2t8wtmeYhp1ouX9Upt05TQAiAddfjg2E6B96QtJboJlakUkG1Te01w3XzGuM7lTJx3NWTeEHEN2xIU9WYggYU57Q/RcE= DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=taugh.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=147eb.4d2415a3.k1101; olt=johnl@user.iecc.com; bh=57tIgQ/3tKO5mqzfBZhYhWHAPdoYlWwo3wJcDFMy13M=; b=OTKj40wIz7Tl13DUh2zpjH4Zvy/gHs+FpN/wfawk7OkKwruSAQ9Vkm6TOGg/iuvWdAJBowykz6kfJatvncoqnprqtWVuTDaI1wsQ78D1VUU85x0Hs53AwFnHXCMHZc0lCFatIe48W81evLNGAOoCCp9nlwtC4xsJRb5n/JJ+VfA= VBR-Info: md=iecc.com; mc=all; mv=dwl.spamhaus.org Date: 5 Jan 2011 06:54:26 -0000 Message-ID: <20110105065426.83946.qmail@joyce.lan> From: John Levine To: asrg@irtf.org In-Reply-To: <4D240FBD.5020104@nortel.com> Organization: Cc: X-Headerized: yes Mime-Version: 1.0 Content-type: text/plain; charset=utf-8 Content-transfer-encoding: 7bit Subject: Re: [Asrg] NXDOMAIN cache behavior, was draft-levine-iprangepub-01 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 06:53:32 -0000 >Er. But isn't this (a) whether you interpret NODATA (aka No Error) as >NXDOMAIN or otherwise a DNS client library and/or DNSBL query function >decision, and (b) does DNS really return "No Error" for b.a if there's >no RR for b.a, but there is one for c.b.a? Per my previous message, the answer to (b) is yes, according to RFC 4592, but there's a lot of buggy software that gets it wrong. >>> dnscache has a fixed size outstanding query buffer (compile-time 100 >>> queries IIRC), and throws queries away if you overflow it. > >> For the handful of computers that use my cache, it hasn't been a problem. > >Note "handful". Indeed. If I find that I have to run the mail system for a major multinational corporation, I expect I'll use something else. djbdns hasn't been updated since 2001, when its query limits were reasonable. But that's Dan for you. >... I'm talking about a single mail server doing 4-8 DNS queries per >email, and 20-100 emails/second blowing a dnscache instance's brains out. That reminds me, any chance of getting running some IP/timestamp logs through my DNSBL analyzer and see how much a cache helps a busy setup like yours? R's, John From CLEWIS@nortel.com Tue Jan 4 23:09:07 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 88E2E3A6B45 for ; Tue, 4 Jan 2011 23:09:07 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.482 X-Spam-Level: X-Spam-Status: No, score=-6.482 tagged_above=-999 required=5 tests=[AWL=0.117, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7wneQV2aaMqc for ; Tue, 4 Jan 2011 23:09:06 -0800 (PST) Received: from zcars04e.nortel.com (zcars04e.nortel.com [47.129.242.56]) by core3.amsl.com (Postfix) with ESMTP id 1CD7F3A6B42 for ; Tue, 4 Jan 2011 23:09:06 -0800 (PST) Received: from zrtphxs1.corp.nortel.com (zrtphxs1.corp.nortel.com [47.140.202.46]) by zcars04e.nortel.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id p057BAA29055 for ; Wed, 5 Jan 2011 07:11:10 GMT Received: from zrtphx5h0.corp.nortel.com ([47.140.202.65]) by zrtphxs1.corp.nortel.com with Microsoft SMTPSVC(6.0.3790.4675); Wed, 5 Jan 2011 02:11:09 -0500 Received: from [47.130.64.29] (47.130.64.29) by zrtphx5h0.corp.nortel.com (47.140.202.65) with Microsoft SMTP Server (TLS) id 8.1.340.0; Wed, 5 Jan 2011 02:11:09 -0500 Message-ID: <4D24198C.7040707@nortel.com> Date: Wed, 5 Jan 2011 02:11:08 -0500 From: "Chris Lewis" Organization: Nortel User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.23) Gecko/20090812 Thunderbird/2.0.0.23 Mnenhy/0.7.6.666 MIME-Version: 1.0 To: References: <20110105065426.83946.qmail@joyce.lan> In-Reply-To: <20110105065426.83946.qmail@joyce.lan> Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 05 Jan 2011 07:11:10.0119 (UTC) FILETIME=[BAA6A370:01CBACA7] Subject: Re: [Asrg] NXDOMAIN cache behavior, was draft-levine-iprangepub-01 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 07:09:07 -0000 On 1/5/2011 1:54 AM, John Levine wrote: > That reminds me, any chance of getting running some IP/timestamp logs > through my DNSBL analyzer and see how much a cache helps a busy setup > like yours? I'll run it on something moderately large tomorrow. Send me the script. I'll also see if I can get it run on something humungous. From scott@doc.net.au Tue Jan 4 23:15:40 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 8CEAC3A6B83 for ; Tue, 4 Jan 2011 23:15:40 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.976 X-Spam-Level: X-Spam-Status: No, score=-2.976 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tiu2n5ZTMtRh for ; Tue, 4 Jan 2011 23:15:39 -0800 (PST) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by core3.amsl.com (Postfix) with ESMTP id 6AD613A6B45 for ; Tue, 4 Jan 2011 23:15:39 -0800 (PST) Received: by wwf26 with SMTP id 26so16074689wwf.19 for ; Tue, 04 Jan 2011 23:17:45 -0800 (PST) MIME-Version: 1.0 Received: by 10.216.181.141 with SMTP id l13mr14243531wem.22.1294211865324; Tue, 04 Jan 2011 23:17:45 -0800 (PST) Received: by 10.216.242.139 with HTTP; Tue, 4 Jan 2011 23:17:45 -0800 (PST) In-Reply-To: <20110105064424.81429.qmail@joyce.lan> References: <20110105064424.81429.qmail@joyce.lan> Date: Tue, 4 Jan 2011 23:17:45 -0800 Message-ID: From: Scott Howard To: Anti-Spam Research Group - IRTF Content-Type: multipart/alternative; boundary=0016367b60fa7f2d4d0499142ef8 Subject: Re: [Asrg] NXDOMAIN cache behavior, was draft-levine-iprangepub-01 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 07:15:40 -0000 --0016367b60fa7f2d4d0499142ef8 Content-Type: text/plain; charset=ISO-8859-1 On Tue, Jan 4, 2011 at 10:44 PM, John Levine wrote: > >NXDOMAIN for c.b.a simply means that there is no RR for c.b.a. > > >It tells you absolutely nothing about the existence of an RR for > >d.c.b.a, and any recursive resolver that synthesized results for such > >based on any RRset for c.b.a would be, simply, broken. > > You might want to look at RFC 4592, particularly section 2.2.2, > which explains this corner of DNS arcana and the difference between > NOERROR and NXDOMAIN. > That's the joy of standards - you can never have too many of them. RFC 1034 implies that NXDOMAIN means that the domain does not exist, and thus it would not be possible for any sub-domains to exist (although if you squint hard enough whilst reading it you can actually make it say the opposite). RFC 2308 states that a response of NXDOMAIN does NOT means that the domain doesn't exist, and thus it's feasible for a there to be sub-domains even on an NXDOMAIN response. Then there's RFC 2535, 4592 and probably a few others which even further muddy the waters. At best, this is a vague area, and the only correct thing to do is to not presume any specific behavior - the only guarantee is that different nameservers are going to act differently. Scott. --0016367b60fa7f2d4d0499142ef8 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On Tue, Jan 4, 2011 at 10:44 PM, John Levine <johnl@taugh.com> wrote:
>NXDOMAIN for c.b.a simply means that there is no RR for c.b.a.

>It tells you absolutely nothing about the existence of an RR for
>d.c.b.a, and any recursive resolver that synthesized results for such >based on any RRset for c.b.a would be, simply, broken.

You might want to look at RFC 4592, particularly section 2.2.2,
which explains this corner of DNS arcana and the difference between
NOERROR and NXDOMAIN.

That's the joy of standa= rds - you can never have too many of them.

RFC 1034 implies that NXD= OMAIN means that the domain does not exist, and thus it would not be possib= le for any sub-domains to exist (although if you squint hard enough whilst = reading it you can actually make it say the opposite).

RFC 2308 states that a response of NXDOMAIN does NOT means that the dom= ain doesn't exist, and thus it's feasible for a there to be sub-dom= ains even on an NXDOMAIN response.

Then there's RFC 2535, 4592 a= nd probably a few others which even further muddy the waters.

At best, this is a vague area, and the only correct thing to do is to n= ot presume any specific behavior - the only guarantee is that different nam= eservers are going to act differently.

=A0 Scott.
--0016367b60fa7f2d4d0499142ef8-- From steve@blighty.com Tue Jan 4 23:39:48 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 3001F3A6A2C for ; Tue, 4 Jan 2011 23:39:48 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.401 X-Spam-Level: X-Spam-Status: No, score=-1.401 tagged_above=-999 required=5 tests=[AWL=1.198, BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gi0NeaLqzz4H for ; Tue, 4 Jan 2011 23:39:47 -0800 (PST) Received: from m.wordtothewise.com (b.wordtothewise.com [208.187.80.136]) by core3.amsl.com (Postfix) with ESMTP id F0B5B3A6CA1 for ; Tue, 4 Jan 2011 23:39:46 -0800 (PST) Received: from platter.wordtothewise.com (184.wordtothewise.com [208.187.80.184]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: steve) by m.wordtothewise.com (Postfix) with ESMTPSA id 1B7D12DDD9 for ; Tue, 4 Jan 2011 23:41:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wordtothewise.com; s=1.wttw; t=1294213314; bh=seDyppqP8KXEpWw0y3PEKMh+04SVvzawsnvoZkXYMsc=; h=Content-Type:Mime-Version:Subject:From:In-Reply-To:Date: Content-Transfer-Encoding:Message-Id:References:To; b=fzz28nGNSbL4FWB+k2/kbQh535BxSdPE8Kzj7HVvAy3M70/6IkgQF+3tLitmDz9Ld tQ+Y0dJKeU3lCLckbXwyPeTCezOFLYq/Zwdtt+9A/BApe/kVOrEi9wISCLxQPcfMuS 3mzpK+amzRzfCQtrHnV+8ke11xmM3ehN11TqqdFo= Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Apple Message framework v1082) From: Steve Atkins In-Reply-To: <20110105064424.81429.qmail@joyce.lan> Date: Tue, 4 Jan 2011 23:41:53 -0800 Content-Transfer-Encoding: 7bit Message-Id: References: <20110105064424.81429.qmail@joyce.lan> To: Anti-Spam Research Group - IRTF X-Mailer: Apple Mail (2.1082) Subject: Re: [Asrg] NXDOMAIN cache behavior, was draft-levine-iprangepub-01 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 07:39:48 -0000 On Jan 4, 2011, at 10:44 PM, John Levine wrote: >> NXDOMAIN for c.b.a simply means that there is no RR for c.b.a. > >> It tells you absolutely nothing about the existence of an RR for >> d.c.b.a, and any recursive resolver that synthesized results for such >> based on any RRset for c.b.a would be, simply, broken. > > You might want to look at RFC 4592, particularly section 2.2.2, > which explains this corner of DNS arcana and the difference between > NOERROR and NXDOMAIN. 4592 is contradicted by some other RFCs and, more importantly, by common behavior. What's "correct" behaviour would be an interesting thing to bring up with the IETF DNS folks (pick a standard, any standard), but what's implemented is what's important when it comes to relying on that behaviour. > I agree that synthesizing results would be risky, since there is a > substantial amount of DNS software that doesn't properly report the > difference between NOERROR and NXDOMAIN. If DNS servers followed the > specs, it should work, perhaps in a more perfect world than this one. The authoritative nameservers I've used recently all do the "right thing" there: no RRs => NXDOMAIN. And the recursive servers I have handy also do the "right thing" and don't synthesize fake answers. For layering something on top of DNS the deployed implementation is the only spec that matters - what's implemented trumps what would be nice. > > With that said, I still like my b-tree hack, which makes queries that > shouldn't get either NOERROR or NXDOMAIN, a lot better as a way to > publish ranges of addresses in a DNSxL. If you're going to the effort of adding that much functionality at the client end, switching to a better matched protocol instead might be better. All the high traffic DNSBL users already use a push protocol of sorts. Moving to a better, more standardized one might be a win. A local server for protocol X could still offer a DNS interface to the MTA to ease implementation Cheers, Steve [1] and potentially CNAMEs and suchlike. [2] or maybe a delegation, if there's a domain cut somewhere above c.b.a. From asrg3@billmail.scconsult.com Wed Jan 5 09:51:04 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id D40C63A6CDB for ; Wed, 5 Jan 2011 09:51:04 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ol2ZwyMWeZys for ; Wed, 5 Jan 2011 09:51:03 -0800 (PST) Received: from toaster.scconsult.com (www.scconsult.com [66.73.230.185]) by core3.amsl.com (Postfix) with ESMTP id 9170E3A6AF4 for ; Wed, 5 Jan 2011 09:51:03 -0800 (PST) Received: from deepfield.scconsult.com (unknown [192.168.254.22]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by toaster.scconsult.com (Postfix) with ESMTP id A6FA7C0A1A4 for ; Wed, 5 Jan 2011 12:53:09 -0500 (EST) Message-ID: <4D24AFF7.6090701@billmail.scconsult.com> Date: Wed, 05 Jan 2011 12:52:55 -0500 From: Bill Cole User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.1.9) Gecko/20100722 Lightning/1.0b1 Eudora/3.0.4 MIME-Version: 1.0 To: asrg@irtf.org References: <20110105065426.83946.qmail@joyce.lan> In-Reply-To: <20110105065426.83946.qmail@joyce.lan> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Asrg] NXDOMAIN cache behavior, was draft-levine-iprangepub-01 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: asrg@irtf.org List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 17:51:05 -0000 John Levine wrote, On 1/5/11 1:54 AM: >> Er. But isn't this (a) whether you interpret NODATA (aka No Error) as >> NXDOMAIN or otherwise a DNS client library and/or DNSBL query function >> decision, and (b) does DNS really return "No Error" for b.a if there's >> no RR for b.a, but there is one for c.b.a? > > Per my previous message, the answer to (b) is yes, according to > RFC 4592, but there's a lot of buggy software that gets it wrong. FWIW, a very quick randomish check of the DNSBL's I find interesting shows exactly one (tor.dnsbl.sectoor.de) that answers such queries correctly other than my own local ones which are served by BIND 9. As a cache, BIND 9 will cache and serve a NXDOMAIN answer that should really be NODATA/NOERROR, even if it has cached records under the supposed NXDOMAIN. The point: getting this wrong is the norm in the DNSBL space. It may be due to buggy software, but the bug is endemic. From davidnicol@gmail.com Wed Jan 5 11:03:06 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id BF8033A6CDB for ; Wed, 5 Jan 2011 11:03:06 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.856 X-Spam-Level: X-Spam-Status: No, score=-2.856 tagged_above=-999 required=5 tests=[AWL=-0.258, BAYES_00=-2.599, HTML_MESSAGE=0.001] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YEK0GSSxUYtu for ; Wed, 5 Jan 2011 11:03:05 -0800 (PST) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by core3.amsl.com (Postfix) with ESMTP id CC7333A6C9F for ; Wed, 5 Jan 2011 11:03:05 -0800 (PST) Received: by iyb26 with SMTP id 26so14817998iyb.13 for ; Wed, 05 Jan 2011 11:05:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:mime-version:received:in-reply-to :references:from:date:message-id:subject:to:content-type; bh=XwllTCv7Y2CDNzwunOUdQaZIyx8EnxQm4SIXJGUh9ZE=; b=lr7hJYEOimJiDEu10P3eUdnIJZAYob+I+SsGegf+WQ96Mu7jVJkFt4L85YN2JaJdS5 aSAriOQ7SpBN+QC6jrAWjHkMkQq0ngE7fIOGAgi0PnIg5LkDTUdEn21/kNWabWosLdZa scRCiWYP8y9i9/+tIon8QDyPrkWpWFw+jBzeU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; b=AXWYrl90HtngwuccmLuTJ3uYsG3cJRIJti182mVfBd4W8HB+Dbj+5BXHj4a/8/FKfc A409QZpcQDpKpq52te+lTfdLcHwWl6vxmS7Ga6EiE+NtGroZXRFBcKYS0j8eiGt7R1Jx y4l7NFslca3rceQFNtegqNip9MvvAmNrc6sN0= Received: by 10.231.30.68 with SMTP id t4mr7763900ibc.15.1294254312395; Wed, 05 Jan 2011 11:05:12 -0800 (PST) MIME-Version: 1.0 Received: by 10.231.206.148 with HTTP; Wed, 5 Jan 2011 11:04:52 -0800 (PST) In-Reply-To: <20110104210324.21585.qmail@joyce.lan> References: <20110104210324.21585.qmail@joyce.lan> From: David Nicol Date: Wed, 5 Jan 2011 13:04:52 -0600 Message-ID: To: Anti-Spam Research Group - IRTF Content-Type: multipart/alternative; boundary=00032557563a8a21d804991e1073 Subject: Re: [Asrg] draft-levine-iprangepub-01 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 19:03:06 -0000 --00032557563a8a21d804991e1073 Content-Type: text/plain; charset=UTF-8 On Tue, Jan 4, 2011 at 3:03 PM, John Levine wrote: >This means that a list query library would start with the full query and > >work its way up until it gets a result, and the NXDOMAIN at the > appropriate > >part of the glue determination process will prevent the too-long queries > >from reaching the server, as they will be handled by the cache server. > > You might want to try this with a few caches. I'd be surprised if > they synthesized answers for subdomains of NXDOMAIN entries. I just > tried it with dnscache from djbdns, and it doesn't. > okay, so the list query library has to start from the top and drill down. Anyway, the interesting generalization is that once you start supporting ranges, there's no reason to have a particular length or granularity of "items in the list"; the other side of this is imagining other problem domains where a-Btree-in-DNS would be useful. I wasn't able to find a paragraph about ranges with holes in them, or the representation of the data beyond this point is in/not in the list, in iprangepub-01. by "ranges with holes" I mean, the range is all listed except for the sublist. also the concept of increasing resolution with finer-grained querying, useful for reputation-by-address kinds of databases, seemed absent. --00032557563a8a21d804991e1073 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

On Tue, Jan 4, 2011 at 3:03 PM, John Lev= ine <johnl@taugh.co= m> wrote:

>This means that a list query library would start with the full query an= d
>work its way up until it gets a result, and the NXDOMAIN at the appropr= iate
>part of the glue determination process will prevent the too-long querie= s
>from reaching the server, as they will be handled by the cache server.<= br>
You might want to try this with a few caches. =C2=A0I'd be surpri= sed if
they synthesized answers for subdomains of NXDOMAIN entries. =C2=A0I just tried it with dnscache from djbdns, and it doesn't.

okay, so the list query library has to start from the top = and drill down. Anyway, the interesting generalization is that once you sta= rt supporting ranges, there's no reason to have a particular length or = granularity of "items in the list"; the other side of this is ima= gining other problem domains where a-Btree-in-DNS would be useful.

I wasn't able to find a paragraph about ranges with= holes in them, or the representation of the data beyond this point is in/n= ot in the list, in iprangepub-01.

by "ranges = with holes" I mean, the range is all listed except for the sublist.

also the concept of increasing resolution with finer-gr= ained querying, useful for reputation-by-address kinds of databases, seemed= absent.
--00032557563a8a21d804991e1073-- From davidnicol@gmail.com Wed Jan 5 11:20:02 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 8F8513A6CBB for ; Wed, 5 Jan 2011 11:20:02 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.339 X-Spam-Level: X-Spam-Status: No, score=-3.339 tagged_above=-999 required=5 tests=[AWL=0.259, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hAb4A6APr7AE for ; Wed, 5 Jan 2011 11:20:01 -0800 (PST) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by core3.amsl.com (Postfix) with ESMTP id 3D0F03A6C9F for ; Wed, 5 Jan 2011 11:20:01 -0800 (PST) Received: by iwn39 with SMTP id 39so15856716iwn.13 for ; Wed, 05 Jan 2011 11:22:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:mime-version:received:in-reply-to :references:from:date:message-id:subject:to:content-type; bh=LUVagXctBaIhThaLOKNdxt2mzZAPcsX6y0yyNbgoRwk=; b=t5BuMmdlGHq3OlJ7WnDeDlfH5BahKNVp2uKQ9gnSJSfJwLwfyMbfQyJ9jjHHWP5uzH yvVhap04TNmpgIX//lZr0j43xK5zEAckFSNVrLRq5ksl52GF/GUZZlhe6Fu4Qb/xR6wM 9+ORmnTK/AVSYDe98PZvovCmZHdyANnaAYrrA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; b=Ybp8fWBv+fk9vls22cuVUOByjR9qc1q1WgJrKL0Mr4C53cNDbzmdSzggc5PdwEkFVl 7L7wfaftv1pJsP+fuGJOSgEreYp73ZWPZ77+BDSuYtxRpr+jeDpOjOYO6oJ4khHhb//J oc4C6TYQ1OWQpqjXrXxqtpaq47Y1HAPKLZon4= Received: by 10.231.11.72 with SMTP id s8mr7545891ibs.40.1294255327956; Wed, 05 Jan 2011 11:22:07 -0800 (PST) MIME-Version: 1.0 Received: by 10.231.206.148 with HTTP; Wed, 5 Jan 2011 11:21:47 -0800 (PST) In-Reply-To: References: <20110105064424.81429.qmail@joyce.lan> From: David Nicol Date: Wed, 5 Jan 2011 13:21:47 -0600 Message-ID: To: Anti-Spam Research Group - IRTF Content-Type: multipart/alternative; boundary=0003255753da124fcd04991e4d1b Subject: Re: [Asrg] NXDOMAIN cache behavior, was draft-levine-iprangepub-01 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 19:20:02 -0000 --0003255753da124fcd04991e4d1b Content-Type: text/plain; charset=UTF-8 On Wed, Jan 5, 2011 at 1:41 AM, Steve Atkins wrote: > For layering something on top of DNS the deployed implementation is > the only spec that matters - what's implemented trumps what would be nice. it depends if we are layering over "DNS: the installed infrastructure" or "DNS: the protocol." If the second, we are free to demand and enforce a particular semantics. --0003255753da124fcd04991e4d1b Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

On Wed, Jan 5, 2011 at 1:41 AM, Steve At= kins <steve@bligh= ty.com> wrote:
=C2=A0
For layering something on top of DNS the deployed impleme= ntation is
the only spec that matters - what's implemented trumps what would be ni= ce.

it depends if we are layering over &quo= t;DNS: the installed infrastructure" or "DNS: the protocol."= If the second, we are free to demand and enforce a particular semantics.

--0003255753da124fcd04991e4d1b-- From steve@blighty.com Wed Jan 5 11:25:20 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id C8F373A6CBB for ; Wed, 5 Jan 2011 11:25:20 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.7 X-Spam-Level: X-Spam-Status: No, score=-1.7 tagged_above=-999 required=5 tests=[AWL=0.899, BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aTAtwK5Z5clz for ; Wed, 5 Jan 2011 11:25:20 -0800 (PST) Received: from m.wordtothewise.com (b.wordtothewise.com [208.187.80.136]) by core3.amsl.com (Postfix) with ESMTP id F02883A6C17 for ; Wed, 5 Jan 2011 11:25:19 -0800 (PST) Received: from platter.wordtothewise.com (184.wordtothewise.com [208.187.80.184]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: steve) by m.wordtothewise.com (Postfix) with ESMTPSA id A24A72DDD9 for ; Wed, 5 Jan 2011 11:27:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wordtothewise.com; s=1.wttw; t=1294255646; bh=pHpaKZZhwQXeetZdQ9s8T6zKErcytsD2iUGuH/wonNU=; h=Content-Type:Mime-Version:Subject:From:In-Reply-To:Date: Content-Transfer-Encoding:Message-Id:References:To; b=crQieodwhDaYg5V2cAMzorC7I3csX3iOWx5zuLTBPI/UYkZNZPV4RlnKJv1zE2dFF qrGy4iar/WF3Hs8bvNbcQGUZFjdAXCXVypkPoZC2m3me+B5kitMKy9PVzC2mD0OxyJ L7UT5KJj40eWv4AANcxXyPolYRxWl1134R4xjmKM= Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Apple Message framework v1082) From: Steve Atkins In-Reply-To: Date: Wed, 5 Jan 2011 11:27:26 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <6ECCD51E-436B-400C-96A4-8C3DBDFD3E2A@blighty.com> References: <20110105064424.81429.qmail@joyce.lan> To: Anti-Spam Research Group - IRTF X-Mailer: Apple Mail (2.1082) Subject: Re: [Asrg] NXDOMAIN cache behavior, was draft-levine-iprangepub-01 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 19:25:20 -0000 On Jan 5, 2011, at 11:21 AM, David Nicol wrote: >=20 >=20 > On Wed, Jan 5, 2011 at 1:41 AM, Steve Atkins = wrote: > =20 > For layering something on top of DNS the deployed implementation is > the only spec that matters - what's implemented trumps what would be = nice. >=20 > it depends if we are layering over "DNS: the installed infrastructure" = or "DNS: the protocol." If the second, we are free to demand and enforce = a particular semantics. If the second... we could do much, much better than DNS for the WAN part = of any protocol. If we're building the infrastructure from scratch, we can demand any = semantics we like, but unless the existing software happens to implement = them, we're still out of luck unless we're planning on implementing = replacements from scratch. The only reason DNS was originally used was because it was easy to jam = into sendmail. The only reason it's used now is that the infrastructure = for it is ubiquitous (where infrastructure includes auth servers, caches = and MTA support). I think DNS still makes sense as a protocol to use between the MTA and a = local server. It makes much less sense for WAN distribution. Cheers, Steve From johnl@iecc.com Wed Jan 5 12:00:33 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 9D8853A6E66 for ; Wed, 5 Jan 2011 12:00:33 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -109.866 X-Spam-Level: X-Spam-Status: No, score=-109.866 tagged_above=-999 required=5 tests=[AWL=-0.967, BAYES_00=-2.599, HABEAS_ACCREDITED_SOI=-4.3, MANGLED_SPAM=2.3, RCVD_IN_BSP_TRUSTED=-4.3, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dQ1hcJJKbnMo for ; Wed, 5 Jan 2011 12:00:31 -0800 (PST) Received: from gal.iecc.com (gal.iecc.com [64.57.183.53]) by core3.amsl.com (Postfix) with ESMTP id B35773A6D1E for ; Wed, 5 Jan 2011 12:00:28 -0800 (PST) Received: (qmail 88264 invoked from network); 5 Jan 2011 20:02:35 -0000 Received: from mail1.iecc.com (64.57.183.56) by mail1.iecc.com with QMQP; 5 Jan 2011 20:02:35 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=iecc.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=142b4.4d24ce5a.k1101; i=johnl@user.iecc.com; bh=xlaf0LhkySGbVoxX/c+qTAEk6chCuIJxg66pmbnumRI=; b=ghe5pwxBWAPqv+eduWXd2KtxpqR/Xi2rNEgC5vPO1WJupNZNsy82PbXNoFF9nZp+U+Q2ujF2GZxb0iKZu/4rigWtKRGwWPxIYDcknEzsXHRcLT2P2EgotkE8SdS8YICPZfF3jGAXeYRTNEz45s0WZBbE1UydSuXaysUHMlbIS7c= DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=taugh.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=142b4.4d24ce5a.k1101; olt=johnl@user.iecc.com; bh=xlaf0LhkySGbVoxX/c+qTAEk6chCuIJxg66pmbnumRI=; b=UnD+c26pfzLmNhH1wqC5bbW1vwvQQ0klazl6jBGWEPebF9KLMkHXMNTBOF1EXTxhlGOzIUsH5ErtuVvq/B5+J14f++gpGt4yoTasXX5uBJp6gScKo68nXP+DUYW0xP303svOfAmdvvg2t+B/Vu9/iNzn58JXKXtDnnXAfE5DDlw= VBR-Info: md=iecc.com; mc=all; mv=dwl.spamhaus.org Date: 5 Jan 2011 20:02:34 -0000 Message-ID: <20110105200234.82611.qmail@joyce.lan> From: John Levine To: asrg@irtf.org In-Reply-To: Organization: Cc: X-Headerized: yes Mime-Version: 1.0 Content-type: text/plain; charset=utf-8 Content-transfer-encoding: 7bit Subject: Re: [Asrg] NXDOMAIN cache behavior, was draft-levine-iprangepub-01 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 20:00:33 -0000 >The authoritative nameservers I've used recently all do the "right thing" >there: no RRs => NXDOMAIN. FYI, BIND 9, which must be the most popular authoritative server on the net, does what 4592 says, NOERROR at an empty nonterminal: $ host -t txt foo.bar.sp.am light.lightlink.com Using domain server: Name: light.lightlink.com Address: 64.57.176.2#53 Aliases: foo.bar.sp.am descriptive text "Hi there" [johnl@joyce ~]$ host -t txt bar.sp.am light.lightlink.com Using domain server: Name: light.lightlink.com Address: 64.57.176.2#53 Aliases: bar.sp.am has no TXT record [johnl@joyce ~]$ host -t txt har.bar.sp.am light.lightlink.com Using domain server: Name: light.lightlink.com Address: 64.57.176.2#53 Aliases: Host har.bar.sp.am not found: 3(NXDOMAIN) > And the recursive servers I have handy also do the "right thing" and >don't synthesize fake answers. No argument there, no cache synthesizes answers, even the ones that have enough DNSSEC data to do so reliably. >For layering something on top of DNS the deployed implementation is >the only spec that matters - what's implemented trumps what would be >nice. Well, for the third time, yeah. >> With that said, I still like my b-tree hack, ... > >If you're going to the effort of adding that much functionality >at the client end, switching to a better matched protocol instead >might be better. All the high traffic DNSBL users already use >a push protocol of sorts. Moving to a better, more standardized >one might be a win. Huh? The functionality to the client is the same, it looks up an IP address and finds out whether or not it's listed. The internal representation is different, and it should cache a lot better. The extra work at the client is actually quite small, a simple loop to walk through some easily parsed records. R's, John From matthias@leisi.net Wed Jan 5 12:05:33 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 165093A6BDB for ; Wed, 5 Jan 2011 12:05:33 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.816 X-Spam-Level: X-Spam-Status: No, score=-2.816 tagged_above=-999 required=5 tests=[AWL=0.161, BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PzIXhqKKtnYs for ; Wed, 5 Jan 2011 12:05:32 -0800 (PST) Received: from mail-qy0-f175.google.com (mail-qy0-f175.google.com [209.85.216.175]) by core3.amsl.com (Postfix) with ESMTP id 47CCE3A6A94 for ; Wed, 5 Jan 2011 12:05:32 -0800 (PST) Received: by qyk8 with SMTP id 8so15927785qyk.13 for ; Wed, 05 Jan 2011 12:07:39 -0800 (PST) Received: by 10.229.227.129 with SMTP id ja1mr14895134qcb.107.1294258058819; Wed, 05 Jan 2011 12:07:38 -0800 (PST) MIME-Version: 1.0 Received: by 10.220.193.2 with HTTP; Wed, 5 Jan 2011 12:07:17 -0800 (PST) In-Reply-To: <20110105200234.82611.qmail@joyce.lan> References: <20110105200234.82611.qmail@joyce.lan> From: Matthias Leisi Date: Wed, 5 Jan 2011 21:07:17 +0100 Message-ID: To: Anti-Spam Research Group - IRTF Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: [Asrg] NXDOMAIN cache behavior, was draft-levine-iprangepub-01 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 20:05:33 -0000 On Wed, Jan 5, 2011 at 9:02 PM, John Levine wrote: > representation is different, and it should cache a lot better. =A0The > extra work at the client is actually quite small, a simple loop to > walk through some easily parsed records. The current proposal is not really a text format, so "easily parsed" does not apply under all circumstances. -- Matthias From johnl@iecc.com Wed Jan 5 12:06:14 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 98E973A6D03 for ; Wed, 5 Jan 2011 12:06:14 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -110.989 X-Spam-Level: X-Spam-Status: No, score=-110.989 tagged_above=-999 required=5 tests=[AWL=0.210, BAYES_00=-2.599, HABEAS_ACCREDITED_SOI=-4.3, RCVD_IN_BSP_TRUSTED=-4.3, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ilbNZ3EcCXIs for ; Wed, 5 Jan 2011 12:06:13 -0800 (PST) Received: from gal.iecc.com (gal.iecc.com [64.57.183.53]) by core3.amsl.com (Postfix) with ESMTP id 723C13A6A94 for ; Wed, 5 Jan 2011 12:06:13 -0800 (PST) Received: (qmail 89261 invoked from network); 5 Jan 2011 20:08:20 -0000 Received: from mail1.iecc.com (64.57.183.56) by mail1.iecc.com with QMQP; 5 Jan 2011 20:08:20 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=iecc.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=14863.4d24cfb4.k1101; i=johnl@user.iecc.com; bh=5++7z6XiR1/4ZDbdnlQFFpYQLFhBjbcwfaQtvaNu1yE=; b=bk+rnNWHqBCeU4yYcRhOY1AxEy/H2+YETafM5LX/+s41dqUKvZP25IilAKLyhUgFBBZOT3W+hSJjIehY2Qp7DQwDKZhQU5AYgge+bsa9CPco2+WVNjevP9N6784SQZUaY62+8Qj64jsFAp1bBUB1IUCc+Jx4FWJswgKpq12tNB8= DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=taugh.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=14863.4d24cfb4.k1101; olt=johnl@user.iecc.com; bh=5++7z6XiR1/4ZDbdnlQFFpYQLFhBjbcwfaQtvaNu1yE=; b=Tagq/XBAdAqNDAXm/HW32HYSQpg895C+1YzgJcd6cMm6vX+yW0CkoCWSzfySqIrswoRJkUldBqIKjp9ArC0FzJhJu/uyDM0pvBnjHcFlZJbzRomY2StocZ+9KTE5wGMNZGADg0gCS1FlM57Q4ySBlJ9l5/T2+PSVyjqkRuqtHrE= VBR-Info: md=iecc.com; mc=all; mv=dwl.spamhaus.org Date: 5 Jan 2011 20:08:20 -0000 Message-ID: <20110105200820.84066.qmail@joyce.lan> From: John Levine To: asrg@irtf.org In-Reply-To: Organization: Cc: X-Headerized: yes Mime-Version: 1.0 Content-type: text/plain; charset=utf-8 Content-transfer-encoding: 7bit Subject: Re: [Asrg] draft-levine-iprangepub-01 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 20:06:14 -0000 >I wasn't able to find a paragraph about ranges with holes in them, or the >representation of the data beyond this point is in/not in the list, in >iprangepub-01. It doesn't handle ranges with holes. If you want a hole, you divide the range. I don't understand what you mean by "data beyond this point is in/not in the list." If you mean checking that an address is less than the first entry in the list or greater than the last entry, the lookup process handles that automatically, as it notes in section 3. >also the concept of increasing resolution with finer-grained querying, >useful for reputation-by-address kinds of databases, seemed absent. Sorry, don't understand what that means, either. If you're asking whether it could return the prefix in which an entry was found rather than just saying the entry was there, of course it can. Regards, John Levine, johnl@iecc.com, Primary Perpetrator of "The Internet for Dummies", Please consider the environment before reading this e-mail. http://jl.ly From davidnicol@gmail.com Wed Jan 5 12:58:33 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id CDFC43A6B8E for ; Wed, 5 Jan 2011 12:58:33 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.855 X-Spam-Level: X-Spam-Status: No, score=-2.855 tagged_above=-999 required=5 tests=[AWL=-0.257, BAYES_00=-2.599, HTML_MESSAGE=0.001] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id p-wz9WXmuoOf for ; Wed, 5 Jan 2011 12:58:32 -0800 (PST) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by core3.amsl.com (Postfix) with ESMTP id 7A7233A6B50 for ; Wed, 5 Jan 2011 12:58:32 -0800 (PST) Received: by iyb26 with SMTP id 26so14912995iyb.13 for ; Wed, 05 Jan 2011 13:00:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:mime-version:received:in-reply-to :references:from:date:message-id:subject:to:content-type; bh=xTZhQ/YaNWiI97ev6bqYcNdoBlEegcaw5Uk4dgIOTK4=; b=oQVW1excuG1H4f+qIZrqpF1/67xknZEEiD1pkQ+ivKXqh6im8Qw/TbhelVNO7QMYjN OiAvuHqyQ10fG0iD47TieD5vVUMfE+KIRf/oZp0l417SJPH+qJ5OCzw3MiVzCVj5lqRi rhTEnNOKn5bgDko0GaNip3Axb71ANigFF5Ono= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; b=QrEYFPZnZKtnKARSyU+HStDoQdDYpf3/EsKi3IxlPU7Y89QbNFm9AkLx6K83jSiIK0 itBqEPfzxKh2/D9uwDZo+kKRG7w4wTONFCSvVe6WIFfRSvAUIfanmtF7kVTdvgyBmoc+ Zlh5g7flnnUZLU3eaWZMFUPoVEzUbEaMwJAFY= Received: by 10.231.40.13 with SMTP id i13mr4663262ibe.190.1294261239386; Wed, 05 Jan 2011 13:00:39 -0800 (PST) MIME-Version: 1.0 Received: by 10.231.206.148 with HTTP; Wed, 5 Jan 2011 13:00:19 -0800 (PST) In-Reply-To: <20110105200820.84066.qmail@joyce.lan> References: <20110105200820.84066.qmail@joyce.lan> From: David Nicol Date: Wed, 5 Jan 2011 15:00:19 -0600 Message-ID: To: Anti-Spam Research Group - IRTF Content-Type: multipart/alternative; boundary=0003255763fe6b912c04991fad94 Subject: Re: [Asrg] draft-levine-iprangepub-01 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 20:58:33 -0000 --0003255763fe6b912c04991fad94 Content-Type: text/plain; charset=UTF-8 On Wed, Jan 5, 2011 at 2:08 PM, John Levine wrote: >also the concept of increasing resolution with finer-grained querying, > >useful for reputation-by-address kinds of databases, seemed absent. > > Sorry, don't understand what that means, either. If you're asking > whether it could return the prefix in which an entry was found rather > than just saying the entry was there, of course it can. Differentiating between a good in a range of bad and a bad in a range of good, or getting some composite information about a range before drilling down further. This feature is most likely a YAGNI. --0003255763fe6b912c04991fad94 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

On Wed, Jan 5, 2011 at 2:08 PM, John Lev= ine <johnl@taugh.co= m> wrote:

>also the concept of increasing resolution with finer-grained querying,<= br> >useful for reputation-by-address kinds of databases, seemed absent.

Sorry, don't understand what that means, either. =C2=A0If you'= ;re asking
whether it could return the prefix in which an entry was found rather
than just saying the entry was there, of course it can.
Differentiating between a good in a range of bad and a bad in = a range of good, or getting some composite information about a range before= drilling down further. This feature is most likely a YAGNI.
--0003255763fe6b912c04991fad94-- From johnl@iecc.com Wed Jan 5 15:18:57 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id E80F23A6E23 for ; Wed, 5 Jan 2011 15:18:57 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -110.995 X-Spam-Level: X-Spam-Status: No, score=-110.995 tagged_above=-999 required=5 tests=[AWL=0.204, BAYES_00=-2.599, HABEAS_ACCREDITED_SOI=-4.3, RCVD_IN_BSP_TRUSTED=-4.3, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gxodb6gpJsRM for ; Wed, 5 Jan 2011 15:18:56 -0800 (PST) Received: from gal.iecc.com (gal.iecc.com [64.57.183.53]) by core3.amsl.com (Postfix) with ESMTP id 9657D3A6CD9 for ; Wed, 5 Jan 2011 15:18:56 -0800 (PST) Received: (qmail 27467 invoked from network); 5 Jan 2011 23:21:02 -0000 Received: from mail1.iecc.com (64.57.183.56) by mail1.iecc.com with QMQP; 5 Jan 2011 23:21:02 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=iecc.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=7fc2.4d24fcde.k1101; i=johnl@user.iecc.com; bh=JHLUOiZfFAVi6VUFqRUiYpDONiiCYTVvtgFy/mkSNkA=; b=kmkdzCfv2tjv4bBun6whPpz6u+POOd0Ei+159holSsjH+REHF5YOB+PKo7xmRljODSgRV7WzufeU2YAsKB3InAkpt7WOhg1xenEz74lpFtbxcFYWSVHDlKOcTQ8DvWn+52ApLtav2dwscgVFFw3s2Wp8CfZfSzDyqbWJG4dhABY= DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=taugh.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=7fc2.4d24fcde.k1101; olt=johnl@user.iecc.com; bh=JHLUOiZfFAVi6VUFqRUiYpDONiiCYTVvtgFy/mkSNkA=; b=QprRd4TmzFwNuhW+LCz8LAMnI9oO6ffst/xZByV3JxmrxGAhpPQ2hWA1zVD8sGjjckqT1h5kJS9R+UuafK+OsAO6XCNZ9H9aU0IAVQp0qmWg0AEzJ8uTgM81cZtDwyWwxTlN9/C/Ja+gzLzPJGPmixmqWt9jKMXfR9nGFx94VoE= VBR-Info: md=iecc.com; mc=all; mv=dwl.spamhaus.org Date: 5 Jan 2011 23:21:02 -0000 Message-ID: <20110105232102.32705.qmail@joyce.lan> From: John Levine To: asrg@irtf.org In-Reply-To: Organization: Cc: X-Headerized: yes Mime-Version: 1.0 Content-type: text/plain; charset=utf-8 Content-transfer-encoding: 7bit Subject: Re: [Asrg] draft-levine-iprangepub-01 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 23:18:58 -0000 >Differentiating between a good in a range of bad and a bad in a range of >good, or getting some composite information about a range before drilling >down further. This feature is most likely a YAGNI. That sounds like an interesting problem, but it's not one that DNSBLs have tried to address, and it's not one that I'm trying to solve. Back when we were running DAC, after we did VBR which lets you publish what is basically a single bit saying "this signer is good", we looked at ways to publish more complex reputation info, and what we found was that we don't understand the issues well enough to try to standardize anything. Regards, John Levine, johnl@iecc.com, Primary Perpetrator of "The Internet for Dummies", Please consider the environment before reading this e-mail. http://jl.ly From CLEWIS@nortel.com Wed Jan 5 15:59:29 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id C8AE73A6D0E for ; Wed, 5 Jan 2011 15:59:29 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.499 X-Spam-Level: X-Spam-Status: No, score=-6.499 tagged_above=-999 required=5 tests=[AWL=0.100, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eN6NesOMWNN9 for ; Wed, 5 Jan 2011 15:59:28 -0800 (PST) Received: from zrtps0kp.nortel.com (zrtps0kp.nortel.com [47.140.192.56]) by core3.amsl.com (Postfix) with ESMTP id 707EC3A6CD9 for ; Wed, 5 Jan 2011 15:59:28 -0800 (PST) Received: from zrtphxs1.corp.nortel.com (zrtphxs1.corp.nortel.com [47.140.202.46]) by zrtps0kp.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id p0601Ui27557 for ; Thu, 6 Jan 2011 00:01:31 GMT Received: from zrtphx5h0.corp.nortel.com ([47.140.202.65]) by zrtphxs1.corp.nortel.com with Microsoft SMTPSVC(6.0.3790.4675); Wed, 5 Jan 2011 19:00:24 -0500 Received: from [47.130.64.29] (47.130.64.29) by zrtphx5h0.corp.nortel.com (47.140.202.65) with Microsoft SMTP Server (TLS) id 8.1.340.0; Wed, 5 Jan 2011 19:00:23 -0500 Message-ID: <4D250616.4090305@nortel.com> Date: Wed, 5 Jan 2011 19:00:22 -0500 From: "Chris Lewis" Organization: Nortel User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.23) Gecko/20090812 Thunderbird/2.0.0.23 Mnenhy/0.7.6.666 MIME-Version: 1.0 To: References: <20110105064424.81429.qmail@joyce.lan> In-Reply-To: Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 06 Jan 2011 00:00:24.0488 (UTC) FILETIME=[B7DE1280:01CBAD34] Subject: Re: [Asrg] NXDOMAIN cache behavior, was draft-levine-iprangepub-01 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 23:59:29 -0000 On 1/5/2011 2:41 AM, Steve Atkins wrote: > > On Jan 4, 2011, at 10:44 PM, John Levine wrote: >> With that said, I still like my b-tree hack, which makes queries that >> shouldn't get either NOERROR or NXDOMAIN, a lot better as a way to >> publish ranges of addresses in a DNSxL. > > If you're going to the effort of adding that much functionality > at the client end, switching to a better matched protocol instead > might be better. All the high traffic DNSBL users already use > a push protocol of sorts. Moving to a better, more standardized > one might be a win. A local server for protocol X could still offer > a DNS interface to the MTA to ease implementation I must confess to a bit of confusion. We seem to be talking about really naive IPv6 DNSBL implementations blowing out caches on high volume, and yet, at the same time, we acknowledge that the large DNSBL users already download zone files. Rsync is essentially the defacto standard for bulk DNSBL transfer, and as you say it's "not awful". So, we don't seem to have a significant difficulty with that. I do know of one (commercial) blacklisting system that does use a more explicitly "incremental" distribution mechanism, but it's probably not that much better than rsync, and in fact it may be worse. Even if we were to do something as simplistic as chop IPv6 queries at the /64, given that the number of spammers and bots doesn't magically go up simply because there's more bits to hide in, the caching problem appears to not that much worse than it already is with IPv4. IOW, some of the discussion threads here seem to be solutions looking for problems. Clearly things are going to shift somewhat. But, it doesn't look like the real future is much more than: 1) Some mechanism for CBL/XBL single-IP DNSBLs to remain useful (eg: hardcoded /64 truncation or some mechanism like John's) for Internet query from small sites. 2) Zone download (Rsync or perhaps something better) becoming more prevalent. 3) DNSBL operators will be more conscious of query load and will more forcefully terminate abusers. Perhaps we might do more work in (2), to specify zone formats for download. While the tradeoff volumes for query versus zone downloads/incrementals may well shift, it will just about be never advantageous for small sites doing a few dozen emails per day to take a whole zone of something as big as the XBL. Besides, in many cases, that introduces latency delays. From steve@blighty.com Wed Jan 5 17:04:59 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 27D223A6D79 for ; Wed, 5 Jan 2011 17:04:59 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.88 X-Spam-Level: X-Spam-Status: No, score=-1.88 tagged_above=-999 required=5 tests=[AWL=0.719, BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Z2ViRDnxwYhJ for ; Wed, 5 Jan 2011 17:04:57 -0800 (PST) Received: from m.wordtothewise.com (b.wordtothewise.com [208.187.80.136]) by core3.amsl.com (Postfix) with ESMTP id AC8F93A6D1C for ; Wed, 5 Jan 2011 17:04:57 -0800 (PST) Received: from platter.wordtothewise.com (184.wordtothewise.com [208.187.80.184]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: steve) by m.wordtothewise.com (Postfix) with ESMTPSA id A07212DDD6 for ; Wed, 5 Jan 2011 17:07:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wordtothewise.com; s=1.wttw; t=1294276024; bh=FcZGrx5J7h5FxsG/+HkFGdpnZWHBF7PmN042MRRmLHI=; h=Content-Type:Mime-Version:Subject:From:In-Reply-To:Date: Content-Transfer-Encoding:Message-Id:References:To; b=cEnfzRpYiUFJw9sreY52m8ECVq3DQqD3IbuGj9Ig7nN59VpkEAIsCjl8j2+ttBoG0 VaPB9Yd0iFwk/ZCah2SNTNO3iWBD0COo97x2UexT35rSoBxyGShL3khMFDBB93JE2Q cFYInhAbxOtdDUjLtb1q6g8HXoo/9JPhbHO01864= Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Apple Message framework v1082) From: Steve Atkins In-Reply-To: <4D250616.4090305@nortel.com> Date: Wed, 5 Jan 2011 17:07:04 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20110105064424.81429.qmail@joyce.lan> <4D250616.4090305@nortel.com> To: Anti-Spam Research Group - IRTF X-Mailer: Apple Mail (2.1082) Subject: Re: [Asrg] NXDOMAIN cache behavior, was draft-levine-iprangepub-01 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Jan 2011 01:04:59 -0000 On Jan 5, 2011, at 4:00 PM, Chris Lewis wrote: > On 1/5/2011 2:41 AM, Steve Atkins wrote: >>=20 >> On Jan 4, 2011, at 10:44 PM, John Levine wrote: >=20 >>> With that said, I still like my b-tree hack, which makes queries = that >>> shouldn't get either NOERROR or NXDOMAIN, a lot better as a way to >>> publish ranges of addresses in a DNSxL. >>=20 >> If you're going to the effort of adding that much functionality >> at the client end, switching to a better matched protocol instead >> might be better. All the high traffic DNSBL users already use >> a push protocol of sorts. Moving to a better, more standardized >> one might be a win. A local server for protocol X could still offer >> a DNS interface to the MTA to ease implementation >=20 > I must confess to a bit of confusion. We seem to be talking about = really naive IPv6 DNSBL implementations blowing out caches on high = volume, and yet, at the same time, we acknowledge that the large DNSBL = users already download zone files. IPv6 is more likely to blow out caches than IPv4, I think. That's going = to increase the load an end-site places on a central server quite = significantly. And _that_ is going to mean that an IPv6 based list may = end up with more "large" DNSBL users than current IPv4 based systems, as = the same end-site might generate significantly more IPv6 queries than = IPv4. Or maybe it won't. Heck, I'm expecting there to be very little IPv6 = mail, legitimate or otherwise, for quite a long time. > Rsync is essentially the defacto standard for bulk DNSBL transfer, and = as you say it's "not awful". So, we don't seem to have a significant = difficulty with that. AIUI, rsync is reasonably expensive to offer (in terms of CPU load), and = it adds significant latency to listing and delisting (as you say below). = Do you have any numbers on system load vs users vs update rate = conveniently to hand? > I do know of one (commercial) blacklisting system that does use a more = explicitly "incremental" distribution mechanism, but it's probably not = that much better than rsync, and in fact it may be worse. Google push a large chunk of their entire domain blacklist to every = end-user browser (using a mix of incremental updates, broad blocks = pushed to the browser, with queries to a central server for more = specific checks when there's a broad block match, explicit invalidation = of locally stored entries and so on). It's not the same problem being = solved as IP based blacklist distribution, but it's something that gives = some good ideas. I don't think that looking at an alternate WAN distribution protocol is = something we *have* to do, but if we're looking at changing behaviour at = the MTA location anyway, now's a good time to think about whether we do = want to do that or not, and what the tradeoffs might be. We're a = research group, are we not? :) > Even if we were to do something as simplistic as chop IPv6 queries at = the /64, given that the number of spammers and bots doesn't magically go = up simply because there's more bits to hide in, the caching problem = appears to not that much worse than it already is with IPv4. The number of spammers may not go up, but the number of addresses they = use may. Or may not, it depends on their behaviour. But I'm not betting = that it won't. > IOW, some of the discussion threads here seem to be solutions looking = for problems. >=20 > Clearly things are going to shift somewhat. But, it doesn't look like = the real future is much more than: >=20 > 1) Some mechanism for CBL/XBL single-IP DNSBLs to remain useful (eg: = hardcoded /64 truncation or some mechanism like John's) for Internet = query from small sites. I think this bit is really tricky, and has lots of possibilities for = over-thinking the problem. Blocking by /64 seems both a good idea, and = probably good enough for small sites. > 2) Zone download (Rsync or perhaps something better) becoming more = prevalent. Yup. I think zone download is going to become more prevalent (to the = extent that I think even rsync is going to get too painful to offer in = some cases, and something that looks more like IXFR is worth another = look). > 3) DNSBL operators will be more conscious of query load and will more = forcefully terminate abusers. >=20 > Perhaps we might do more work in (2), to specify zone formats for = download. >=20 > While the tradeoff volumes for query versus zone = downloads/incrementals may well shift, it will just about be never = advantageous for small sites doing a few dozen emails per day to take a = whole zone of something as big as the XBL. No, it wouldn't. (Then again, there's no such thing as a site doing a = few dozen emails a day, I don't believe. The spam alone is going to be a = couple of orders of magnitude more than that :) ). > Besides, in many cases, that introduces latency delays. It only introduces latency delays because rsync is not a good protocol = for it - it's not dreadful, but it's not great. Cheers, Steve= From johnl@iecc.com Wed Jan 5 17:05:08 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id BEBF53A6DDA for ; Wed, 5 Jan 2011 17:05:08 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -111 X-Spam-Level: X-Spam-Status: No, score=-111 tagged_above=-999 required=5 tests=[AWL=0.199, BAYES_00=-2.599, HABEAS_ACCREDITED_SOI=-4.3, RCVD_IN_BSP_TRUSTED=-4.3, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id C6zYyTMgfcnt for ; Wed, 5 Jan 2011 17:05:07 -0800 (PST) Received: from gal.iecc.com (gal.iecc.com [64.57.183.53]) by core3.amsl.com (Postfix) with ESMTP id DF05B3A6E2C for ; Wed, 5 Jan 2011 17:05:06 -0800 (PST) Received: (qmail 44138 invoked from network); 6 Jan 2011 01:07:13 -0000 Received: from mail1.iecc.com (64.57.183.56) by mail1.iecc.com with QMQP; 6 Jan 2011 01:07:13 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=iecc.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=e8b3.4d2515c1.k1101; i=johnl@user.iecc.com; bh=pYAkzDKW9sLQlmGrWGq9ed5c2at7b3moyDXyDbRSW04=; b=LLJmC9KBvzuGKIrQHBitdVrJgVDp7FlNmlzH3/5iA9UT9EtJL9Nk3OXSf4iX/uvXSZdtF48q4dRXz8iFgJnmHYR7SsCi8RLIbXqXMOa1CnmRcZU7k9Y8pWbvM6nLw3ZEWtDFD2E7szsQjZ7eGTNZPk/K6PyehUjrbSz4i5LW3Aw= DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=taugh.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=e8b3.4d2515c1.k1101; olt=johnl@user.iecc.com; bh=pYAkzDKW9sLQlmGrWGq9ed5c2at7b3moyDXyDbRSW04=; b=CdD38WDlkdqCGzdfsLD1vniM+l/Elg87lwO+dNLK70AIP46I/DiMRgoJK7oUZTlP02KwEcSWO1iwiTwGHUzSG8VVgZjplj5z59LzfNm1f6/DH/kAJyE27TmPHUlzdj5DppcQcEM0pEi/Oiex/mFZwOL408wNVHKNfRLLomD25tY= VBR-Info: md=iecc.com; mc=all; mv=dwl.spamhaus.org Date: 6 Jan 2011 01:07:13 -0000 Message-ID: <20110106010713.59570.qmail@joyce.lan> From: John Levine To: asrg@irtf.org In-Reply-To: <4D250616.4090305@nortel.com> Organization: Cc: X-Headerized: yes Mime-Version: 1.0 Content-type: text/plain; charset=utf-8 Content-transfer-encoding: 7bit Subject: Re: [Asrg] NXDOMAIN cache behavior, was draft-levine-iprangepub-01 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Jan 2011 01:05:08 -0000 >Even if we were to do something as simplistic as chop IPv6 queries at >the /64, given that the number of spammers and bots doesn't magically go >up simply because there's more bits to hide in, the caching problem >appears to not that much worse than it already is with IPv4. Everyone tells me that in IPv6 we're going to be using a lot of whitelists. I don't think it's a good idea to whitelist /64s, and people are checking individual IPs and spammers change addresses which IPv6 makes easy, you'll have load problems. I agree that blacklists can probably do OK just by truncating to /64s, but my thinking is that if everyone is going to change their filtering software to handle IPv6, they'll do it by using some lookup library or other, so this is the time to develop libraries that will work for all the tasks they're likely to be used for. R's, John From CLEWIS@nortel.com Thu Jan 6 23:17:26 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 8B7723A67D1 for ; Thu, 6 Jan 2011 23:17:26 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.512 X-Spam-Level: X-Spam-Status: No, score=-6.512 tagged_above=-999 required=5 tests=[AWL=0.087, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zwIGuVo6KtFj for ; Thu, 6 Jan 2011 23:17:25 -0800 (PST) Received: from zrtps0kp.nortel.com (zrtps0kp.nortel.com [47.140.192.56]) by core3.amsl.com (Postfix) with ESMTP id 1E8CB3A67C2 for ; Thu, 6 Jan 2011 23:17:24 -0800 (PST) Received: from zrtphxs1.corp.nortel.com (zrtphxs1.corp.nortel.com [47.140.202.46]) by zrtps0kp.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id p077JSv18886 for ; Fri, 7 Jan 2011 07:19:28 GMT Received: from zrtphx5h0.corp.nortel.com ([47.140.202.65]) by zrtphxs1.corp.nortel.com with Microsoft SMTPSVC(6.0.3790.4675); Fri, 7 Jan 2011 02:19:27 -0500 Received: from [47.130.80.173] (47.130.80.173) by zrtphx5h0.corp.nortel.com (47.140.202.65) with Microsoft SMTP Server (TLS) id 8.1.340.0; Fri, 7 Jan 2011 02:19:26 -0500 Message-ID: <4D26BE7C.3080203@nortel.com> Date: Fri, 7 Jan 2011 02:19:24 -0500 From: "Chris Lewis" Organization: Nortel User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.23) Gecko/20090812 Thunderbird/2.0.0.23 Mnenhy/0.7.6.666 MIME-Version: 1.0 To: References: <20110105064424.81429.qmail@joyce.lan> <4D250616.4090305@nortel.com> In-Reply-To: Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 07 Jan 2011 07:19:27.0144 (UTC) FILETIME=[37BA2280:01CBAE3B] Subject: Re: [Asrg] NXDOMAIN cache behavior, was draft-levine-iprangepub-01 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Jan 2011 07:17:26 -0000 On 1/5/2011 8:07 PM, Steve Atkins wrote: > > On Jan 5, 2011, at 4:00 PM, Chris Lewis wrote: >> Rsync is essentially the defacto standard for bulk DNSBL transfer, and as you say it's "not awful". So, we don't seem to have a significant difficulty with that. > > AIUI, rsync is reasonably expensive to offer (in terms of CPU load), and it adds significant latency to listing and delisting (as you say below). Do you have any numbers on system load vs users vs update rate conveniently to hand? AIUI, rsync really isn't that bad on large zone files (with one caveat, see below), unless compression is turned on (CPU), and even then, you can support several hundred downloaders (mixed compression and non-compression) on reasonably modest hardware. The rsyncd can also force non-compression if deemed necessary. Of course bandwidth can also be an issue. I wouldn't want to do the above with anything less than a 20Mb/S Internet link... Rsync behaves quite badly, however, on large zone files if you're updating random locations in the zone. Eg: if you're doing time-based expiration, you damn well better sort the file in entry "first seen" timestamp order, so the bulk part of the adds and deletes are contiguous. Don't change, say, existing TXT entries - once the entry goes in, it shouldn't change until it disappears. The performance numbers I have are from a strangeish case, so, you'll want to get representative numbers from a more straightforward one. Like Spamhaus. > I don't think that looking at an alternate WAN distribution protocol is something we *have* to do, but if we're looking at changing behaviour at the MTA location anyway, now's a good time to think about whether we do want to do that or not, and what the tradeoffs might be. We're a research group, are we not? :) No. Other than, maybe, zone file formats. This can encourage the development of common ancilliary tools to manipulate the zone files before internal publication. For example, as a performance improvement, we download almost all of our DNSBLs and merge them into one zone, with locally assigned return codes for each of the originating DNSBLs. One query, and you can get many A records back, each of which is uniquely identifiable (by the A value) as to which DNSBL it came from, so you can score them independently (which we do). It would have been nice if the DNSBL file formats was better standardized. The merge code is a bit smelly. It'll work on (most) rbldnsd-compatible zones, but everything else gives the coder (me) heartburn ;-) >> Even if we were to do something as simplistic as chop IPv6 queries at the /64, given that the number of spammers and bots doesn't magically go up simply because there's more bits to hide in, the caching problem appears to not that much worse than it already is with IPv4. > > The number of spammers may not go up, but the number of addresses they use may. Or may not, it depends on their behaviour. But I'm not betting that it won't. Given the experience and impressions I've acquired - at least in the botspam space, IP hopping is actually quite rare in IPv4. The cases I've seen are actually hard to tell whether they're due to load balanced multiple exit NATs, or real and explicit IP hopping by the bot. As it's been explained to me, "real" IP hopping is usually just about impossible (on ISPs) with more modern hardware that enforces which IP you've authenticated to on _that_ wire, and you lose connectivity altogether if you change your IP (and this is even _within_ the subnet your local ISP switch is operating in). If each IPv6 user is getting a, say, /64 for their allocation, but the ISP hardware capabilities don't change, they can easily change their IP within the /64, but not outside of it. Hence, if we fix the DNSBL cutpoint at the same level, they can't hop far enough to get out from under the listing... This doesn't necessitate that the cutpoint is 100% applicable, only that it's the majority. Mind you, if IPv6 is so plentiful, snow-shoers will likely be able to get far more IP blocks than they can now, and get harder to block. I'm really beginning to think that it's still quite premature to do much in this space. We have to wait and see how the environment evolves before we can predict what will work. And as you say, it's probably still quite some time off before it matters. It's nice to see public experiments in IPv6 DNSBLs, but, I really have to wonder about the usefulness of it at the present time. Of the hundreds of DNSBLs that exist, only a dozen or two block that much on their own. Couple that with the dearth of IPv6 MTAs, I can't help thinking that such DNSBLs are essentially useless except as proof-of-concept. >> 1) Some mechanism for CBL/XBL single-IP DNSBLs to remain useful (eg: hardcoded /64 truncation or some mechanism like John's) for Internet query from small sites. > I think this bit is really tricky, and has lots of possibilities for over-thinking the problem. Blocking by /64 seems both a good idea, and probably good enough for small sites. It strikes me that it may well be advantageous to pick an arbitrary cut-point and encourage even the medium to large sites to continue using it at that level. You'd be building an additional (but small) factor of possible FPs into it, giving the hosters more incentive to clean up the bad IPs faster, because they may well have someone else yelling at them. Consider the following thought experiment: if the XBL rounded off every entry to a /31 (in IPv4), every listing has a (actually quite small) possibility of blocking an innocent IP that someone will be impacted by. I think providers would have a disproportionally higher incentive to fix the bad IP. But the real FP impact would be extremely low. >> 2) Zone download (Rsync or perhaps something better) becoming more prevalent. > Yup. I think zone download is going to become more prevalent (to the extent that I think even rsync is going to get too painful to offer in some cases, and something that looks more like IXFR is worth another look). I thought IXFR was abandoned because it was too expensive. Or perhaps that support was pretty poor. AXFR is obviously out of the question. >> While the tradeoff volumes for query versus zone downloads/incrementals may well shift, it will just about be never advantageous for small sites doing a few dozen emails per day to take a whole zone of something as big as the XBL. > > No, it wouldn't. (Then again, there's no such thing as a site doing a few dozen emails a day, I don't believe. Sure there are servers that small. Mine - it does even less ;-) > The spam alone is going to be a couple of orders of magnitude more than that :) ). Spammers haven't found it yet. Open relay probers have tho. >> Besides, in many cases, that introduces latency delays. > > It only introduces latency delays because rsync is not a good protocol for it Rsync's batch-ish nature isn't quite why I mentioned latency. The latency is zone rebuild time at the DNSBL server. With a zone file as large as the XBL, say, even if you did some sort of incremental update directly off a core DBMS in realish-time, I think you'd run into performance problems. I've been doing experiments aimed towards UDP-based DNSBL adds, and even that starts to get nasty at the levels I think major DNSBLs would see. I know of systems that can get "bad IPs" from detectors thru to moderately large numbers of clients within a handful of seconds. But I don't know how big the system is (clients nor magnitude of bad IPs), nor am I confident of its stability in a heterogenous client environment with multiple code authors. These are commercial for-pay environments, who are obviously much smaller audiences than the good "free" ones. From johnl@iecc.com Fri Jan 7 14:08:28 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 9DA0528C0F2 for ; Fri, 7 Jan 2011 14:08:28 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -111.005 X-Spam-Level: X-Spam-Status: No, score=-111.005 tagged_above=-999 required=5 tests=[AWL=0.194, BAYES_00=-2.599, HABEAS_ACCREDITED_SOI=-4.3, RCVD_IN_BSP_TRUSTED=-4.3, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id P-5-R3C+fpUU for ; Fri, 7 Jan 2011 14:08:27 -0800 (PST) Received: from gal.iecc.com (gal.iecc.com [64.57.183.53]) by core3.amsl.com (Postfix) with ESMTP id 7A25C3A6945 for ; Fri, 7 Jan 2011 14:08:26 -0800 (PST) Received: (qmail 59064 invoked from network); 7 Jan 2011 22:08:51 -0000 Received: from mail1.iecc.com (64.57.183.56) by mail1.iecc.com with QMQP; 7 Jan 2011 22:08:51 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=iecc.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=e1c2.4d278ef2.k1101; i=johnl@user.iecc.com; bh=Ldwch0YDruOQGBKCJ8Vp7GCyuMHIChrLtj7ZYkMxC94=; b=KcJ364DwymPXmg7xswaPsgOvsshUm0RCbTTbImJpD9lphOcOYCDELaKRyFWjONvyLw/kyuVY2lbSL2Q7Kq8ls/bEzM+08wd91V/uUV3T4dwqRuF132XRuoJw9//Jypjxi3JqpD0AU+LPs/VJk/eh1y088+KTuF/lrc7scPc6N8w= DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=taugh.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=e1c2.4d278ef2.k1101; olt=johnl@user.iecc.com; bh=Ldwch0YDruOQGBKCJ8Vp7GCyuMHIChrLtj7ZYkMxC94=; b=OqDbbEcWfOgluK8yPBowFgIdcaklK2yBsT5zI9D3spNhxQ9ylnFd1D3TR1ydiOU2nXalthJ0pOCtrgKrCxx73K72FQ2xjbpv0M7fOBA5fKwb9hoS3ivWeTj6GtxIYjQhX5hpFVJvLSwbHdo1ctbDaKoiPQKwymGdIgcTHDymzyg= VBR-Info: md=iecc.com; mc=all; mv=dwl.spamhaus.org Date: 7 Jan 2011 22:08:50 -0000 Message-ID: <20110107220850.57793.qmail@joyce.lan> From: John Levine To: asrg@irtf.org In-Reply-To: <4D26BE7C.3080203@nortel.com> Organization: Cc: X-Headerized: yes Mime-Version: 1.0 Content-type: text/plain; charset=utf-8 Content-transfer-encoding: 7bit Subject: Re: [Asrg] download approaches X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Jan 2011 22:08:28 -0000 >I thought IXFR was abandoned because it was too expensive. Or >perhaps that support was pretty poor. AXFR is obviously out of the >question. Does anyone know if any DNS software other than BIND supports IXFR? Supported or not, it's not very clever, just ships a list of NS records to delete and a list to add, relative to the version of the zone that the client says it has. The server has to have (or be prepared to generate) the diffs from all the stale versions of the zone that a client might ask for. If we know that a file is a list of IP addresses or a list of prefixes and lengths, we can presumably design an efficient representation of the differences. I'm more interested in how we'd distribute it. For example, the oft-cited ClamAV distributes via a very large network of HTTP mirrors. That's not awful, but I wouldn't want to have to come up with such a network for every BL in the world. It seems to me that there's always going to be a point below which it's better to query for individual answers, and above which it's better to maintain a mirror, so whatever we do we should be prepared for both. I also think that it wouldn't be hard to come up with something that caches a lot better than current BLs, which would move the query/mirror line up considerably. R's, John From franck@avonsys.com Fri Jan 7 14:41:13 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id E3E8D3A6948 for ; Fri, 7 Jan 2011 14:41:13 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ofWsJmQuasCV for ; Fri, 7 Jan 2011 14:41:08 -0800 (PST) Received: from seine.avonsys.com (unknown [IPv6:2001:df0:67:1::129]) by core3.amsl.com (Postfix) with ESMTP id 9D7DF3A6935 for ; Fri, 7 Jan 2011 14:41:05 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by seine.avonsys.com (Postfix) with ESMTP id E266BDFC002 for ; Sat, 8 Jan 2011 11:43:04 +1300 (FJST) X-Virus-Scanned: amavisd-new at avonsys.com Received: from seine.avonsys.com ([127.0.0.1]) by localhost (seine.avonsys.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YQhHLS6fOFiE for ; Sat, 8 Jan 2011 11:42:58 +1300 (FJST) Received: from seine.avonsys.com (seine.avonsys.com [192.168.69.129]) by seine.avonsys.com (Postfix) with ESMTP id 4EAEADFC001 for ; Sat, 8 Jan 2011 11:42:58 +1300 (FJST) Date: Sat, 8 Jan 2011 10:42:58 +1200 (FJT) From: Franck Martin To: Anti-Spam Research Group - IRTF Message-ID: <1059310.221.1294440172110.JavaMail.franck@franck-martins-macbook-pro.local> In-Reply-To: <20110107220850.57793.qmail@joyce.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [192.168.69.112] X-Mailer: Zimbra 6.0.5_GA_2213.UBUNTU6 (Zimbra Desktop/2.0.1_10659_Mac) Subject: Re: [Asrg] download approaches X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Jan 2011 22:41:14 -0000 Seems GIT with puppet could be a better fit? Or some kind of revision control system... Git allows you to share patches (between repositories), and puppet can make it automatic... http://en.wikipedia.org/wiki/Git_%28software%29 Franck Martin http://www.avonsys.com/ http://www.facebook.com/Avonsys twitter: FranckMartin Avonsys Check your domain reputation: http://gurl.im/b69d4o ----- Original Message ----- From: "John Levine" To: asrg@irtf.org Sent: Saturday, 8 January, 2011 11:08:50 AM Subject: Re: [Asrg] download approaches >I thought IXFR was abandoned because it was too expensive. Or >perhaps that support was pretty poor. AXFR is obviously out of the >question. Does anyone know if any DNS software other than BIND supports IXFR? Supported or not, it's not very clever, just ships a list of NS records to delete and a list to add, relative to the version of the zone that the client says it has. The server has to have (or be prepared to generate) the diffs from all the stale versions of the zone that a client might ask for. If we know that a file is a list of IP addresses or a list of prefixes and lengths, we can presumably design an efficient representation of the differences. I'm more interested in how we'd distribute it. For example, the oft-cited ClamAV distributes via a very large network of HTTP mirrors. That's not awful, but I wouldn't want to have to come up with such a network for every BL in the world. It seems to me that there's always going to be a point below which it's better to query for individual answers, and above which it's better to maintain a mirror, so whatever we do we should be prepared for both. I also think that it wouldn't be hard to come up with something that caches a lot better than current BLs, which would move the query/mirror line up considerably. R's, John _______________________________________________ Asrg mailing list Asrg@irtf.org http://www.irtf.org/mailman/listinfo/asrg From steve@blighty.com Fri Jan 7 14:57:41 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 6866B28C17F for ; Fri, 7 Jan 2011 14:57:41 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[AWL=0.599, BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RaCLQKCOyPJK for ; Fri, 7 Jan 2011 14:57:40 -0800 (PST) Received: from m.wordtothewise.com (b.wordtothewise.com [208.187.80.136]) by core3.amsl.com (Postfix) with ESMTP id 5F27128C17E for ; Fri, 7 Jan 2011 14:57:40 -0800 (PST) Received: from platter.wordtothewise.com (184.wordtothewise.com [208.187.80.184]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: steve) by m.wordtothewise.com (Postfix) with ESMTPSA id 324B72DDD9 for ; Fri, 7 Jan 2011 14:59:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wordtothewise.com; s=1.wttw; t=1294441186; bh=yPGFEqg+RnvQUPkkzJrZhMwYE7P23nsFo+/rEoXnmAw=; h=Content-Type:Mime-Version:Subject:From:In-Reply-To:Date: Content-Transfer-Encoding:Message-Id:References:To; b=FiWxBrK+hn8IAOyJYBoDiv1mb6HyUhRCyqtzC+KBVGVF8xA22vY18oNYZZmcYDkSM CoULEeKTPwAEvNOYHLibHSOFnh9i/0x/0hoexXn/wnz24YEQuXdY3m10rU/JIBIKsA Kgz4tgpgeflEY1Er+23hj1DAWVRT8PL+5LmKmtNw= Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Apple Message framework v1082) From: Steve Atkins In-Reply-To: <20110107220850.57793.qmail@joyce.lan> Date: Fri, 7 Jan 2011 14:59:39 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <9C920AAC-225A-49E0-80F0-F61F81E330A2@blighty.com> References: <20110107220850.57793.qmail@joyce.lan> To: Anti-Spam Research Group - IRTF X-Mailer: Apple Mail (2.1082) Subject: Re: [Asrg] download approaches X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Jan 2011 22:57:41 -0000 On Jan 7, 2011, at 2:08 PM, John Levine wrote: >> I thought IXFR was abandoned because it was too expensive. Or >> perhaps that support was pretty poor. AXFR is obviously out of the >> question. >=20 > Does anyone know if any DNS software other than BIND supports IXFR? I'm not even convinced that bind really *supports* it. :) IXFR isn't useful unless you're maintaining your entire zone in a transactional (database or dynamic update driven) way, and that's not that common a case for a typical zone. (It is a common case for a DNSBL, or could be quite easily.) > Supported or not, it's not very clever, just ships a list of NS > records to delete and a list to add, relative to the version of the > zone that the client says it has. The server has to have (or be > prepared to generate) the diffs from all the stale versions of the > zone that a client might ask for. Yes. For any decently designed server that's going to be a *lot* cheaper to do than rsync, especially for zones that change a lot. There are some obvious tweaks you can do to make those deltas more cacheable, but they'll be pretty easy to generate (and cache) on = the fly, if you're using a database backend, anyway (blacklist operators using flat files and scripts to process their data, rather than = something transactional, are likely to be out of luck). > If we know that a file is a list of IP addresses or a list of prefixes > and lengths, we can presumably design an efficient representation of > the differences. I'm more interested in how we'd distribute it. For > example, the oft-cited ClamAV distributes via a very large network of > HTTP mirrors. That's not awful, but I wouldn't want to have to come > up with such a network for every BL in the world. There are a bunch of ways to do that, but there's one distinction between them - that's access control. Distributing the list widely and cheaply without access control is one problem. Distributing it with access control, along a controlled distribution path is another. I've not thought about the second case much, but there are some ways to layer the first case on top of a (cacheable, CDN-able) http distribution layer fairly efficiently, and that's a content = distribution platform that "everybody" already supports cheaply. > It seems to me that there's always going to be a point below which > it's better to query for individual answers, and above which it's > better to maintain a mirror, so whatever we do we should be prepared > for both. I also think that it wouldn't be hard to come up with > something that caches a lot better than current BLs, Yes, it would. I'm less convinced that it's easy to come up with something that's both significantly better (in terms of query load and implementation complexity tradeoffs) and also layered on top of a simple key-value store such as DNS. Well worth a try, though. > which would move > the query/mirror line up considerably. Yup. OTOH, a cheap, efficient way of running a local mirror would move it in the other direction. Cheers, Steve From matthias@leisi.net Sat Jan 8 12:59:03 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 14E9D3A6831 for ; Sat, 8 Jan 2011 12:59:03 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.828 X-Spam-Level: X-Spam-Status: No, score=-2.828 tagged_above=-999 required=5 tests=[AWL=0.148, BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, NORMAL_HTTP_TO_IP=0.001, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jgJYbiy5b6c3 for ; Sat, 8 Jan 2011 12:59:01 -0800 (PST) Received: from mail-qy0-f175.google.com (mail-qy0-f175.google.com [209.85.216.175]) by core3.amsl.com (Postfix) with ESMTP id 75F803A6830 for ; Sat, 8 Jan 2011 12:59:01 -0800 (PST) Received: by qyk8 with SMTP id 8so388656qyk.13 for ; Sat, 08 Jan 2011 13:01:09 -0800 (PST) Received: by 10.229.236.196 with SMTP id kl4mr23953685qcb.69.1294520468512; Sat, 08 Jan 2011 13:01:08 -0800 (PST) MIME-Version: 1.0 Received: by 10.220.193.2 with HTTP; Sat, 8 Jan 2011 13:00:48 -0800 (PST) In-Reply-To: <4D26BE7C.3080203@nortel.com> References: <20110105064424.81429.qmail@joyce.lan> <4D250616.4090305@nortel.com> <4D26BE7C.3080203@nortel.com> From: Matthias Leisi Date: Sat, 8 Jan 2011 22:00:48 +0100 Message-ID: To: Anti-Spam Research Group - IRTF Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: [Asrg] NXDOMAIN cache behavior, was draft-levine-iprangepub-01 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Jan 2011 20:59:03 -0000 On Fri, Jan 7, 2011 at 8:19 AM, Chris Lewis wrote: > It's nice to see public experiments in IPv6 DNSBLs, but, I really have to > wonder about the usefulness of it at the present time. =A0Of the hundreds= of > DNSBLs that exist, only a dozen or two block that much on their own. =A0C= ouple > that with the dearth of IPv6 MTAs, I can't help thinking that such DNSBLs > are essentially useless except as proof-of-concept. Even having proof-of-concepts is valuable -- otherwise, everybody will wait for everybody else. Chicken, meet Egg :) There has been quite some discussion on whether (more-or-less-) arbitrary limits should be chosen ("only allow /64" etc), and some proposals have appeared. I now implemented an extremely simple proposal (note that this is experimental only and not useful for production purposes). The basic idea is that the server will *always* return multiple answers (up to 3): 1) The listed entry "below" the one requested 2) The listed entry "above" the one requested 3) If available, a "matching" entry If two of the above overlap, only one will be returned. In general, always at least two RRs should be returned. Each of the responses will have a netmask, and additional data (ie, trust levels, reputation score etc). With that information, a "intelligent" application / query library can greatly reduce the number of lookups required (eg when a spammer enumerates all IPs in a IPv6-/64). On the other hand, this makes it mathematically feasible to enumerate a complete DNSxL zone -- but I do not consider this as an issue, at least not for dnswl.org. The experimental server supports TXT and SRV records with essentially the same information. It is single-threaded and not very efficiently coded. But please feel free to test it: :~> dig @ns-exp1.dnswl.org -t txt 2.0.0.127.exp.dnswl.org ;; ANSWER SECTION: 2.0.0.127.exp.dnswl.org. 3600 IN TXT "32" "10" "0" 34.244.255.125.exp.dnswl.org. 3600 IN TXT "32" "8" "2" 254.254.254.127.exp.dnswl.org. 3600 IN TXT "32" "10" "0" :~> dig @ns-exp1.dnswl.org -t srv 2.0.0.127.exp.dnswl.org ;; ANSWER SECTION: 2.0.0.127.exp.dnswl.org. 3600 IN SRV 32 10 0 2.0.0.127.exp.dnswl= .org. 2.0.0.127.exp.dnswl.org. 3600 IN SRV 32 8 2 34.244.255.125.exp.dnswl.org. 2.0.0.127.exp.dnswl.org. 3600 IN SRV 32 10 0 254.254.254.127.exp.dnswl.org. The information for the two identical examples above is to be interpreted as follows: * Request for 127.0.0.2 * 127.0.0.2 has an exact match in the first of the three return RRs. * 125.255.244.34/32 is the next entry "below", with category 8 and trustlev= el 2 * 127.254.254.254/32 is the next entry "above" (a further test entry), with cat 10 and trust 0 Another example: :~> dig @ns-exp1.dnswl.org -t srv 254.5.210.128.exp.dnswl.org 254.5.210.128.exp.dnswl.org. 3600 IN SRV 24 11 2 0.5.210.128.exp.dnswl.org. 254.5.210.128.exp.dnswl.org. 3600 IN SRV 32 11 2 121.9.210.128.exp.dnswl.org. * Request 128.210.5.254 * 128.210.5.254 is contained in the range 128.210.5.0/24, category 11, trus= t 2 * Next entry "above" is 128.210.9.121/32, cat 11, trust 2 * No next entry "below" is listed, because the range already serves that purpose. The list has two synthetical records to indicate the lower and upper bands: :~> dig @ns-exp1.dnswl.org -t srv 0.0.0.0.exp.dnswl.org :~> dig @ns-exp1.dnswl.org -t srv 255.255.255.255.exp.dnswl.org Since I just hacked this together using existing Perl modules, the data is only in IPv4, but the concept can easily be extended to IPv6 payload. This server will serve current dnswl.org data. This approach has a number of benefits: * No arbitrary cut-points * Enables "intelligent" applications to avoid a lot of unnecessary requests * Does require an application to walk down the tree from a root point (and does not require DNSxL operators to artificially create such a tree with synthesized records) Some disadvantages: * I don't know of a way to serve this off of a generic nameserver (other than that nameserver being proxy to a "hidden master"). * DNSxL operators have less valuable logs because less specific traffic hits their infrastructure. I approached naturally from a whitelist point-of-view, so someone with more experience in blacklists may have different approaches. > Rsync's batch-ish nature isn't quite why I mentioned latency. =A0The late= ncy > is zone rebuild time at the DNSBL server. =A0With a zone file as large as= the > XBL, say, even if you did some sort of incremental update directly off a > core DBMS in realish-time, I think you'd run into performance problems. The latency is also in TTL. The lower the latency requirement, the less is DNS the right tool for the job. -- Matthias From johnl@iecc.com Sat Jan 8 13:24:13 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 11AA73A6831 for ; Sat, 8 Jan 2011 13:24:13 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -111.023 X-Spam-Level: X-Spam-Status: No, score=-111.023 tagged_above=-999 required=5 tests=[AWL=0.176, BAYES_00=-2.599, HABEAS_ACCREDITED_SOI=-4.3, RCVD_IN_BSP_TRUSTED=-4.3, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QUP2a6eVu3fb for ; Sat, 8 Jan 2011 13:24:11 -0800 (PST) Received: from gal.iecc.com (gal.iecc.com [64.57.183.53]) by core3.amsl.com (Postfix) with ESMTP id 836443A6827 for ; Sat, 8 Jan 2011 13:24:11 -0800 (PST) Received: (qmail 80246 invoked from network); 8 Jan 2011 21:26:20 -0000 Received: from mail1.iecc.com (64.57.183.56) by mail1.iecc.com with QMQP; 8 Jan 2011 21:26:20 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=iecc.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=1336c.4d28d67a.k1101; i=johnl@user.iecc.com; bh=Mzctxh06EQQb08O9C+g1RurJ9+UPxErHhfNuLZIZUHg=; b=gqQ79AGTFHD3PYUZoflsRCOIxrkwYMFTd1mxFKPnJSbCWgszFT6wFZoCeWSUckEKKCrjkmR1zQiIHRzPj1eClwJm+QPP8nAeGmj7iEnbQNxak6ECYt+H9I4cz8YQVNx2VuMFAHKc/D7lGZUqfrO61kBO5oBOxyHxBdoSzArJkwM= DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=taugh.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=1336c.4d28d67a.k1101; olt=johnl@user.iecc.com; bh=Mzctxh06EQQb08O9C+g1RurJ9+UPxErHhfNuLZIZUHg=; b=USTK5DrFNzZW/6erMkFEwQklo7k/pC39fgsJcxvdHgmv1TYdcLUvrDMNCy7j54xdizlIXQ7dU9uUtNwEH8sgCXXk9RcoxG8btN+OipQgqYY0DUaaWveT2Zjyr9C2AUjb8rzNGCsm98hVwK1P6FLTyr+ibuOcPC+c0dYWp6QCSUE= VBR-Info: md=iecc.com; mc=all; mv=dwl.spamhaus.org Date: 8 Jan 2011 21:26:18 -0000 Message-ID: <20110108212618.78699.qmail@joyce.lan> From: John Levine To: asrg@irtf.org In-Reply-To: Organization: Cc: X-Headerized: yes Mime-Version: 1.0 Content-type: text/plain; charset=utf-8 Content-transfer-encoding: 7bit Subject: Re: [Asrg] NXDOMAIN cache behavior, was draft-levine-iprangepub-01 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Jan 2011 21:24:13 -0000 >1) The listed entry "below" the one requested >2) The listed entry "above" the one requested >3) If available, a "matching" entry Interesting approach, and as always, experiments are informative. >With that information, a "intelligent" application / query library can >greatly reduce the number of lookups required Well, that's the problem. Some MTAs keep shared context for all incoming connections, some (particularly on Unix-ish systems) don't. Even on systems that share context, it's common to have a cluster of servers each with their own context. If you don't have the shared context, this has triple the queries of a naive system. That's why I designed my B-tree scheme so that the shared context is in the DNS cache. Nearby lookups will fetch exactly the same records and get good cache behavior, without having to remember anything in the MTA. With a system like yours, you could certainly posit a shared local daemon that caches the info, but if you're going to do that, you might as well use a daemon-to-server query system that gets 40 ranges per query rather one that only gets two or three. R's, John PS: >* I don't know of a way to serve this off of a generic nameserver >(other than that nameserver being proxy to a "hidden master"). Oh. that's easy, you just have some offline process make up the zone files and poke the server to load them. It's conceptually no different from putting files of CIDRs into rbldnsd. From CLEWIS@nortel.com Sat Jan 8 16:14:34 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id E663D28C120 for ; Sat, 8 Jan 2011 16:14:34 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RsPdaPJ7hLJ8 for ; Sat, 8 Jan 2011 16:14:34 -0800 (PST) Received: from zrtps0kp.nortel.com (zrtps0kp.nortel.com [47.140.192.56]) by core3.amsl.com (Postfix) with ESMTP id F0FB128C0D8 for ; Sat, 8 Jan 2011 16:14:33 -0800 (PST) Received: from zrtphxs1.corp.nortel.com (casmtp.ca.nortel.com [47.140.202.46]) by zrtps0kp.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id p090Gbg00347 for ; Sun, 9 Jan 2011 00:16:37 GMT Received: from zrtphx5h0.corp.nortel.com ([47.140.202.65]) by zrtphxs1.corp.nortel.com with Microsoft SMTPSVC(6.0.3790.4675); Sat, 8 Jan 2011 19:16:37 -0500 Received: from [47.102.168.6] (47.102.168.6) by zrtphx5h0.corp.nortel.com (47.140.202.65) with Microsoft SMTP Server (TLS) id 8.1.340.0; Sat, 8 Jan 2011 19:16:36 -0500 Message-ID: <4D28FE61.40808@nortel.com> Date: Sat, 8 Jan 2011 19:16:33 -0500 From: "Chris Lewis" Organization: Nortel User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.23) Gecko/20090812 Thunderbird/2.0.0.23 Mnenhy/0.7.6.666 MIME-Version: 1.0 To: References: <20110108212618.78699.qmail@joyce.lan> In-Reply-To: <20110108212618.78699.qmail@joyce.lan> Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 09 Jan 2011 00:16:37.0190 (UTC) FILETIME=[7AE1DE60:01CBAF92] Subject: Re: [Asrg] NXDOMAIN cache behavior, was draft-levine-iprangepub-01 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Jan 2011 00:14:35 -0000 On 1/8/2011 4:26 PM, John Levine wrote: >> 1) The listed entry "below" the one requested >> 2) The listed entry "above" the one requested >> 3) If available, a "matching" entry > > Interesting approach, and as always, experiments are informative. Yes. >> With that information, a "intelligent" application / query library can >> greatly reduce the number of lookups required > > Well, that's the problem. Some MTAs keep shared context for all > incoming connections, some (particularly on Unix-ish systems) don't. > Even on systems that share context, it's common to have a cluster of > servers each with their own context. If you don't have the shared > context, this has triple the queries of a naive system. Consider the situation with many larger server infrastructures. Not only do you have multiple clusters of machines, but each machine may be using threads. While there are ways to share server context, in the large, they can still be bottlenecks even on a single machine. I can't help thinking that divide and conquer - as in, a highly efficient distinct cache (whether DNS or not) is better. >> * I don't know of a way to serve this off of a generic nameserver >> (other than that nameserver being proxy to a "hidden master"). > > Oh. that's easy, you just have some offline process make up the zone > files and poke the server to load them. It's conceptually no different > from putting files of CIDRs into rbldnsd. That's how my local rbldnsd infrastructure works. Single query to a zone that's a merge of the DNSBLs we use. Spfilter is old (now impractical) code that can do this. I don't know of any free implementation of anything better - so I had to write my own. From matthias@leisi.net Mon Jan 10 05:14:29 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 38D893A698A for ; Mon, 10 Jan 2011 05:14:29 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.839 X-Spam-Level: X-Spam-Status: No, score=-2.839 tagged_above=-999 required=5 tests=[AWL=0.138, BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5dqW6Tdd2zmg for ; Mon, 10 Jan 2011 05:14:28 -0800 (PST) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by core3.amsl.com (Postfix) with ESMTP id 5F8E43A6975 for ; Mon, 10 Jan 2011 05:14:27 -0800 (PST) Received: by qwj9 with SMTP id 9so18989948qwj.13 for ; Mon, 10 Jan 2011 05:16:41 -0800 (PST) Received: by 10.224.80.138 with SMTP id t10mr13726166qak.389.1294665401128; Mon, 10 Jan 2011 05:16:41 -0800 (PST) MIME-Version: 1.0 Received: by 10.220.193.2 with HTTP; Mon, 10 Jan 2011 05:16:19 -0800 (PST) In-Reply-To: <20110108212618.78699.qmail@joyce.lan> References: <20110108212618.78699.qmail@joyce.lan> From: Matthias Leisi Date: Mon, 10 Jan 2011 14:16:19 +0100 Message-ID: To: Anti-Spam Research Group - IRTF Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: [Asrg] NXDOMAIN cache behavior, was draft-levine-iprangepub-01 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Jan 2011 13:14:29 -0000 On Sat, Jan 8, 2011 at 10:26 PM, John Levine wrote: >>* I don't know of a way to serve this off of a generic nameserver >>(other than that nameserver being proxy to a "hidden master"). > > Oh. that's easy, you just have some offline process make up the zone > files and poke the server to load them. =A0It's conceptually no different > from putting files of CIDRs into rbldnsd. No, it's very much different. It will always return the "nearest" records, regardless of whether you have a match in the list data or not (ie, querying for none-existing "a" will return "a-1" and "a+1" if they exist). -- Matthias From johnl@iecc.com Mon Jan 10 12:55:09 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 3BB683A6765 for ; Mon, 10 Jan 2011 12:55:09 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -111.031 X-Spam-Level: X-Spam-Status: No, score=-111.031 tagged_above=-999 required=5 tests=[AWL=0.168, BAYES_00=-2.599, HABEAS_ACCREDITED_SOI=-4.3, RCVD_IN_BSP_TRUSTED=-4.3, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BI9nwWOqmhvg for ; Mon, 10 Jan 2011 12:55:07 -0800 (PST) Received: from gal.iecc.com (gal.iecc.com [64.57.183.53]) by core3.amsl.com (Postfix) with ESMTP id A01703A672F for ; Mon, 10 Jan 2011 12:55:06 -0800 (PST) Received: (qmail 18936 invoked from network); 10 Jan 2011 20:57:19 -0000 Received: from mail1.iecc.com (64.57.183.56) by mail1.iecc.com with QMQP; 10 Jan 2011 20:57:19 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=iecc.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=7ede.4d2b72ae.k1101; i=johnl@user.iecc.com; bh=ax70uLnDdZRK/LmRIf1v+4sX23mEQnVLYwDCahKpJ1k=; b=mWe5FkshBnY+M/bqskfjr03H2A3OQCB0w37RMqRfD6yGiBE8668r+1qizysliA0LP94oR5TjBMjP4+It4bO6FqhpfRg+oO5OTmjEv9DMnn+Ac+apfYyg2cxhC3HZ1dW3rJCDHsEnknajfHXO4Rq+d7zavcyM7E5bk75obaMZ0pA= DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=taugh.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=7ede.4d2b72ae.k1101; olt=johnl@user.iecc.com; bh=ax70uLnDdZRK/LmRIf1v+4sX23mEQnVLYwDCahKpJ1k=; b=loaAgUO72prM9ruVjpwvbm0LQYx+HuWKIF4VQ/llK+4VGVkG5oKz7qRBy7indSeQHjW15lqHOJ069CqLKRrDn7AUdoAt2yrcuTUrM6SohrPzVKqm4at5IFR+M7XoSQ/JeRwvwvc/jrBXOHPQ2LsGZmFiAPPDaCe7WkQQYuNwoz0= VBR-Info: md=iecc.com; mc=all; mv=dwl.spamhaus.org Date: 10 Jan 2011 20:57:17 -0000 Message-ID: <20110110205717.32477.qmail@joyce.lan> From: John Levine To: asrg@irtf.org In-Reply-To: Organization: Cc: X-Headerized: yes Mime-Version: 1.0 Content-type: text/plain; charset=utf-8 Content-transfer-encoding: 7bit Subject: Re: [Asrg] NXDOMAIN cache behavior, was draft-levine-iprangepub-01 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Jan 2011 20:55:09 -0000 >> Oh. that's easy, you just have some offline process make up the zone >> files and poke the server to load them.  It's conceptually no different >> from putting files of CIDRs into rbldnsd. > >No, it's very much different. It will always return the "nearest" >records, regardless of whether you have a match in the list data or >not (ie, querying for none-existing "a" will return "a-1" and "a+1" if >they exist). It's not that hard. The concept is that you precompute all of the possible answers and put them in the zone. To keep the zone size down you use wildcards to cover ranges with the same answer. On the other hand, if there's something I'm missing and this approach really does need a stunt DNS server, how about making a b-tree out of the ranges and serving it from a bog standard DNS server instead? R's, John From jgriebsch@gmx.de Wed Jan 12 03:55:45 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 9D5F328C10F for ; Wed, 12 Jan 2011 03:55:45 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.025 X-Spam-Level: X-Spam-Status: No, score=-1.025 tagged_above=-999 required=5 tests=[AWL=1.574, BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id b8-JUqZ4vh3T for ; Wed, 12 Jan 2011 03:55:44 -0800 (PST) Received: from mail.gmx.net (mailout-de.gmx.net [213.165.64.22]) by core3.amsl.com (Postfix) with SMTP id 48DDD28C10C for ; Wed, 12 Jan 2011 03:55:43 -0800 (PST) Received: (qmail invoked by alias); 12 Jan 2011 11:58:01 -0000 Received: from p7223e3d4.np.gmx.net (EHLO jgdev) [212.227.35.114] by mail.gmx.net (mp058) with SMTP; 12 Jan 2011 12:58:01 +0100 X-Authenticated: #57931487 X-Provags-ID: V01U2FsdGVkX1/+DXs/fLr+EeQOdk7sBaGp4CGRtHN1IImY+UBttJ run17BLSYXwohl Content-Type: text/plain; charset=utf-8; format=flowed; delsp=yes To: asrg@irtf.org Date: Wed, 12 Jan 2011 12:54:30 +0100 MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: "Jan Griebsch" Message-ID: User-Agent: Opera Mail/11.00 (Linux) X-Y-GMX-Trusted: 3 Subject: [Asrg] IPv6 Reputation / DNSBL X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Jan 2011 12:36:49 -0000 Hi, having read through discussion threads here and on spamassassin-users@apache regarding John Levine's DNSBL implementation proposal, I have a related question. I post separately, since my question is more general, and I dont want to pollute these threads. Motivation for IP-based reputation today is broader than SMTP traffic. It is used as an resource-efficent first estimate, wherever service providers decide on letting users use their resources (create accounts etc) or on taking responsibility for user's content. Even for the mail context, it seems to me short sighted, to only consider SMTP traffic. For example, a lot of the private-user-to-private-user mail traffic is generated by web-/freemailers. If their account creation process is over-run (because no functioning IPv6 reputation solution + CAPTCHA is broken anyways), outbound spam through their IPs will increase accordingly, and the DNSWL approach discussed in aforementioned threads will tend to become meaningless for private-to-private email. If you accept the above premise, a more general approach to the problem could be subdivided into 3 parts: 1) Which entities (/128s, /64, variable prefixes..) do we assign reputation to ? 2) What are suitable backend datastructures/implementations for efficient lookup/updates/caching for the assumed (number of) entities ? 3) What are suitable mechs/protocols for querying and updating/distributing the entity-reputation data ? In my mind, the central, totally unresolved part is 1). If we can come up with a reasonable (see below) solution for 1), then 2) should be solvable as well. 3), as discussed in Levine's threads, I see in this context, as a negotiation/convention issue (with tricky tech. problems). Not to diminish it - but I want to make clear my point, that, if 1) is not solved, 2) and 3) are moot - at least for blacklisting. on 1): criteria for reasonable reputation entities * the number of entities must be limited to something much smaller then 2^64 <- for computational cost <- for meaning; in the end, "reputation" means judging the history of behavoir of a real-world communication/inter-action partner * corollary: creating entities must incur some type of cost <- e.g. simply rotating your hosts /64 suffix is for free <- e.g. money for registering a domain <- e.g. effort for an attacker to take over a host on 1): a more concrete question Can we manage to deduce (for traffic seen) * the internal partitioning of ISP's (e.g) /32s into subnets * classify each subnet into static vs. dynamic IP address assignment * for static SNs, the end-customer prefix (length) ? Then, I think, we'd have gone a long way to solving 1). We would assign reputation to prefixes, that are tied to hosts/end-customers and could, deny dialup/dyn.IP communication as desired (today via e.g. PBL). 2) would then amount to solving the LCP (longest common prefix) problem for a reasonable (order of 10^8) number of prefixes. A lot o academic work exists for that, and router do it, too. Any ideas ? Regards, --jan From mark@streamservice.nl Wed Jan 12 05:48:24 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 71D5128C0EE for ; Wed, 12 Jan 2011 05:48:24 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.113 X-Spam-Level: X-Spam-Status: No, score=-3.113 tagged_above=-999 required=5 tests=[AWL=1.391, BAYES_00=-2.599, HELO_EQ_NL=0.55, HOST_EQ_NL=1.545, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aKJXKzyssJin for ; Wed, 12 Jan 2011 05:48:23 -0800 (PST) Received: from spamfilter01.streamservice.nl (spamfilter01.streamservice.nl [91.199.167.162]) by core3.amsl.com (Postfix) with ESMTP id EC5F93A6B27 for ; Wed, 12 Jan 2011 05:48:22 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by spamfilter01.streamservice.nl (Postfix) with ESMTP id 1780C21D51 for ; Wed, 12 Jan 2011 14:50:40 +0100 (CET) X-Virus-Scanned: by amavisd-new-2.6.2 (20081215) (Debian) at streamservice.nl Received: from spamfilter01.streamservice.nl ([127.0.0.1]) by localhost (spamfilter01.streamservice.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bEKN53AEuXpr for ; Wed, 12 Jan 2011 14:50:38 +0100 (CET) Received: from shared.streamservice.nl (shared.streamservice.nl [91.199.167.26]) by spamfilter01.streamservice.nl (Postfix) with ESMTPS id 800C521CCD for ; Wed, 12 Jan 2011 14:50:38 +0100 (CET) Received: from 524bc59e.cm-4-4d.dynamic.ziggo.nl ([82.75.197.158] helo=laptoptruus) by shared.streamservice.nl with esmtpa (Exim 4.72) (envelope-from ) id 1Pd15z-00012j-Vn for asrg@irtf.org; Wed, 12 Jan 2011 14:50:40 +0100 From: "Mark Scholten" To: "'Anti-Spam Research Group - IRTF'" References: In-Reply-To: Date: Wed, 12 Jan 2011 14:50:31 +0100 Message-ID: <08de01cbb25f$ad8ff0c0$08afd240$@nl> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AcuyVb6bnEkPiwHQSuyzZPte4yxQdgAB50sw Content-Language: nl Subject: Re: [Asrg] IPv6 Reputation / DNSBL X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Jan 2011 13:48:24 -0000 Hello Jan, > From: asrg-bounces@irtf.org [mailto:asrg-bounces@irtf.org] On Behalf Of > Jan Griebsch > Sent: Wednesday, January 12, 2011 12:55 PM > > Hi, > > having read through discussion threads here and on > spamassassin-users@apache > regarding John Levine's DNSBL implementation proposal, I have a > related > question. I post separately, since my question is more general, and I > dont > want to pollute these threads. > > Motivation for IP-based reputation today is broader than SMTP traffic. > It is used as an resource-efficent first estimate, > wherever service providers decide on letting users use their resources > (create accounts etc) or on taking responsibility for user's content. > > Even for the mail context, it seems to me short sighted, to only > consider > SMTP > traffic. For example, a lot of the private-user-to-private-user mail > traffic is > generated by web-/freemailers. If their account creation process is > over-run > (because no functioning IPv6 reputation solution + CAPTCHA is broken > anyways), > outbound spam through their IPs will increase accordingly, and the > DNSWL > approach discussed in aforementioned threads will tend to become > meaningless > for private-to-private email. This is a problem that depends on how the web-/freemailers setup their systems. As long as the traffic stays within the mail servers of that company I think they could use some in-house tool to filter probably abusive users. As soon as they send it to others their reputation is on the line and they should act to defend it accordingly (by trying to prevent abusive users to sign up/block email/filter outbound/etc.). > > If you accept the above premise, a more general approach > to the problem could be subdivided into 3 parts: > 1) Which entities (/128s, /64, variable prefixes..) do we assign > reputation to ? If you ask me we should do it for multiple ranges. Let me explain it below: - 2 spam messages per [time] per /128 could be "acceptable", with more block that /128. For a good reputation a /128 should send at least XX messages per [time] without X being marked as spam. - 10 spam messages per [time] per /64 could be "acceptable", with more block that /64. For a good reputation a /64 should send at least XX messages per [time] without X being marked as spam. - 50 spam messages per [time] per /32 could be "acceptable", with more block that /32. For a good reputation a /32 should send at least XXX messages per [time] without XX being marked as spam. > 2) What are suitable backend datastructures/implementations > for efficient lookup/updates/caching for the assumed (number of) > entities ? PowerDNS with MySQL backend would work for DNS based lookups probably (it does caching, it is efficient, it is easy to update, lookups are easy and adding capacity is easy). > 3) What are suitable mechs/protocols for querying and > updating/distributing > the entity-reputation data ? Querying: DNS or some http/https form is probably the easiest solution (with DNS you could cache answers more easily). Distributing: Depending on the solution I'd look for AXFR/zone files/MySQL master-slave replication/CSV files. I think we'll first have to decide what is good for 1) and 2) and let 3) depend on that answer. > > In my mind, the central, totally unresolved part is 1). > If we can come up with a reasonable (see below) solution for 1), then > 2) > should be solvable as well. 3), as discussed in Levine's threads, I see > in > this context, as a negotiation/convention issue (with tricky tech. > problems). > Not to diminish it - but I want to make clear my point, that, if 1) > is not solved, 2) and 3) are moot - at least for blacklisting. > > on 1): criteria for reasonable reputation entities > * the number of entities must be limited to something much smaller then > 2^64 > <- for computational cost > <- for meaning; in the end, "reputation" means judging the history > of > behavoir > of a real-world communication/inter-action partner > * corollary: creating entities must incur some type of cost > <- e.g. simply rotating your hosts /64 suffix is for free > <- e.g. money for registering a domain > <- e.g. effort for an attacker to take over a host That is why I'd look for some reputation based on multiple ranges to look for the best action to take. If you've an ISP that has paying clients that are hurting others with their bad reputation they are more likely to say good bye to the clients that are hurting business. Or do you've a better idea for this? > > > on 1): a more concrete question > > Can we manage to deduce (for traffic seen) > > * the internal partitioning of ISP's (e.g) /32s into subnets Difficult if the ISP isn't cooperating/updating data sources. > * classify each subnet into static vs. dynamic IP address assignment Depends on the ISP and that makes it difficult to use I think. > * for static SNs, the end-customer prefix (length) > ? Difficult if the ISP isn't cooperating/updating data sources. > > Then, I think, we'd have gone a long way to solving 1). We would assign > reputation > to prefixes, that are tied to hosts/end-customers and could, deny > dialup/dyn.IP > communication as desired (today via e.g. PBL). > 2) would then amount to solving the LCP (longest common prefix) problem > for a > reasonable (order of 10^8) number of prefixes. A lot o academic work > exists for that, and router do it, too. With a good DNS based solution this shouldn't be a really big problem I think. Regards, Mark > > Any ideas ? > > Regards, > --jan > _______________________________________________ > Asrg mailing list > Asrg@irtf.org > http://www.irtf.org/mailman/listinfo/asrg From jgriebsch@gmx.de Thu Jan 13 03:22:33 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 2F0C83A6B76 for ; Thu, 13 Jan 2011 03:22:33 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.774 X-Spam-Level: X-Spam-Status: No, score=-1.774 tagged_above=-999 required=5 tests=[AWL=0.825, BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1MoBQ2X4f9UC for ; Thu, 13 Jan 2011 03:22:32 -0800 (PST) Received: from mail.gmx.net (mailout-de.gmx.net [213.165.64.22]) by core3.amsl.com (Postfix) with SMTP id B8D0E3A6B58 for ; Thu, 13 Jan 2011 03:22:30 -0800 (PST) Received: (qmail invoked by alias); 13 Jan 2011 11:24:52 -0000 Received: from p7223e3d4.np.gmx.net (EHLO jgdev) [212.227.35.114] by mail.gmx.net (mp072) with SMTP; 13 Jan 2011 12:24:52 +0100 X-Authenticated: #57931487 X-Provags-ID: V01U2FsdGVkX19c6d8psy9cha2YAXd31LsjYT49xPp187+rCK5CZM w4biiFupa1QJRH Content-Type: text/plain; charset=utf-8; format=flowed; delsp=yes To: "'Anti-Spam Research Group - IRTF'" , "Mark Scholten" References: <08de01cbb25f$ad8ff0c0$08afd240$@nl> Date: Thu, 13 Jan 2011 12:21:04 +0100 MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: "Jan Griebsch" Message-ID: In-Reply-To: <08de01cbb25f$ad8ff0c0$08afd240$@nl> User-Agent: Opera Mail/11.00 (Linux) X-Y-GMX-Trusted: 3 Subject: Re: [Asrg] IPv6 Reputation / DNSBL X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Jan 2011 11:22:33 -0000 Hi Mark, > This is a problem that depends on how the web-/freemailers setup their > systems. As long as the traffic stays within the mail servers of that > company I think they could use some in-house tool to filter probably > abusive > users. As soon as they send it to others their reputation is on the line > and > they should act to defend it accordingly (by trying to prevent abusive > users > to sign up/block email/filter outbound/etc.). True, but you're essentially saying ESPs will have to cleanup spam(mers) later, expensively, that they could have prevented earlier if they had had working IP filters/reputation. True, IP filters do little/nothing that one couldnt do without them. But the issue here is being efficient (or: being overrun if you are not). Also, outgoing spam-filtering is just more error prone (worse filter training, user-feedback, lack of viable consequences if unsure of classification). > If you ask me we should do it for multiple ranges. Let me explain it > below: > - 2 spam messages per [time] per /128 could be "acceptable", with more > block > that /128. For a good reputation a /128 should send at least XX messages > per > [time] without X being marked as spam. > - 10 spam messages per [time] per /64 could be "acceptable", with more > block > that /64. For a good reputation a /64 should send at least XX messages > per > [time] without X being marked as spam. > - 50 spam messages per [time] per /32 could be "acceptable", with more > block > that /32. For a good reputation a /32 should send at least XXX messages > per > [time] without XX being marked as spam. 2 problems: * You dont want to punish whole groups for a few bad apples. If "group" means another ESP (/32) this could hurt your bottom line outright. * You dont want to give spammers freebies either. AFAIK, some ISPs even give /48s to "end-customers". If your count onto /64 thats, according to your example 10*2^16 freebies. >> on 1): criteria for reasonable reputation entities >> * the number of entities must be limited to something much smaller then >> 2^64 >> <- for computational cost >> <- for meaning; in the end, "reputation" means judging the history >> of >> behavoir >> of a real-world communication/inter-action partner >> * corollary: creating entities must incur some type of cost >> <- e.g. simply rotating your hosts /64 suffix is for free >> <- e.g. money for registering a domain >> <- e.g. effort for an attacker to take over a host > That is why I'd look for some reputation based on multiple ranges to look > for the best action to take. If you've an ISP that has paying clients > that > are hurting others with their bad reputation they are more likely to say > good bye to the clients that are hurting business. Or do you've a better > idea for this? What I was trying to say (in a rather overly generic way) is, that it doesnt make sense to count & give *bad* reputation to entities that are (almost) infinitely available (/128s and even /64s). Even if you could, computationally. Because attackers would simply keep creating new entities. >> on 1): a more concrete question >> >> Can we manage to deduce (for traffic seen) >> >> * the internal partitioning of ISP's (e.g) /32s into subnets > Difficult if the ISP isn't cooperating/updating data sources. > >> * classify each subnet into static vs. dynamic IP address assignment > Depends on the ISP and that makes it difficult to use I think. > >> * for static SNs, the end-customer prefix (length) >> ? > Difficult if the ISP isn't cooperating/updating data sources. .."Difficult".. Exactely. ** But somewhat/somehow doable ? ** If we cant aggregate /128s /64s etc. onto prefixes that are tied to meaningful, costly entites (hosts, customers, companies..), then, I think, (bad) IP blacklisting is dead, independently of all clever solutions to 2) and 3). Regards, --jan From mark@streamservice.nl Thu Jan 13 05:21:45 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id E61E13A6B85 for ; Thu, 13 Jan 2011 05:21:45 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.891 X-Spam-Level: X-Spam-Status: No, score=-1.891 tagged_above=-999 required=5 tests=[AWL=-0.387, BAYES_00=-2.599, HELO_EQ_NL=0.55, HOST_EQ_NL=1.545, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id S5o+LUbcVQ3M for ; Thu, 13 Jan 2011 05:21:44 -0800 (PST) Received: from spamfilter01.streamservice.nl (spamfilter01.streamservice.nl [91.199.167.162]) by core3.amsl.com (Postfix) with ESMTP id 8DD6B3A6B8A for ; Thu, 13 Jan 2011 05:21:44 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by spamfilter01.streamservice.nl (Postfix) with ESMTP id 8B28F21D4A; Thu, 13 Jan 2011 14:24:04 +0100 (CET) X-Virus-Scanned: by amavisd-new-2.6.2 (20081215) (Debian) at streamservice.nl Received: from spamfilter01.streamservice.nl ([127.0.0.1]) by localhost (spamfilter01.streamservice.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eHEQKHy0T6dC; Thu, 13 Jan 2011 14:24:03 +0100 (CET) Received: from shared.streamservice.nl (shared.streamservice.nl [91.199.167.26]) by spamfilter01.streamservice.nl (Postfix) with ESMTPS id 1D9FF21CA3; Thu, 13 Jan 2011 14:24:03 +0100 (CET) Received: from 524bc59e.cm-4-4d.dynamic.ziggo.nl ([82.75.197.158] helo=laptoptruus) by shared.streamservice.nl with esmtpa (Exim 4.72) (envelope-from ) id 1PdN9o-0001Rq-Bd; Thu, 13 Jan 2011 14:24:04 +0100 From: "Mark Scholten" To: "'Jan Griebsch'" , "'Anti-Spam Research Group - IRTF'" References: <08de01cbb25f$ad8ff0c0$08afd240$@nl> In-Reply-To: Date: Thu, 13 Jan 2011 14:23:49 +0100 Message-ID: <09dc01cbb325$1db00d70$59102850$@nl> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AcuzFIFJI1+w58MVTxOrcaSxHG5UjQADtP5g Content-Language: nl Subject: Re: [Asrg] IPv6 Reputation / DNSBL X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Jan 2011 13:21:46 -0000 > From: Jan Griebsch [mailto:jgriebsch@gmx.de] > Sent: Thursday, January 13, 2011 12:21 PM > Hi Mark, >=20 > > This is a problem that depends on how the web-/freemailers setup > their > > systems. As long as the traffic stays within the mail servers of = that > > company I think they could use some in-house tool to filter probably > > abusive > > users. As soon as they send it to others their reputation is on the > line > > and > > they should act to defend it accordingly (by trying to prevent > abusive > > users > > to sign up/block email/filter outbound/etc.). >=20 > True, but you're essentially saying ESPs will have to cleanup > spam(mers) > later, expensively, that they could have prevented earlier if they had > had working IP filters/reputation. True, IP filters do little/nothing > that one couldnt do without them. But the issue here is being = efficient > (or: being overrun if you are not). Also, outgoing spam-filtering is > just > more error prone (worse filter training, user-feedback, lack of viable > consequences if unsure of classification). I know that outgoing spam-filtering is more difficult. However I believe = that it is at least required to filter some abuse that is clear for most = filters (viruses/spam with high scores with default spam assassin setups = and the like). Lists could also be used to limit number of email = accounts per IP (IP range) or block bad users. In the beginning this = will be difficult (lack of data). >=20 > > If you ask me we should do it for multiple ranges. Let me explain it > > below: > > - 2 spam messages per [time] per /128 could be "acceptable", with > more > > block > > that /128. For a good reputation a /128 should send at least XX > messages > > per > > [time] without X being marked as spam. > > - 10 spam messages per [time] per /64 could be "acceptable", with > more > > block > > that /64. For a good reputation a /64 should send at least XX > messages > > per > > [time] without X being marked as spam. > > - 50 spam messages per [time] per /32 could be "acceptable", with > more > > block > > that /32. For a good reputation a /32 should send at least XXX > messages > > per > > [time] without XX being marked as spam. >=20 > 2 problems: > * You dont want to punish whole groups for a few bad apples. If = "group" > means another ESP (/32) this could hurt your bottom line outright. If blocking per /128 didn't work you should look for bigger ranges to = block. Also limiting it to x bad /128 before calling a /64 bad is an = idea. In that case you could increase the range you block/do extra = checks on when multiple /64 ranges are considered bad (/56 or /48 or = even /32). I don't prefer the last option, but if the ISP isn't = providing valid whois data you really don't have another choice than = consider the ISP as being bad. > * You dont want to give spammers freebies either. AFAIK, some ISPs = even > give /48s to "end-customers". If your count onto /64 thats, according > to > your example 10*2^16 freebies. In my example I increase the range to block when there is more abuse = coming from that range. For that you've of course multiple options. >=20 > >> on 1): criteria for reasonable reputation entities > >> * the number of entities must be limited to something much smaller > then > >> 2^64 > >> <- for computational cost > >> <- for meaning; in the end, "reputation" means judging the > history > >> of > >> behavoir > >> of a real-world communication/inter-action partner > >> * corollary: creating entities must incur some type of cost > >> <- e.g. simply rotating your hosts /64 suffix is for free > >> <- e.g. money for registering a domain > >> <- e.g. effort for an attacker to take over a host > > That is why I'd look for some reputation based on multiple ranges to > look > > for the best action to take. If you've an ISP that has paying = clients > > that > > are hurting others with their bad reputation they are more likely to > say > > good bye to the clients that are hurting business. Or do you've a > better > > idea for this? >=20 > What I was trying to say (in a rather overly generic way) is, that > it doesnt make sense to count & give *bad* reputation to entities > that are (almost) infinitely available (/128s and even /64s). > Even if you could, computationally. Because attackers would simply > keep creating new entities. I'd increase the range when more IP's are used for bad behavior. That is = also an option currently in use at some location for IPv4 and I don't = see a reason to not do that again with IPv6. There are just too many = /128s or /64s or /48s for that matter available in a /32. >=20 > >> on 1): a more concrete question > >> > >> Can we manage to deduce (for traffic seen) > >> > >> * the internal partitioning of ISP's (e.g) /32s into subnets > > Difficult if the ISP isn't cooperating/updating data sources. > > > >> * classify each subnet into static vs. dynamic IP address = assignment > > Depends on the ISP and that makes it difficult to use I think. > > > >> * for static SNs, the end-customer prefix (length) > >> ? > > Difficult if the ISP isn't cooperating/updating data sources. >=20 > .."Difficult".. Exactely. ** But somewhat/somehow doable ? ** It is possible to aggregate it based on whois data, however if an ISP = isn't providing the correct data (or has to many bad users) other action = should be taken if you ask me. >=20 > If we cant aggregate /128s /64s etc. onto prefixes that are > tied to meaningful, costly entites (hosts, customers, companies..), > then, I think, (bad) IP blacklisting is dead, independently of all > clever solutions to 2) and 3). The option I did mention doesn't have to be extremely costly. You could = apply some other rules before accepting the traffic (for example that it = comes from space that is officially given out by a RIR). After that it = is indeed difficult to block ranges without problems (if you don't also = have a whitelist with known "normal" mailservers). This system could be = used against botnets I think, for normal mailservers you need to look = for some % of legit mail to never block that IP/range. For us this limit = would probably be around 50% (or higher). Regards, Mark From davidnicol@gmail.com Tue Jan 18 06:43:39 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id C466728C13A for ; Tue, 18 Jan 2011 06:43:39 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.341 X-Spam-Level: X-Spam-Status: No, score=-3.341 tagged_above=-999 required=5 tests=[AWL=0.258, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id m9R39w1eed6w for ; Tue, 18 Jan 2011 06:43:39 -0800 (PST) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by core3.amsl.com (Postfix) with ESMTP id 1267328C119 for ; Tue, 18 Jan 2011 06:43:39 -0800 (PST) Received: by iwn39 with SMTP id 39so5797325iwn.13 for ; Tue, 18 Jan 2011 06:46:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:from:date :message-id:subject:to:content-type; bh=ZGNrQ8hSozzTikGCouja9ZpUxPC9E5782BBEh0Li7xY=; b=ojWxcaNp7jMFiukMVWPo6uOroYyc1PFyBRvOy9XJg/Im+0BK+UxgV7rt+0zvKHS1uw ktxBrIRXZBaWyKMYP9HRVJXc6+oCIXkDuSUXyEaPYWNF0UMO0XlJnrkN6ESLt1FX9sLM pGYDpDwM1/+dWd2ECO1JAzC6Ti+x+8+F1OJWY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; b=jBk7z2B9eZepgWLSZ9nN8Ie5J/e25RlAkSK5z8ysulnaNtbxKkCYiTiZKDdy6DdlMW sK0WWwoMCrEBwQSjyMLuLBatu35yCNsd/KfFEJqNKAwjz2eeqMIxWfbhuJnN+mkKihjr g2uzZ2HWuAtm/fWja2q7bNVYAc6RaBMOIPYXg= Received: by 10.231.199.81 with SMTP id er17mr6142299ibb.115.1295361975754; Tue, 18 Jan 2011 06:46:15 -0800 (PST) MIME-Version: 1.0 Received: by 10.231.206.148 with HTTP; Tue, 18 Jan 2011 06:45:55 -0800 (PST) In-Reply-To: References: From: David Nicol Date: Tue, 18 Jan 2011 08:45:55 -0600 Message-ID: To: Anti-Spam Research Group - IRTF Content-Type: text/plain; charset=UTF-8 Subject: Re: [Asrg] IPv6 Reputation / DNSBL X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Jan 2011 14:43:39 -0000 On Wed, Jan 12, 2011 at 5:54 AM, Jan Griebsch wrote: > 1) Which entities (/128s, /64, variable prefixes..) do we assign reputation > to ? Variable prefixes. Duh. Does this point need expansion? From johnl@iecc.com Tue Jan 18 07:40:56 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id B899428C16F for ; Tue, 18 Jan 2011 07:40:56 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -111.056 X-Spam-Level: X-Spam-Status: No, score=-111.056 tagged_above=-999 required=5 tests=[AWL=0.143, BAYES_00=-2.599, HABEAS_ACCREDITED_SOI=-4.3, RCVD_IN_BSP_TRUSTED=-4.3, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RUN9LqxQsX6Z for ; Tue, 18 Jan 2011 07:40:55 -0800 (PST) Received: from gal.iecc.com (gal.iecc.com [64.57.183.53]) by core3.amsl.com (Postfix) with ESMTP id 7ECA83A7023 for ; Tue, 18 Jan 2011 07:40:55 -0800 (PST) Received: (qmail 4398 invoked from network); 18 Jan 2011 15:43:32 -0000 Received: from mail1.iecc.com (64.57.183.56) by mail1.iecc.com with QMQP; 18 Jan 2011 15:43:32 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=iecc.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=9da2.4d35b524.k1101; i=johnl@user.iecc.com; bh=bu6hMnp1mNu3IDBCWeakx+8KRBFNuJwWBpbkgU8cnio=; b=XUl12qC4QT7eE30Yqsz1+ohGxSG9fDtfLfyKfGtTthI5y6rJztB3Da80oleaiZ2P77E2Ct23czWvtsPkTKuedi1FSdph+paqaOwW8jBn4pwySf/VnriS2JxbCNWokmz2TK4mH5+TNATHW84leRSEo/iGWMu/ps58UTdX6ZtCURQ= DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=taugh.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=9da2.4d35b524.k1101; olt=johnl@user.iecc.com; bh=bu6hMnp1mNu3IDBCWeakx+8KRBFNuJwWBpbkgU8cnio=; b=PxKTrJCB328h7coWRtefgyIU1PajQ26TAc4bykgo7yN9iYQ72RxGPjc585mI4Mw5FfCYXSMsYzKBDf2o6LGZpWXltGvOP21O6vVeyv7eehmDUFb9W3tzNx5FOhhyN0Qj0qkpi6Bz7611nifW/RrIfyUhBCaiiaq7+HVqzgsRV0k= VBR-Info: md=iecc.com; mc=all; mv=dwl.spamhaus.org Date: 18 Jan 2011 15:43:31 -0000 Message-ID: <20110118154331.40353.qmail@joyce.lan> From: John Levine To: asrg@irtf.org In-Reply-To: Organization: Cc: X-Headerized: yes Mime-Version: 1.0 Content-type: text/plain; charset=utf-8 Content-transfer-encoding: 7bit Subject: Re: [Asrg] IPv6 Reputation / DNSBL X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Jan 2011 15:40:56 -0000 >> 1) Which entities (/128s, /64, variable prefixes..) do we assign reputation >> to ? > >Variable prefixes. Duh. Does this point need expansion? Depends on the application. For a whitelist of mail hosts, probably /128s. For a blacklist of zombies, probably /64s. For a blacklist of evildoers, arbitrary ranges depending on their network sizes. I agree that any technical approach to publishing BLs and WLs needs to be able to handle different prefix lengths. R's, John From johnl@iecc.com Tue Jan 18 07:44:23 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 71D3F28C1BE for ; Tue, 18 Jan 2011 07:44:23 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -110.66 X-Spam-Level: X-Spam-Status: No, score=-110.66 tagged_above=-999 required=5 tests=[AWL=-0.260, BAYES_00=-2.599, HABEAS_ACCREDITED_SOI=-4.3, RCVD_IN_BSP_TRUSTED=-4.3, SARE_SUB_RAND_LETTRS4=0.799, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id A4fj2FrKUkNK for ; Tue, 18 Jan 2011 07:44:22 -0800 (PST) Received: from gal.iecc.com (gal.iecc.com [64.57.183.53]) by core3.amsl.com (Postfix) with ESMTP id 092A428C1C1 for ; Tue, 18 Jan 2011 07:44:21 -0800 (PST) Received: (qmail 6329 invoked from network); 18 Jan 2011 15:46:59 -0000 Received: from mail1.iecc.com (64.57.183.56) by mail1.iecc.com with QMQP; 18 Jan 2011 15:46:59 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=iecc.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=a121.4d35b5f3.k1101; i=johnl@user.iecc.com; bh=B+utI0NWIaIlzS/BK0NOYDYrl154MAeX0dJmAHY7g/I=; b=i8g4r918G4YLcivQmVqLc2VezCWEV/KF0jLMrQ8DCtal+9g3JrA6tivlWxTlS43/LFmAcE3rcVo/o4Bn1utlpQ6U8+//HcdKLtp4WRzqDROT+3KJ01HwjUSj+6dbom7DTpHa1+ghQCN7o5xuk++cyqK3y48lsqEBQkd8xCb5wiQ= DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=taugh.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=a121.4d35b5f3.k1101; olt=johnl@user.iecc.com; bh=B+utI0NWIaIlzS/BK0NOYDYrl154MAeX0dJmAHY7g/I=; b=Fc635lJFeog0rl7aB82rMs8kxup/dYdLAEtk7R6Eebzt+1fRQSdJgMs5h/WpDV/iwp7KvqjWlyOT7h9yAnMTXFnzjAZXKrgrR4KJv4otkcocWlKScTIjKmonQ9c4PibshQ/KJEYQLHr0dGlu8UM8lHHETcauY4wNB0GaZ0QQf0w= VBR-Info: md=iecc.com; mc=all; mv=dwl.spamhaus.org Date: 18 Jan 2011 15:46:59 -0000 Message-ID: <20110118154659.41248.qmail@joyce.lan> From: John Levine To: asrg@irtf.org In-Reply-To: <4D230C50.8050508@nortel.com> Organization: Cc: X-Headerized: yes Mime-Version: 1.0 Content-type: text/plain; charset=utf-8 Content-transfer-encoding: 7bit Subject: [Asrg] please review draft-irtf-asrg-bcp-blacklists-07 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Jan 2011 15:44:23 -0000 >New and I hope final revision of the DNSBL BCP. >https://datatracker.ietf.org/doc/draft-irtf-asrg-bcp-blacklists/ I'd like to publish this as an ASRG document sooner rather than later. So please take a look at it and say: a) I agree with what it says, OK for ASRG consensus. b) I have minor reservations but don't object to publishing it, OK for ASRG partial consensus. c) It's so wrong that you shouldn't publish it unless you change X to Y. (Be sure to fill in X and Y.) R's, John, ASRG chair From feenberg@nber.org Tue Jan 18 08:24:06 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 73B2528C144 for ; Tue, 18 Jan 2011 08:24:06 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.199 X-Spam-Level: X-Spam-Status: No, score=-6.199 tagged_above=-999 required=5 tests=[AWL=-0.399, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, SARE_SUB_RAND_LETTRS4=0.799] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QTBJiVvtgWcb for ; Tue, 18 Jan 2011 08:24:05 -0800 (PST) Received: from mail2.nber.org (mail2.nber.org [66.251.72.79]) by core3.amsl.com (Postfix) with ESMTP id 329D328C154 for ; Tue, 18 Jan 2011 08:24:04 -0800 (PST) Received: from nber6.nber.org (nber6.nber.org [66.251.72.76]) by mail2.nber.org (8.14.4/8.14.4) with ESMTP id p0IGQdtr032067; Tue, 18 Jan 2011 11:26:39 -0500 (EST) (envelope-from feenberg@nber.org) Received: from localhost (feenberg@localhost) by nber6.nber.org (8.14.4/8.14.4/Submit) with ESMTP id p0IGNFbN016372; Tue, 18 Jan 2011 11:23:16 -0500 (EST) X-Authentication-Warning: nber6.nber.org: feenberg owned process doing -bs Date: Tue, 18 Jan 2011 11:23:15 -0500 (EST) From: Daniel Feenberg X-X-Sender: feenberg@nber6 To: Anti-Spam Research Group - IRTF In-Reply-To: <20110118154659.41248.qmail@joyce.lan> Message-ID: References: <20110118154659.41248.qmail@joyce.lan> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Anti-Virus: Kaspersky Anti-Virus for Linux Mail Server 5.6.39/RELEASE, bases: 20110118 #4672796, check: 20110118 clean Subject: Re: [Asrg] please review draft-irtf-asrg-bcp-blacklists-07 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Jan 2011 16:24:06 -0000 On Tue, 18 Jan 2011, John Levine wrote: >> New and I hope final revision of the DNSBL BCP. >> https://datatracker.ietf.org/doc/draft-irtf-asrg-bcp-blacklists/ > > I'd like to publish this as an ASRG document sooner rather than later. > So please take a look at it and say: > > a) I agree with what it says, OK for ASRG consensus. > > b) I have minor reservations but don't object to publishing it, OK for > ASRG partial consensus. > > c) It's so wrong that you shouldn't publish it unless you change X to > Y. (Be sure to fill in X and Y.) I believe the syntax for queries should be specified in this document. While the syntax for IP address queries is standardized in practice (and shown in examples in the text) I am not sure the syntax (especially translation of invalid characters) of URIs is fully standardized. I haven't used such services yet, so perhaps I am wrong about practice. Isn't this the opportunity to do so? Standardization of the query makes it simple for clients to switch among servers. I should add that I am in agreement with policy advice in the document and wouldn't actually oppose it if asked. Daniel Feenberg > > R's, > John, ASRG chair > _______________________________________________ > Asrg mailing list > Asrg@irtf.org > http://www.irtf.org/mailman/listinfo/asrg > From CLEWIS@nortel.com Tue Jan 18 09:11:15 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 0B1EF28C1E8 for ; Tue, 18 Jan 2011 09:11:15 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.899 X-Spam-Level: X-Spam-Status: No, score=-4.899 tagged_above=-999 required=5 tests=[AWL=-1.700, BAYES_50=0.001, RCVD_IN_DNSWL_MED=-4, SARE_SUB_RAND_LETTRS4=0.799] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id c+BCwnajqOPP for ; Tue, 18 Jan 2011 09:11:13 -0800 (PST) Received: from zrtps0kp.nortel.com (zrtps0kp.nortel.com [47.140.192.56]) by core3.amsl.com (Postfix) with ESMTP id 38D3328C103 for ; Tue, 18 Jan 2011 09:11:12 -0800 (PST) Received: from zrtphxs1.corp.nortel.com (zrtphxs1.corp.nortel.com [47.140.202.46]) by zrtps0kp.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id p0IHDkT12534 for ; Tue, 18 Jan 2011 17:13:46 GMT Received: from zrtphx5h0.corp.nortel.com ([47.140.202.65]) by zrtphxs1.corp.nortel.com with Microsoft SMTPSVC(6.0.3790.4675); Tue, 18 Jan 2011 12:13:44 -0500 Received: from [47.102.152.48] (47.102.152.48) by zrtphx5h0.corp.nortel.com (47.140.202.65) with Microsoft SMTP Server (TLS) id 8.1.340.0; Tue, 18 Jan 2011 12:13:44 -0500 Message-ID: <4D35CA46.8050202@nortel.com> Date: Tue, 18 Jan 2011 12:13:42 -0500 From: "Chris Lewis" Organization: Nortel User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.23) Gecko/20090812 Thunderbird/2.0.0.23 Mnenhy/0.7.6.666 MIME-Version: 1.0 To: References: <20110118154659.41248.qmail@joyce.lan> In-Reply-To: Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 18 Jan 2011 17:13:45.0064 (UTC) FILETIME=[100C2E80:01CBB733] Subject: Re: [Asrg] please review draft-irtf-asrg-bcp-blacklists-07 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Jan 2011 17:11:15 -0000 On 1/18/2011 11:23 AM, Daniel Feenberg wrote: > I believe the syntax for queries should be specified in this document. It's in RFC 5782. From feenberg@nber.org Tue Jan 18 09:21:11 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id CD56028C1E8 for ; Tue, 18 Jan 2011 09:21:11 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.066 X-Spam-Level: X-Spam-Status: No, score=-6.066 tagged_above=-999 required=5 tests=[AWL=-0.266, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, SARE_SUB_RAND_LETTRS4=0.799] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cJ8wgqYTwhhQ for ; Tue, 18 Jan 2011 09:21:10 -0800 (PST) Received: from mail2.nber.org (mail2.nber.org [66.251.72.79]) by core3.amsl.com (Postfix) with ESMTP id 71CD628C0F1 for ; Tue, 18 Jan 2011 09:21:10 -0800 (PST) Received: from nber6.nber.org (nber6.nber.org [66.251.72.76]) by mail2.nber.org (8.14.4/8.14.4) with ESMTP id p0IHNiT8050308; Tue, 18 Jan 2011 12:23:45 -0500 (EST) (envelope-from feenberg@nber.org) Received: from localhost (feenberg@localhost) by nber6.nber.org (8.14.4/8.14.4/Submit) with ESMTP id p0IHKLJ1023809; Tue, 18 Jan 2011 12:20:21 -0500 (EST) X-Authentication-Warning: nber6.nber.org: feenberg owned process doing -bs Date: Tue, 18 Jan 2011 12:20:21 -0500 (EST) From: Daniel Feenberg X-X-Sender: feenberg@nber6 To: Anti-Spam Research Group - IRTF In-Reply-To: <4D35CA46.8050202@nortel.com> Message-ID: References: <20110118154659.41248.qmail@joyce.lan> <4D35CA46.8050202@nortel.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Anti-Virus: Kaspersky Anti-Virus for Linux Mail Server 5.6.39/RELEASE, bases: 20110118 #4672796, check: 20110118 clean Subject: Re: [Asrg] please review draft-irtf-asrg-bcp-blacklists-07 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Jan 2011 17:21:12 -0000 On Tue, 18 Jan 2011, Chris Lewis wrote: > On 1/18/2011 11:23 AM, Daniel Feenberg wrote: > >> I believe the syntax for queries should be specified in this document. > > It's in RFC 5782. Then I will withdraw my suggestion. Thanks Daniel Feenberg > _______________________________________________ > Asrg mailing list > Asrg@irtf.org > http://www.irtf.org/mailman/listinfo/asrg > From martijn.grooten@virusbtn.com Tue Jan 18 09:31:23 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id BA2A23A7030 for ; Tue, 18 Jan 2011 09:31:23 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.9 X-Spam-Level: X-Spam-Status: No, score=-4.9 tagged_above=-999 required=5 tests=[AWL=0.900, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, SARE_SUB_RAND_LETTRS4=0.799] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YutCBu9bokn7 for ; Tue, 18 Jan 2011 09:31:23 -0800 (PST) Received: from mx2.sophos.com (mx2.sophos.com [195.166.81.53]) by core3.amsl.com (Postfix) with ESMTP id D45F43A6F6C for ; Tue, 18 Jan 2011 09:31:21 -0800 (PST) Received: from mx2.sophos.com (localhost.localdomain [127.0.0.1]) by localhost (Postfix) with SMTP id C75369580E1 for ; Tue, 18 Jan 2011 17:33:58 +0000 (GMT) Received: from uk-exch1.green.sophos (uk-exch1.green.sophos [10.100.199.16]) by mx2.sophos.com (Postfix) with ESMTP id 8EA4E9580DF for ; Tue, 18 Jan 2011 17:33:58 +0000 (GMT) Received: from UK-EXCHMBX1.green.sophos ([fe80:0000:0000:0000:e1bd:d3c1:23.222.229.221]) by uk-exch1.green.sophos ([10.100.199.16]) with mapi; Tue, 18 Jan 2011 17:33:58 +0000 From: Martijn Grooten To: Anti-Spam Research Group - IRTF Date: Tue, 18 Jan 2011 17:33:57 +0000 Thread-Topic: [Asrg] please review draft-irtf-asrg-bcp-blacklists-07 Thread-Index: Acu3JvZL4KJm6gdzSNaYgf43jsXtiQADiI1Q Message-ID: <18B53BA2A483AD45962AAD1397BE132537AE83E800@UK-EXCHMBX1.green.sophos> References: <4D230C50.8050508@nortel.com> <20110118154659.41248.qmail@joyce.lan> In-Reply-To: <20110118154659.41248.qmail@joyce.lan> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [Asrg] please review draft-irtf-asrg-bcp-blacklists-07 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Jan 2011 17:31:23 -0000 > >New and I hope final revision of the DNSBL BCP. > >https://datatracker.ietf.org/doc/draft-irtf-asrg-bcp-blacklists/ > > I'd like to publish this as an ASRG document sooner rather than later. > So please take a look at it and say: > > a) I agree with what it says, OK for ASRG consensus. a) Nice piece of work. Martijn. Virus Bulletin Ltd, The Pentagon, Abingdon, OX14 3YP, England. Company Reg No: 2388295. VAT Reg No: GB 532 5598 33. From jjohnson@jdmc.org Tue Jan 18 09:41:38 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id B0FAF3A6E7B for ; Tue, 18 Jan 2011 09:41:38 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.8 X-Spam-Level: X-Spam-Status: No, score=-1.8 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, SARE_SUB_RAND_LETTRS4=0.799] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tcIzediP3pgY for ; Tue, 18 Jan 2011 09:41:38 -0800 (PST) Received: from secure.jdmc.org (secure.jdmc.org [164.58.70.160]) by core3.amsl.com (Postfix) with ESMTP id E4DF03A6E78 for ; Tue, 18 Jan 2011 09:41:37 -0800 (PST) Received: from [164.58.70.150] (mail.jdmc.org [164.58.70.150]) by secure.jdmc.org with ESMTP id p0IHiF78027303 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Tue, 18 Jan 2011 11:44:15 -0600 X-Client-IP: 192.168.3.212 X-EnvRcpt: X-EnvFrom: jjohnson@jdmc.org X-LDomain: mail.jdmc.org X-LCount: 0 X-Vsender: dhcp3-212.jdmc.org [192.168.3.212] Received: from [192.168.3.212] (dhcp3-212.jdmc.org [192.168.3.212]) (authenticated bits=0) by mail.jdmc.org (8.14.4/8.14.4) with ESMTP id p0IHiBVN026912 for ; Tue, 18 Jan 2011 11:44:14 -0600 Message-ID: <4D35D16B.3020805@jdmc.org> Date: Tue, 18 Jan 2011 11:44:11 -0600 From: John Johnson User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.13) Gecko/20101207 Lightning/1.0b2 Thunderbird/3.1.7 MIME-Version: 1.0 To: Anti-Spam Research Group - IRTF References: <20110118154659.41248.qmail@joyce.lan> In-Reply-To: <20110118154659.41248.qmail@joyce.lan> X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Apparently-To: X-Scanned-By: MIMEDefang 2.71 on 164.58.70.150 X-GlobalCerts-Milter: secure.jdmc.org 18Jan2011-11:44:15.180 Subject: Re: [Asrg] please review draft-irtf-asrg-bcp-blacklists-07 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Jan 2011 17:41:38 -0000 On 01/18/2011 09:46 AM, John Levine wrote: >> New and I hope final revision of the DNSBL BCP. >> https://datatracker.ietf.org/doc/draft-irtf-asrg-bcp-blacklists/ > > I'd like to publish this as an ASRG document sooner rather than later. > So please take a look at it and say: > > a) I agree with what it says, OK for ASRG consensus. I would choose a) for my circumstances. Thank you John and all involved. From dotis@mail-abuse.org Tue Jan 18 11:34:00 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id B99D33A7028 for ; Tue, 18 Jan 2011 11:34:00 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.18 X-Spam-Level: X-Spam-Status: No, score=-106.18 tagged_above=-999 required=5 tests=[AWL=-0.380, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, SARE_SUB_RAND_LETTRS4=0.799, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eNjsxfjE9APc for ; Tue, 18 Jan 2011 11:34:00 -0800 (PST) Received: from harry.mail-abuse.org (harry.mail-abuse.org [168.61.5.27]) by core3.amsl.com (Postfix) with ESMTP id E5E593A7021 for ; Tue, 18 Jan 2011 11:33:59 -0800 (PST) Received: from US-DOUGLASO-MB.local (gateway1.sjc.mail-abuse.org [168.61.5.81]) by harry.mail-abuse.org (Postfix) with ESMTP id DE78DA94462 for ; Tue, 18 Jan 2011 19:37:31 +0000 (UTC) Message-ID: <4D35EBBE.6050402@mail-abuse.org> Date: Tue, 18 Jan 2011 11:36:30 -0800 From: Douglas Otis User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: asrg@irtf.org References: <20110118154659.41248.qmail@joyce.lan> In-Reply-To: <20110118154659.41248.qmail@joyce.lan> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Asrg] please review draft-irtf-asrg-bcp-blacklists-07 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Jan 2011 19:34:00 -0000 On 1/18/11 7:46 AM, John Levine wrote: >> New and I hope final revision of the DNSBL BCP. >> https://datatracker.ietf.org/doc/draft-irtf-asrg-bcp-blacklists/ > I'd like to publish this as an ASRG document sooner rather than later. > So please take a look at it and say: > > a) I agree with what it says, OK for ASRG consensus. > > b) I have minor reservations but don't object to publishing it, OK for > ASRG partial consensus. > > c) It's so wrong that you shouldn't publish it unless you change X to > Y. (Be sure to fill in X and Y.) John, C. This draft continues the delusion of IP address listings for IPv6. An important feature of IPv6 is the ability to rapidly renumber. Any scheme that attempts to apply policy against IPv6 addresses removes this important feature. Suggestions related to IPv6 address listings are almost certainly representative of poor advice. For example how might one determine what would be a "reasonable" rescan period for the v6 address space? What would be a reasonable listing duration when a different IP address can be used for centuries for every message ever sent? Frankly, anything that suggests IP address listing criteria for IPv6 would represent advice not based upon any practical experience, nor supported by reasonable justifications. There is nothing that even suggests reactive block-listings of IPv6 addresses can ever be disseminated at a rate effective at mitigating abuse. This draft should caution against assumptions that suggest IPv4 practices can be extended for use with IPv6! -Doug From john@jlc.net Tue Jan 18 12:42:55 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id AB93828C142 for ; Tue, 18 Jan 2011 12:42:55 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.041 X-Spam-Level: X-Spam-Status: No, score=-106.041 tagged_above=-999 required=5 tests=[AWL=-0.241, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, SARE_SUB_RAND_LETTRS4=0.799, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ofrLywaVfkTu for ; Tue, 18 Jan 2011 12:42:54 -0800 (PST) Received: from mailhost.jlc.net (mailhost.jlc.net [199.201.159.4]) by core3.amsl.com (Postfix) with ESMTP id BF2B228C14E for ; Tue, 18 Jan 2011 12:42:54 -0800 (PST) Received: by mailhost.jlc.net (Postfix, from userid 104) id DF68E33C29; Tue, 18 Jan 2011 15:45:32 -0500 (EST) Date: Tue, 18 Jan 2011 15:45:32 -0500 From: John Leslie To: Anti-Spam Research Group - IRTF Message-ID: <20110118204532.GC68297@verdi> References: <20110118154659.41248.qmail@joyce.lan> <4D35EBBE.6050402@mail-abuse.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4D35EBBE.6050402@mail-abuse.org> User-Agent: Mutt/1.4.1i Subject: Re: [Asrg] please review draft-irtf-asrg-bcp-blacklists-07 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Jan 2011 20:42:55 -0000 Douglas Otis wrote: > On 1/18/11 7:46 AM, John Levine wrote: > >>>https://datatracker.ietf.org/doc/draft-irtf-asrg-bcp-blacklists/ > > C. This draft continues the delusion of IP address listings for IPv6. > An important feature of IPv6 is the ability to rapidly renumber. Any > scheme that attempts to apply policy against IPv6 addresses removes this > important feature. I'm afraid Doug is a bit in the rough here: most of us seem hell-bent on _making_ IPv6 DNSBLs work. (I do, however, share Doug's doubt that we'll succeed.) > This draft should caution against assumptions that suggest IPv4 > practices can be extended for use with IPv6! +1 (I didn't stumble upon any exactly contrary statement; but I think we need an explicit explanation that IPv6 DNSBLs are a work-in-progress at best, and this documents cannot make recommendations specific to IPv6.) ==== Other than that, I stumbled upon some of the MUSTard. In general, we cannot enforce any behavior on list maintainers. Specifically, the "MUST NOT charge" in Section 2.5 is inappropriate. DNSBLs that do not charge for access will necessarily need to recover costs for manual actions that exceed de-minimis, or simply not do them. A strong "SHOULD NOT charge" is appropriate. We could probably even get away with a "MUST NOT use" such DNSBLs in that paragraph, although personally I'd prefer "SHOULD NOT charge" and "SHOULD NOT use". In Section 2.2.2, the "MUST NOT use..." such that removal requests would be blocked is a bit strong for my taste. DDoS attacks are very real, and defensive action _is_ needed from time to time. I think a strong SHOULD NOT is appropriate, and I'd suggest adding text to the effect that the website "SHOULD list short-lived alternatives" whenever the usual removal-request path may be blocked. In Section 3.4, the "MUST NOT list the entire Internet" is a bit strong as well. While I think you have given plenty of suggested alternatives to this, folks _do_ paint themselves into corners where such a doomsday-weapon becomes needed. (Listing the entire Internet _does_ mean they're outside the specification we're writing; but it comes across as an attempt to control a behavior we can't control. Perhaps we could suggest alternatives, not use the MUSTard, and say that listing the entire Internet puts a DNSBL list out of conformance with this set of practices.) Whatever, we should make sure that language about testing for list-the-entire-Internet SHOULD be done is found within this Section. ==== Overall, very good work! -- John Leslie From CLEWIS@nortel.com Tue Jan 18 13:54:31 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id B300A3A6EDA for ; Tue, 18 Jan 2011 13:54:31 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.35 X-Spam-Level: X-Spam-Status: No, score=-5.35 tagged_above=-999 required=5 tests=[AWL=0.450, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, SARE_SUB_RAND_LETTRS4=0.799] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Oahp29qt0E24 for ; Tue, 18 Jan 2011 13:54:30 -0800 (PST) Received: from zrtps0kp.nortel.com (zrtps0kp.nortel.com [47.140.192.56]) by core3.amsl.com (Postfix) with ESMTP id 55A073A6DCE for ; Tue, 18 Jan 2011 13:54:29 -0800 (PST) Received: from zrtphxs1.corp.nortel.com (casmtp.ca.nortel.com [47.140.202.46]) by zrtps0kp.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id p0ILv4T24594 for ; Tue, 18 Jan 2011 21:57:04 GMT Received: from zrtphx5h0.corp.nortel.com ([47.140.202.65]) by zrtphxs1.corp.nortel.com with Microsoft SMTPSVC(6.0.3790.4675); Tue, 18 Jan 2011 16:57:02 -0500 Received: from [47.102.152.48] (47.102.152.48) by zrtphx5h0.corp.nortel.com (47.140.202.65) with Microsoft SMTP Server (TLS) id 8.1.340.0; Tue, 18 Jan 2011 16:57:02 -0500 Message-ID: <4D360CAC.4050902@nortel.com> Date: Tue, 18 Jan 2011 16:57:00 -0500 From: "Chris Lewis" Organization: Nortel User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.23) Gecko/20090812 Thunderbird/2.0.0.23 Mnenhy/0.7.6.666 MIME-Version: 1.0 To: References: <20110118154659.41248.qmail@joyce.lan> <4D35EBBE.6050402@mail-abuse.org> <20110118204532.GC68297@verdi> In-Reply-To: <20110118204532.GC68297@verdi> Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 18 Jan 2011 21:57:02.0904 (UTC) FILETIME=[A38CBB80:01CBB75A] Subject: Re: [Asrg] please review draft-irtf-asrg-bcp-blacklists-07 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Jan 2011 21:54:31 -0000 On 1/18/2011 3:45 PM, John Leslie wrote: > Douglas Otis wrote: >> On 1/18/11 7:46 AM, John Levine wrote: >> >>>> https://datatracker.ietf.org/doc/draft-irtf-asrg-bcp-blacklists/ >> >> C. This draft continues the delusion of IP address listings for IPv6. >> An important feature of IPv6 is the ability to rapidly renumber. Any >> scheme that attempts to apply policy against IPv6 addresses removes this >> important feature. > > I'm afraid Doug is a bit in the rough here: most of us seem hell-bent > on _making_ IPv6 DNSBLs work. > > (I do, however, share Doug's doubt that we'll succeed.) > >> This draft should caution against assumptions that suggest IPv4 >> practices can be extended for use with IPv6! > +1 > (I didn't stumble upon any exactly contrary statement; but I think we > need an explicit explanation that IPv6 DNSBLs are a work-in-progress at > best, and this documents cannot make recommendations specific to IPv6.) This is suggestive of a disclaimer to that effect up in the introduction. I'm, however, hesitant in doing so. My reasoning is as follows: a) If IPv6 DNSBLs are impractical as some suggest, no useful ones will exist, the issue is moot, and no disclaimers are necessary. b) If IPv6 DNSBLs are practical, then most of the policies are entirely IPvX neutral (and are in fact extensible to non-DNSBLs at that), and a disclaimer of that type would unnecessarily deprecate the useful policies that _should_ still apply to IPv6. c) If IPv6 DNSBLs are practical, but some of the policies don't make technical sense, again the issue is moot, and no disclaimers are necessary. Eg: rescan intervals: there's nothing that implies that rescans are necessary. Hence, comments about rescanning of IPv6 entries being impractical or impossible doesn't invalidate a "minimum rescan interval" policy. If you can't rescan, then, the minimum rescan interval policy is moot - by not rescanning at all, you still meet the BCP. So, I don't see a hard conflict nor a necessity for such a disclaimer - at least not a blanket one. Is there any MUST/SHOULD that _cannot_ be met by a plausible IPv6 DNSBL implementation and needs a specific disclaimer? > Other than that, I stumbled upon some of the MUSTard. > > In general, we cannot enforce any behavior on list maintainers. No, but we can encourage, and note/discourage what behaviours cause problems in the real world. > Specifically, the "MUST NOT charge" in Section 2.5 is inappropriate. > DNSBLs that do not charge for access will necessarily need to recover > costs for manual actions that exceed de-minimis, or simply not do them. There are DNSBLs that do not charge for access and do not charge for manual actions. In fact, there's only one DNSBL that I know of that still imposes financial costs surrounding delistings. It ain't SORBS... Michelle _herself_ suggested "MUST NOT". But charging for removals demonstrably causes problems, such as people who refuse to bother with perhaps mistaken DNSBL listings because they view it as extortion. Thus increasing the FP rate and impairing the accuracy for everyone else. Not to mention the harm to the reputation of the mechanism as a whole. A strong statement that this isn't acceptable behaviour can do nothing but good. > A strong "SHOULD NOT charge" is appropriate. > > We could probably even get away with a "MUST NOT use" such DNSBLs > in that paragraph, although personally I'd prefer "SHOULD NOT charge" > and "SHOULD NOT use". The goal of the BCP is to strongly encourage the best practises on the DNSBL admin side, but at the same time recognize that receivers can do anything they feel like as long as they know what they're doing. IOW: a receiver can be in compliance even while using a DNSBL that's out of compliance. Splitting between strong prohibition aimed at DNSBL admins and mild discouragement aimed at receivers was quite deliberate here. Given that the "MUST NOT" only puts one known DNSBL out of compliance that I'm aware of, I'm comfortable with it. > In Section 2.2.2, the "MUST NOT use..." such that removal requests > would be blocked is a bit strong for my taste. DDoS attacks are very > real, and defensive action _is_ needed from time to time. I think a > strong SHOULD NOT is appropriate, and I'd suggest adding text to the > effect that the website "SHOULD list short-lived alternatives" > whenever the usual removal-request path may be blocked. Remember that DDoS attacks are generally temporary, and there's nothing in the BCP that precludes changing the removal request process if/when it becomes necessary - see section 3.2. The section is meant to mean that the DNSBL is privately contactable and not subject to the obvious screwup that the listee cannot contact the lister due to the listing or other routine spam filtering, not that it cannot be changed as circumstances necessitate. > In Section 3.4, the "MUST NOT list the entire Internet" is a bit > strong as well. While I think you have given plenty of suggested > alternatives to this, folks _do_ paint themselves into corners where > such a doomsday-weapon becomes needed. (Listing the entire Internet > _does_ mean they're outside the specification we're writing; but it > comes across as an attempt to control a behavior we can't control. > Perhaps we could suggest alternatives, not use the MUSTard, and say > that listing the entire Internet puts a DNSBL list out of conformance > with this set of practices.) Isn't that exactly what the section means? Besides, heretofore, any DNSBL utilizing such a doomsday weapon was heralding their own doom - compliance becomes moot. We have to really really _really_ discourage the practise so that receivers will know that listing-world is something they should not need to worry about. But in the end, a DNSBL having to resort to it ain't going to care about compliance, and some of the provisioning recommendations are about making sure that list-the-world scenarios are not necessary. > Whatever, we should make sure that language about testing for > list-the-entire-Internet SHOULD be done is found within this Section. It's the next section ... isn't that enough? > Overall, very good work! Thanks! From davidnicol@gmail.com Tue Jan 18 14:14:20 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id E56BC3A6F5F for ; Tue, 18 Jan 2011 14:14:20 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.956 X-Spam-Level: X-Spam-Status: No, score=-2.956 tagged_above=-999 required=5 tests=[AWL=-0.156, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1, SARE_SUB_RAND_LETTRS4=0.799] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EWBf8Hkooaja for ; Tue, 18 Jan 2011 14:14:20 -0800 (PST) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by core3.amsl.com (Postfix) with ESMTP id 028933A6F5C for ; Tue, 18 Jan 2011 14:14:19 -0800 (PST) Received: by iwn39 with SMTP id 39so158147iwn.13 for ; Tue, 18 Jan 2011 14:16:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:from:date :message-id:subject:to:content-type:content-transfer-encoding; bh=WzRWw9oQZPErQuyMUqqDwj85stmVWJtEM0TVXO0lNyo=; b=pQ1ydMZguYjXNW1BMV/f2WXcjhBw68DfMiD5JF74olKxCLcQhUxTbDvpHc2rS9+i99 qTmFkOPKH8oAkKWCUGiJw1qgZCPz05qLPl5grI6muYL330XFSjWt7bV5L2gOSreyvN8J vbH2zmi9NVK1LMC6ghlHm4PXi9O0yesA6DbNo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type:content-transfer-encoding; b=Lzebmj3NDnaTvNNFgenq2o6nGI7QbPwtKSSCoJLLOufGmGOpaySWHn1Fz4PDAwoRUw evinyNAlh9eGBR77RSL8iD2SzpYio0LvTkvbehTMT/RDvo9Q7YTt1CJf5fNLG6sZnf4s vo07Ji4e6EmZ39XIsw/R3G0OyPnE+xuJX6qNg= Received: by 10.231.30.70 with SMTP id t6mr6730500ibc.165.1295389018089; Tue, 18 Jan 2011 14:16:58 -0800 (PST) MIME-Version: 1.0 Received: by 10.231.206.148 with HTTP; Tue, 18 Jan 2011 14:16:37 -0800 (PST) In-Reply-To: <20110118154659.41248.qmail@joyce.lan> References: <4D230C50.8050508@nortel.com> <20110118154659.41248.qmail@joyce.lan> From: David Nicol Date: Tue, 18 Jan 2011 16:16:37 -0600 Message-ID: To: Anti-Spam Research Group - IRTF Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [Asrg] please review draft-irtf-asrg-bcp-blacklists-07 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Jan 2011 22:14:21 -0000 On Tue, Jan 18, 2011 at 9:46 AM, John Levine wrote: > > b) I have minor reservations but don't object to publishing it, OK for > =C2=A0 ASRG partial consensus. My minor reservation is =C2=A0that I would like to see a "why DNS?" section that describes the characteristics of an information distribution scenario that make DNS a good choice of technology. This comes up often here, usually in the form of millennial rants prophesying eventual replacement by public (or non-public) rsync servers or such, more or less. The characteristics of the problem domains that DNSxL technology has been applied to are implicit in the introduction, but are not enumerated in such a way that one might come away from https://datatracker.ietf.org/doc/draft-irtf-asrg-bcp-blacklists with vocabulary allowing the reader to classify information distribution problems in a way that she might be able to confidently answer the question "Is a DNSxL the correct distribution technology?" when selecting. Towards drafting a section to satisfy this criticism, the characteristics to enumerate would be the characteristics of DNS that make DNS appropriate for this application, from which we must go through the motion of enumerating the (currently implied) characteristics of the application that make DNS a good fit for it. These characteristics include, but are not limited to: * setting of time-to-live -- variable time to cache, at record grain siz= e * data only transmitted on as-requested basis * central administration, with possibility to explicitly delegate parts * existing well-understood software infrastructure The inclusion of such a section could elevate the document from the specific to the general, and set an example as a model for a best practice in the composition of best practices documents: before getting down and dirty with details confusing to the newcomer, spend a paragraph answering the "why?" behind the deer-in-the-headlights glaze. Currently all we have in this realm is "Due to the rising amount of spam and other forms of network abuse on the Internet, many community members and companies began to create, publish and maintain DNS-based reputation systems (DNS-Based Lists) of IP addresses or domain names and make reputation suggestions or assertions about email sourced from these IP addresses or domain names." I would like to see that paragraph expanded to explain what it is about reputation data that makes DNS a good technology for distributing it, and what it is about DNS that makes it a good technology for distributing reputation data, or other data (aside from, of course, name to number mappings and their reverse) that makes, or might make sense, to distribute by DNS. In conclusion, start reading a biography of Mark Twain first thing in the morning and you'll be loquacious all day. From CLEWIS@nortel.com Tue Jan 18 14:34:09 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id C6A5A28C100 for ; Tue, 18 Jan 2011 14:34:09 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.5 X-Spam-Level: X-Spam-Status: No, score=-5.5 tagged_above=-999 required=5 tests=[AWL=0.300, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, SARE_SUB_RAND_LETTRS4=0.799] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id w+ttapyM-kBB for ; Tue, 18 Jan 2011 14:34:08 -0800 (PST) Received: from zrtps0kp.nortel.com (zrtps0kp.nortel.com [47.140.192.56]) by core3.amsl.com (Postfix) with ESMTP id 998AF28C108 for ; Tue, 18 Jan 2011 14:34:08 -0800 (PST) Received: from zrtphxs1.corp.nortel.com (casmtp.ca.nortel.com [47.140.202.46]) by zrtps0kp.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id p0IMafT02294 for ; Tue, 18 Jan 2011 22:36:42 GMT Received: from zrtphx5h0.corp.nortel.com ([47.140.202.65]) by zrtphxs1.corp.nortel.com with Microsoft SMTPSVC(6.0.3790.4675); Tue, 18 Jan 2011 17:36:40 -0500 Received: from [47.102.152.48] (47.102.152.48) by zrtphx5h0.corp.nortel.com (47.140.202.65) with Microsoft SMTP Server (TLS) id 8.1.340.0; Tue, 18 Jan 2011 17:36:40 -0500 Message-ID: <4D3615F6.5010206@nortel.com> Date: Tue, 18 Jan 2011 17:36:38 -0500 From: "Chris Lewis" Organization: Nortel User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.23) Gecko/20090812 Thunderbird/2.0.0.23 Mnenhy/0.7.6.666 MIME-Version: 1.0 To: References: <4D230C50.8050508@nortel.com> <20110118154659.41248.qmail@joyce.lan> In-Reply-To: Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 18 Jan 2011 22:36:40.0716 (UTC) FILETIME=[2CD60CC0:01CBB760] Subject: Re: [Asrg] please review draft-irtf-asrg-bcp-blacklists-07 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Jan 2011 22:34:09 -0000 On 1/18/2011 5:16 PM, David Nicol wrote: > The inclusion of such a section could elevate the document from the > specific to the general, and set an example as a model for a best > practice in the composition of best practices documents: before > getting down and dirty with details confusing to the newcomer, spend a > paragraph answering the "why?" behind the deer-in-the-headlights > glaze. > > Currently all we have in this realm is "Due to the rising amount of > spam and other forms of network abuse on the Internet, many community > members and companies began to create, publish and maintain DNS-based > reputation systems (DNS-Based Lists) of IP addresses or domain names > and make reputation suggestions or assertions about email sourced from > these IP addresses or domain names." > > I would like to see that paragraph expanded to explain what it is > about reputation data that makes DNS a good technology for > distributing it, and what it is about DNS that makes it a good > technology for distributing reputation data, or other data (aside > from, of course, name to number mappings and their reverse) that > makes, or might make sense, to distribute by DNS. > > In conclusion, start reading a biography of Mark Twain first thing in > the morning and you'll be loquacious all day. Heh. I can see the point. A subsection of the introduction entitled "But why DNS?", and listing out your points plus a few more (like "DNS already exists and is ubiquitous", "it distributes in a global manner", "like wow, it works well"). A bit of a paeon to Vixie's serendipitous insight at a time when few would have thought of it. But, I'm not sure that the BCP is exactly the place for that much detail of the historical justification of _why_ we do it this way. Especially one that could start up a religious war with the purists who've already screamed "but it doesn't return an IP address of a host!" even when it wasn't rubbed in their faces. I dunno. Other opinions? From feenberg@nber.org Tue Jan 18 14:44:11 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 9EC4728C114 for ; Tue, 18 Jan 2011 14:44:11 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6 X-Spam-Level: X-Spam-Status: No, score=-6 tagged_above=-999 required=5 tests=[AWL=-0.200, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, SARE_SUB_RAND_LETTRS4=0.799] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FmRRKiwRkNxr for ; Tue, 18 Jan 2011 14:44:10 -0800 (PST) Received: from mail2.nber.org (mail2.nber.org [66.251.72.79]) by core3.amsl.com (Postfix) with ESMTP id 563BA28C0FC for ; Tue, 18 Jan 2011 14:44:08 -0800 (PST) Received: from nber6.nber.org (nber6.nber.org [66.251.72.76]) by mail2.nber.org (8.14.4/8.14.4) with ESMTP id p0IMkfsF026099; Tue, 18 Jan 2011 17:46:42 -0500 (EST) (envelope-from feenberg@nber.org) Received: from localhost (feenberg@localhost) by nber6.nber.org (8.14.4/8.14.4/Submit) with ESMTP id p0IMhIVw006812; Tue, 18 Jan 2011 17:43:19 -0500 (EST) X-Authentication-Warning: nber6.nber.org: feenberg owned process doing -bs Date: Tue, 18 Jan 2011 17:43:18 -0500 (EST) From: Daniel Feenberg X-X-Sender: feenberg@nber6 To: Anti-Spam Research Group - IRTF In-Reply-To: <4D360CAC.4050902@nortel.com> Message-ID: References: <20110118154659.41248.qmail@joyce.lan> <4D35EBBE.6050402@mail-abuse.org> <20110118204532.GC68297@verdi> <4D360CAC.4050902@nortel.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Anti-Virus: Kaspersky Anti-Virus for Linux Mail Server 5.6.39/RELEASE, bases: 20110118 #4672796, check: 20110118 clean Subject: Re: [Asrg] please review draft-irtf-asrg-bcp-blacklists-07 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Jan 2011 22:44:11 -0000 On Tue, 18 Jan 2011, Chris Lewis wrote: > On 1/18/2011 3:45 PM, John Leslie wrote: >> Douglas Otis wrote: >>> On 1/18/11 7:46 AM, John Levine wrote: >>> >>>>> https://datatracker.ietf.org/doc/draft-irtf-asrg-bcp-blacklists/ >>> >>> C. This draft continues the delusion of IP address listings for IPv6. >>> An important feature of IPv6 is the ability to rapidly renumber. Any >>> scheme that attempts to apply policy against IPv6 addresses removes this >>> important feature. >> >> I'm afraid Doug is a bit in the rough here: most of us seem hell-bent >> on _making_ IPv6 DNSBLs work. >> >> (I do, however, share Doug's doubt that we'll succeed.) >> >>> This draft should caution against assumptions that suggest IPv4 >>> practices can be extended for use with IPv6! > >> +1 > >> (I didn't stumble upon any exactly contrary statement; but I think we >> need an explicit explanation that IPv6 DNSBLs are a work-in-progress at >> best, and this documents cannot make recommendations specific to IPv6.) > IPv6 is mentioned only once, in Section 3.5, and not in a way that suggests DNSBLs for IPv6 are appropriate or practical. The document covers whitelists as well as blacklists, and it is not unreasonable to suppose that a whitelist could exist for IPv6 mail hosts. While I might prefer a vigourous denunciation of IPv6 blacklists, there isn't anything really objectionable in the draft on this topic. Daniel Feenberg From dotis@mail-abuse.org Tue Jan 18 15:59:01 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id AA90A3A6E0A for ; Tue, 18 Jan 2011 15:59:01 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.165 X-Spam-Level: X-Spam-Status: No, score=-106.165 tagged_above=-999 required=5 tests=[AWL=-0.365, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, SARE_SUB_RAND_LETTRS4=0.799, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id oI-EVD2P6KKO for ; Tue, 18 Jan 2011 15:59:00 -0800 (PST) Received: from harry.mail-abuse.org (harry.mail-abuse.org [168.61.5.27]) by core3.amsl.com (Postfix) with ESMTP id E45A93A6DB6 for ; Tue, 18 Jan 2011 15:59:00 -0800 (PST) Received: from US-DOUGLASO-MB.local (gateway1.sjc.mail-abuse.org [168.61.5.81]) by harry.mail-abuse.org (Postfix) with ESMTP id 4027FA94451 for ; Wed, 19 Jan 2011 00:02:40 +0000 (UTC) Message-ID: <4D3629E2.4050307@mail-abuse.org> Date: Tue, 18 Jan 2011 16:01:38 -0800 From: Douglas Otis User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: asrg@irtf.org References: <20110118154659.41248.qmail@joyce.lan> <4D35EBBE.6050402@mail-abuse.org> <20110118204532.GC68297@verdi> <4D360CAC.4050902@nortel.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Asrg] please review draft-irtf-asrg-bcp-blacklists-07 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Jan 2011 23:59:02 -0000 On 1/18/11 2:43 PM, Daniel Feenberg wrote: > IPv6 is mentioned only once, in Section 3.5, and not in a way that > suggests DNSBLs for IPv6 are appropriate or practical. The document > covers whitelists as well as blacklists, and it is not unreasonable to > suppose that a whitelist could exist for IPv6 mail hosts. While I > might prefer a vigourous denunciation of IPv6 blacklists, there isn't > anything really objectionable in the draft on this topic. Daniel, It is not the number of times that IPv6 has been mentioned in the draft. It is the number of times an example proves wholly unreasonable when related to an IPv6 service. In the rather critical matter, "2.2.1. Listings SHOULD Be Temporary", this makes a questionable assumption that listing/de-listing churn will not become damaging whenever expiration is used. Whether one is talking about v6 prefixes, or interface addresses, bad actors have access to virtually an endless supply of prefixes and interface addresses. This means an address may never repeat over a bad actor's life. Even the expectation that white-listing should be fine for v6 fails to consider cases where undefined translations may occur using shared resources. Efforts aimed at codifying use of DNSxBLs would normally be commendable. At this time, during the transition to IPv6, this draft only distracts from practical and well considered methods necessary to properly defend services operating within the v6 address space. Implications that DNSxL offers a reasonable service for use with v6 is instead counter productive and satisfies no immediate need. Especially when attempts to use DNSxBL's for IPv6 address reputations are likely to prove damaging and problematic. -Doug From john@jlc.net Tue Jan 18 16:49:08 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 51FED3A6FDB for ; Tue, 18 Jan 2011 16:49:08 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.039 X-Spam-Level: X-Spam-Status: No, score=-106.039 tagged_above=-999 required=5 tests=[AWL=-0.239, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, SARE_SUB_RAND_LETTRS4=0.799, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DWVl+upBgKqz for ; Tue, 18 Jan 2011 16:49:06 -0800 (PST) Received: from mailhost.jlc.net (mailhost.jlc.net [199.201.159.4]) by core3.amsl.com (Postfix) with ESMTP id 657073A6F68 for ; Tue, 18 Jan 2011 16:49:06 -0800 (PST) Received: by mailhost.jlc.net (Postfix, from userid 104) id BCAFA33C36; Tue, 18 Jan 2011 19:51:44 -0500 (EST) Date: Tue, 18 Jan 2011 19:51:44 -0500 From: John Leslie To: Anti-Spam Research Group - IRTF Message-ID: <20110119005144.GE68297@verdi> References: <20110118154659.41248.qmail@joyce.lan> <4D35EBBE.6050402@mail-abuse.org> <20110118204532.GC68297@verdi> <4D360CAC.4050902@nortel.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4D360CAC.4050902@nortel.com> User-Agent: Mutt/1.4.1i Subject: Re: [Asrg] please review draft-irtf-asrg-bcp-blacklists-07 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Jan 2011 00:49:08 -0000 Chris Lewis wrote: > On 1/18/2011 3:45 PM, John Leslie wrote: >> Douglas Otis wrote: >> >>> This draft should caution against assumptions that suggest IPv4 >>> practices can be extended for use with IPv6! > >> +1 > >> (I didn't stumble upon any exactly contrary statement; but I think we >> need an explicit explanation that IPv6 DNSBLs are a work-in-progress at >> best, and this documents cannot make recommendations specific to IPv6.) > > This is suggestive of a disclaimer to that effect up in the introduction. It could go in Section 1.4 "Background", for example. > I'm, however, hesitant in doing so. My reasoning is as follows: > > a) If IPv6 DNSBLs are impractical as some suggest, no useful ones will > exist, the issue is moot, and no disclaimers are necessary. Not a helpful argument. It supposes this document being read much later than it's being issued (which could be true, but isn't our main intent). > b) If IPv6 DNSBLs are practical, then most of the policies are entirely > IPvX neutral (and are in fact extensible to non-DNSBLs at that), and a > disclaimer of that type would unnecessarily deprecate the useful > policies that _should_ still apply to IPv6. This is false. > c) If IPv6 DNSBLs are practical, but some of the policies don't make > technical sense, again the issue is moot, and no disclaimers are > necessary. Eg: rescan intervals: there's nothing that implies that > rescans are necessary. Hence, comments about rescanning of IPv6 entries > being impractical or impossible doesn't invalidate a "minimum rescan > interval" policy. If you can't rescan, then, the minimum rescan > interval policy is moot - by not rescanning at all, you still meet the BCP. We don't even know what "rescan" will mean in an IPv6 world -- it's just too darn premature. I asked for a statement that IPv6 DNSBLs are "a work in progress" and that this document "cannot make recommendations specific to IPv6" -- both inarguably true. And we see plenty of evidence _within_ this RG that folks are thinking s/4/6/ is an adequate heuristic -- which is inarguably false. > So, I don't see a hard conflict nor a necessity for such a disclaimer - > at least not a blanket one. Is there any MUST/SHOULD that _cannot_ be > met by a plausible IPv6 DNSBL implementation and needs a specific > disclaimer? We don't know. We don't even know what a "plausible IPv6 DNSBL implementation" may turn out to be. (But it is very suggestive that there is essentially nothing keeping a compromised IPv6 host from using a different /128 for every connection it makes...) >> Other than that, I stumbled upon some of the MUSTard. >> >> In general, we cannot enforce any behavior on list maintainers. > > No, but we can encourage, and note/discourage what behaviours cause > problems in the real world. MUSTard isn't about "encouraging" and "discouraging". Actually, MUSTard is an IETF thing, not an IRTF thing. We incorporate RFC 2119 by reference, but we can't entirely know what baggage our readers will bring. >> Specifically, the "MUST NOT charge" in Section 2.5 is inappropriate. >> DNSBLs that do not charge for access will necessarily need to recover >> costs for manual actions that exceed de-minimis, or simply not do them. > > There are DNSBLs that do not charge for access and do not charge for > manual actions. For the most part, they don't _do_ manual actions. > In fact, there's only one DNSBL that I know of that still imposes > financial costs surrounding delistings. It ain't SORBS... > Michelle _herself_ suggested "MUST NOT". And I disagree with Michelle -- though that's really not the point. Chances are, the "offending" DNSBL will abandon that business model soon, quite regardless of what we write here. The practice is being abandoned because it's an unprofitable business model. > But charging for removals demonstrably causes problems, such as people > who refuse to bother with perhaps mistaken DNSBL listings because they > view it as extortion. That's a business choice on their part. Even in IPv4, renumbering a SMTP client/server is simple enough -- and I find myself willing to do this to avoid even a $10 charge. (This is _not_ a particularly rational tradeoff for my time!) > Thus increasing the FP rate and impairing the accuracy for everyone else. "Everyone" ??? > Not to mention the harm to the reputation of the mechanism as a whole. (That sounds like a null set to me...) > A strong statement that this isn't acceptable behaviour can do nothing > but good. I don't mind a strong statement (though I don't expect it to have any effect) -- I mind MUSTard which makes us look foolish. Specifically, consider: " " Therefore, negative-connotation DNSBLs MUST not charge fees or " require donations for delisting or "faster handling", and it is " RECOMMENDED that such DNSBLs that do charge fees or require donations " not be used. (Why is there a need to RECOMMEND not using what we declare to be a null set?) You're letting your emotions rule here. There's a perfectly reasonable business model I call "Vouching Service" where human judgment is needed to verify that an event or practice which violated the rules has been sufficiently rectified to again vouch for the offending whatever. This sort of service _really_should_not_ be prohibited. But to say there may be no charge involved comes arbitrarily close to prohibiting it. :^( >> A strong "SHOULD NOT charge" is appropriate. >> >> We could probably even get away with a "MUST NOT use" such DNSBLs >> in that paragraph, although personally I'd prefer "SHOULD NOT charge" >> and "SHOULD NOT use". > > The goal of the BCP is to strongly encourage the best practises on the > DNSBL admin side, but at the same time recognize that receivers can do > anything they feel like as long as they know what they're doing. "MUST" in 2119-speak isn't "strongly encourage". Surely you can find words (such as "strongly" and "encourage") to say what you mean. > IOW: a receiver can be in compliance even while using a DNSBL that's out > of compliance. Splitting between strong prohibition aimed at DNSBL > admins and mild discouragement aimed at receivers was quite deliberate > here. (I wasn't asking for "MUST NOT use"...) > Given that the "MUST NOT" only puts one known DNSBL out of compliance > that I'm aware of, I'm comfortable with it. I'm uncomfortable "putting one out of compliance" in order to make a statment likely to rather-soon reference the null set. If we say "MUST NOT charge", we should be talking about future practices, not past ones. I feel pretty strongly about Vouching Services being a useful tool. >> In Section 3.4, the "MUST NOT list the entire Internet" is a bit >> strong as well. While I think you have given plenty of suggested >> alternatives to this, folks _do_ paint themselves into corners where >> such a doomsday-weapon becomes needed. (Listing the entire Internet >> _does_ mean they're outside the specification we're writing; but it >> comes across as an attempt to control a behavior we can't control. >> Perhaps we could suggest alternatives, not use the MUSTard, and say >> that listing the entire Internet puts a DNSBL list out of conformance >> with this set of practices.) > > Isn't that exactly what the section means? To whom ?? Seriously, it bothers me how often a reviewer notes a point of confusion, and the response amounts to "What kind of fool could ever be confused by that?" and not a single word of the document changes. Another set of eyeballs looking at a document is a Good Thing (TM). A good Document Editor welcomes the opportunity to reduce confusion, even at the expense of adding words to a document. > Besides, heretofore, any DNSBL utilizing such a doomsday weapon was > heralding their own doom - compliance becomes moot. Quite true -- so why do we need any MUSTard here? > We have to really really _really_ discourage the practise so that > receivers will know that listing-world is something they should not > need to worry about. Is "Pollyanna" your middle name? Face it: this happens. MUSTard won't change that. Receivers will be better off to understand what happened _when_ they are hit by a case of this. > But in the end, a DNSBL having to resort to it ain't going to care > about compliance, Agreed. > and some of the provisioning recommendations are about making sure > that list-the-world scenarios are not necessary. Howzzat? >> Whatever, we should make sure that language about testing for >> list-the-entire-Internet SHOULD be done is found within this Section. > > It's the next section ... isn't that enough? A forward-reference would be plenty. -- John Leslie From feenberg@nber.org Tue Jan 18 16:54:37 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 36C3D3A7081 for ; Tue, 18 Jan 2011 16:54:37 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.96 X-Spam-Level: X-Spam-Status: No, score=-5.96 tagged_above=-999 required=5 tests=[AWL=-0.160, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, SARE_SUB_RAND_LETTRS4=0.799] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RXz+nkSaszMw for ; Tue, 18 Jan 2011 16:54:36 -0800 (PST) Received: from mail2.nber.org (mail2.nber.org [66.251.72.79]) by core3.amsl.com (Postfix) with ESMTP id 49FAB28B797 for ; Tue, 18 Jan 2011 16:54:36 -0800 (PST) Received: from nber6.nber.org (nber6.nber.org [66.251.72.76]) by mail2.nber.org (8.14.4/8.14.4) with ESMTP id p0J0vBT3050921; Tue, 18 Jan 2011 19:57:11 -0500 (EST) (envelope-from feenberg@nber.org) Received: from localhost (feenberg@localhost) by nber6.nber.org (8.14.4/8.14.4/Submit) with ESMTP id p0J0rmH3023897; Tue, 18 Jan 2011 19:53:48 -0500 (EST) X-Authentication-Warning: nber6.nber.org: feenberg owned process doing -bs Date: Tue, 18 Jan 2011 19:53:48 -0500 (EST) From: Daniel Feenberg X-X-Sender: feenberg@nber6 To: Anti-Spam Research Group - IRTF In-Reply-To: <4D3629E2.4050307@mail-abuse.org> Message-ID: References: <20110118154659.41248.qmail@joyce.lan> <4D35EBBE.6050402@mail-abuse.org> <20110118204532.GC68297@verdi> <4D360CAC.4050902@nortel.com> <4D3629E2.4050307@mail-abuse.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Anti-Virus: Kaspersky Anti-Virus for Linux Mail Server 5.6.39/RELEASE, bases: 20110118 #4673006, check: 20110119 clean Subject: Re: [Asrg] please review draft-irtf-asrg-bcp-blacklists-07 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Jan 2011 00:54:37 -0000 On Tue, 18 Jan 2011, Douglas Otis wrote: > On 1/18/11 2:43 PM, Daniel Feenberg wrote: >> IPv6 is mentioned only once, in Section 3.5, and not in a way that suggests >> DNSBLs for IPv6 are appropriate or practical. The document covers >> whitelists as well as blacklists, and it is not unreasonable to suppose >> that a whitelist could exist for IPv6 mail hosts. While I might prefer a >> vigourous denunciation of IPv6 blacklists, there isn't anything really >> objectionable in the draft on this topic. > Daniel, > > It is not the number of times that IPv6 has been mentioned in the draft. It > is the number of times an example proves wholly unreasonable when related to > an IPv6 service. > > In the rather critical matter, "2.2.1. Listings SHOULD Be Temporary", this > makes a questionable assumption that listing/de-listing churn will not become > damaging whenever expiration is used. Whether one is talking about v6 > prefixes, or interface addresses, bad actors have access to virtually an > endless supply of prefixes and interface addresses. This means an address > may never repeat over a bad actor's life. I am confused about why 2.2.1 is critical. Whether listings are temporary or permanent, an IPv6 DNSBL is totally impractical and probably worthless. I believe we agree there. There are lots of things that can't be successfully distributed over DNS, this is only one of them. I might favor singling it out on the grounds that there are a few people who want to try it, but I wonder why it is so necessary to do so, since circumstances will defeat them soon enough. Is there a special reason an IPv6 DNSBL which listed spam sources would interfere with legitimate traffic? Wouldn't it just get very large without actually ever blocking much traffic, legitimate or spam? Daniel Feenberg From steve@blighty.com Tue Jan 18 17:02:57 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 5CA813A7093 for ; Tue, 18 Jan 2011 17:02:57 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.686 X-Spam-Level: X-Spam-Status: No, score=-1.686 tagged_above=-999 required=5 tests=[AWL=0.114, BAYES_00=-2.599, SARE_SUB_RAND_LETTRS4=0.799] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Lvk6DtwEOp0H for ; Tue, 18 Jan 2011 17:02:55 -0800 (PST) Received: from m.wordtothewise.com (b.wordtothewise.com [208.187.80.136]) by core3.amsl.com (Postfix) with ESMTP id AEDCF3A708F for ; Tue, 18 Jan 2011 17:02:55 -0800 (PST) Received: from platter.wordtothewise.com (184.wordtothewise.com [208.187.80.184]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: steve) by m.wordtothewise.com (Postfix) with ESMTPSA id 3CE4C2E9B9 for ; Tue, 18 Jan 2011 17:05:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wordtothewise.com; s=1.wttw; t=1295399134; bh=e6RkkL5dDu86+amo7QIoxye8z6hVGOWRY6p5dsKRQJs=; h=Content-Type:Mime-Version:Subject:From:In-Reply-To:Date: Content-Transfer-Encoding:Message-Id:References:To; b=UOIK6T9YaABU3qzpYKW7QgZnONwLyC10A151Pi/h829j1NLlCCC8tqzN3HT/wGDdd NN+xzbdxb3COkljuEjZq1exBlpL8w30yNq4rJhRwIBL6vRDdKCMgXQMInr+FleZusI KQU44grkSqWnKzqMdKyDu1Wk5wYdEu7HU3kqW/+o= Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Apple Message framework v1082) From: Steve Atkins In-Reply-To: Date: Tue, 18 Jan 2011 17:05:33 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <8FDE16FA-C5A6-454C-AE37-699C9231FB44@blighty.com> References: <20110118154659.41248.qmail@joyce.lan> <4D35EBBE.6050402@mail-abuse.org> <20110118204532.GC68297@verdi> <4D360CAC.4050902@nortel.com> <4D3629E2.4050307@mail-abuse.org> To: Anti-Spam Research Group - IRTF X-Mailer: Apple Mail (2.1082) Subject: Re: [Asrg] please review draft-irtf-asrg-bcp-blacklists-07 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Jan 2011 01:02:57 -0000 On Jan 18, 2011, at 4:53 PM, Daniel Feenberg wrote: >=20 >=20 > On Tue, 18 Jan 2011, Douglas Otis wrote: >=20 >> On 1/18/11 2:43 PM, Daniel Feenberg wrote: >>> IPv6 is mentioned only once, in Section 3.5, and not in a way that = suggests DNSBLs for IPv6 are appropriate or practical. The document = covers whitelists as well as blacklists, and it is not unreasonable to = suppose that a whitelist could exist for IPv6 mail hosts. While I might = prefer a vigourous denunciation of IPv6 blacklists, there isn't anything = really objectionable in the draft on this topic. >> Daniel, >>=20 >> It is not the number of times that IPv6 has been mentioned in the = draft. It is the number of times an example proves wholly unreasonable = when related to an IPv6 service. >>=20 >> In the rather critical matter, "2.2.1. Listings SHOULD Be = Temporary", this makes a questionable assumption that listing/de-listing = churn will not become damaging whenever expiration is used. Whether = one is talking about v6 prefixes, or interface addresses, bad actors = have access to virtually an endless supply of prefixes and interface = addresses. This means an address may never repeat over a bad actor's = life. >=20 > I am confused about why 2.2.1 is critical. Whether listings are = temporary or permanent, an IPv6 DNSBL is totally impractical and = probably worthless. I believe we agree there. =20 You both seem to assume that an IPv6 based blacklist would only list = /128s. If you start from that assumption, you're reaching the right = conclusion. > There are lots of things that can't be successfully distributed over = DNS, this is only one of them. I might favor singling it out on the = grounds that there are a few people who want to try it, but I wonder why = it is so necessary to do so, since circumstances will defeat them soon = enough. >=20 > Is there a special reason an IPv6 DNSBL which listed spam sources = would interfere with legitimate traffic? Wouldn't it just get very large = without actually ever blocking much traffic, legitimate or spam? No. DNS isn't a good way of distributing blacklist data, for v4 or v6, = but it's not entirely broken.=20 As an example, an IPv6 blacklist that was distributed via DNS and = typically listed in units of /48 would block quite a lot of traffic[1] - = whether that would be primarily spam or primarily legitimate would = depend on listing policy. The blacklist zone itself wouldn't necessarily be any bigger than an = equivalent v4 zone. Where you might see some bloat would be in caching = resolvers, and you'd only see it there if a significant fraction of = sources of email changed their IPv6 address often (where "often" means = "several times during the blacklist TTL"). Cheers, Steve [1] Assuming there'll be a lot of IPv6 SMTP traffic.= From johnl@iecc.com Tue Jan 18 19:49:42 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 13E4128C105 for ; Tue, 18 Jan 2011 19:49:42 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -110.665 X-Spam-Level: X-Spam-Status: No, score=-110.665 tagged_above=-999 required=5 tests=[AWL=-0.265, BAYES_00=-2.599, HABEAS_ACCREDITED_SOI=-4.3, RCVD_IN_BSP_TRUSTED=-4.3, SARE_SUB_RAND_LETTRS4=0.799, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5Ca8YQZSnjBE for ; Tue, 18 Jan 2011 19:49:40 -0800 (PST) Received: from gal.iecc.com (gal.iecc.com [64.57.183.53]) by core3.amsl.com (Postfix) with ESMTP id 9EFD228C101 for ; Tue, 18 Jan 2011 19:49:40 -0800 (PST) Received: (qmail 43883 invoked from network); 19 Jan 2011 03:52:18 -0000 Received: from mail1.iecc.com (64.57.183.56) by mail1.iecc.com with QMQP; 19 Jan 2011 03:52:18 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=iecc.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=13725.4d365ff2.k1101; i=johnl@user.iecc.com; bh=tlXQyODAArUFWg4zB4l9zLtqw3pxrpekShM/ue7jm0E=; b=PfGxJ5O3EC6UDTIj/2Tv0G6z3+mPFcZReZJYm1FqOxOau1ZP1lGafza/OzU52f3Dw5MXzZDAswliig3H3XD910h6u8+oyJ450c6cQcHoq33GHBI+8bGiXcMF/r/toNZYcFsVnnQHVTiGkOSwTGIL3SFYU4ZoFJPGjkWoVONMqA4= DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=taugh.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=13725.4d365ff2.k1101; olt=johnl@user.iecc.com; bh=tlXQyODAArUFWg4zB4l9zLtqw3pxrpekShM/ue7jm0E=; b=ey2G0a4EPgcXvPyUw0lXDcVhjTJL58euOjcUiumyQU8uUyytjNwRVDye33v71ityOxeKFeK55xZGApKIZ7LsvsFOZzb2LtxoTQwgKBGpey1XjwVRzmWtlU8BQ9afJH8AHO4cW1Igts/iErtdP6NQddNxSrVc+RwDNcLYd45ImcE= VBR-Info: md=iecc.com; mc=all; mv=dwl.spamhaus.org Date: 19 Jan 2011 03:52:18 -0000 Message-ID: <20110119035218.79652.qmail@joyce.lan> From: John Levine To: asrg@irtf.org In-Reply-To: <4D3615F6.5010206@nortel.com> Organization: Cc: X-Headerized: yes Mime-Version: 1.0 Content-type: text/plain; charset=utf-8 Content-transfer-encoding: 7bit Subject: Re: [Asrg] please review draft-irtf-asrg-bcp-blacklists-07 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Jan 2011 03:49:42 -0000 >I can see the point. A subsection of the introduction entitled "But why >DNS?", and listing out your points plus a few more (like "DNS already >exists and is ubiquitous", "it distributes in a global manner", "like >wow, it works well"). A bit of a paeon to Vixie's serendipitous insight >at a time when few would have thought of it. FYI, it wasn't Vixie. He asked me to submit an erratum to 5782: > Eric Ziegast, who was a system administrator for Vixie Enterprises > where the RBL was hosted, created a DNS-based distribution scheme > that quickly became more popular than the original BGP distribution. I agree that this isn't a great place for a justification of DNS distribution of BL/WL beyond the obvious fact that we've done it that way since 1997, and there isn't anything else available off the shelf that will do a lookup in a single UDP round trip and has widely deployed caches. I've been doing some simulations of DNSBL cache behavior, to compare conventional record-per-IP DNSBLs with other approaches. That may evolve into an I-D and an RFC, so the merits or lack thereof of various publication techniques could go in there. R's, John PS: "paean" From dotis@mail-abuse.org Tue Jan 18 20:04:40 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id CC92628C113 for ; Tue, 18 Jan 2011 20:04:40 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.152 X-Spam-Level: X-Spam-Status: No, score=-106.152 tagged_above=-999 required=5 tests=[AWL=-0.352, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, SARE_SUB_RAND_LETTRS4=0.799, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZX-Q4Ei42tKE for ; Tue, 18 Jan 2011 20:04:39 -0800 (PST) Received: from harry.mail-abuse.org (harry.mail-abuse.org [168.61.5.27]) by core3.amsl.com (Postfix) with ESMTP id DBA3328C101 for ; Tue, 18 Jan 2011 20:04:39 -0800 (PST) Received: from US-DOUGLASO-MB.local (gateway1.sjc.mail-abuse.org [168.61.5.81]) by harry.mail-abuse.org (Postfix) with ESMTP id 115C5A9443C for ; Wed, 19 Jan 2011 04:08:20 +0000 (UTC) Message-ID: <4D366376.7050406@mail-abuse.org> Date: Tue, 18 Jan 2011 20:07:18 -0800 From: Douglas Otis User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: asrg@irtf.org References: <20110118154659.41248.qmail@joyce.lan> <4D35EBBE.6050402@mail-abuse.org> <20110118204532.GC68297@verdi> <4D360CAC.4050902@nortel.com> <4D3629E2.4050307@mail-abuse.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Asrg] please review draft-irtf-asrg-bcp-blacklists-07 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Jan 2011 04:04:40 -0000 On 1/18/11 4:53 PM, Daniel Feenberg wrote: >> In the rather critical matter, "2.2.1. Listings SHOULD Be >> Temporary", this makes a questionable assumption that >> listing/de-listing churn will not become damaging whenever expiration >> is used. Whether one is talking about v6 prefixes, or interface >> addresses, bad actors have access to virtually an endless supply of >> prefixes and interface addresses. This means an address may never >> repeat over a bad actor's life. > > I am confused about why 2.2.1 is critical. Back in October of 2008, it was apparent this group had strong expectation that DNSBL listings are to be fully automated with an emphasis on listing expiry. Unfortunately, automated listings make it easy for bad actors to determine spam trap locations, where expiry also permits routine reuse of an often large and diverse IP address space. Some spammers cycle targets into different geographic regions often served by different listing services as well. The Introduction states: ,-- This document is not applicable to some DNSBLs in some areas (mentioned as appropriate) but it is the authors' belief that most of the practices are applicable to almost all DNSBLs. '-- In addition, this draft describes a requirement to delineate listing criteria with auditing to be made available. It even suggests exclusion of information that might reveal the location of a spam trap. Any audit information reveals spam trap locations. There is now an increase level of abuse emitted by bulk senders that "wash" their lists of detected spam traps. :^( The draft also wants disclosure whether a range of IP addresses might be listed that could include undetected addresses, and then describes such listings as a "mistake". It also makes _no_ reference to a boundary of such ranges not crossing those controlled by different Autonomous Systems (network providers). It also claims fast listing mechanisms benefit from very short expiration intervals. Once again, this wholly ignores problems related to the disclosure of the spam trap infrastructure. This draft also claims that listings based upon longer time periods have higher error rates and states that responses to removal requests SHOULD occur within 2 days and MUST occur within 7 days. This draft also suggests that network operators should not be required to resolve abuse issues because this requires "impractical amounts of effort" and even recommends "no questions asked" removals. This draft further states that removals should be possible in absence of DNSBL Operators. This does not describe policies that are appropriate for IPv4, and certainly it would be wrong to suggest these policies might also be applied against IPv6. The only reference to listings methods references rfc5782 which does not directly expose CIDRs used to make the listings. > Whether listings are temporary or permanent, an IPv6 DNSBL is totally > impractical and probably worthless. I believe we agree there. There > are lots of things that can't be successfully distributed over DNS, > this is only one of them. I might favor singling it out on the grounds > that there are a few people who want to try it, but I wonder why it is > so necessary to do so, since circumstances will defeat them soon enough. This nonsense serves as a distraction where valuable time is lost in getting ready for the use of IPv6. > Is there a special reason an IPv6 DNSBL which listed spam sources > would interfere with legitimate traffic? Wouldn't it just get very > large without actually ever blocking much traffic, legitimate or spam? Yes. Standards organizations need to clearly state that IPv6 REQUIRES the use of domain name authentication as a basis for acceptance. Negative listings represent an impossibility, and positive listings will likely be problematic unless based upon cryptographically authenticated domain names. -Doug From CLEWIS@nortel.com Tue Jan 18 20:36:44 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 7869D3A70CA for ; Tue, 18 Jan 2011 20:36:44 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.575 X-Spam-Level: X-Spam-Status: No, score=-5.575 tagged_above=-999 required=5 tests=[AWL=0.225, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, SARE_SUB_RAND_LETTRS4=0.799] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yY+4gptxkJEI for ; Tue, 18 Jan 2011 20:36:41 -0800 (PST) Received: from zrtps0kp.nortel.com (zrtps0kp.nortel.com [47.140.192.56]) by core3.amsl.com (Postfix) with ESMTP id 32C6B28C128 for ; Tue, 18 Jan 2011 20:36:41 -0800 (PST) Received: from zrtphxs1.corp.nortel.com (zrtphxs1.corp.nortel.com [47.140.202.46]) by zrtps0kp.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id p0J4dGF15688 for ; Wed, 19 Jan 2011 04:39:16 GMT Received: from zrtphx5h0.corp.nortel.com ([47.140.202.65]) by zrtphxs1.corp.nortel.com with Microsoft SMTPSVC(6.0.3790.4675); Tue, 18 Jan 2011 23:39:15 -0500 Received: from [47.102.152.48] (47.102.152.48) by zrtphx5h0.corp.nortel.com (47.140.202.65) with Microsoft SMTP Server (TLS) id 8.1.340.0; Tue, 18 Jan 2011 23:39:15 -0500 Message-ID: <4D366AF1.3000307@nortel.com> Date: Tue, 18 Jan 2011 23:39:13 -0500 From: "Chris Lewis" Organization: Nortel User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.23) Gecko/20090812 Thunderbird/2.0.0.23 Mnenhy/0.7.6.666 MIME-Version: 1.0 To: References: <20110118154659.41248.qmail@joyce.lan> <4D35EBBE.6050402@mail-abuse.org> <20110118204532.GC68297@verdi> <4D360CAC.4050902@nortel.com> <20110119005144.GE68297@verdi> In-Reply-To: <20110119005144.GE68297@verdi> Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 19 Jan 2011 04:39:15.0899 (UTC) FILETIME=[D3EF88B0:01CBB792] Subject: Re: [Asrg] please review draft-irtf-asrg-bcp-blacklists-07 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Jan 2011 04:36:44 -0000 On 1/18/2011 7:51 PM, John Leslie wrote: > Chris Lewis wrote: >> I'm, however, hesitant in doing so. My reasoning is as follows: >> >> a) If IPv6 DNSBLs are impractical as some suggest, no useful ones will >> exist, the issue is moot, and no disclaimers are necessary. > > Not a helpful argument. It supposes this document being read much > later than it's being issued (which could be true, but isn't our main > intent). I don't see how _when_ the document is being read is relevant. If they read it in the future when IPv6's are known to be impractical, they'll know it's moot. If they read it now, and IPv6 DNSBLs turn out to be impractical, they'll run into the reality wall that proves it's moot. We don't _know_ IPv6 DNSBLs are impossible. And frankly I don't think they are, unless you presuppose that listing /128s are the only way. Listing only /32s isn't true in IPv4 land, there is no reason to believe that that would change in IPv6. In fact, the contrary it will almost certainly become necessary for any DNSBL to be "profitable". Let me cut to the chase here: I can see a relatively easy way to maintain the usefulness of even (present) /32-listing style DNSBLs in an IPv6 world (eg: either fixed or adjustable granularity to match or roughly match real/likely end-user allocation sizes) with only complexity being to deal with the cache blow-up problem. And I don't see any of that which would force IPv6 DNSBLs implemented that way violating any of the MUSTs or SHOULDs. Some of that may obviously be impacted by IPv6 DNSBL standardization efforts, or I may be wrong about some of the MUSTage/SHOULDage, but there's nothing to stop the issuance of a new BCP around IPv6 specifically that deals with that. Better to have general guidance in place that will probably work for the most part and might need a bit of later tweakage, than to throw up our hands and provide no guidance at all. >> b) If IPv6 DNSBLs are practical, then most of the policies are entirely >> IPvX neutral (and are in fact extensible to non-DNSBLs at that), and a >> disclaimer of that type would unnecessarily deprecate the useful >> policies that _should_ still apply to IPv6. > This is false. You're going to have to explain a bit more than that. >> c) If IPv6 DNSBLs are practical, but some of the policies don't make >> technical sense, again the issue is moot, and no disclaimers are >> necessary. Eg: rescan intervals: there's nothing that implies that >> rescans are necessary. Hence, comments about rescanning of IPv6 entries >> being impractical or impossible doesn't invalidate a "minimum rescan >> interval" policy. If you can't rescan, then, the minimum rescan >> interval policy is moot - by not rescanning at all, you still meet the BCP. > We don't even know what "rescan" will mean in an IPv6 world -- it's > just too darn premature. I asked for a statement that IPv6 DNSBLs are > "a work in progress" and that this document "cannot make recommendations > specific to IPv6" -- both inarguably true. There aren't any recommendations specific to IPv6, all of the examples are specific to IPv4, so is the statement even necessary? I don't object to the statement _except_ for the reason given in (b) above where you didn't elaborate beyond "this is false". [Can I say that I don't like discussion by bare assertion? ;-)] "rescan" is used within the document in a very specific context - that of probing systems for security faults. It could only be used in the same context in IPv6. If /128 hopping is truly going to be as ubiquitous as people may think, the form of probing inferred by the context will be impossibly expensive. Hence, specifying a minimum interval is moot. They'll either know that ahead of time, or find out quickly enough. In terms you use later - "it'll be unprofitable". > And we see plenty of evidence _within_ this RG that folks are thinking > s/4/6/ is an adequate heuristic -- which is inarguably false. And DNSBLs constructed in that basis would, even I would assume, just about useless and probably wouldn't live long enough for the BCP to matter. We don't have to make policy for the unworkable. > (But it is very suggestive that > there is essentially nothing keeping a compromised IPv6 host from using > a different /128 for every connection it makes...) Not really suggestive. Postulate Doug's scenario. Every compromised host spits out a spam on a different /128 every time. DNSBLs confounded by this will be entirely "unprofitable" and they won't exist for long. Most of the policy is technology/query format neutral - and those that aren't will either be implementable in some sort of IPv6 analogue (which is up to a format specification for IPv6 DNSBLs), or not implementable at all. >> There are DNSBLs that do not charge for access and do not charge for >> manual actions. > For the most part, they don't _do_ manual actions. I have significant background in this area. I assure you, except for the toy ones, even the free ones do manual action, and some fairly extensively. [Oddly enough, it's the commercial ventures that seem to do the least. Many of the commercial ventures make even SORBS look impossibly responsive. Steve might yell about C&C warnings, but I think he'd have to admit it's true ;-)] >> In fact, there's only one DNSBL that I know of that still imposes >> financial costs surrounding delistings. It ain't SORBS... >> Michelle _herself_ suggested "MUST NOT". > And I disagree with Michelle -- though that's really not the point. > Chances are, the "offending" DNSBL will abandon that business model > soon, quite regardless of what we write here. The practice is being > abandoned because it's an unprofitable business model. Actually, that's not why Michelle abandoned it. She was always uncomfortable with it, and said so, but thought it necessary due to technical implementation limitations and the avoidance of gaming. When the implementation changed that calculation changed, and the perceived necessity went away. Even so, most disagreed with her necessity calculation including virtually all of the DNSBL admin community I know (and I know rather a lot of it). >> Thus increasing the FP rate and impairing the accuracy for everyone else. > "Everyone" ??? Yes, everyone. A, say, ISP's broad refusal to deal with a DNSBL's false positives because of orthogonal perceptions of extortion, blackmail or protection racket worsens the false positive rate of that DNSBL. This can only be untrue if you assume that every DNSBL is perfect. Unfortunately, they aren't. >> Not to mention the harm to the reputation of the mechanism as a whole. > > (That sounds like a null set to me...) I think it's provably not a null set ;-) >> A strong statement that this isn't acceptable behaviour can do nothing >> but good. > I don't mind a strong statement (though I don't expect it to have any > effect) -- I mind MUSTard which makes us look foolish. H'mm - beginning of parenthetical comment - see below. > Specifically, consider: > " > " Therefore, negative-connotation DNSBLs MUST not charge fees or > " require donations for delisting or "faster handling", and it is > " RECOMMENDED that such DNSBLs that do charge fees or require donations > " not be used. > (Why is there a need to RECOMMEND not using what we declare to be a > null set?) It's demonstrably not a null set. > You're letting your emotions rule here. There's a perfectly reasonable > business model I call "Vouching Service" where human judgment is needed > to verify that an event or practice which violated the rules has been > sufficiently rectified to again vouch for the offending whatever. > This sort of service _really_should_not_ be prohibited. You're right, it shouldn't. And it isn't. > But to say > there may be no charge involved comes arbitrarily close to prohibiting > it. :^( I disagree with that through existence proofs ;-) >>> A strong "SHOULD NOT charge" is appropriate. >>> >>> We could probably even get away with a "MUST NOT use" such DNSBLs >>> in that paragraph, although personally I'd prefer "SHOULD NOT charge" >>> and "SHOULD NOT use". >> >> The goal of the BCP is to strongly encourage the best practises on the >> DNSBL admin side, but at the same time recognize that receivers can do >> anything they feel like as long as they know what they're doing. > > "MUST" in 2119-speak isn't "strongly encourage". Surely you can find > words (such as "strongly" and "encourage") to say what you mean. Parenthetical comment (from "H'mmm"): Lemme think a bit more/experiment on using non-RFC 2119 language for this and I'll get back to y'all. > If we say "MUST NOT charge", we should be talking about future > practices, not past ones. Right. > I feel pretty strongly about Vouching Services being a useful tool. Me too, but that doesn't necessitate charging for repair of involuntary inclusion in negative connotation DNSBLs. We have to very much keep in mind some of the wording from the BCP surrounding negative connotation DNSBLs and the fundamental difference between DNSBL and more traditional potentially negative vouching services we know of. Namely, the human factor. When a restaurant reviewer says someone's restaurant sucks, the restaurant goer has to make a conscious choice to accept/reject that advice for that restaurant. In contrast, a DNSBL can literally make the site disappear for all of their users, and the user wouldn't necessarily know it then or ever. If the DNSBL does "repair" of that unasked for negative vouch for profit, it becomes more an extra-judicial fine with little if any oversight. Imagine if you will restaurant reviewers charging restaurants for a re-review of bad vouches. Just how well do you think that would be received by _anyone_? Even more so with DNSBLs. >>> In Section 3.4, the "MUST NOT list the entire Internet" is a bit >>> strong as well. While I think you have given plenty of suggested >>> alternatives to this, folks _do_ paint themselves into corners where >>> such a doomsday-weapon becomes needed. (Listing the entire Internet >>> _does_ mean they're outside the specification we're writing; but it >>> comes across as an attempt to control a behavior we can't control. >>> Perhaps we could suggest alternatives, not use the MUSTard, and say >>> that listing the entire Internet puts a DNSBL list out of conformance >>> with this set of practices.) >> >> Isn't that exactly what the section means? > To whom ?? > Seriously, it bothers me how often a reviewer notes a point of > confusion, and the response amounts to "What kind of fool could ever > be confused by that?" and not a single word of the document changes. I assure you it wasn't intended that way. The problem is that I don't see suggestions that aren't already implemented. Without a "MUST NOT", how can we say that doing so puts someone out of compliance? Alternatives are given, etc. Is it intra-document pointers I'm missing? Is it just unclear? What? > Another set of eyeballs looking at a document is a Good Thing (TM). Absolutely. > A good Document Editor welcomes the opportunity to reduce confusion, > even at the expense of adding words to a document. I'm just a bit confused as to which words. >> Besides, heretofore, any DNSBL utilizing such a doomsday weapon was >> heralding their own doom - compliance becomes moot. > Quite true -- so why do we need any MUSTard here? Unlike most other "standardization" (or whatever) efforts, DNSBLs are sometimes a highly charged emotional/political issue. As such, it is of significant value to reduce the FUDage: - Demonstrating to DNSBL users and others that such behaviour is out of bounds. - Demonstrating that a bit of fore-thought in DNSBL setup from the beginning obviates the possibility of being painted into a corner where such may become necessary. [The howzzat is below] >> We have to really really _really_ discourage the practise so that >> receivers will know that listing-world is something they should not >> need to worry about. > Is "Pollyanna" your middle name? I'm Canadian. Does that answer your question? ;-) >> and some of the provisioning recommendations are about making sure >> that list-the-world scenarios are not necessary. > Howzzat? Virtually all list-the-world scenarios come about because of name server blowout/overload due to DNSBL shutdown and an attempt to keep the base domain useful for other purposes in future. Listing the world is an attempt to get everybody still using the DNSBL to stop using it and thereby shed DNS load. If the DNSBL is implemented using the recommendations in the BCP, you can mitigate load/avoid server blowout _without_ listing the world and _still_ communicate (in a non-destructive manner) a fairly obvious signal that the users should stop using it. The other scenario is if the domain simply expires, someone else picks it up, and promptly gets blown out by NS queries and fumbles along far enough to learn about DNS wildcarding and letting fly. This too has happened. At least domain owners can be pointed to this document as a means by which they can do the same rather than barging around blowing up random mail servers. [I should point out that this section of previous drafts of this document have already proven valuable in real-world occurrences of _both_ scenarios.] >>> Whatever, we should make sure that language about testing for >>> list-the-entire-Internet SHOULD be done is found within this Section. >> >> It's the next section ... isn't that enough? > > A forward-reference would be plenty. Sorry, should have asked that instead. Okay. From davidnicol@gmail.com Wed Jan 19 07:50:16 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 286333A7016 for ; Wed, 19 Jan 2011 07:50:16 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.947 X-Spam-Level: X-Spam-Status: No, score=-2.947 tagged_above=-999 required=5 tests=[AWL=-0.147, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1, SARE_SUB_RAND_LETTRS4=0.799] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JzearfehThft for ; Wed, 19 Jan 2011 07:50:11 -0800 (PST) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by core3.amsl.com (Postfix) with ESMTP id 751C33A6FEF for ; Wed, 19 Jan 2011 07:50:11 -0800 (PST) Received: by iyb26 with SMTP id 26so892766iyb.13 for ; Wed, 19 Jan 2011 07:52:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:from:date :message-id:subject:to:content-type:content-transfer-encoding; bh=JF1pnCVtBZdVjni0wjCrA7BSK0WNakn/XJt0kmarvAo=; b=NU/agpFOuxrAkWd2oMdGLgVjqB7BbMoKrOo1t91u6wZlDbHU+wtv/UJZqm9ZUrNBN1 1Br7eayXcqxPIBSZxRex16Jyc2RjV1X9meOM/p412gl9B4hX33ZU/a8/HbD9T5AxJvzk /r/7KBVbLhzWGDNwXvyJP3Mko/rnn7OjQja/M= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type:content-transfer-encoding; b=fBxd9Sy/k35Iu3bLAerDP69ei1E5D/75lgeICK4PZ0KfJdwgVDqbr8Odl7L0xcx6IO 2DZQMPPv+mEgYA7G2bRMA6VsM7WKNGuo6ASEE6gYC7GWRaUrLzIPSi/xFLgW8B64sZ/C +wbVv9hkKgvOP4vs/0OYEbY99p7f1Z29M0zfo= Received: by 10.231.14.134 with SMTP id g6mr1035058iba.65.1295452371650; Wed, 19 Jan 2011 07:52:51 -0800 (PST) MIME-Version: 1.0 Received: by 10.231.206.148 with HTTP; Wed, 19 Jan 2011 07:52:31 -0800 (PST) In-Reply-To: <4D366AF1.3000307@nortel.com> References: <20110118154659.41248.qmail@joyce.lan> <4D35EBBE.6050402@mail-abuse.org> <20110118204532.GC68297@verdi> <4D360CAC.4050902@nortel.com> <20110119005144.GE68297@verdi> <4D366AF1.3000307@nortel.com> From: David Nicol Date: Wed, 19 Jan 2011 09:52:31 -0600 Message-ID: To: Anti-Spam Research Group - IRTF Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [Asrg] please review draft-irtf-asrg-bcp-blacklists-07 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Jan 2011 15:50:16 -0000 On Tue, Jan 18, 2011 at 10:39 PM, Chris Lewis wrote: > Imagine if you will restaurant reviewers charging restaurants for a > re-review of bad vouches. =C2=A0Just how well do you think that would be > received by _anyone_? =C2=A0 Even more so with DNSBLs. Restaurant reviewing is luxury, not infrastructure. Revise to "health departments" and it seems entirely reasonable. >> =C2=A0 =C2=A0Is "Pollyanna" your middle name? > > I'm Canadian. =C2=A0Does that answer your question? ;-) I for one eagerly await the Canadian invasion, with their quaint paper ballots. I shall throw flowers from my balcony. --=20 =E2=80=9CThe aeroplane is fatally defective. It is merely a toy=E2=80=94a s= porting play-thing. =C2=A0It can never become commercially practical." -- Nikola Tesla From davidnicol@gmail.com Wed Jan 19 08:15:31 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 5943D3A7024 for ; Wed, 19 Jan 2011 08:15:31 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.94 X-Spam-Level: X-Spam-Status: No, score=-2.94 tagged_above=-999 required=5 tests=[AWL=-0.140, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1, SARE_SUB_RAND_LETTRS4=0.799] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FIqk9MIIwS3A for ; Wed, 19 Jan 2011 08:15:30 -0800 (PST) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by core3.amsl.com (Postfix) with ESMTP id 8F9433A7130 for ; Wed, 19 Jan 2011 08:15:30 -0800 (PST) Received: by iyb26 with SMTP id 26so916154iyb.13 for ; Wed, 19 Jan 2011 08:18:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:from:date :message-id:subject:to:content-type; bh=Ulqc3UeZo6uaE4msb+8lcIgk3ob8WMnaV5xNYD3rEvw=; b=Y1xZp6AeXcZxkqFi60BPhHR/6e6rSXXLZmJH9V7k6i2qUlzw+EWL2HocTV4T0I/xQd RVO6u27S/fOrVrUEvNnVU+LT7TyPdDaWH/tM+SziqFbuUhGYer+SB2aA94amt61TdiEU IINFkOt0XnajCqrXOYwWZTAscpGOchTvH6IKg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; b=KeptLZpTwlOA5CDdWqTiTYCFAA4aWq1biUddSLYdzBLtaht2S/ah9Yi2O/mLJtqM+p wZ5YFYJHap1wYZEmFDdQLxbovGYi0y+esd2y+x4hEj+AVqr8l0NkqEIBWk6iRchvU9G1 v1RCPhhd2r3jM/AiavWQtdp5kg+g/gyHNwI0Y= Received: by 10.231.14.134 with SMTP id g6mr1073716iba.65.1295453890950; Wed, 19 Jan 2011 08:18:10 -0800 (PST) MIME-Version: 1.0 Received: by 10.231.206.148 with HTTP; Wed, 19 Jan 2011 08:17:50 -0800 (PST) In-Reply-To: <20110119035218.79652.qmail@joyce.lan> References: <4D3615F6.5010206@nortel.com> <20110119035218.79652.qmail@joyce.lan> From: David Nicol Date: Wed, 19 Jan 2011 10:17:50 -0600 Message-ID: To: Anti-Spam Research Group - IRTF Content-Type: text/plain; charset=UTF-8 Subject: Re: [Asrg] please review draft-irtf-asrg-bcp-blacklists-07 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Jan 2011 16:15:31 -0000 On Tue, Jan 18, 2011 at 9:52 PM, John Levine wrote: > I agree that this isn't a great place for a justification of DNS > distribution of BL/WL beyond the obvious fact that we've done it that > way since 1997, and there isn't anything else available off the shelf > that will do a lookup in a single UDP round trip and has widely > deployed caches. Here's a draft extended version of the introduction: "The rising amount of spam and other forms of network abuse on the Internet led to sys-admins compiling lists of sources from which e-mail was usually undesired. As DNS provides quick distribution of fine-grained updates and performs lookups in a single UDP round trip and only as needed, many community members and companies began to share their data through creation, publish and maintainence of DNS-based reputation systems. These "DNS-Based Lists" of IP addresses or domain names contain entries with reputation ratings useful as suggestions or assertions about e-mail originating from these IP addresses or domain names." END OF REVISED PARAGRAPH But wait, there's more! Furthermore, there apparently exist back-channels for mirroring the whole database that are currently in use in high-volume situations. Perhaps that deserves a mention? A new section on sharing one's list via methods other than DNS? From CLEWIS@nortel.com Wed Jan 19 09:03:36 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 08D8828C0EB for ; Wed, 19 Jan 2011 09:03:36 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.32 X-Spam-Level: X-Spam-Status: No, score=-5.32 tagged_above=-999 required=5 tests=[AWL=-0.120, BAYES_00=-2.599, J_CHICKENPOX_33=0.6, RCVD_IN_DNSWL_MED=-4, SARE_SUB_RAND_LETTRS4=0.799] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7uApqFPz2nLZ for ; Wed, 19 Jan 2011 09:03:35 -0800 (PST) Received: from zrtps0kp.nortel.com (zrtps0kp.nortel.com [47.140.192.56]) by core3.amsl.com (Postfix) with ESMTP id E5A8B28C0FB for ; Wed, 19 Jan 2011 09:03:34 -0800 (PST) Received: from zrtphxs1.corp.nortel.com (zrtphxs1.corp.nortel.com [47.140.202.46]) by zrtps0kp.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id p0JH6BF27895 for ; Wed, 19 Jan 2011 17:06:11 GMT Received: from zrtphx5h0.corp.nortel.com ([47.140.202.65]) by zrtphxs1.corp.nortel.com with Microsoft SMTPSVC(6.0.3790.4675); Wed, 19 Jan 2011 12:06:11 -0500 Received: from [47.102.168.100] (47.102.168.100) by zrtphx5h0.corp.nortel.com (47.140.202.65) with Microsoft SMTP Server (TLS) id 8.1.340.0; Wed, 19 Jan 2011 12:06:10 -0500 Message-ID: <4D3719DF.7040304@nortel.com> Date: Wed, 19 Jan 2011 12:05:35 -0500 From: "Chris Lewis" Organization: Nortel User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.23) Gecko/20090812 Thunderbird/2.0.0.23 Mnenhy/0.7.6.666 MIME-Version: 1.0 To: References: <4D3615F6.5010206@nortel.com> <20110119035218.79652.qmail@joyce.lan> In-Reply-To: Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 19 Jan 2011 17:06:11.0057 (UTC) FILETIME=[2BD9F610:01CBB7FB] Subject: Re: [Asrg] please review draft-irtf-asrg-bcp-blacklists-07 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Jan 2011 17:03:36 -0000 On 1/19/2011 11:17 AM, David Nicol wrote: > On Tue, Jan 18, 2011 at 9:52 PM, John Levine wrote: > >> I agree that this isn't a great place for a justification of DNS >> distribution of BL/WL beyond the obvious fact that we've done it that >> way since 1997, and there isn't anything else available off the shelf >> that will do a lookup in a single UDP round trip and has widely >> deployed caches. > > Here's a draft extended version of the introduction: > > "The rising amount of spam and other forms of network abuse on the > Internet led to sys-admins compiling lists of sources from which > e-mail was usually undesired. As DNS provides quick distribution of > fine-grained updates and performs lookups in a single UDP round trip > and only as needed, many community members and companies began to > share their data through creation, publish and maintainence of > DNS-based reputation systems. These "DNS-Based Lists" of IP addresses > or domain names contain entries with reputation ratings useful as > suggestions or assertions about e-mail originating from these IP > addresses or domain names." > > END OF REVISED PARAGRAPH > But wait, there's more! > Furthermore, there apparently exist back-channels for mirroring the > whole database that are currently in use in high-volume situations. > Perhaps that deserves a mention? A new section on sharing one's list > via methods other than DNS? I think I like the first part - slipping in more justification of DNS without making a big/distinct thing about it. As for the second: Yes, it could be mentioned, but I think it should be in the introduction with a mention that standardizing and/or BCP'ing zone transfer is beyond scope. Would it be appropriate to go as far as mentioning "commonly via rsync of a DNS server zone file format"? Not to force everything into that mold, just to indicate the most common way? From CLEWIS@nortel.com Wed Jan 19 09:40:36 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id D89463A7038 for ; Wed, 19 Jan 2011 09:40:36 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.6 X-Spam-Level: X-Spam-Status: No, score=-5.6 tagged_above=-999 required=5 tests=[AWL=0.200, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, SARE_SUB_RAND_LETTRS4=0.799] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WRuoIwvHWx+z for ; Wed, 19 Jan 2011 09:40:36 -0800 (PST) Received: from zrtps0kp.nortel.com (zrtps0kp.nortel.com [47.140.192.56]) by core3.amsl.com (Postfix) with ESMTP id CEFE03A702C for ; Wed, 19 Jan 2011 09:40:35 -0800 (PST) Received: from zrtphxs1.corp.nortel.com (zrtphxs1.corp.nortel.com [47.140.202.46]) by zrtps0kp.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id p0JHhDF05841 for ; Wed, 19 Jan 2011 17:43:13 GMT Received: from zrtphx5h0.corp.nortel.com ([47.140.202.65]) by zrtphxs1.corp.nortel.com with Microsoft SMTPSVC(6.0.3790.4675); Wed, 19 Jan 2011 12:43:13 -0500 Received: from [47.102.168.100] (47.102.168.100) by zrtphx5h0.corp.nortel.com (47.140.202.65) with Microsoft SMTP Server (TLS) id 8.1.340.0; Wed, 19 Jan 2011 12:43:12 -0500 Message-ID: <4D3722AE.2020701@nortel.com> Date: Wed, 19 Jan 2011 12:43:10 -0500 From: "Chris Lewis" Organization: Nortel User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.23) Gecko/20090812 Thunderbird/2.0.0.23 Mnenhy/0.7.6.666 MIME-Version: 1.0 To: References: <20110118154659.41248.qmail@joyce.lan> <4D35EBBE.6050402@mail-abuse.org> <20110118204532.GC68297@verdi> <4D360CAC.4050902@nortel.com> <20110119005144.GE68297@verdi> <4D366AF1.3000307@nortel.com> In-Reply-To: Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 19 Jan 2011 17:43:13.0009 (UTC) FILETIME=[583CEE10:01CBB800] Subject: Re: [Asrg] please review draft-irtf-asrg-bcp-blacklists-07 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Jan 2011 17:40:36 -0000 On 1/19/2011 10:52 AM, David Nicol wrote: > On Tue, Jan 18, 2011 at 10:39 PM, Chris Lewis wrote: > >> Imagine if you will restaurant reviewers charging restaurants for a >> re-review of bad vouches. Just how well do you think that would be >> received by _anyone_? Even more so with DNSBLs. > Restaurant reviewing is luxury, not infrastructure. Revise to "health > departments" and it seems entirely reasonable. Health inspections are a regulatory/legal requirement - compliance is legally mandatory, and if there are fines, that's also part of the legislative framework. Not to mention that there's usually only one you have to deal with. It's part of the implicit (or explicit) societal contract that a restaurant undertakes to comply with by existing. That's a huge qualitative difference between that and some random self-selected guy showing up at the cash register who says "your bacon was horrible, pay me money or I'll shut you down". Those guys usually carry baseball bats. >>> Is "Pollyanna" your middle name? >> I'm Canadian. Does that answer your question? ;-) > I for one eagerly await the Canadian invasion, with their quaint paper > ballots. I shall throw flowers from my balcony. We aim to please ;-) From davidnicol@gmail.com Wed Jan 19 10:51:42 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id D86693A70E0 for ; Wed, 19 Jan 2011 10:51:42 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.933 X-Spam-Level: X-Spam-Status: No, score=-2.933 tagged_above=-999 required=5 tests=[AWL=-0.133, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1, SARE_SUB_RAND_LETTRS4=0.799] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UOMWJjRLLTIV for ; Wed, 19 Jan 2011 10:51:41 -0800 (PST) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by core3.amsl.com (Postfix) with ESMTP id 8A5F03A705E for ; Wed, 19 Jan 2011 10:51:41 -0800 (PST) Received: by iyb26 with SMTP id 26so1070311iyb.13 for ; Wed, 19 Jan 2011 10:54:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:from:date :message-id:subject:to:content-type:content-transfer-encoding; bh=EI+7Z691TVaemej/fGy+eD+bHHNe772Q12z8dGD5tZs=; b=j2onAtLWVdO7f0s+bxNfFUH8gTXKMYLBsXGrFxnFJz7v1qVCe2uDpICvsTRvXIVphW mNm9YV25iNeg4ogiCcIxUVjWLfFKKvZYdftYcnLzFUvBiypLV8aSwXPMsAPZxxOHxzly +2YU5jrE4Ar0h5BSe3QmfDv1+KP4KeDDMuIng= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type:content-transfer-encoding; b=TuvVbPaiRicd9qOAXBh1KTa//vQD37OgeaN8oo3AnzynGEM0bRj0wtMWwLXtoZZ/BX B2/cGzV+UHaxdnSL+VqMHsFyp/y2op9rxlQ61L3oqCDUmZ48DvBpcRgWNet5rgnaagma 8LsAHUyO81yHJq52HBpQlVATZlA7iDwMakz8M= Received: by 10.231.31.4 with SMTP id w4mr1278302ibc.140.1295463006723; Wed, 19 Jan 2011 10:50:06 -0800 (PST) MIME-Version: 1.0 Received: by 10.231.206.148 with HTTP; Wed, 19 Jan 2011 10:49:46 -0800 (PST) In-Reply-To: <4D3722AE.2020701@nortel.com> References: <20110118154659.41248.qmail@joyce.lan> <4D35EBBE.6050402@mail-abuse.org> <20110118204532.GC68297@verdi> <4D360CAC.4050902@nortel.com> <20110119005144.GE68297@verdi> <4D366AF1.3000307@nortel.com> <4D3722AE.2020701@nortel.com> From: David Nicol Date: Wed, 19 Jan 2011 12:49:46 -0600 Message-ID: To: Anti-Spam Research Group - IRTF Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [Asrg] please review draft-irtf-asrg-bcp-blacklists-07 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Jan 2011 18:51:43 -0000 On Wed, Jan 19, 2011 at 11:43 AM, Chris Lewis wrote: >> Restaurant reviewing is luxury, not infrastructure. Revise to "health >> departments" and it seems entirely reasonable. > > Health inspections are a regulatory/legal requirement - compliance is > legally mandatory, and if there are fines, that's also part of the > legislative framework. =C2=A0Not to mention that there's usually only one= you > have to deal with. =C2=A0It's part of the implicit (or explicit) societal > contract that a restaurant undertakes to comply with by existing. > > That's a huge qualitative difference between that and some random > self-selected guy showing up at the cash register who says "your bacon wa= s > horrible, pay me money or I'll shut you down". =C2=A0Those guys usually c= arry > baseball bats. I will not be silenced by creative use of violent imagery! The question under discussion is something like "should the BCP document contain guidance concerning compensation for preferential treatment?" in specific, should the section now numbered 2.2.5: 2.2.5. Conflict of Interest Some DNSBLs used for blocking/negative reputation have had a practise of requiring fees or donations to charities from the listee for delisting. It is generally considered entirely appropriate for a DNSBL to charge for access to it by its users - the definition of a commercial DNSBL. However, the practise of requiring a listee to pay for delisting from a negative connotation DNSBL steers perilously close to notions of extortion, blackmail or a "protection racket". Even if such accusations are entirely unjustified the practise causes uproar and damage to the DNSBLs reputation, if not the entire DNSBL mechanism as a whole. Colloquially, "it smells bad". Therefore, negative-connotation DNSBLs MUST not charge fees or require donations for delisting or "faster handling", and it is RECOMMENDED that such DNSBLs that do charge fees or require donations not be used. END SEC. 2.2.5 EXTRACT be changed? It looks okay to me, as long as we are painfully, explicitly clear that the "users" are sys-admins and not end-users. The thing is, the random self-selected guy doesn't have all the power he says he has. Were he honest, instead of "your bacon was horrible, pay me money or I'll shut you down" he would be threatening "your bacon was horrible, pay me money or I'll list you on my list of places where I didn't like the bacon." A good example of real-life reputation listing services are lists that, for instance, enumerate places that were not able to satisfy requirements of a food intolerance. Such as this whitelist: http://www.yelp.com/search?find_desc=3Dgluten+free&find_loc=3DChicago,+IL From CLEWIS@nortel.com Wed Jan 19 12:57:47 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 550533A71CE for ; Wed, 19 Jan 2011 12:57:47 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.629 X-Spam-Level: X-Spam-Status: No, score=-5.629 tagged_above=-999 required=5 tests=[AWL=0.171, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, SARE_SUB_RAND_LETTRS4=0.799] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xygJk9R-k4ln for ; Wed, 19 Jan 2011 12:57:46 -0800 (PST) Received: from zrtps0kp.nortel.com (zrtps0kp.nortel.com [47.140.192.56]) by core3.amsl.com (Postfix) with ESMTP id 0EC403A71D0 for ; Wed, 19 Jan 2011 12:57:45 -0800 (PST) Received: from zrtphxs1.corp.nortel.com (zrtphxs1.corp.nortel.com [47.140.202.46]) by zrtps0kp.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id p0JL0MF17610 for ; Wed, 19 Jan 2011 21:00:23 GMT Received: from zrtphx5h0.corp.nortel.com ([47.140.202.65]) by zrtphxs1.corp.nortel.com with Microsoft SMTPSVC(6.0.3790.4675); Wed, 19 Jan 2011 16:00:22 -0500 Received: from [47.102.168.100] (47.102.168.100) by zrtphx5h0.corp.nortel.com (47.140.202.65) with Microsoft SMTP Server (TLS) id 8.1.340.0; Wed, 19 Jan 2011 16:00:22 -0500 Message-ID: <4D3750E0.5040605@nortel.com> Date: Wed, 19 Jan 2011 16:00:16 -0500 From: "Chris Lewis" Organization: Nortel User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.23) Gecko/20090812 Thunderbird/2.0.0.23 Mnenhy/0.7.6.666 MIME-Version: 1.0 To: References: <20110118154659.41248.qmail@joyce.lan> <4D35EBBE.6050402@mail-abuse.org> <20110118204532.GC68297@verdi> <4D360CAC.4050902@nortel.com> <20110119005144.GE68297@verdi> <4D366AF1.3000307@nortel.com> <4D3722AE.2020701@nortel.com> In-Reply-To: Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 19 Jan 2011 21:00:22.0647 (UTC) FILETIME=[E3407070:01CBB81B] Subject: Re: [Asrg] please review draft-irtf-asrg-bcp-blacklists-07 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Jan 2011 20:57:47 -0000 On 1/19/2011 1:49 PM, David Nicol wrote: > I will not be silenced by creative use of violent imagery! Canadians? Violence? Michael Moore says there's no crime here, so it can't be so ;-) > [2.2.5] be changed? It looks okay to me, as long as we are painfully, > explicitly clear that the "users" are sys-admins and not end-users. I think it is, isn't it? > The thing is, the random self-selected guy doesn't have all the power > he says he has. Were he honest, instead of "your bacon was horrible, > pay me money or I'll shut you down" he would be threatening "your > bacon was horrible, pay me money or I'll list you on my list of places > where I didn't like the bacon." In the DNSBL world, a popular blacklist listing can cause more than 50% of their email to get blocked in such a way that most recipients don't know that it's happening. They've abruptly disappeared email-wise, for a huge chunk of the Internet. This is far more than midway between an end-user advisory that an end-user can choose to ignore or accept, and a health-department mandated door shutting. > A good example of real-life reputation listing services are lists > that, for instance, enumerate places that were not able to satisfy > requirements of a food intolerance. > > Such as this whitelist: > http://www.yelp.com/search?find_desc=gluten+free&find_loc=Chicago,+IL The phraseology I used was for blacklists, not whitelists, these are end-user advisories that can be ignored/accepted by the end-user, and do they take kickbacks? ;-) Do you know of negative advisories that take money for delistings of entities that _didn't_ subscribe to the advisory service? I don't, except for the criminal code ;-) From davidnicol@gmail.com Wed Jan 19 13:25:49 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 529613A71C8 for ; Wed, 19 Jan 2011 13:25:49 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.927 X-Spam-Level: X-Spam-Status: No, score=-2.927 tagged_above=-999 required=5 tests=[AWL=-0.127, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1, SARE_SUB_RAND_LETTRS4=0.799] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WTGxFSW++JFE for ; Wed, 19 Jan 2011 13:25:48 -0800 (PST) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by core3.amsl.com (Postfix) with ESMTP id 6E0E13A71BE for ; Wed, 19 Jan 2011 13:25:48 -0800 (PST) Received: by iwn39 with SMTP id 39so1283337iwn.13 for ; Wed, 19 Jan 2011 13:28:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:from:date :message-id:subject:to:content-type:content-transfer-encoding; bh=BZvRYESCgPQnCjhf4wUYIuhh6UuJXi9n7/J9dCm6dGE=; b=DIxefrPyAaYGcex1CZX4a9a+lXzH7CfuHrelHU5USq5IeOLDMsVnME9Hm8/Chasaer VGMRS1Ap67UOVpjP2AFramiUi0DEN8LwcDflw+C3T2DJ50bO8GoFgjSMVPZ/l/Nhbw3A O/ZpAFKrshUbdTi/mGRToWkN+QmYiX64OVFbs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type:content-transfer-encoding; b=bvCS7t6pf4geSkjyG9YBLnPUuLV0Qyn3Vs+z+/qWKbt66rHGZz4gYWKKuNcrnZVhBZ nNcEHHFafKO2oMvKZULUDup41P9ojYHBEBVRMauaa4plgIjuR08f88jZJ04Bu6sXLy1M Lr6NnVJIEjWCyXtxzj08oynVsUiVjeYQQeBQg= Received: by 10.231.190.8 with SMTP id dg8mr1523355ibb.15.1295472509301; Wed, 19 Jan 2011 13:28:29 -0800 (PST) MIME-Version: 1.0 Received: by 10.231.206.148 with HTTP; Wed, 19 Jan 2011 13:28:03 -0800 (PST) In-Reply-To: <4D3750E0.5040605@nortel.com> References: <20110118154659.41248.qmail@joyce.lan> <4D35EBBE.6050402@mail-abuse.org> <20110118204532.GC68297@verdi> <4D360CAC.4050902@nortel.com> <20110119005144.GE68297@verdi> <4D366AF1.3000307@nortel.com> <4D3722AE.2020701@nortel.com> <4D3750E0.5040605@nortel.com> From: David Nicol Date: Wed, 19 Jan 2011 15:28:03 -0600 Message-ID: To: Anti-Spam Research Group - IRTF Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [Asrg] please review draft-irtf-asrg-bcp-blacklists-07 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Jan 2011 21:25:49 -0000 On Wed, Jan 19, 2011 at 3:00 PM, Chris Lewis wrote: >> [2.2.5] be changed? =C2=A0It looks okay to me, as long as we are painful= ly, >> explicitly clear that the "users" are sys-admins and not end-users. > > I think it is, isn't it? I don't know if it is nor not in the draft BCP document, but it's crazy muddled in this discussion. > In the DNSBL world, a popular blacklist listing can cause more than 50% o= f > their email to get blocked in such a way that most recipients don't know > that it's happening. =C2=A0They've abruptly disappeared email-wise, for a= huge > chunk of the Internet. =C2=A0This is far more than midway between an end-= user > advisory that an end-user can choose to ignore or accept, and a > health-department mandated door shutting. And is the system working as designed, "protecting" the recipients from alleged annoyance. Does the BCP draft have a section advising the consumers of the lists, as well as the providers of the lists? Recommending that a list consumer provide per-recipient configuration WRT protection by the various ones --- some of the people on the qpsmtpd list discuss their per-recipient configuration database hacks, I don't know what the general state of the practice is in 2011 > Do you know of negative advisories that take money for delistings of > entities that _didn't_ subscribe to the advisory service? =C2=A0I don't, = except > for the criminal code ;-) http://yourbiz.msnbc.msn.com/_news/2008/05/13/4355660-staying-off-the-bette= r-business-bureaus-bad-boy-list suggests "... take the time to sit down with their local BBB and discuss ..." which is not free. Of course if it became known that the BBB accepted bribes for delisting, that would hurt their reputation. I think attempting to mandate the mechanics of reputation will be futile, maybe 2.2.5 could be weakened to be strictly descriptive, yet provide advice for the reputation-mechanically-impaired? From dotis@mail-abuse.org Wed Jan 19 14:28:44 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 7E8E428C12A for ; Wed, 19 Jan 2011 14:28:44 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -105.981 X-Spam-Level: X-Spam-Status: No, score=-105.981 tagged_above=-999 required=5 tests=[AWL=-0.496, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, SARE_MILLIONSOF=0.315, SARE_SUB_RAND_LETTRS4=0.799, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Zods-myIyLKI for ; Wed, 19 Jan 2011 14:28:43 -0800 (PST) Received: from harry.mail-abuse.org (harry.mail-abuse.org [168.61.5.27]) by core3.amsl.com (Postfix) with ESMTP id 2126628C126 for ; Wed, 19 Jan 2011 14:28:43 -0800 (PST) Received: from US-DOUGLASO-MB.local (gateway1.sjc.mail-abuse.org [168.61.5.81]) by harry.mail-abuse.org (Postfix) with ESMTP id 542D3A9443B for ; Wed, 19 Jan 2011 22:32:19 +0000 (UTC) Message-ID: <4D376635.8000602@mail-abuse.org> Date: Wed, 19 Jan 2011 14:31:17 -0800 From: Douglas Otis User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: asrg@irtf.org References: <20110118154659.41248.qmail@joyce.lan> <4D35EBBE.6050402@mail-abuse.org> <20110118204532.GC68297@verdi> <4D360CAC.4050902@nortel.com> <20110119005144.GE68297@verdi> <4D366AF1.3000307@nortel.com> <4D3722AE.2020701@nortel.com> <4D3750E0.5040605@nortel.com> In-Reply-To: <4D3750E0.5040605@nortel.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Asrg] please review draft-irtf-asrg-bcp-blacklists-07 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Jan 2011 22:28:44 -0000 On 1/19/11 1:00 PM, Chris Lewis wrote: >> he says he has. Were he honest, instead of "your bacon was horrible, >> pay me money or I'll shut you down" he would be threatening "your >> bacon was horrible, pay me money or I'll list you on my list of places >> where I didn't like the bacon." > The thing is, the random self-selected guy doesn't have all the power > > In the DNSBL world, a popular blacklist listing can cause more than > 50% of their email to get blocked in such a way that most recipients > don't know that it's happening. They've abruptly disappeared > email-wise, for a huge chunk of the Internet. This is far more than > midway between an end-user advisory that an end-user can choose to > ignore or accept, and a health-department mandated door shutting. Chris, Networks containing millions of individual sources of abuse are often most effectively handled by policies enforced by their network providers. Policies promoted by this draft overlook network provider's role. Only network providers are able to block abuse and determine whether abuse has been mitigated by their customer. Third-parties monitoring network traffic see only a tiny fraction of any overall abuse. Listing stratagems that aid list-washing techniques further reduce the effectiveness of important third-party monitoring. It seems this draft has a goal to make third-party monitoring ineffective, and to avoid the corrective actions that may need to be taken by network providers. It is not a mistake to list networks replete with abuse. When a network provider is unwilling to stop commerce of abuse (tainted bacon), listing their network gets their attention. Their angry customers are free to seek services from other responsive providers. The original abuse listings were distributed as BGP filters. It seems future listings of IP addresses will likely need to return to this model in order to scale to the larger IP address space. Of course, positive reputations by domain name work well with DNS, and would be independent of the network provider. It seems this draft misses an opportunity to better lay out how services can be supported in the future. We continually need to counter the erosion caused by "automated" abuse listings and mailing-list washing. IPv6 will make obscuring spam traps and selectively reacting to individual IP addresses impossible. :^( -Doug From CLEWIS@nortel.com Wed Jan 19 22:50:25 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 82CAD3A6EFA for ; Wed, 19 Jan 2011 22:50:25 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.65 X-Spam-Level: X-Spam-Status: No, score=-5.65 tagged_above=-999 required=5 tests=[AWL=0.150, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, SARE_SUB_RAND_LETTRS4=0.799] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RpTOALuWzw4M for ; Wed, 19 Jan 2011 22:50:22 -0800 (PST) Received: from zrtps0kp.nortel.com (zrtps0kp.nortel.com [47.140.192.56]) by core3.amsl.com (Postfix) with ESMTP id 90B703A70AF for ; Wed, 19 Jan 2011 22:50:21 -0800 (PST) Received: from zrtphxs1.corp.nortel.com (casmtp.ca.nortel.com [47.140.202.46]) by zrtps0kp.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id p0K6qxF08099 for ; Thu, 20 Jan 2011 06:52:59 GMT Received: from zrtphx5h0.corp.nortel.com ([47.140.202.65]) by zrtphxs1.corp.nortel.com with Microsoft SMTPSVC(6.0.3790.4675); Thu, 20 Jan 2011 01:52:58 -0500 Received: from [47.102.168.100] (47.102.168.100) by zrtphx5h0.corp.nortel.com (47.140.202.65) with Microsoft SMTP Server (TLS) id 8.1.340.0; Thu, 20 Jan 2011 01:52:57 -0500 Message-ID: <4D37DBC7.20803@nortel.com> Date: Thu, 20 Jan 2011 01:52:55 -0500 From: "Chris Lewis" Organization: Nortel User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.23) Gecko/20090812 Thunderbird/2.0.0.23 Mnenhy/0.7.6.666 MIME-Version: 1.0 To: References: <20110118154659.41248.qmail@joyce.lan> <4D35EBBE.6050402@mail-abuse.org> <20110118204532.GC68297@verdi> <4D360CAC.4050902@nortel.com> <20110119005144.GE68297@verdi> <4D366AF1.3000307@nortel.com> <4D3722AE.2020701@nortel.com> <4D3750E0.5040605@nortel.com> In-Reply-To: Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 20 Jan 2011 06:52:58.0491 (UTC) FILETIME=[AC2FACB0:01CBB86E] Subject: Re: [Asrg] please review draft-irtf-asrg-bcp-blacklists-07 X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jan 2011 06:50:25 -0000 On 1/19/2011 4:28 PM, David Nicol wrote: > On Wed, Jan 19, 2011 at 3:00 PM, Chris Lewis wrote: > >>> [2.2.5] be changed? It looks okay to me, as long as we are painfully, >>> explicitly clear that the "users" are sys-admins and not end-users. >> >> I think it is, isn't it? > I don't know if it is nor not in the draft BCP document, but it's > crazy muddled in this discussion. Really? I don't think end-user came up at all, and the BCP doesn't talk about it. It's quite clear in the BCP that it's talking about list-users and list-owners, not end-users. > And is the system working as designed, "protecting" the recipients > from alleged annoyance. Does the BCP draft have a section advising > the consumers of the lists, as well as the providers of the lists? Yes. List-owner vs list-user distinctions are covered in the draft BCP quite substantively in a number of areas. > Recommending that a list consumer provide per-recipient configuration > WRT protection by the various ones --- some of the people on the > qpsmtpd list discuss their per-recipient configuration database hacks, > I don't know what the general state of the practice is in 2011. Recipient filtering policy, mechanisms and implementation are well out of scope for this document _except_ where they directly relate to DNSBLs. I've been wanting to write a more general BCP on filtering from the receiving perspective, but it'd probably be quite hard to achieve something reaching consensus without diluting it so far that it doesn't mean anything. There are strong feelings in a number of different directions. It may more end up as a series of scenarios enumerating pros and cons of each. [Aside: As an educated guess (and knowing that we're not entirely atypical ;-), most sites will have the same filtering for everyone. Many do provide a recipient-adjustable whitelist-by-sender knob and either a blacklist-by-sender knob or a TiS button, but seldom more than that. Commercial offerings (esp. 3rd party MX-handlers) tend to not have much per-user configurability beyond sender whitelisting. Not because it's hard (it isn't), but because few are asking for it, hence it's not worth implementing. There are some practitioners who wax quite extreme on substantive per-recipient configurability (eg: some of those on the qpsmtpd list ;-), but the reality is that the number of people who'd actually _use_ such knobs would be extremely small, and still fewer who could adjust it effectively. If they had such knobs, after the bombardment, experience suggests that they'd usually turn it right back on again (pushing the knobs too far ;-) and/or scream we should be filtering _that_ crap. Or sue. Or the admins get arrested[+]. Sigh. :-( Corporates, and GOV sites generally do not have end-user configurability at all except for perhaps individual sender partial whitelisting. Audit/CorpSec tend not to like people potentially turning off A-V or policy-imposed "not on our equipment!" filtering (which may or may not fit the definition of spam). Our filtering is across the board. We have the ability to whitelist senders (in any of a variety of ways) or bypass filtering for specific recipients. End-users can ask for such, but they can't set it themselves. Each request is checked. Things causing false positives with business operation are dealt with routinely - it's part of the filtering process _itself_ - our filtering is good enough that these are rare. Similarly with most non-business related stuff (we have a pretty liberal policy about reasonable personal use - many corps don't). Full blown bypass every filter sender whitelisting is almost never done (I can count the number of instances on one hand, most of them are on anti-spam/anti-virus mailing lists that either I or the virus team lead are on). There's only two recipients that don't get filtering - our filter FP handling contact address and our registration contact address. In 13 years of filtering, we've only had three people (peak user population: 120,000) ask to get out from behind filtering. We said no. It's not that big an issue if you do sender-signalling on rejects and/or provide methods for receivers to find out what was blocked and how to deal with it. As I understand it, govs (particularly LE) often don't have any whitelisting/filter bypass capabilities whatsoever and/or refuse to use them by policy. It's awesomely annoying sometimes when you can't get them a sample of what they asked for, even if via a mailing list intended for the purpose. end of aside] > Of course if it became known that the BBB accepted bribes for > delisting, that would hurt their reputation. I think attempting to > mandate the mechanics of reputation will be futile, maybe 2.2.5 could > be weakened to be strictly descriptive, yet provide advice for the > reputation-mechanically-impaired? This is the same argument as saying "we don't need a law against X, because it won't stop people from doing X". Well of course. So what? Without it, you don't know that X is bad, and you have no way to deal with people who did X anyway. A BCP does the former. Obviously, a BCP can at most wave a "bad boy!" wand (is that non-violent enough? ;-) over the latter. But that's important too. [+] Read your sexual harassment in the workplace legislation sometime. If it's anything like what we have, the admins can go to jail, there is no "corporate shield" on that bit. From johnl@iecc.com Thu Jan 20 09:10:26 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id E93DA3A713C for ; Thu, 20 Jan 2011 09:10:25 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -110.119 X-Spam-Level: X-Spam-Status: No, score=-110.119 tagged_above=-999 required=5 tests=[AWL=-0.806, BAYES_00=-2.599, HABEAS_ACCREDITED_SOI=-4.3, IMPOTENCE=1.886, RCVD_IN_BSP_TRUSTED=-4.3, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cCOQHIiuFjWI for ; Thu, 20 Jan 2011 09:10:22 -0800 (PST) Received: from gal.iecc.com (gal.iecc.com [64.57.183.53]) by core3.amsl.com (Postfix) with ESMTP id 29E5B3A7037 for ; Thu, 20 Jan 2011 09:10:22 -0800 (PST) Received: (qmail 19433 invoked from network); 20 Jan 2011 17:13:04 -0000 Received: from mail1.iecc.com (64.57.183.56) by mail1.iecc.com with QMQP; 20 Jan 2011 17:13:04 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=iecc.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=c896.4d386d20.k1101; i=johnl@user.iecc.com; bh=cPK61HvsCrgPQ6DfifhHourBNzDoax0bJURKA5kXBl0=; b=ca8RfcbC9OlfuViqKwqAgk/E9Icgd2fejUy5uCCsNZbmpXnoXp4gjoQlCi9XYysNW83S8H+ZrCU59JpGRuEixAB6Q4kK4e8PAC9lILlgFqqnH2GIcWQYnKTz88NfVLFWVq+Te3Fa+Brsa21Mzit1TgjbKWOuxbgPuBdQSTZfIVc= DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=taugh.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=c896.4d386d20.k1101; olt=johnl@user.iecc.com; bh=cPK61HvsCrgPQ6DfifhHourBNzDoax0bJURKA5kXBl0=; b=FM65xbDxE6ulbgRFvING4l05spUX8HDxeJaUfQVt6YexhnwrA/ZLaT6ysKfJt+DI1wNntH9HMmtFHk+4BOkw7OaIelkmDhbPoXYJYw902j1uA8lAHbSKhAXSUQcvcCEWNbEvzP+xek6yGq/eF52KTUgcf+hVECxL0pX702SHxd0= VBR-Info: md=iecc.com; mc=all; mv=dwl.spamhaus.org Date: 20 Jan 2011 17:13:04 -0000 Message-ID: <20110120171304.51349.qmail@joyce.lan> From: John Levine To: asrg@irtf.org In-Reply-To: <4D37DBC7.20803@nortel.com> Organization: Cc: X-Headerized: yes Mime-Version: 1.0 Content-type: text/plain; charset=utf-8 Content-transfer-encoding: 7bit Subject: Re: [Asrg] knobs and dials, was please review draft X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jan 2011 17:10:26 -0000 >There are some practitioners who wax quite extreme on substantive >per-recipient configurability (eg: some of those on the qpsmtpd list >;-), but the reality is that the number of people who'd actually >_use_ such knobs would be extremely small, and still fewer who could >adjust it effectively. Some years ago I was talking to abuse people at a large cable ISP. They said their old filters didn't work very well, and their users were constantly demanding configurable whitelists and special cases. Then they switched to a new filter that worked a lot better, and the requests just stopped. It shouldn't come as a big surprise that users' mental models are not the same as ours, and frequently don't match what's really going on. It generally doesn't occur to them that the problem is that their filters just don't work very well, so they assume that whatever they've got is the state of the art, and the only way to make them work better is to add special cases to fix the mistakes. But in fact, a well tuned filter makes very few mistakes. I suppose that if you are the only researcher at your institution working on erectile dysfunction, you might need special exceptions, but other than that, special cases tell us that the filters are broken. R's, John From davidnicol@gmail.com Fri Jan 21 09:31:32 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 411893A6A58 for ; Fri, 21 Jan 2011 09:31:32 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.321 X-Spam-Level: X-Spam-Status: No, score=-3.321 tagged_above=-999 required=5 tests=[AWL=0.278, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qWQNZoXA4FLf for ; Fri, 21 Jan 2011 09:31:31 -0800 (PST) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by core3.amsl.com (Postfix) with ESMTP id 670A83A69F5 for ; Fri, 21 Jan 2011 09:31:31 -0800 (PST) Received: by iyb26 with SMTP id 26so1932768iyb.13 for ; Fri, 21 Jan 2011 09:34:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:from:date:message-id:subject:to :content-type; bh=v6NJVOjgQ/RG5qxVJYMDhddRoQsXXYLQEjompf9ZG+4=; b=mIrVMGgqij5hIhwMwA5DTfiDYG06L3vgNSmc3hInueG5tYe3uSYIVrQFHjt1dW3dgm +7Su8SeBpmWz9uqXa+WNfRvid4mKuve4/v77tw47VkG3CeyVaifqxR692uxcjJmkGIVu bLd1O+Q/4qLamql4j3ZlQAsN3qrcK550pr2lM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:from:date:message-id:subject:to:content-type; b=IIlyr7YfJPP9jNhYIVtCmgR92OHSh7nr8p9Qa1MTIJAkVaMJxZxF8SAwn1azKTRE4O WuocT2WlPMZ7mH77ee16uRe+V+FqhpwPUWNKPNvzpeAx4SqLn5CKJM6XPBZ7LiSZsOvy zrF0dN7EzXGzil4HX3Blcg1s6kcuJKsxs6UOo= Received: by 10.231.190.8 with SMTP id dg8mr1106283ibb.15.1295631256401; Fri, 21 Jan 2011 09:34:16 -0800 (PST) MIME-Version: 1.0 Received: by 10.231.206.148 with HTTP; Fri, 21 Jan 2011 09:33:09 -0800 (PST) From: David Nicol Date: Fri, 21 Jan 2011 11:33:09 -0600 Message-ID: To: ASRG Content-Type: text/plain; charset=UTF-8 Subject: [Asrg] ipv6 macro expansion example in SPF specification, DNS ranges... X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Jan 2011 17:31:32 -0000 IPv6: %{ir}.%{v}._spf.%{d2} 1.0.B.C.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.8.B.D.0.1.0.0.2.ip6._spf.example.com cached NXDOMAIN suppressing subdomain lookup attempts seems like the way to go, if one refers to a DNSWL in a SPF record. From johnl@iecc.com Fri Jan 21 18:08:54 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id AB7863A6858 for ; Fri, 21 Jan 2011 18:08:54 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -111.054 X-Spam-Level: X-Spam-Status: No, score=-111.054 tagged_above=-999 required=5 tests=[AWL=0.145, BAYES_00=-2.599, HABEAS_ACCREDITED_SOI=-4.3, RCVD_IN_BSP_TRUSTED=-4.3, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vmw-tiE-+8gK for ; Fri, 21 Jan 2011 18:08:53 -0800 (PST) Received: from gal.iecc.com (gal.iecc.com [64.57.183.53]) by core3.amsl.com (Postfix) with ESMTP id DE1913A6836 for ; Fri, 21 Jan 2011 18:08:52 -0800 (PST) Received: (qmail 91008 invoked from network); 22 Jan 2011 02:11:39 -0000 Received: from mail1.iecc.com (64.57.183.56) by mail1.iecc.com with QMQP; 22 Jan 2011 02:11:39 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=iecc.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=c014.4d3a3cdb.k1101; i=johnl@user.iecc.com; bh=PdGLdwzSRAnFRLFsfmG8aNxPakVrP9hbyqQLa+SzQ7w=; b=SxWl8s0+vZFl3nzwG8Cbr3Tt3d9QXZilAayrPiyg93Kpabbt76EeqPfRY1kbF0lFBNhBt9RtgzUWmVWEdX0ldCFE5DtHM8A9sH0yepRjNFylgYThbD8Y4/xLvbov5KHNFEUU+mqDCtj67kIgfzjNmM4ZHaPn1TOK7R9FafGI5bg= DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=taugh.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=c014.4d3a3cdb.k1101; olt=johnl@user.iecc.com; bh=PdGLdwzSRAnFRLFsfmG8aNxPakVrP9hbyqQLa+SzQ7w=; b=e4OzzhMBW4Z8pabINAcdI3353hxsPm5MZfpBc3u2QBQkKBEbseFlj8ZGvhhzfliwYrYT36rokhFcyJX3/6Ly6cBjryjx1smTnxscFZP995gGJm5hk0pgr7av9A7WfzaXHni29hsRT26Cy3Jq4qO7D4Xtt5E6PDDxRkh3ef37b4M= VBR-Info: md=iecc.com; mc=all; mv=dwl.spamhaus.org Date: 22 Jan 2011 02:11:39 -0000 Message-ID: <20110122021139.49171.qmail@joyce.lan> From: John Levine To: asrg@irtf.org In-Reply-To: Organization: Cc: X-Headerized: yes Mime-Version: 1.0 Content-type: text/plain; charset=utf-8 Content-transfer-encoding: 7bit Subject: Re: [Asrg] ipv6 macro expansion example in SPF specification, DNS ranges... X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Jan 2011 02:08:54 -0000 >cached NXDOMAIN suppressing subdomain lookup attempts seems like the >way to go, if one refers to a DNSWL in a SPF record. Are you assuming that whatever does the SPF lookups remembers all of its previous queries? R's, John From davidnicol@gmail.com Fri Jan 21 19:41:12 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 8316B3A6AB2 for ; Fri, 21 Jan 2011 19:41:12 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.333 X-Spam-Level: X-Spam-Status: No, score=-3.333 tagged_above=-999 required=5 tests=[AWL=0.266, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4UtWO+C+sF25 for ; Fri, 21 Jan 2011 19:41:11 -0800 (PST) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by core3.amsl.com (Postfix) with ESMTP id B1D633A68AD for ; Fri, 21 Jan 2011 19:41:11 -0800 (PST) Received: by iyb26 with SMTP id 26so2382452iyb.13 for ; Fri, 21 Jan 2011 19:43:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:from:date :message-id:subject:to:content-type:content-transfer-encoding; bh=M2Ag0CknsJh8O7YItg/jgRZhNbB6tYJ1rUF3DZAffrQ=; b=tYDe3H8CJkX6QVy/Es6MsAP7nHH7zeNmosH3F0zb2WEJ/8Zo/tvgCzLdieVo5dAfr2 2MNnwFB+3v4vKXobISxQu181/OgHGrD+Iol7d6jgkVT083fifXukpWYBruTymRh1e/BK ci9iGs8b2h2u6epQWhNHHW6SxQcm//IfW2yvQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type:content-transfer-encoding; b=TAJG3gRss7KYyPlIt3H3jTmd9slbjnQSHYQe/E5y1uCKcya1HlDsrnN71R6ayUbzdV PV1NibPA2gSeaZ8AVjVFDXyE+sBMAV74VoiyrM63a62L9AW3qr2RVIdCfhvmryU8WU9p uSetVbME2bCY4jN6lSLwPYm85Pa7prwqA47Dw= Received: by 10.231.14.134 with SMTP id g6mr1750696iba.65.1295667837202; Fri, 21 Jan 2011 19:43:57 -0800 (PST) MIME-Version: 1.0 Received: by 10.231.206.148 with HTTP; Fri, 21 Jan 2011 19:43:36 -0800 (PST) In-Reply-To: <20110122021139.49171.qmail@joyce.lan> References: <20110122021139.49171.qmail@joyce.lan> From: David Nicol Date: Fri, 21 Jan 2011 21:43:36 -0600 Message-ID: To: Anti-Spam Research Group - IRTF Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [Asrg] ipv6 macro expansion example in SPF specification, DNS ranges... X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Jan 2011 03:41:12 -0000 that the cache would. The special strict NXDOMAIN cache for use by DNSxL clients. On Fri, Jan 21, 2011 at 8:11 PM, John Levine wrote: >>cached NXDOMAIN suppressing subdomain lookup attempts seems like the >>way to go, if one refers to a DNSWL in a SPF record. > > Are you assuming that whatever does the SPF lookups remembers all of > its previous queries? > > R's, > John > _______________________________________________ > Asrg mailing list > Asrg@irtf.org > http://www.irtf.org/mailman/listinfo/asrg > --=20 =E2=80=9CThe aeroplane is fatally defective. It is merely a toy=E2=80=94a s= porting play-thing. =C2=A0It can never become commercially practical." -- Nikola Tesla From johnl@iecc.com Fri Jan 21 20:26:32 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id B96F53A6AC3 for ; Fri, 21 Jan 2011 20:26:32 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -111.056 X-Spam-Level: X-Spam-Status: No, score=-111.056 tagged_above=-999 required=5 tests=[AWL=0.143, BAYES_00=-2.599, HABEAS_ACCREDITED_SOI=-4.3, RCVD_IN_BSP_TRUSTED=-4.3, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dFNZ5qFPq2N5 for ; Fri, 21 Jan 2011 20:26:31 -0800 (PST) Received: from gal.iecc.com (gal.iecc.com [64.57.183.53]) by core3.amsl.com (Postfix) with ESMTP id 4D6F63A6AC1 for ; Fri, 21 Jan 2011 20:26:31 -0800 (PST) Received: (qmail 13502 invoked from network); 22 Jan 2011 04:29:17 -0000 Received: from mail1.iecc.com (64.57.183.56) by mail1.iecc.com with QMQP; 22 Jan 2011 04:29:17 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=iecc.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=14735.4d3a5d1d.k1101; i=johnl@user.iecc.com; bh=IY2ogy4zxP1JOHx5WvWmD0Y314mmJ8HodF+4y/JWlts=; b=mPO8lBoOXa/bRQkJaLkuEEiIVsN6iHgyAxk7OYvDUZ7QfMZIFgQGm5sqG/VJ7DAnT5SvOJUpN40zEUV2gxpnFkWDWad1O3goWhzeYg3QWVddoHS1HtiOKoXFJuT/1wm7uvZo6GEBQrGb0bzGr92+Si8fuPjyckzppDrEPkUwJAk= DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=taugh.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=14735.4d3a5d1d.k1101; olt=johnl@user.iecc.com; bh=IY2ogy4zxP1JOHx5WvWmD0Y314mmJ8HodF+4y/JWlts=; b=Rw26usFPF3957ko+DQkIESlt4y7xt5B60sz2ukcjJ2GjousVz58ecNmEN7qCYrIxWOPFgFJC6wktGCrJLJ62nh7pOWRSQwC86RGW7cKye2LgakCLsLOEX+RoviUMUzijarCJYioCye/pLEOL76yFAP1X1hNiP+xbikpLBr6alFE= VBR-Info: md=iecc.com; mc=all; mv=dwl.spamhaus.org Date: 22 Jan 2011 04:29:17 -0000 Message-ID: <20110122042917.83764.qmail@joyce.lan> From: John Levine To: asrg@irtf.org In-Reply-To: Organization: Cc: X-Headerized: yes Mime-Version: 1.0 Content-type: text/plain; charset=utf-8 Content-transfer-encoding: 7bit Subject: Re: [Asrg] ipv6 macro expansion example in SPF specification, DNS ranges... X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Jan 2011 04:26:32 -0000 >>>cached NXDOMAIN suppressing subdomain lookup attempts seems like the >>>way to go, if one refers to a DNSWL in a SPF record. >> Are you assuming that whatever does the SPF lookups remembers all of >> its previous queries? >that the cache would. The special strict NXDOMAIN cache for use by >DNSxL clients. So you're expecting something other than a normal DNS cache that exists now. If you're doing a special purpose cache, why not do a smarter protocol between the cache and the master server? As far as the client's concerned, once you need anything other than a vanilla DNS setup, it's all new special purpose code anyway. R's, John From davidnicol@gmail.com Sat Jan 22 17:48:32 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 89CF43A6BC5 for ; Sat, 22 Jan 2011 17:48:32 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.343 X-Spam-Level: X-Spam-Status: No, score=-3.343 tagged_above=-999 required=5 tests=[AWL=0.256, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HVKDBLFVo1r7 for ; Sat, 22 Jan 2011 17:48:31 -0800 (PST) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by core3.amsl.com (Postfix) with ESMTP id B496F3A6BC3 for ; Sat, 22 Jan 2011 17:48:30 -0800 (PST) Received: by iyb26 with SMTP id 26so2987178iyb.13 for ; Sat, 22 Jan 2011 17:51:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:from:date :message-id:subject:to:content-type:content-transfer-encoding; bh=hdmIrvIVbOBGsP09SrWoXFgKqk5eCkknvcBC8HlET38=; b=lurq7ceLXLBgdF3O4C1eFOS9tiulwZqGTx4M/HIyC99deW7AsXDzl8cR6ts2qNF8FC ZXUjLkEWsvf1rHP9N8QmY97JT22YUoNDB1lnV0UsuoQc8mBeIxLo3GmwIpYSqL9jRssW lfpSMizoYq8PCt7/74ggExPogOFNm8t/DUSjs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type:content-transfer-encoding; b=iYA0E0oZ3wXRtKbn9CDHVoupVvR36JgyFPhuI+y/f9DllmopiAt9ufrxYVVI4f4e6M s9aZqZP8Sic0ioRNW1oIJFUZdaSqeqPFIukTdOGmDisHMAMfBsqS9k1kZTeFki9dGv20 1gznfSjzxLWbck9Dc4mj4MztZt3gZgmBK5X7s= Received: by 10.231.208.83 with SMTP id gb19mr2852152ibb.40.1295747480186; Sat, 22 Jan 2011 17:51:20 -0800 (PST) MIME-Version: 1.0 Received: by 10.231.206.148 with HTTP; Sat, 22 Jan 2011 17:51:00 -0800 (PST) In-Reply-To: <20110122042917.83764.qmail@joyce.lan> References: <20110122042917.83764.qmail@joyce.lan> From: David Nicol Date: Sat, 22 Jan 2011 19:51:00 -0600 Message-ID: To: Anti-Spam Research Group - IRTF Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [Asrg] ipv6 macro expansion example in SPF specification, DNS ranges... X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Jan 2011 01:48:32 -0000 On Fri, Jan 21, 2011 at 10:29 PM, John Levine wrote: >>that the cache would. The special strict NXDOMAIN cache for use by >>DNSxL clients. > > So you're expecting something other than a normal DNS cache that > exists now. Small patch to existing cache. > If you're doing a special purpose cache, why not do a smarter protocol > between the cache and the master server? =C2=A0As far as the client's > concerned, once you need anything other than a vanilla DNS setup, it's > all new special purpose code anyway. How would one represent "e-mail may appear from sources appearing in this whitelist" in an SPF record, with your b-tree proposal? Popularize the b-tree protocol, then have a way to refer to it included in SPFv3? From johnl@iecc.com Sat Jan 22 18:28:11 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 077423A6BC5 for ; Sat, 22 Jan 2011 18:28:11 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -111.062 X-Spam-Level: X-Spam-Status: No, score=-111.062 tagged_above=-999 required=5 tests=[AWL=0.137, BAYES_00=-2.599, HABEAS_ACCREDITED_SOI=-4.3, RCVD_IN_BSP_TRUSTED=-4.3, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QuF4Xy22UpFh for ; Sat, 22 Jan 2011 18:28:09 -0800 (PST) Received: from gal.iecc.com (gal.iecc.com [64.57.183.53]) by core3.amsl.com (Postfix) with ESMTP id 04BC93A6878 for ; Sat, 22 Jan 2011 18:28:08 -0800 (PST) Received: (qmail 2958 invoked from network); 23 Jan 2011 02:30:57 -0000 Received: from mail1.iecc.com (64.57.183.56) by mail1.iecc.com with QMQP; 23 Jan 2011 02:30:57 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=iecc.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=4315.4d3b92e1.k1101; i=johnl@user.iecc.com; bh=Z1madoD2FzRLVyWYIVvtBPqG3Onmaq9ulOgAOfj6Xbk=; b=hQ6p7HlRZYt10yVnTXGlSo26RrUmXgZVhDUmQvrLW5NleHVkXp6kQcUr555hb5NU9QHS7DUGfH5G3Dq6s9+LdSZQJi9o27NH77MDQuJ05e/cnmMQ7VKZUKBLSS6WQvtw8j8bA2vG9zkXCAfIFcSrRZ8yoN8s6gKKxRRhSo81e3w= DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=taugh.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=4315.4d3b92e1.k1101; olt=johnl@user.iecc.com; bh=Z1madoD2FzRLVyWYIVvtBPqG3Onmaq9ulOgAOfj6Xbk=; b=mvrH2PIL5QYAEfzZzU7gcGK42VSDnoNCBUahlEHBUzPdY7T5H2uU/3a88XLmbAYL+UTfAc8Z8f7iWFJjES9urKerIdRVAgm/WljsFeYN2zXYy72nN0OrZt/uJzIIKDbps+Qe7zKP6X0LEqKgRRCNlC1e0mNEM0fyZDhHJIlNBsA= VBR-Info: md=iecc.com; mc=all; mv=dwl.spamhaus.org Date: 23 Jan 2011 02:30:57 -0000 Message-ID: <20110123023057.17172.qmail@joyce.lan> From: John Levine To: asrg@irtf.org In-Reply-To: Organization: Cc: X-Headerized: yes Mime-Version: 1.0 Content-type: text/plain; charset=utf-8 Content-transfer-encoding: 7bit Subject: Re: [Asrg] ipv6 macro expansion example in SPF specification, DNS ranges... X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Jan 2011 02:28:11 -0000 >>>that the cache would. The special strict NXDOMAIN cache for use by >>>DNSxL clients. >> So you're expecting something other than a normal DNS cache that >> exists now. >Small patch to existing cache. Hmmn. As a proof of concept, how about making the appropriate changes to BIND 9.7 and send us the patch file. Or for that matter do it to dnscache in djbdns, so you only have to look at 12K lines of code rather than 400K lines. >> If you're doing a special purpose cache, why not do a smarter protocol >> between the cache and the master server? As far as the client's >> concerned, once you need anything other than a vanilla DNS setup, it's >> all new special purpose code anyway. > >How would one represent "e-mail may appear from sources appearing in >this whitelist" in an SPF record, with your b-tree proposal? My suggestion is that if you find SPF to be useful and you want a macro that means look up an address in a DNSxL, you should add a macro that means look up this address in that DNSxL, rather than trying to do it indirectly with %{ir} or whatever. Since the installed base of IPv6 whitelists and blacklists is nonexistent, the backward compatibility problem is limited. R's, John From dotis@mail-abuse.org Mon Jan 24 11:49:43 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id D46AD3A694D for ; Mon, 24 Jan 2011 11:49:43 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.521 X-Spam-Level: X-Spam-Status: No, score=-106.521 tagged_above=-999 required=5 tests=[AWL=0.078, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vjsavWUfNG+c for ; Mon, 24 Jan 2011 11:49:42 -0800 (PST) Received: from harry.mail-abuse.org (harry.mail-abuse.org [168.61.5.27]) by core3.amsl.com (Postfix) with ESMTP id 826C93A6904 for ; Mon, 24 Jan 2011 11:49:42 -0800 (PST) Received: from US-DOUGLASO-MB.local (gateway1.sjc.mail-abuse.org [168.61.5.81]) by harry.mail-abuse.org (Postfix) with ESMTP id 8CC55A94448 for ; Mon, 24 Jan 2011 19:53:30 +0000 (UTC) Message-ID: <4D3DD87C.5040208@mail-abuse.org> Date: Mon, 24 Jan 2011 11:52:28 -0800 From: Douglas Otis User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: asrg@irtf.org References: <20110122042917.83764.qmail@joyce.lan> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Asrg] ipv6 macro expansion example in SPF specification, DNS ranges... X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jan 2011 19:49:43 -0000 On 1/22/11 5:51 PM, David Nicol wrote: > On Fri, Jan 21, 2011 at 10:29 PM, John Levine wrote: > How would one represent "e-mail may appear from sources appearing in > this whitelist" in an SPF record, with your b-tree proposal? > Popularize the b-tree protocol, then have a way to refer to it > included in SPFv3? There was general consensus at MAAWG, SPF does not serve as a basis for mitigating spam. Use of SPF was limited to determining which reputation services currently list an SPF authorized IP address when handling complaints, or whether the authorized IP address can be included in feedback. This expects SPF records will resolve specific IP addresses. This is not necessarily true, nor is it always safe to assume a domain controls the authorized IP addresses. When SMTP Authentication becomes possible with DANE resource records, this will remove a need to collect sets of DNS resource records representing both authorized v4 and v6 IP addresses. Many records sets that only list v4 already have reached their "mechanism" limits. DANE based authentication would provide a safer basis for directing feedback as well. Unfortunately, it is not practical to list all possible v6 addresses or domain names that might instigate abuse. Defensive strategies must develop a generous list of "conforming" domains instead. Since billions of new domains are used every day by malefactors, neither the possession of a domain, nor a v6 IP address, offers a safe basis for acceptance when screened with negative reputations unable to keep pace. Efforts that attempt to relate domains to IP addresses seen by targets to confirm legitimate use of a domain are also likely to prove problematic, which leaves cryptographic authentication. In addition, SPF may support a denial of service attack with its processing of encoded macros based upon email address local-parts. Cryptographic authentication will not threaten DNS, and when based upon "conforming" domains, is unlikely to serve as a denial of service mechanism. Of course, DNS already handles domain names effectively without a need to redesign the query mechanism to obtain which domains "conform" to rules of reasonable behavior. It is also likely there will be a need to authorize other domains to better ensure acceptance of third-party transactions. -Doug From dotzero@gmail.com Mon Jan 24 12:51:27 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 712A13A6B19 for ; Mon, 24 Jan 2011 12:51:27 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.599 X-Spam-Level: X-Spam-Status: No, score=-3.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OPdFr6kLQdfq for ; Mon, 24 Jan 2011 12:51:26 -0800 (PST) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by core3.amsl.com (Postfix) with ESMTP id 3B1533A693C for ; Mon, 24 Jan 2011 12:51:26 -0800 (PST) Received: by vws9 with SMTP id 9so1993185vws.13 for ; Mon, 24 Jan 2011 12:54:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type:content-transfer-encoding; bh=+QBoQTIyDMFBecRxHOC89/mmXCP1mnhhcDts8WyWIDY=; b=mlS5Za1W0W1yN5m/cdf3gPRp1T6Rno/TTVMObErlT25wmTwSfWEJCn3X52+7zYgxfq B9YfbPInmY33DD8va1lF9MCk3OdQMuDO0/nWbdkkosJINT0YfiXIzZuLp+fPAXubPnOl s5Aif4uC+allxCK2ZbiIxkBRlISbUmGpAtCuw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=cjpTrnOKweVGvbM7Q9Grukp/B7M50u5z2o1cvYFeBdnBiPO9uitXqGNZosOVdxJrcb tYE+gDpBwiO/XV6XHEz6MbeUs3lMhzHsxQagyBCJgAoKDtxWfHAJ4zQsCE0xkrk++JsZ +0W2OuU52xnLfpbEpoamQiwo96FUfuAoKzKCs= MIME-Version: 1.0 Received: by 10.229.185.1 with SMTP id cm1mr4315332qcb.81.1295902393486; Mon, 24 Jan 2011 12:53:13 -0800 (PST) Received: by 10.229.188.200 with HTTP; Mon, 24 Jan 2011 12:53:13 -0800 (PST) In-Reply-To: <4D3DD87C.5040208@mail-abuse.org> References: <20110122042917.83764.qmail@joyce.lan> <4D3DD87C.5040208@mail-abuse.org> Date: Mon, 24 Jan 2011 15:53:13 -0500 Message-ID: From: Dotzero To: Anti-Spam Research Group - IRTF Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: [Asrg] ipv6 macro expansion example in SPF specification, DNS ranges... X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jan 2011 20:51:27 -0000 On Mon, Jan 24, 2011 at 2:52 PM, Douglas Otis wrote: > On 1/22/11 5:51 PM, David Nicol wrote: >> >> On Fri, Jan 21, 2011 at 10:29 PM, John Levine =A0wrote: >> How would one represent "e-mail may appear from sources appearing in >> this whitelist" in an SPF record, with your b-tree proposal? >> Popularize the b-tree protocol, then have a way to refer to it >> included in SPFv3? > > There was general consensus at MAAWG, SPF does not serve as a basis for > mitigating spam. =A0Use of SPF was limited to determining which reputatio= n > services currently list an SPF authorized IP address when handling > complaints, or whether the authorized IP address can be included in > feedback. =A0This expects SPF records will resolve specific IP addresses. > =A0This is not necessarily true, nor is it always safe to assume a domain > controls the authorized IP addresses. > Doug, I'm not sure that you are correct when you state that there was a consensus for what you describe above. There WAS a consensus that many mailbox providers are not using SPF exactly the way that many originally expected. For example, many do not automatically use a hardfail as a basis to reject mail or discard. The breakage on forwarded/relayed/vanity/lists is in many cases too painful for both sending domains and mailbox providers (think complaint handling/customer service costs). Instead, many mailbox providers are considering SPF results in conjunction with DKIM results in a combination whitelist/blacklist approach. For example, if I get an SPF pass on a record that ends in -all and/or I get a DKIM pass on a first party signature then I have a high confidence level as to where the mail came from at either the transport layer or the message layer. If I have both then I have an even higher confidence level. This does not speak to the "goodness" or badness" of the domain, only relationship to the domain. On the other hand, if a domain is publishing a record ending in a -all AND consistently DKIM signing (even in the absense of an ADSP record although an ALL would be useful), we can have a high confidence level of minimal false positives (that is, legitimate mail that is rejected or discarded) if we throw away mail that fails both SPF and DKIM checks. This does not necessarily presume that a mailbox provider is looking at external reputation services. From dotis@mail-abuse.org Mon Jan 24 14:35:07 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id E57D93A69B0 for ; Mon, 24 Jan 2011 14:35:07 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.524 X-Spam-Level: X-Spam-Status: No, score=-106.524 tagged_above=-999 required=5 tests=[AWL=0.075, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Imj7Hg+K9zfI for ; Mon, 24 Jan 2011 14:35:05 -0800 (PST) Received: from harry.mail-abuse.org (harry.mail-abuse.org [168.61.5.27]) by core3.amsl.com (Postfix) with ESMTP id CD88B3A69A6 for ; Mon, 24 Jan 2011 14:35:04 -0800 (PST) Received: from US-DOUGLASO-MB.local (gateway1.sjc.mail-abuse.org [168.61.5.81]) by harry.mail-abuse.org (Postfix) with ESMTP id 2AF35A9446E for ; Mon, 24 Jan 2011 22:38:57 +0000 (UTC) Message-ID: <4D3DFF43.2020502@mail-abuse.org> Date: Mon, 24 Jan 2011 14:37:55 -0800 From: Douglas Otis User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: asrg@irtf.org References: <20110122042917.83764.qmail@joyce.lan> <4D3DD87C.5040208@mail-abuse.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Asrg] ipv6 macro expansion example in SPF specification, DNS ranges... X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jan 2011 22:35:08 -0000 On 1/24/11 12:53 PM, Dotzero wrote: > On Mon, Jan 24, 2011 at 2:52 PM, Douglas Otis wrote: >> There was general consensus at MAAWG, SPF does not serve as a basis for >> mitigating spam. Use of SPF was limited to determining which reputation >> services currently list an SPF authorized IP address when handling >> complaints, or whether the authorized IP address can be included in >> feedback. This expects SPF records will resolve specific IP addresses. >> This is not necessarily true, nor is it always safe to assume a domain >> controls the authorized IP addresses. > Doug, I'm not sure that you are correct when you state that there was > a consensus for what you describe above. There WAS a consensus that > many mailbox providers are not using SPF exactly the way that many > originally expected. > > For example, many do not automatically use a hardfail as a basis to > reject mail or discard. The breakage on forwarded/relayed/vanity/lists > is in many cases too painful for both sending domains and mailbox > providers (think complaint handling/customer service costs). > > Instead, many mailbox providers are considering SPF results in > conjunction with DKIM results in a combination whitelist/blacklist > approach. For example, if I get an SPF pass on a record that ends in > -all and/or I get a DKIM pass on a first party signature then I have a > high confidence level as to where the mail came from at either the > transport layer or the message layer. If I have both then I have an > even higher confidence level. This does not speak to the "goodness" or > badness" of the domain, only relationship to the domain. > > On the other hand, if a domain is publishing a record ending in a -all > AND consistently DKIM signing (even in the absense of an ADSP record > although an ALL would be useful), we can have a high confidence level > of minimal false positives (that is, legitimate mail that is rejected > or discarded) if we throw away mail that fails both SPF and DKIM > checks. > > This does not necessarily presume that a mailbox provider is looking > at external reputation services. Mike, An SPF failure can not be trusted to be an indicator of spam. DKIM signing is almost never assured, especially when handled by third-party services. As such, these mechanisms failing alone or together still do not offer a safe basis for rejection. Of course both passing means nothing as well. -Doug From dotzero@gmail.com Mon Jan 24 18:12:04 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id B2C973A6867 for ; Mon, 24 Jan 2011 18:12:04 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.599 X-Spam-Level: X-Spam-Status: No, score=-3.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9jMCzWW5qgMW for ; Mon, 24 Jan 2011 18:12:03 -0800 (PST) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by core3.amsl.com (Postfix) with ESMTP id B063B3A6847 for ; Mon, 24 Jan 2011 18:12:03 -0800 (PST) Received: by qwj9 with SMTP id 9so4606712qwj.13 for ; Mon, 24 Jan 2011 18:14:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type:content-transfer-encoding; bh=I+eN3Lpxju6zDWRw3VNctSg2MkXHUaZ3lCoIO2UfBAc=; b=kFO0pL9dC9KDfr6E5wlcd7ApMKGLUbkEgv/aKs79gf9NZQTusYnBcedxDjrgzE7oKm yWuC73s8tkHK4H13UoJ0EFxbXqYPtS8cKB6H3rRIYJsL5WPsrOcilvEbUsn9FiuR9wbH 8QzCJqOa/DLs3ApiY9/s2gvmDUXnTeRX8Ofh4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=IZnbHxTQX8y1iLyECfYKm0PKioao5D9xTj8n/7bwR3DMpfWkLgVRgG5DhKnJceOTiu MotxnBLJYQicRHUWhgUB9cYfxY2n+wNG21Y5h5VikVEf3SaDOmSw/LUz3oB7zAkKOCm8 cQbIiikIiRDf11DjfWIEs7jBnE7Wtk88PSW+A= MIME-Version: 1.0 Received: by 10.229.89.208 with SMTP id f16mr4368870qcm.43.1295921699456; Mon, 24 Jan 2011 18:14:59 -0800 (PST) Received: by 10.229.188.200 with HTTP; Mon, 24 Jan 2011 18:14:59 -0800 (PST) In-Reply-To: <4D3DFF43.2020502@mail-abuse.org> References: <20110122042917.83764.qmail@joyce.lan> <4D3DD87C.5040208@mail-abuse.org> <4D3DFF43.2020502@mail-abuse.org> Date: Mon, 24 Jan 2011 21:14:59 -0500 Message-ID: From: Dotzero To: Anti-Spam Research Group - IRTF Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: [Asrg] ipv6 macro expansion example in SPF specification, DNS ranges... X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jan 2011 02:12:04 -0000 On Mon, Jan 24, 2011 at 5:37 PM, Douglas Otis wrote: > On 1/24/11 12:53 PM, Dotzero wrote: >> >> On Mon, Jan 24, 2011 at 2:52 PM, Douglas Otis >> =A0wrote: >>> >>> There was general consensus at MAAWG, SPF does not serve as a basis for >>> mitigating spam. =A0Use of SPF was limited to determining which reputat= ion >>> services currently list an SPF authorized IP address when handling >>> complaints, or whether the authorized IP address can be included in >>> feedback. =A0This expects SPF records will resolve specific IP addresse= s. >>> =A0This is not necessarily true, nor is it always safe to assume a doma= in >>> controls the authorized IP addresses. >> >> Doug, I'm not sure that you are correct when you state that there was >> a consensus for what you describe above. There WAS a consensus that >> many mailbox providers are not using SPF exactly the way that many >> originally expected. >> >> For example, many do not automatically use a hardfail as a basis to >> reject mail or discard. The breakage on forwarded/relayed/vanity/lists >> is in many cases too painful for both sending domains and mailbox >> providers (think complaint handling/customer service costs). >> >> Instead, many mailbox providers are considering SPF results in >> conjunction with DKIM results in a combination whitelist/blacklist >> approach. For example, if I get an SPF pass on a record that ends in >> -all and/or I get a DKIM pass on a first party signature then I have a >> high confidence level as to where the mail came from at either the >> transport layer or the message layer. If I have both then I have an >> even higher confidence level. This does not speak to the "goodness" or >> badness" of the domain, only relationship to the domain. >> >> On the other hand, if a domain is publishing a record ending in a -all >> AND consistently DKIM signing (even in the absense of an ADSP record >> although an ALL would be useful), we can have a high confidence level >> of minimal false positives (that is, legitimate mail that is rejected >> or discarded) if we throw away mail that fails both SPF and DKIM >> checks. >> >> This does not necessarily presume that a mailbox provider is looking >> at external reputation services. > > Mike, > > An SPF failure can not be trusted to be an indicator of spam. =A0DKIM sig= ning > is almost never assured, especially when handled by third-party services. > =A0As such, these mechanisms failing alone or together still do not offer= a > safe basis for rejection. =A0Of course both passing means nothing as well= . > Doug, there are plenty of people with real world operational experience that would disagree with you. You state that failing means nothing and passing means nothing. If that is true, why are there a significant number of implementers using this approach successfully? From fergdawgster@gmail.com Mon Jan 24 18:18:16 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id E0A053A6867 for ; Mon, 24 Jan 2011 18:18:16 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.05 X-Spam-Level: X-Spam-Status: No, score=-2.05 tagged_above=-999 required=5 tests=[AWL=1.549, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZuDV-vNa1NM1 for ; Mon, 24 Jan 2011 18:18:16 -0800 (PST) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by core3.amsl.com (Postfix) with ESMTP id 624753A6840 for ; Mon, 24 Jan 2011 18:18:15 -0800 (PST) Received: by iyb26 with SMTP id 26so4791651iyb.13 for ; Mon, 24 Jan 2011 18:21:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type:content-transfer-encoding; bh=ys1h3EQ/Yq2TAWcijbxDhS9NFld7K8pDklZT0aoGV2s=; b=xS2QHe7GssFRJ1ANQgPh4XXoOKuW+qbRQvOEQZVcgG1TJY3y9/pLu7njFqwO8CUJ1X ed7XuylbO5D8Eou5gtfzx4W+xvDHa5mIgz5z0povIA3a01lXu436X64bcAS78ieP3WmR hvWhji7s1rJ0vkKSVFiGA9SDcenQNACwU73vw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=n8V9Wh50Cx4XzH4xWTjz4AgFil6pHMHjMF5jeTBCUa/qRit1hKowvzNk9tw0moDHpz OVTvX07JgmiTvNHbPfT3dMABKBx9m5eMj5yhh+mCakUDCbYiK03u6409m6h+WVWmgtPF EyNqcBS4lhnIIy7cP/inkwYvASRbTZ46rh7+Y= MIME-Version: 1.0 Received: by 10.42.167.198 with SMTP id t6mr5837801icy.432.1295922071267; Mon, 24 Jan 2011 18:21:11 -0800 (PST) Received: by 10.42.221.73 with HTTP; Mon, 24 Jan 2011 18:21:11 -0800 (PST) In-Reply-To: References: <20110122042917.83764.qmail@joyce.lan> <4D3DD87C.5040208@mail-abuse.org> <4D3DFF43.2020502@mail-abuse.org> Date: Mon, 24 Jan 2011 18:21:11 -0800 Message-ID: From: Paul Ferguson To: Anti-Spam Research Group - IRTF Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: [Asrg] ipv6 macro expansion example in SPF specification, DNS ranges... X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jan 2011 02:18:17 -0000 On Mon, Jan 24, 2011 at 6:14 PM, Dotzero wrote: > On Mon, Jan 24, 2011 at 5:37 PM, Douglas Otis wrot= e: >> An SPF failure can not be trusted to be an indicator of spam. =A0DKIM si= gning >> is almost never assured, especially when handled by third-party services= . >> =A0As such, these mechanisms failing alone or together still do not offe= r a >> safe basis for rejection. =A0Of course both passing means nothing as wel= l. >> > > Doug, there are plenty of people with real world operational > experience that would disagree with you. =A0You state that failing means > nothing and passing means nothing. If that is true, why are there a > significant number of implementers using this approach successfully? Hi Mike, While I'm just catching up on this thread, I can also chime in that -- as far as I can tell, neither spf or dkim has had any impact on spam, at least from what I can determine. If that was the original intent of spf and/or dkim, then I would be inclined to say that neither have properly addressed the spam problem. $.02, - ferg --=20 "Fergie", a.k.a. Paul Ferguson =A0Engineering Architecture for the Internet =A0fergdawgster(at)gmail.com =A0ferg's tech blog: http://fergdawg.blogspot.com/ From johnl@iecc.com Mon Jan 24 19:04:14 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 14C4B3A6A16 for ; Mon, 24 Jan 2011 19:04:14 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -111.07 X-Spam-Level: X-Spam-Status: No, score=-111.07 tagged_above=-999 required=5 tests=[AWL=0.129, BAYES_00=-2.599, HABEAS_ACCREDITED_SOI=-4.3, RCVD_IN_BSP_TRUSTED=-4.3, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zuzvoCQSrveP for ; Mon, 24 Jan 2011 19:04:13 -0800 (PST) Received: from gal.iecc.com (gal.iecc.com [64.57.183.53]) by core3.amsl.com (Postfix) with ESMTP id 3EADD3A699F for ; Mon, 24 Jan 2011 19:04:12 -0800 (PST) Received: (qmail 75411 invoked from network); 25 Jan 2011 03:07:08 -0000 Received: from mail1.iecc.com (64.57.183.56) by mail1.iecc.com with QMQP; 25 Jan 2011 03:07:08 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=iecc.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=5328.4d3e3e5b.k1101; i=johnl@user.iecc.com; bh=gb0DNsC/l2RNcspH/YLMlY9Yyp+7nqrkv8XwHl5Zs90=; b=Ul6ykUIH34v9ripls1SVpC7lSnKDcLsdrhXsTqFcPqPGFfD3c8FB7LiN9YHzCmCcgB9ZaE1/5guaI5v3DoZipbilJxGdSb943JPHSFiorDTgckkbwKdquvr7fzrKe6ReuLCWE5WB1ccbxBlnKeRAWUDrjhk3KabB7W6EBFFhPr8= DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=taugh.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=5328.4d3e3e5b.k1101; olt=johnl@user.iecc.com; bh=gb0DNsC/l2RNcspH/YLMlY9Yyp+7nqrkv8XwHl5Zs90=; b=Tte4BBDAxnGpQ3VHbWVADa4LAK0VSn3y3IkRvDt9Wj/BcvRDPlOB9EJ2KzrudwKZL6U2NiEqhkDC7gDPExals/x+ihJjXDzFrX5f5prebFXe/CXVG6GFWio4VuB5Vo7Ov9x0YDpclJN9FvNwcHjYqtv25B6kqZ/6m6H8ei67JUU= VBR-Info: md=iecc.com; mc=all; mv=dwl.spamhaus.org Date: 25 Jan 2011 03:07:07 -0000 Message-ID: <20110125030707.21285.qmail@joyce.lan> From: John Levine To: asrg@irtf.org In-Reply-To: Organization: Cc: X-Headerized: yes Mime-Version: 1.0 Content-type: text/plain; charset=utf-8 Content-transfer-encoding: 7bit Subject: Re: [Asrg] into the woods X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jan 2011 03:04:14 -0000 >While I'm just catching up on this thread, I can also chime in that -- >as far as I can tell, neither spf or dkim has had any impact on spam, >at least from what I can determine. > >If that was the original intent of spf and/or dkim, then I would be >inclined to say that neither have properly addressed the spam problem. I can't speak for the SPF crowd, but the goal of DKIM is to make it easier to recognize mail from people you know, for some definition of "people" and "know" which should make filtering more accurate. It's also alleged to be useful to deter forgery, although I happen to think that any forgery it deters is already covered under the category of mail from people you don't know. The only thing that's going to make the amount of spam drop is major legal and technical advances that get rid of botnets. Don't hold your breath. R's, John From dotzero@gmail.com Mon Jan 24 19:07:08 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 1A6063A6A16 for ; Mon, 24 Jan 2011 19:07:08 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.599 X-Spam-Level: X-Spam-Status: No, score=-3.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id y7InEH1PJtNE for ; Mon, 24 Jan 2011 19:07:07 -0800 (PST) Received: from mail-qy0-f175.google.com (mail-qy0-f175.google.com [209.85.216.175]) by core3.amsl.com (Postfix) with ESMTP id C23373A699F for ; Mon, 24 Jan 2011 19:07:06 -0800 (PST) Received: by qyk8 with SMTP id 8so3393232qyk.13 for ; Mon, 24 Jan 2011 19:10:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type:content-transfer-encoding; bh=hnCwIKdGCP4wIJxIlFbQT2WJuU0DfkCt/CJoxSroqlM=; b=n7gK05+0fbPKs5hgf03kjCKTOBsFSE7gjjsIjb1b7uHChWHEt6tV1IOyu8pZRfJFi9 BVlNEo4IcjSoXYBaMymn5lpRcMZAsQUUKWK71SLVmiMrORxq+b1n0beZ1oaf7M+urtkU 1y3Pje24kECmZ7YJ5huJuZYhNMcjk/AlxHlVc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=h10gbVcbno/yZi/6Oo5NEbg3GXUByPel4N6/8SYG/D9kYmLfdrdcd7MwiswIJAwfA/ ggvuTYIU7ajzsrQq8HOE9h3ss+X835XJVYmF50jBDWHSxsQsdW84DngfEUGHqSwASWA3 IU6MWbJ75r+c1AFJZrVeerkKOLtUYF9qm8bkk= MIME-Version: 1.0 Received: by 10.229.240.66 with SMTP id kz2mr4344016qcb.233.1295925002753; Mon, 24 Jan 2011 19:10:02 -0800 (PST) Received: by 10.229.188.200 with HTTP; Mon, 24 Jan 2011 19:10:02 -0800 (PST) In-Reply-To: References: <20110122042917.83764.qmail@joyce.lan> <4D3DD87C.5040208@mail-abuse.org> <4D3DFF43.2020502@mail-abuse.org> Date: Mon, 24 Jan 2011 22:10:02 -0500 Message-ID: From: Dotzero To: Anti-Spam Research Group - IRTF Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: [Asrg] ipv6 macro expansion example in SPF specification, DNS ranges... X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jan 2011 03:07:08 -0000 On Mon, Jan 24, 2011 at 9:21 PM, Paul Ferguson wro= te: > On Mon, Jan 24, 2011 at 6:14 PM, Dotzero wrote: > >> On Mon, Jan 24, 2011 at 5:37 PM, Douglas Otis wro= te: > >>> An SPF failure can not be trusted to be an indicator of spam. =A0DKIM s= igning >>> is almost never assured, especially when handled by third-party service= s. >>> =A0As such, these mechanisms failing alone or together still do not off= er a >>> safe basis for rejection. =A0Of course both passing means nothing as we= ll. >>> >> >> Doug, there are plenty of people with real world operational >> experience that would disagree with you. =A0You state that failing means >> nothing and passing means nothing. If that is true, why are there a >> significant number of implementers using this approach successfully? > > Hi Mike, > > While I'm just catching up on this thread, I can also chime in that -- > as far as I can tell, neither spf or dkim has had any impact on spam, > at least from what I can determine. > > If that was the original intent of spf and/or dkim, then I would be > inclined to say that neither have properly addressed the spam problem. > Hey Paul, Thanks for chiming in. As I pointed out in my original response to Doug, I inicated that in and of themselves, SPF or DKIM succeses/failures do not tell you about whether a message is SPAM. What they do allow you to know is whether that message is tied to a particular IP Address or Domain. The best description might be that they are enabling technologies. For domains that have good control over their mail flows (which mailbox providers can determine for themselves without relying on 3rd party reputation services), my experience and the experience of others is that one can (for example) recommend to mailbox providers that they can drop mail on a double fail (SPF/DKIM) with very little risk of dropping legitimate mail. Because each of these approaches break in tehir own ways, they are actually somewhat complimentary. This does have the side effect of reducing SPAM purporting to be from those domains at mailbox providers using this approach. There are other permutations/combinations that allow varying degrees of whitelisting (confidence) or skepticism that the message really did/did not come from the purported domain. Without naming any names, I am comfortable stating that a number of financial organizations have been working on this sort of implementation in conjunction with various large receivers with successful outcomes. I know that folks from some of those organizations lurk here (jump right in, the water's fine). The focus is more on phishing attacks and less on generic spam. Part of the issue that has been percolating for a while is that most of the interaction has been through private peering arrangements rather than through open standards that allow anyone to play. This also means that the data isn't publicly available. I (and others) get feeds on authentication failures from various receivers. Some of the senders are comparing notes and we are seeing similar outcomes. Again, mileage may vary based on the specific mail flows and the control exercised by those controlling the domain. Unfortunately, because the data is being shared under contractual arrangements you just aren't going to see detailed data being disclosed in public forums. Particularly sensitive data would be data on individual message authentication failures provided in ARF format. The best I have seen is including "cleansed" headers (RFC5322 To and Return Path redacted) with links from the message body included. At least for me, if I remove from the dataset our own legitimate links (a relatively small number), all I am left with is maliciousness. I guess the crude way to put it is that nobody is accidently purporting to send mail using our domains and accidently putting in links. (This case would be described as DKIM None and SPF fail.) If you want to hook up during RSA we can share notes. I could also make an introduction to some folks (I know some of them you already know) who could be in a position to make much more extensive data sets available to you. I will qualify my statements by pointing out that I play in the space known as potentially highly abused brands. Mileage may vary significantly for domains that allow their users to send mail from whatever mail server is handy. This of course does not address the issue of subversion of the emitting domains systems which would be problematic under any scenario. To sum up, this approach works well for some domains but is not a general solution to the problem of SPAM (or phishing). To the extent someone suceeds in controlling abuse against domains their work, the abusers simply move on to abuse another domain which is not as aggressively defended. I think the difference between what Doug is saying and what I am saying is that he is only willing to accept a solution which is works across the board. I'll take the wins where I can get them and accept that the abusers will go commit their abuse somewhere else. From your perspective, this approach may not be particularly useful because it might be considered a wash (substitute one abused domain for another). For a mailbox provider/ISP this approach is one more tool in their toolkit. What can I say, Darwin was right. I think we can also all agree that there are no magic bullets. In closing, the reason I'm saying things on the list that are directed to you is that I think it is useful for the wider group to understand that data is out there even if the detailed data is not openly available. Mike From dotis@mail-abuse.org Mon Jan 24 19:22:02 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id A16663A6A1B for ; Mon, 24 Jan 2011 19:22:02 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.526 X-Spam-Level: X-Spam-Status: No, score=-106.526 tagged_above=-999 required=5 tests=[AWL=0.073, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mILjrYAACm-G for ; Mon, 24 Jan 2011 19:22:01 -0800 (PST) Received: from harry.mail-abuse.org (harry.mail-abuse.org [168.61.5.27]) by core3.amsl.com (Postfix) with ESMTP id E900C3A6A21 for ; Mon, 24 Jan 2011 19:22:01 -0800 (PST) Received: from US-DOUGLASO-MB.local (gateway1.sjc.mail-abuse.org [168.61.5.81]) by harry.mail-abuse.org (Postfix) with ESMTP id E3199A9445E for ; Tue, 25 Jan 2011 03:25:50 +0000 (UTC) Message-ID: <4D3E4281.8080700@mail-abuse.org> Date: Mon, 24 Jan 2011 19:24:49 -0800 From: Douglas Otis User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: asrg@irtf.org References: <20110122042917.83764.qmail@joyce.lan> <4D3DD87C.5040208@mail-abuse.org> <4D3DFF43.2020502@mail-abuse.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Asrg] ipv6 macro expansion example in SPF specification, DNS ranges... X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jan 2011 03:22:02 -0000 On 1/24/11 6:14 PM, Dotzero wrote: >> Mike, >> > >> > An SPF failure can not be trusted to be an indicator of spam. DKIM signing >> > is almost never assured, especially when handled by third-party services. >> > As such, these mechanisms failing alone or together still do not offer a >> > safe basis for rejection. Of course both passing means nothing as well. > Doug, there are plenty of people with real world operational > experience that would disagree with you. You state that failing means > nothing and passing means nothing. If that is true, why are there a > significant number of implementers using this approach successfully? Defeating spam requires the reputation of SMTP clients be weighed for rejection or acceptance! SPF failures say little about an SMTP client. DKIM failures also say little about the SMTP client because either mechanism MUST be allowed to fail to retain email delivery integrity. When failure of SPF or DKIM offers scant basis for judging an SMTP client, they are useless as a mitigation tool. Ipso facto, their passing therefore provides little meaning as well, since mitigation must be based upon reliable mechanisms. Overlapping results must be considered a mere distraction from what is needed to mitigate spam. In rare cases, DKIM may play a role in preventing spoofing, but this can not be considered a significant component of spam mitigation. -Doug From dotzero@gmail.com Mon Jan 24 19:52:06 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 131AF3A6A17 for ; Mon, 24 Jan 2011 19:52:06 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.599 X-Spam-Level: X-Spam-Status: No, score=-3.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EKVIaQoSs8vD for ; Mon, 24 Jan 2011 19:52:05 -0800 (PST) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by core3.amsl.com (Postfix) with ESMTP id CB4643A6A16 for ; Mon, 24 Jan 2011 19:52:04 -0800 (PST) Received: by qwj9 with SMTP id 9so4673700qwj.13 for ; Mon, 24 Jan 2011 19:55:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type:content-transfer-encoding; bh=qEl3bPKq4meEaLdocPyPRxrObpZ7HpZVeHiZ2cM5M7k=; b=qVf1YWoAbBBuLugeHx3jHN6plbrLvJteqUyL1rg8NjHuYpcpjF+YfPRbWMTf71XXPV 0yXPafg5pGGqC4eU3Qy6Q6hl8yPA58SShNo8J7Plczq2c5qLX0F6H5s6pdIop0NlD4mu 5FbuICJzxpnx/pwsxKKGedddUc/a2cjuA3vt0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=mnfGSEPL+9DQmF7KXaagmV17iuQEN5bIEdrsVmvERA30ZY7XqAlHDUGV7Vm4sd6tie PTirvX3aYMJBfRbOAy/6oMhucKLe4TUBc+qcTyz18Q+8NCyfwevCnfcYs3HPtDEMrn+w OO/MHXGnvCR9ExgJoIjlrn1048u9rSAdysvFo= MIME-Version: 1.0 Received: by 10.229.248.198 with SMTP id mh6mr4426601qcb.5.1295927700921; Mon, 24 Jan 2011 19:55:00 -0800 (PST) Received: by 10.229.188.200 with HTTP; Mon, 24 Jan 2011 19:55:00 -0800 (PST) In-Reply-To: <4D3E4281.8080700@mail-abuse.org> References: <20110122042917.83764.qmail@joyce.lan> <4D3DD87C.5040208@mail-abuse.org> <4D3DFF43.2020502@mail-abuse.org> <4D3E4281.8080700@mail-abuse.org> Date: Mon, 24 Jan 2011 22:55:00 -0500 Message-ID: From: Dotzero To: Anti-Spam Research Group - IRTF Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: [Asrg] ipv6 macro expansion example in SPF specification, DNS ranges... X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jan 2011 03:52:06 -0000 On Mon, Jan 24, 2011 at 10:24 PM, Douglas Otis wrote= : > On 1/24/11 6:14 PM, Dotzero wrote: >>> >>> Mike, >>> > >>> > =A0An SPF failure can not be trusted to be an indicator of spam. =A0D= KIM >>> > signing >>> > =A0is almost never assured, especially when handled by third-party >>> > services. >>> > =A0As such, these mechanisms failing alone or together still do not o= ffer >>> > a >>> > =A0safe basis for rejection. =A0Of course both passing means nothing = as >>> > well. >> >> Doug, there are plenty of people with real world operational >> experience that would disagree with you. =A0You state that failing means >> nothing and passing means nothing. If that is true, why are there a >> significant number of implementers using this approach successfully? > > Defeating spam requires the reputation of SMTP clients be weighed for > rejection or acceptance! Doug, could you share with us what reputation is? As far the mailbox providers are concerned (that I have spoken with) it is reduced to "What have you done to me today". Their systems don't care whether you have had a good reputation for the past 2 years. If you start spewing badness today and you start generating complaints today then you will be throttled or blocked. If the time interval of reputation is that short then reputation systems are not particularly useful as an absolute requirement for defeating SPAM. >SPF failures say little about an SMTP client. They may or may not say something about the client in and of itself but they do say something about the relationship of the SMTP client to the purported Mail From domain when a well constructed SPF record ends in a -ALL > =A0DKIM failures also say little about the SMTP client because either > mechanism MUST be allowed to fail to retain email delivery integrity. A purist always brings a smile to my face. I know quite a few folks at large receivers who would disagree with your "MUST". What they are concerned about is ensuring the delivery integrity of mail their users want. As far as delivery integrity of mail their users don't want.....not so much. And that includes marketing emails their users don't want. The issue for them is not generic email delivery integrity but rather, how to find the sweet spot of rejecting/blocking/discarding while minimizing (not eliminating every single instance) performing such actions on mail their users actually want. >When > failure of SPF or DKIM offers scant basis for judging an SMTP client, the= y > are useless as a mitigation tool. =A0Ipso facto, their passing therefore > provides little meaning as well, since mitigation must be based upon > reliable mechanisms. Reliability is not a generic. It is a measure. One mailbox provider may choose a 99.99% reliability level and it's users may find that acceptable. Another may choose 99.999% reliability and it's users might find that acceptable. It may be that the mailbox provider segregates various inbound mail streams in making these determinations. Here is a truth for you. Mailbox providers will determine for themselves what works for them irrespective of any "MUST" you seek to impose on them. They will use standards based approaches and they will use non-standards based approaches. They will discard mail in a manner that offends your sensibilities. At the end of the day it is not about you....it is about them and their relationship with their customers. >Overlapping results must be considered a mere > distraction from what is needed to mitigate spam. =A0In rare cases, DKIM = may > play a role in preventing spoofing, but this can not be considered a > significant component of spam mitigation. > "cannot" is a very strong word in the manner that you use it. From vesely@tana.it Tue Jan 25 07:24:48 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id AD9813A67EA for ; Tue, 25 Jan 2011 07:24:48 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.117 X-Spam-Level: X-Spam-Status: No, score=-5.117 tagged_above=-999 required=5 tests=[AWL=-0.398, BAYES_00=-2.599, HELO_EQ_IT=0.635, HOST_EQ_IT=1.245, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FZtBnhAdaIc2 for ; Tue, 25 Jan 2011 07:24:47 -0800 (PST) Received: from wmail.tana.it (wmail.tana.it [62.94.243.226]) by core3.amsl.com (Postfix) with ESMTP id 85EAE3A67E7 for ; Tue, 25 Jan 2011 07:24:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tana.it; s=test; t=1295969263; bh=9Hg+PqsdrYi5VPGrEQ63z4NDrN1doxF7sDR2lePm8lc=; l=708; h=Message-ID:Date:From:MIME-Version:To:References:In-Reply-To: Content-Transfer-Encoding; b=Fc/ZwKwycCMxm8wP0z6ISocnj90AoXPtB4SnKOImxvxsRT/pzhfmXZYoqr5CanyTT 7KXWfmJv91QKv7MVWx9327ooQI0ias5lUW2LUJRXnSSI9fScPx89AEpUgZuTR1Zvzd qPUbZkPZTw6HnS6At9gXkYlTGcaHlpxepy+dxFBc= Received: from [172.25.197.158] (pcale.tana [172.25.197.158]) (AUTH: CRAM-MD5 515, TLS: TLS1.0,256bits,RSA_AES_256_CBC_SHA1) by wmail.tana.it with ESMTPSA; Tue, 25 Jan 2011 16:27:43 +0100 id 00000000005DC033.000000004D3EEBEF.000069B5 Message-ID: <4D3EEBEF.5080401@tana.it> Date: Tue, 25 Jan 2011 16:27:43 +0100 From: Alessandro Vesely User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101207 Lightning/1.0b2 Thunderbird/3.1.7 MIME-Version: 1.0 To: asrg@irtf.org References: <20110125030707.21285.qmail@joyce.lan> In-Reply-To: <20110125030707.21285.qmail@joyce.lan> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: Re: [Asrg] into the woods X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jan 2011 15:24:48 -0000 On 25/Jan/11 04:07, John Levine wrote: > I can't speak for the SPF crowd Why not? SPF users are just a subset of SMTP users, and hence spam victims, who thus deserve being considered by the ASRG. > the goal of DKIM is to make it easier to recognize mail from people > you know, for some definition of "people" and "know" which should > make filtering more accurate. The same is obviously true for SPF. I concur they are both enabling authentication technologies. > The only thing that's going to make the amount of spam drop is major > legal and technical advances that get rid of botnets. Don't hold your > breath. Isn't that an overly renunciative approach to reputation systems? From dotzero@gmail.com Tue Jan 25 07:29:19 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 5A3693A67ED for ; Tue, 25 Jan 2011 07:29:19 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.599 X-Spam-Level: X-Spam-Status: No, score=-3.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nwkEspSKdCXX for ; Tue, 25 Jan 2011 07:29:18 -0800 (PST) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by core3.amsl.com (Postfix) with ESMTP id 457913A67E9 for ; Tue, 25 Jan 2011 07:29:18 -0800 (PST) Received: by vws9 with SMTP id 9so2223682vws.13 for ; Tue, 25 Jan 2011 07:32:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type:content-transfer-encoding; bh=zKPFUNAickx1Mtq7V8yUN+I90dqZCWiSYw6vqH7sFIE=; b=AYyFOqSrdQB0IfEJ/ZsU1lWR2K0dHoNwp20uzHzVR0HpVNfO4cJNmnZ834w1jfpIPW qdrTqg5NtvGEX0EYxRkZpI8SwUlCEGi8R9bsK6MWVysvKPKoPfQ1JeYt5JMpvpEgr/U9 20LYV/t1lyPO2E4b1nplntYea2mLd/nfTspIE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=XUbQNIPW0KAoPqy4JovuxeTxwP54JdmJlyvUuP2kX3injpPNS3MZaLHqmhFkLpxnNP omaFpZfNvuYVIugvKb9MECAzc14nrYfjx4JhV3H/aiPNq2rypRpbL4VHbIMLF67Sb9Cx A//r9+LvmKfbb7EXVh6lybQ10wwg2VdSSOHig= MIME-Version: 1.0 Received: by 10.229.231.21 with SMTP id jo21mr4998442qcb.119.1295969535152; Tue, 25 Jan 2011 07:32:15 -0800 (PST) Received: by 10.229.188.200 with HTTP; Tue, 25 Jan 2011 07:32:15 -0800 (PST) In-Reply-To: <20110125030707.21285.qmail@joyce.lan> References: <20110125030707.21285.qmail@joyce.lan> Date: Tue, 25 Jan 2011 10:32:15 -0500 Message-ID: From: Dotzero To: Anti-Spam Research Group - IRTF Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: [Asrg] into the woods X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jan 2011 15:29:19 -0000 On Mon, Jan 24, 2011 at 10:07 PM, John Levine wrote: > > > The only thing that's going to make the amount of spam drop is major > legal and technical advances that get rid of botnets. =A0Don't hold your > breath. > The only thing that's going to make the amount of spam drop is heads on pikes. Don't hold your breath. From rsk@gsp.org Tue Jan 25 08:37:30 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 8D90A3A6800 for ; Tue, 25 Jan 2011 08:37:30 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.701 X-Spam-Level: X-Spam-Status: No, score=-5.701 tagged_above=-999 required=5 tests=[AWL=0.898, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AbKje-YMulgX for ; Tue, 25 Jan 2011 08:37:29 -0800 (PST) Received: from taos.firemountain.net (taos.firemountain.net [207.114.3.54]) by core3.amsl.com (Postfix) with ESMTP id B712F3A67F0 for ; Tue, 25 Jan 2011 08:37:29 -0800 (PST) Received: from squonk.gsp.org (bltmd-207.114.17.175.dsl.charm.net [207.114.17.175]) by taos.firemountain.net (8.14.4/8.14.4) with ESMTP id p0PGePrN006942 for ; Tue, 25 Jan 2011 11:40:26 -0500 (EST) Received: from avatar.gsp.org (avatar.gsp.org [192.168.0.11]) by squonk.gsp.org (8.14.3/8.14.3) with ESMTP id p0PGjRbB030667 for ; Tue, 25 Jan 2011 11:45:27 -0500 (EST) Received: from avatar.gsp.org (localhost [127.0.0.1]) by avatar.gsp.org (8.14.3/8.14.3/Debian-9ubuntu1) with ESMTP id p0PGeKT1024472 for ; Tue, 25 Jan 2011 11:40:20 -0500 Received: (from rsk@localhost) by avatar.gsp.org (8.14.3/8.14.3/Submit) id p0PGeKOL024471 for asrg@irtf.org; Tue, 25 Jan 2011 11:40:20 -0500 Date: Tue, 25 Jan 2011 11:40:20 -0500 From: Rich Kulawiec To: Anti-Spam Research Group - IRTF Message-ID: <20110125164020.GA24052@gsp.org> References: <20110122042917.83764.qmail@joyce.lan> <4D3DD87C.5040208@mail-abuse.org> <4D3DFF43.2020502@mail-abuse.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) Subject: Re: [Asrg] ipv6 macro expansion example in SPF specification, DNS ranges... X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jan 2011 16:37:30 -0000 On Mon, Jan 24, 2011 at 06:21:11PM -0800, Paul Ferguson wrote: > While I'm just catching up on this thread, I can also chime in that -- > as far as I can tell, neither spf or dkim has had any impact on spam, > at least from what I can determine. > > If that was the original intent of spf and/or dkim, then I would be > inclined to say that neither have properly addressed the spam problem. I emphatically concur: SPF and DKIM have no anti-spam value. (And given that SPF was introduced with the thus-far unretracted claim: "Spam as a technical problem is solved by SPF." I think it's quite fair to say that its original intent was anti-spam.) As to whether has an impact on forgeries: that depends on what one considers a "forgery". My own view that is that both (as well as any other similar schemes) *may* have some applicability on a local level, but not at the scale of the entire Internet. And that's not because of any inherent issues with them: the problem is insoluble at the moment no matter what technology is used, because of the underlying persistent and systemic security issues. But as I said, this depends on what one considers a "forgery" and those using different definitions than mine may find DKIM useful in that context. ---rsk From dotis@mail-abuse.org Tue Jan 25 11:26:53 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 5F0BE3A6838 for ; Tue, 25 Jan 2011 11:26:53 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.529 X-Spam-Level: X-Spam-Status: No, score=-106.529 tagged_above=-999 required=5 tests=[AWL=0.070, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id i5kuL+d1Lm5N for ; Tue, 25 Jan 2011 11:26:51 -0800 (PST) Received: from harry.mail-abuse.org (harry.mail-abuse.org [168.61.5.27]) by core3.amsl.com (Postfix) with ESMTP id 37E603A6816 for ; Tue, 25 Jan 2011 11:26:50 -0800 (PST) Received: from US-DOUGLASO-MB.local (gateway1.sjc.mail-abuse.org [168.61.5.81]) by harry.mail-abuse.org (Postfix) with ESMTP id 30BC0A94442 for ; Tue, 25 Jan 2011 19:30:49 +0000 (UTC) Message-ID: <4D3F24AC.2020906@mail-abuse.org> Date: Tue, 25 Jan 2011 11:29:48 -0800 From: Douglas Otis User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: asrg@irtf.org References: <20110122042917.83764.qmail@joyce.lan> <4D3DD87C.5040208@mail-abuse.org> <4D3DFF43.2020502@mail-abuse.org> <4D3E4281.8080700@mail-abuse.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Asrg] ipv6 macro expansion example in SPF specification, DNS ranges... X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jan 2011 19:26:53 -0000 On 1/24/11 7:55 PM, Dotzero wrote: > On Mon, Jan 24, 2011 at 10:24 PM, Douglas Otis > wrote: > > Defeating spam requires the reputation of SMTP clients be weighed > > for rejection or acceptance! > > Doug, could you share with us what reputation is? Reputation Definition: An assessment of an identifier that accurately reflects the behavior of the resource administrator. For example, these identifiers might be the address of an SMTP client or the AS of the network provider. SPF represents a poor choice. SPF represents an unreliable authorization mechanism that also fails at isolating behaviors of resource administrators. Those who attempt to make use of this mechanism often weaken the integrity of email delivery. > As far the mailbox providers are concerned (that I have spoken with) > it is reduced to "What have you done to me today". Sorry, but this statement is vague, as is SPF. No one knows how SPF might be used. A PASS used as a basis to verify an unseen Sender-ID PRA header or Return-Path parameter can be highly deceptive and even place recipients at risk. Malefactors can authorize anyone's resources, where macros contained in SPF can also obfuscate their breath. > Their systems don't care whether you have had a good reputation for > the past 2 years. If you start spewing badness today and you start > generating complaints today then you will be throttled or blocked. SPF does not help with this problem, nor can DKIM be safely used in this manner, as it is designed to permit replays. See Reputation Definition. > If the time interval of reputation is that short then reputation > systems are not particularly useful as an absolute requirement for > defeating SPAM. Systems are often in transition from compromised to cleaned. As such, effective SPAM mitigation MUST be responsive, either by actions of reputation services, or by resource administrators. A responsive resource administrator represents by far the most effective actor in SPAM mitigation. > > SPF failures say little about an SMTP client. > > They may or may not say something about the client in and of itself > but they do say something about the relationship of the SMTP client > to the purported Mail From domain when a well constructed SPF record > ends in a -ALL An "-all" suffix means what? Should MTAs refuse a message that fails a check that may require 111 transactions to verify? An SMTP client issuing a message by properly carrying forward the Return-Path to ensure email delivery integrity may cause this check to fail and has done nothing wrong! > > DKIM failures also say little about the SMTP client because either > > mechanism MUST be allowed to fail to retain email delivery > > integrity. > > A purist always brings a smile to my face. I know quite a few folks > at large receivers who would disagree with your "MUST". Are you saying that all SPF failures MAY be refused? > What they > are concerned about is ensuring the delivery integrity of mail their > users want. As far as delivery integrity of mail their users don't > want.....not so much. And that includes marketing emails their users > don't want. The issue for them is not generic email delivery > integrity but rather, how to find the sweet spot of > rejecting/blocking/discarding while minimizing (not eliminating > every single instance) performing such actions on mail their users > actually want. > > Here is a truth for you. Mailbox providers will determine for > themselves what works for them irrespective of any "MUST" you seek > to impose on them. They will use standards based approaches and they > will use non-standards based approaches. They will discard mail in a > manner that offends your sensibilities. At the end of the day it is > not about you....it is about them and their relationship with their > customers. Attempts to divine a recipient's desire has little to do with SPF. Are you suggesting all failures be rejected, or all passes be accepted? Any provider that hopes to meet the desires of their customers MUST look elsewhere to determine proper message handling. For example, I have advocated use of a third-party authorization scheme to assist with this process. -Doug From davidnicol@gmail.com Tue Jan 25 12:59:40 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id BC58D3A6899 for ; Tue, 25 Jan 2011 12:59:40 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.353 X-Spam-Level: X-Spam-Status: No, score=-3.353 tagged_above=-999 required=5 tests=[AWL=0.246, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bwekyuwVJKuP for ; Tue, 25 Jan 2011 12:59:40 -0800 (PST) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by core3.amsl.com (Postfix) with ESMTP id 0FB353A6838 for ; Tue, 25 Jan 2011 12:59:40 -0800 (PST) Received: by iwn39 with SMTP id 39so225018iwn.13 for ; Tue, 25 Jan 2011 13:02:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:from:date :message-id:subject:to:content-type:content-transfer-encoding; bh=kiRKkFHKzGpv0taeEV09jCwAmqQ/crvhjydpjNPwE0E=; b=sfnGI0dgwzFNobT4Bv9P34ww2rRXoTMpgvEsN4YF9iPRM9apEJ0QUihmTrP9ZSY3ie eqIQmf9otFwWINPeOjPRAAMoQJL3lf2dYBRb6I/4jM29NX8pP2/OqwopjhnoagQHmiWP 8bHOgB8LtdgscReIT+9EshDxdc98UOeP4JLEo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type:content-transfer-encoding; b=EGy3HqqBHSFCybER4RznElttR+Nx5DEpMXDV4jgfxXW5tXsLGZTGeT1xmmuvECT3bT kV50qtpxj9aOvlqW78u7pJIa60wyi5GA59hdQPFn7gsuTNLPI5vGSi+OI6HoA4ynZz0o ZiHLJAJke5lDpSTghgqiJ4CGtq6JAMMzqERiw= Received: by 10.231.206.206 with SMTP id fv14mr7239204ibb.75.1295989358327; Tue, 25 Jan 2011 13:02:38 -0800 (PST) MIME-Version: 1.0 Received: by 10.231.200.138 with HTTP; Tue, 25 Jan 2011 13:02:18 -0800 (PST) In-Reply-To: <4D3F24AC.2020906@mail-abuse.org> References: <20110122042917.83764.qmail@joyce.lan> <4D3DD87C.5040208@mail-abuse.org> <4D3DFF43.2020502@mail-abuse.org> <4D3E4281.8080700@mail-abuse.org> <4D3F24AC.2020906@mail-abuse.org> From: David Nicol Date: Tue, 25 Jan 2011 15:02:18 -0600 Message-ID: To: Anti-Spam Research Group - IRTF Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [Asrg] ipv6 macro expansion example in SPF specification, DNS ranges... X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jan 2011 20:59:40 -0000 On Tue, Jan 25, 2011 at 1:29 PM, Douglas Otis wrote: > An "-all" suffix means what? =C2=A0Should MTAs refuse a message that fail= s a > check that may require 111 transactions to verify? =C2=A0An SMTP client i= ssuing a > message by properly carrying forward the Return-Path to ensure email > delivery integrity may cause this check to fail and has done nothing wron= g! yeah, SPF presumes that everyone will play by the SPF rules, in which case carrying forward a return path is no longer proper. (he said, taking the bait.) From john@jlc.net Tue Jan 25 13:12:00 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 065833A6838 for ; Tue, 25 Jan 2011 13:12:00 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.352 X-Spam-Level: X-Spam-Status: No, score=-106.352 tagged_above=-999 required=5 tests=[AWL=0.247, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5iUlSgEFuxGJ for ; Tue, 25 Jan 2011 13:11:59 -0800 (PST) Received: from mailhost.jlc.net (mailhost.jlc.net [199.201.159.4]) by core3.amsl.com (Postfix) with ESMTP id B4FD83A6873 for ; Tue, 25 Jan 2011 13:11:58 -0800 (PST) Received: by mailhost.jlc.net (Postfix, from userid 104) id 8CC1733C2A; Tue, 25 Jan 2011 16:14:57 -0500 (EST) Date: Tue, 25 Jan 2011 16:14:57 -0500 From: John Leslie To: Anti-Spam Research Group - IRTF Message-ID: <20110125211457.GR68297@verdi> References: <20110122042917.83764.qmail@joyce.lan> <4D3DD87C.5040208@mail-abuse.org> <4D3DFF43.2020502@mail-abuse.org> <4D3E4281.8080700@mail-abuse.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.1i Subject: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jan 2011 21:12:00 -0000 Dotzero wrote: > On Mon, Jan 24, 2011 at 10:24 PM, Douglas Otis wrote: >> On 1/24/11 6:14 PM, Dotzero wrote: >>> >>> Doug, there are plenty of people with real world operational >>> experience that would disagree with you. ?You state that failing means >>> nothing and passing means nothing. If that is true, why are there a >>> significant number of implementers using this approach successfully? Lack of better alternatives... >> Defeating spam requires the reputation of SMTP clients be weighed for >> rejection or acceptance! > > Doug, could you share with us what reputation is? Reputation (as the name implies) is a prediction of the likelihood of near-future behavior. > As far the mailbox providers are concerned (that I have spoken with) > it is reduced to "What have you done to me today". Perhaps... if nothing better is available... The real question for a mailbox-provider is, "How much does the mailbox customer want to see this?" For practical reasons, it's usually reduced to, "How much does my typical mailbox customer want to see this?" And that value is arbitrarily cloe to zero for email that's part of a spam run. So once you have identified a spam rum, you're likely to want to reject it all; and once the spam run is ended, you're likely willing to let email through. > Their systems don't care whether you have had a good reputation for > the past 2 years. As a practical matter, a 2-year-old good reputation doesn't say you won't be compromised and start spewing spam. > If you start spewing badness today and you start generating complaints > today then you will be throttled or blocked. Reputation systems, in principle, could have one-second update cycles. They could, in principle, differentiate the end of a single spam run on a botted computer from the computer being no longer botted. Understandably, the "free" reputation services don't attempt that. But that's a business model: not an inherent limitation. > If the time interval of reputation is that short then reputation > systems are not particularly useful as an absolute requirement for > defeating SPAM. Day-old reputation information is still _somewhat_ useful. And, IMHO, reputation _is_ an absolute requirement for defeating SPAM. We do need to add something equivalent to "incident in progress," where SPAM is likely to be interspersed with HAM, but there's reason to believe the SPAM will be purged from retry queues (and how long that should take). That's venturing into territory which presents challenges gathering the information, absent what I call "vouching" services which are given information about policies and procedures. The vouching services (if trustworthy) can tell reputation services the estimated time to repair a breach. (There are also a number of questions of trust, which require customer relationships: for example, how much needs to be sanitized in a SPAM report.) So, IMHO, _standalone_ reputation services are necessary but far from sufficient for defeating SPAM. -- John Leslie From fergdawgster@gmail.com Tue Jan 25 13:13:57 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 782043A68A0 for ; Tue, 25 Jan 2011 13:13:57 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.566 X-Spam-Level: X-Spam-Status: No, score=-2.566 tagged_above=-999 required=5 tests=[AWL=1.033, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NZMymm6-mi8W for ; Tue, 25 Jan 2011 13:13:56 -0800 (PST) Received: from mail-gy0-f182.google.com (mail-gy0-f182.google.com [209.85.160.182]) by core3.amsl.com (Postfix) with ESMTP id B2F2B3A689F for ; Tue, 25 Jan 2011 13:13:56 -0800 (PST) Received: by gyf3 with SMTP id 3so1992852gyf.13 for ; Tue, 25 Jan 2011 13:16:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type:content-transfer-encoding; bh=iWPvQKMhxEao794MndYo84UwZyjUhUOl0EtUUx+lERA=; b=IQCI9E7J+6YEoFpxx6bKJKkub9KMbGb1EOIAXkPorcTDrNn8Kt1O250cRC99AxP8AN Ri1ka+uMJ8dZfr39ydCUSS5KUCIVSAAxHVPj8MrEmR8i6PHXD7jcq34sH7VKOvdpnsFq QzgqgrsCzQiYvSvjfi99TYXvAzjwVv3ejHq5Y= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=kzNggXIbVM+DOALXrTzl5SmbAoFZ8q9SIrmtIaX47hUFTeffT1m0KbgTrFURP7qmwi fZW0gesODb9K0ATLBw8Zqsqneal4vOYm93wa1EOIUx0I4BxOVnVkNtrdNdhMrs47B0WD n/ddirCuN9E6KjDmTiBbvZYg85BqqtnBz7XyE= MIME-Version: 1.0 Received: by 10.42.172.67 with SMTP id m3mr7352095icz.231.1295990214734; Tue, 25 Jan 2011 13:16:54 -0800 (PST) Received: by 10.42.221.73 with HTTP; Tue, 25 Jan 2011 13:16:54 -0800 (PST) In-Reply-To: <20110125211457.GR68297@verdi> References: <20110122042917.83764.qmail@joyce.lan> <4D3DD87C.5040208@mail-abuse.org> <4D3DFF43.2020502@mail-abuse.org> <4D3E4281.8080700@mail-abuse.org> <20110125211457.GR68297@verdi> Date: Tue, 25 Jan 2011 13:16:54 -0800 Message-ID: From: Paul Ferguson To: Anti-Spam Research Group - IRTF Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jan 2011 21:13:57 -0000 On Tue, Jan 25, 2011 at 1:14 PM, John Leslie wrote: > > =A0 Reputation (as the name implies) is a prediction of the likelihood of > near-future behavior. > ...based on previously observed behavior. - ferg --=20 "Fergie", a.k.a. Paul Ferguson =A0Engineering Architecture for the Internet =A0fergdawgster(at)gmail.com =A0ferg's tech blog: http://fergdawg.blogspot.com/ From dotzero@gmail.com Tue Jan 25 17:03:07 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 5E44D3A68FB for ; Tue, 25 Jan 2011 17:03:07 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.599 X-Spam-Level: X-Spam-Status: No, score=-3.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6i2UsoTi1UxH for ; Tue, 25 Jan 2011 17:03:06 -0800 (PST) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by core3.amsl.com (Postfix) with ESMTP id B1A803A68F7 for ; Tue, 25 Jan 2011 17:03:05 -0800 (PST) Received: by qwj9 with SMTP id 9so437113qwj.13 for ; Tue, 25 Jan 2011 17:06:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type:content-transfer-encoding; bh=07iWG4aJyBfvqUNzjMs4Q5degzLPmoDCsyoWVJ1DyXk=; b=U6V0bvNT3OTMgZXCLGU34eywJ87eIKdKrEvntRjszC4CNMj39Lgw82Ro/rdPWn7TuN rG5WeG0zROs0su1kn0p9lFj22Wtbh4SgwMdnlrnwdT3tiiD+mSjmh8r2KgUXZCZq3+7w q0ASvgQ16DQA++Qs/WXqk0c+saMxbvmZCslkA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=FyV7Wwn7Pv+edLEvjrYhdsdEbLislmANhpxvct2V1ZigCfTurQvbOm+5zTaWykI0hl hy8nfPmaJoFee0ZS5b2YNZHDdRhkH5SbWmeBp1UEQktSrzyTHA8KdYeXroN8neVBlPhv k4fkjLc6VFu4FWwQcHiTiksclu/hEvYhlKk60= MIME-Version: 1.0 Received: by 10.229.240.66 with SMTP id kz2mr5390749qcb.233.1296003963836; Tue, 25 Jan 2011 17:06:03 -0800 (PST) Received: by 10.229.188.200 with HTTP; Tue, 25 Jan 2011 17:06:03 -0800 (PST) In-Reply-To: References: <20110122042917.83764.qmail@joyce.lan> <4D3DD87C.5040208@mail-abuse.org> <4D3DFF43.2020502@mail-abuse.org> <4D3E4281.8080700@mail-abuse.org> <20110125211457.GR68297@verdi> Date: Tue, 25 Jan 2011 20:06:03 -0500 Message-ID: From: Dotzero To: Anti-Spam Research Group - IRTF Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2011 01:03:07 -0000 On Tue, Jan 25, 2011 at 4:16 PM, Paul Ferguson wro= te: > On Tue, Jan 25, 2011 at 1:14 PM, John Leslie wrote: > >> >> =A0 Reputation (as the name implies) is a prediction of the likelihood o= f >> near-future behavior. >> > > ...based on previously observed behavior. > > - ferg > So, what exactly does this mean when behavior suddenly changes? If a domain or IP address (was well behaved yesterday) but begins spewing badness today, what will your company do as an arbiter of whether mail is accepted by your customers? Will you allow that domain or IP address to spew badness? I highly doubt it. As some point once the spewing has subsided you may (automatically or manually) again start allowing traffic through from that domain or IP address. But that isn't really reputation in the traditional sense of the word. But that brings me back to my original question. If reputation doesn't prevent a site from getting throttled or blocked when it goes bad, what does reputation mean? It doesn't particularly protect the site from the immediate consequences of going bad. It appears that the responses are authoritative (this domain or IP is currently emitting badness) rather than reputational (this site has a good reputation so I will accept badness from it on the presumption they are going to address it). I will grant that there may be some small slack cut based on reputation but does it really extend that far? Set aside whether you have personal contacts with someone at that domain or ASN. That approach doesn't scale from either direction. From fergdawgster@gmail.com Tue Jan 25 17:15:34 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 5F9043A68CC for ; Tue, 25 Jan 2011 17:15:34 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.824 X-Spam-Level: X-Spam-Status: No, score=-2.824 tagged_above=-999 required=5 tests=[AWL=0.775, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xjsOJjmcCVtl for ; Tue, 25 Jan 2011 17:15:33 -0800 (PST) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by core3.amsl.com (Postfix) with ESMTP id E00BB3A68B8 for ; Tue, 25 Jan 2011 17:15:32 -0800 (PST) Received: by iyb26 with SMTP id 26so5904357iyb.13 for ; Tue, 25 Jan 2011 17:18:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type:content-transfer-encoding; bh=Yl3zcW49uhFYs4w9VImFJJc+i/giPJIwcU5bz8w8wb0=; b=klgNQIKjHnbmz/6q8xn4hoooYrTxbO97BsbiAPcaxP0/vA2528bxnLrdxQOdac2rXT k8cbniHqevg7Obmr4dRsMATx33AmU2LuUV8eqK84z8ZApzfH5jeBl+3Hcu4FoKTaIIO8 TtHkFt5mKRNbIzeMGS4Iz38Xi8aOCWSvDPzZE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=ahrHY3NjW7d3XPHqQlkVCLN1YXo5fUNzndc7rK7pXq9CnR+Qfzx506WwMPD2YU5vPQ TjRfS4lYlc//IpAlODqAYJ8lCQkezhplcnFLlycdmxwUJOFDTg+hDUGB/CO86AbXia96 gLySgEais1WpHbIdEOkaY0HFZadcAELp6a+90= MIME-Version: 1.0 Received: by 10.42.228.68 with SMTP id jd4mr7622315icb.499.1296004711452; Tue, 25 Jan 2011 17:18:31 -0800 (PST) Received: by 10.42.221.73 with HTTP; Tue, 25 Jan 2011 17:18:31 -0800 (PST) In-Reply-To: References: <20110122042917.83764.qmail@joyce.lan> <4D3DD87C.5040208@mail-abuse.org> <4D3DFF43.2020502@mail-abuse.org> <4D3E4281.8080700@mail-abuse.org> <20110125211457.GR68297@verdi> Date: Tue, 25 Jan 2011 17:18:31 -0800 Message-ID: From: Paul Ferguson To: Anti-Spam Research Group - IRTF Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2011 01:15:34 -0000 On Tue, Jan 25, 2011 at 5:06 PM, Dotzero wrote: > > But that brings me back to my original question. If reputation doesn't > prevent a site from getting throttled or blocked when it goes bad, > what does reputation mean? It doesn't particularly protect the site > from the immediate consequences of going bad. It appears that the > responses are authoritative (this domain or IP is currently emitting > badness) rather than reputational (this site has a good reputation so > I will accept badness from it on the presumption they are going to > address it). I will grant that there may be some small slack cut based > on reputation but does it really extend that far? > Hi Mike, if you recall my presentation at ISOI 8 in Pasadena [1], I gave a presentation (specifically) on domain reputation, followed by Eric Ziegast of ISC on DNS RPZ [2]. One of the main points I was trying to make in that presentation is that there should be specific distinctions made between reputation "policy" and reputation "technical implementation". Each individual organization -- both the ones that provide the "reputation", and the ones that decide to "implement" it -- should determine for themselves what their policies are. The technical implementation should, in my opinion, should be "standardized" (for whatever your definition of standardized may be), such that many different policies can be implemented in the same fashion. That's one reason I'm a big fan of DNS RPZ for implementing domain reputation -- each implementing organization can choose for themselves which reputation providers they choose to implement (e.g. Spamhaus, SURBL, etc.). Let's not get sucked in to a long drawn-out discussion of policy. It generally spirals into a meaningless quagmire. :-) - ferg p.s. Of course, this doesn't necessarily address spam, per se, but more broadly the distinction between policy & technical implementation. [1] http://www.isotf.org/isoi8.html [2] http://www.isc.org/community/blog/201007/taking-back-dns-0 --=20 "Fergie", a.k.a. Paul Ferguson =A0Engineering Architecture for the Internet =A0fergdawgster(at)gmail.com =A0ferg's tech blog: http://fergdawg.blogspot.com/ From steve@blighty.com Tue Jan 25 17:28:10 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id F15F43A6900 for ; Tue, 25 Jan 2011 17:28:09 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.48 X-Spam-Level: X-Spam-Status: No, score=-1.48 tagged_above=-999 required=5 tests=[AWL=-0.121, BAYES_00=-2.599, SARE_LWSHORTT=1.24] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id chLCrFoPp2rp for ; Tue, 25 Jan 2011 17:28:08 -0800 (PST) Received: from m.wordtothewise.com (b.wordtothewise.com [208.187.80.136]) by core3.amsl.com (Postfix) with ESMTP id CFBA23A68B0 for ; Tue, 25 Jan 2011 17:28:08 -0800 (PST) Received: from platter.wordtothewise.com (184.wordtothewise.com [208.187.80.184]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: steve) by m.wordtothewise.com (Postfix) with ESMTPSA id 9D7DE2E9B9 for ; Tue, 25 Jan 2011 17:31:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wordtothewise.com; s=1.wttw; t=1296005468; bh=FjX8GI09KAM0/E2CBmvZRKUN5ZjIQLx4ZYMYljrZwL4=; h=Content-Type:Mime-Version:Subject:From:In-Reply-To:Date: Content-Transfer-Encoding:Message-Id:References:To; b=RhOPbRRmubfcIplkpzFmGOuOhqHmmnlWbmWYQYNOvpqXUXHWX4JOLTxjux2A9a+fq aPrs8C1Qj+rcnO5DH4z2jZCn4KlHP3DndQMJPA4dArI5FcKyk6ThiaZskGTfDMB0bR pXynwaIUXgp5c/m+bLoDFceFLWua08NGWdDlF7JU= Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Apple Message framework v1082) From: Steve Atkins In-Reply-To: Date: Tue, 25 Jan 2011 17:31:05 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <21E51B60-9912-4410-BC98-B62596E28949@blighty.com> References: <20110122042917.83764.qmail@joyce.lan> <4D3DD87C.5040208@mail-abuse.org> <4D3DFF43.2020502@mail-abuse.org> <4D3E4281.8080700@mail-abuse.org> <20110125211457.GR68297@verdi> To: Anti-Spam Research Group - IRTF X-Mailer: Apple Mail (2.1082) Subject: Re: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2011 01:28:10 -0000 On Jan 25, 2011, at 5:06 PM, Dotzero wrote: > On Tue, Jan 25, 2011 at 4:16 PM, Paul Ferguson = wrote: >> On Tue, Jan 25, 2011 at 1:14 PM, John Leslie wrote: >>=20 >>>=20 >>> Reputation (as the name implies) is a prediction of the likelihood = of >>> near-future behavior. >>>=20 >>=20 >> ...based on previously observed behavior. >>=20 >> - ferg >>=20 >=20 > So, what exactly does this mean when behavior suddenly changes? If a > domain or IP address (was well behaved yesterday) but begins spewing > badness today, what will your company do as an arbiter of whether mail > is accepted by your customers? Will you allow that domain or IP > address to spew badness? "badness" is hard to measure at scale once you've removed the obvious botnet spew from your mailstream. > I highly doubt it. Depends. If it's sending a lot of wanted email, a spike of obviously unwanted email and a middle ground of mail which you can't decide about you're always going to want to deliver the wanted email, and you're always going to want to block the obviously unwanted email. But the mail in the middle is harder to decide what to do about, and that's where sender reputation helps. > As some point once the > spewing has subsided you may (automatically or manually) again start > allowing traffic through from that domain or IP address. But that > isn't really reputation in the traditional sense of the word. >=20 > But that brings me back to my original question. If reputation doesn't > prevent a site from getting throttled or blocked when it goes bad, > what does reputation mean? Reputation is, loosely, the past history of the sender from a decade ago through to a second ago. It's not a simple integer (0=3Dbad, = 100=3Dgood) however much people want to map it onto that. It includes, at least, traffic volumes over time and fraction of email that was wanted vs not wanted over time. Comparing the past history of the sender (over a period of months) to the current behaviour of the sender recently (minutes to hours) can help you guess what the sender is up to, categorize them into one of several "bins" automatically and treat mail from them appropriately. If a sender has a history of not sending any email at all and you = suddenly see a lot of email from it then you can categorize it as "probably a compromised end-user machine". If it has a history of sending 1000 emails a day and it suddenly starts spewing 100k then the change in behavior lets you categorize it as "tiny smarthost with compromised system" and maybe block it outright. But if a site has a history of sending you large volumes of wanted email over a long period, and you suddenly see a spike of unwanted email then you're likely to assume that it's a transient problem (bad customer, perhaps) and that once you've notified them of the problem, they'll fix it. Meanwhile you'll keep delivering most of the email from them, on the assumption that it's mostly wanted. They're all reasonable decisions. And they're things that can be implemented as a set of business rules driven by, amongst other things, the short term and long term history of the sender = ("reputation") in an automated way that doesn't require much per-sender configuration. Plug in your policy, let it run, watch your statistics and tweak. > It doesn't particularly protect the site > from the immediate consequences of going bad. It appears that the > responses are authoritative (this domain or IP is currently emitting > badness) rather than reputational (this site has a good reputation so > I will accept badness from it on the presumption they are going to > address it). I will grant that there may be some small slack cut based > on reputation but does it really extend that far? Yes. Filtering out "obvious" spam is easy. Recognizing "obvious" 1:1 email between regular correspondents isn't too hard. Dealing with the big grey area in the middle is hard, and sender reputation is about the only thing that gives you anything to base delivery decisions on there, so for the big middle ground of email good reputation will take you a long way. Cheers, Steve From dotzero@gmail.com Tue Jan 25 17:46:35 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 147593A690B for ; Tue, 25 Jan 2011 17:46:35 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.235 X-Spam-Level: X-Spam-Status: No, score=-2.235 tagged_above=-999 required=5 tests=[AWL=-1.365, BAYES_05=-1.11, RCVD_IN_DNSWL_LOW=-1, SARE_LWSHORTT=1.24] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9OkqRKnjO-jx for ; Tue, 25 Jan 2011 17:46:34 -0800 (PST) Received: from mail-qy0-f175.google.com (mail-qy0-f175.google.com [209.85.216.175]) by core3.amsl.com (Postfix) with ESMTP id AF56E3A6906 for ; Tue, 25 Jan 2011 17:46:33 -0800 (PST) Received: by qyk8 with SMTP id 8so4518242qyk.13 for ; Tue, 25 Jan 2011 17:49:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type:content-transfer-encoding; bh=2dVuP+K14to8X691TfHOO00GExT1qSzZcbNlDo/gQgI=; b=n4ioN3QT62ZoxWW8lLK0ArQYrsaiqp/Jmjj7ugbIoOym68+CQGcOlELN07XSpZuB7J q6aFZThTMtDMY8kh+fGAG7tFwiIGIFzLpZozKnhNKS9NFE9XOV7+WDygKVoT8ojNz2KT VA/bF4zGh+oLzRJyeJslH5XX81AfUw0LxwKnY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=FGFjgVRxUIJsgLBI43mCsXdE/P/hOJxq5clPn112kqjVUhvhN1UfeTiPIJCaM2elam mGXW0Qbo5NtlUh47stGes5lI45L3WOvFmDrZ34TW9NaekpRW9PCNoVrOKk7CwHDKDXdI ASkHxj6CJYL1p3waYC5WQM0h10+CrhhO+kI94= MIME-Version: 1.0 Received: by 10.229.211.138 with SMTP id go10mr5394693qcb.195.1296006572451; Tue, 25 Jan 2011 17:49:32 -0800 (PST) Received: by 10.229.188.200 with HTTP; Tue, 25 Jan 2011 17:49:32 -0800 (PST) In-Reply-To: <21E51B60-9912-4410-BC98-B62596E28949@blighty.com> References: <20110122042917.83764.qmail@joyce.lan> <4D3DD87C.5040208@mail-abuse.org> <4D3DFF43.2020502@mail-abuse.org> <4D3E4281.8080700@mail-abuse.org> <20110125211457.GR68297@verdi> <21E51B60-9912-4410-BC98-B62596E28949@blighty.com> Date: Tue, 25 Jan 2011 20:49:32 -0500 Message-ID: From: Dotzero To: Anti-Spam Research Group - IRTF Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2011 01:46:35 -0000 On Tue, Jan 25, 2011 at 8:31 PM, Steve Atkins wrote: > > On Jan 25, 2011, at 5:06 PM, Dotzero wrote: > >> On Tue, Jan 25, 2011 at 4:16 PM, Paul Ferguson = wrote: >>> On Tue, Jan 25, 2011 at 1:14 PM, John Leslie wrote: >>> >>>> >>>> =A0 Reputation (as the name implies) is a prediction of the likelihood= of >>>> near-future behavior. >>>> >>> >>> ...based on previously observed behavior. >>> >>> - ferg >>> >> >> So, what exactly does this mean when behavior suddenly changes? If a >> domain or IP address (was well behaved yesterday) but begins spewing >> badness today, what will your company do as an arbiter of whether mail >> is accepted by your customers? Will you allow that domain or IP >> address to spew badness? > > "badness" is hard to measure at scale once you've removed the > obvious botnet spew from your mailstream. > >> I highly doubt it. > > Depends. If it's sending a lot of wanted email, a spike of obviously > unwanted email and a middle ground of mail which you can't > decide about you're always going to want to deliver the wanted > email, and you're always going to want to block the obviously > unwanted email. > Wanting and doing are two different things. In many cases the evaluator isn't going to get that granular. This is particularly true where the unwanted mail is somehow malicious, not simply unwanted. > But the mail in the middle is harder to decide what to do about, > and that's where sender reputation helps. > I'll grant you that about the middle. But that's not necessarily "badness". >> As some point once the >> spewing has subsided you may (automatically or manually) again start >> allowing traffic through from that domain or IP address. But that >> isn't really reputation in the traditional sense of the word. >> >> But that brings me back to my original question. If reputation doesn't >> prevent a site from getting throttled or blocked when it goes bad, >> what does reputation mean? > > Reputation is, loosely, the past history of the sender from a decade > ago through to a second ago. It's not a simple integer (0=3Dbad, 100=3Dgo= od) > however much people want to map it onto that. It includes, at least, > traffic volumes over time and fraction of email that was wanted vs > not wanted over time. > > Comparing the past history of the sender (over a period of months) > to the current behaviour of the sender recently (minutes to hours) can > help you guess what the sender is up to, categorize them into one > of several "bins" automatically and treat mail from them appropriately. > But the case that originally triggered the discussion is false positives. That is a case of treating those messages inappropriately. > If a sender has a history of not sending any email at all and you suddenl= y > see a lot of email from it then you can categorize it as "probably a > compromised end-user machine". > A slightly different issue. For emitters of large volume mail streams, the bad stream may still only be a percentage of the overall mail stream. > If it has a history of sending 1000 emails a day and it suddenly starts > spewing 100k then the change in behavior lets you categorize it > as "tiny smarthost with compromised system" and maybe block it outright. > > But if a site has a history of sending you large volumes of wanted > email over a long period, and you suddenly see a spike of unwanted > email then you're likely to assume that it's a transient problem (bad > customer, perhaps) and that once you've notified them of the problem, > they'll fix it. Meanwhile you'll keep delivering most of the email from > them on the assumption that it's mostly wanted. > I think you'll find (At least what I have seen) is that they will throttle the overall stream (for example Yahoo! - 421 too many complaints) or implement a temporary block (AOL), etc. > They're all reasonable decisions. And they're things that can be > implemented as a set of business rules driven by, amongst other > things, the short term and long term history of the sender ("reputation") > in an automated way that doesn't require much per-sender configuration. > Plug n your policy, let it run, watch your statistics and tweak. > I'm still radiating skepticism. While I agree that what you describe is the optimal approach, I'm not sure it matches reality. >> It doesn't particularly protect the site >> from the immediate consequences of going bad. It appears that the >> responses are authoritative (this domain or IP is currently emitting >> badness) rather than reputational (this site has a good reputation so >> I will accept badness from it on the presumption they are going to >> address it). I will grant that there may be some small slack cut based >> on reputation but does it really extend that far? > > Yes. Filtering out "obvious" spam is easy. Recognizing "obvious" > 1:1 email between regular correspondents isn't too hard. > Various mailbox providers exhibit varying degrees of dealing with the issue on a message basis rather than a stream basis. To the extent that messages are diverted to a spam folder, we can infer the confidence level that the receiver has in the capability of their systems. That is, if they felt they could analyze individual messages perfectly thent here wouldn't be a need for a spam folder. Ham would get delierd and spam would go to /dev/null. > Dealing with the big grey area in the middle is hard, and sender > reputation is about the only thing that gives you anything to base > delivery decisions on there, so for the big middle ground of email > good reputation will take you a long way. > Excellent response Steve (As usual) and I agree with you about the middle. Remember the context of the discussion though. The triggering question was whether one can make decisions directly based on SPF, DKIM or a combination of the two. Doug stated that one could not. My position is that you generally can (combinations) for domains that have good control over the mailstreams (particularly abused brands such as financials). From john@jlc.net Tue Jan 25 17:53:33 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 152DF3A6906 for ; Tue, 25 Jan 2011 17:53:33 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.358 X-Spam-Level: X-Spam-Status: No, score=-106.358 tagged_above=-999 required=5 tests=[AWL=0.241, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lhybm8ZdlY5h for ; Tue, 25 Jan 2011 17:53:31 -0800 (PST) Received: from mailhost.jlc.net (mailhost.jlc.net [199.201.159.4]) by core3.amsl.com (Postfix) with ESMTP id 74EB13A6902 for ; Tue, 25 Jan 2011 17:53:31 -0800 (PST) Received: by mailhost.jlc.net (Postfix, from userid 104) id 155A833C31; Tue, 25 Jan 2011 20:56:31 -0500 (EST) Date: Tue, 25 Jan 2011 20:56:31 -0500 From: John Leslie To: Anti-Spam Research Group - IRTF Message-ID: <20110126015631.GT68297@verdi> References: <4D3DD87C.5040208@mail-abuse.org> <4D3DFF43.2020502@mail-abuse.org> <4D3E4281.8080700@mail-abuse.org> <20110125211457.GR68297@verdi> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.1i Subject: Re: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2011 01:53:33 -0000 Dotzero wrote: > On Tue, Jan 25, 2011 at 4:16 PM, Paul Ferguson wrote: >> On Tue, Jan 25, 2011 at 1:14 PM, John Leslie wrote: >> >>> Reputation (as the name implies) is a prediction of the likelihood of >>> near-future behavior. >> >> ...based on previously observed behavior. > > So, what exactly does this mean when behavior suddenly changes? The reputation changes (almost as suddenly). > If a domain or IP address (was well behaved yesterday) but begins > spewing badness today, what will your company do as an arbiter of > whether mail is accepted by your customers? If the badness is sufficient, they get a permanent error, until I get around to clearing it. If the bandess is slight, they probably get a temporary error. Ideally they fix the problem, clear the badness from their retry queue, and everything works fine a few hours later. (OK, so more often they _don't_ clear the badness from their retry queue, and their legitimate senders get SMTP warnings; but they stop getting temporary errors after a week or so anyway.) > Will you allow that domain or IP address to spew badness? I highly > doubt it. In some cases, I _do_ allow local cable providers to spew badness, but that's because they're local and my particular customers are more upset about missing one good email than receiving 100 SPAMs. This is real-world (TM), not academia... > As some point once the spewing has subsided you may (automatically > or manually) again start allowing traffic through from that domain > or IP address. Automatically is much better! > But that isn't really reputation in the traditional sense of the word. I suppose when you use a word, you are entitled to mean "just what I choose it to mean -- neither more nor less." Nonetheless, I contend this is, exactly, "prediction of the likelihood of near-future behavior." In the Internet, reputation can change in a few milliseconds. > But that brings me back to my original question. If reputation doesn't > prevent a site from getting throttled or blocked when it goes bad, > what does reputation mean? I can but guess you think of reputation as "prediction" based on ancient history (several days ago, at least). But that's foolish in an Internet where hardly anything is more than one second away. > It doesn't particularly protect the site from the immediate > consequences of going bad. Why should it? At most, I think it should give a "good" site more leeway in cleaning up the badness. (But to do that well, we need communication of what will be done and when...) > It appears that the responses are authoritative (this domain or IP > is currently emitting badness) rather than reputational (this site > has a good reputation so I will accept badness from it on the > presumption they are going to address it). Theres nothing "wrong" with the second kind of reputation service, but neither is there much demand for it. Perhaps that's because there's so little correlation between "we think they'll clean it up RealSoonNow" and the actual responsiveness. Besides, why should any of us let through known badness even if the prediction were true? Why shouldn't we instead keep giving temporary errors until the badness is removed (or expired) from their retry queue? > I will grant that there may be some small slack cut based > on reputation but does it really extend that far? In a well-designed system, it could certainly extend to stopping the temporary errors after a contracted number of hours to clean things up. (And, BTW, yes, I have gone through postfix retry queues, removing badness. It leaves me feeling _quite_ annoyed with the (ex-)customer.) -- John Leslie From dotis@mail-abuse.org Tue Jan 25 19:05:09 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 39A623A687E for ; Tue, 25 Jan 2011 19:05:09 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.531 X-Spam-Level: X-Spam-Status: No, score=-106.531 tagged_above=-999 required=5 tests=[AWL=0.068, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id P7iVFTs0nGeo for ; Tue, 25 Jan 2011 19:05:08 -0800 (PST) Received: from harry.mail-abuse.org (harry.mail-abuse.org [168.61.5.27]) by core3.amsl.com (Postfix) with ESMTP id 71F353A6840 for ; Tue, 25 Jan 2011 19:05:08 -0800 (PST) Received: from US-DOUGLASO-MB.local (gateway1.sjc.mail-abuse.org [168.61.5.81]) by harry.mail-abuse.org (Postfix) with ESMTP id C19DCA9443B for ; Wed, 26 Jan 2011 03:09:03 +0000 (UTC) Message-ID: <4D3F9012.5040607@mail-abuse.org> Date: Tue, 25 Jan 2011 19:08:02 -0800 From: Douglas Otis User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: asrg@irtf.org References: <20110122042917.83764.qmail@joyce.lan> <4D3DD87C.5040208@mail-abuse.org> <4D3DFF43.2020502@mail-abuse.org> <4D3E4281.8080700@mail-abuse.org> <20110125211457.GR68297@verdi> <21E51B60-9912-4410-BC98-B62596E28949@blighty.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2011 03:05:09 -0000 On 1/25/11 5:49 PM, Dotzero wrote: > > The triggering question was whether one can make decisions directly > based on SPF, DKIM or a combination of the two. Doug stated that one > could not. My position is that you generally can (combinations) for > domains that have good control over the mailstreams (particularly > abused brands such as financial). Mike, Correction. To be clear, may I say one can use the position of stars in the sky when making decisions. However, when making reasonable decisions about how well a provider is managing their mail resource, SPF pass or fail represents a poor measure. SPF pass offers little upon which an administrator can be judged. Malefactors are good at offering an SPF pass. Providers that handle messages where a customer's SPF record ends up failing also does not necessarily mean the administrator has done anything wrong either. Instead, one must use identifiers that accurately track resources being managed by the administrator when assessing their stewardship, not how clever their customers are in deciding which SPF records to publish. We both know customers are likely to get this wrong. Your statements about some strange need to use "-all" suffixes in these records does not help the situation either. Secondly, DKIM can be replayed and will not necessarily indicate the intended recipient of the message. Blindly using DKIM as a basis to assert a signing domain is responsible for sending unsolicited bulk email may prove highly unfair when replayed by some malefactor. When mitigating SPAM, both SPF and DKIM represent poor and often misused tools. In addition, seldom are both DKIM and SPF required for message acceptance, which leaves either mechanism wide-open to abuse. A requirement that both mechanisms pass would reduce the integrity of email delivery, and make recipients unhappy about their lost messages. Use of cryptographic authentication of SMTP clients supported by DANE resource records can operate within IPv6, and then offer truly effective tools able to fairly aid in the mitigation of SPAM. Use of third-party authorization would also permit policies able to cope with how email is normally used, without loosing track of the administrator's resource management. -Doug From dotzero@gmail.com Tue Jan 25 19:57:51 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 9DCEC3A68E7 for ; Tue, 25 Jan 2011 19:57:51 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.404 X-Spam-Level: X-Spam-Status: No, score=-3.404 tagged_above=-999 required=5 tests=[AWL=0.195, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8WKguvdnH6C1 for ; Tue, 25 Jan 2011 19:57:50 -0800 (PST) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by core3.amsl.com (Postfix) with ESMTP id 687C13A689B for ; Tue, 25 Jan 2011 19:57:50 -0800 (PST) Received: by qwj9 with SMTP id 9so551499qwj.13 for ; Tue, 25 Jan 2011 20:00:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type:content-transfer-encoding; bh=7Z2UUEiUCWliOyb/0QApTf1aHwgo79rIXXi+tcUG9vc=; b=ZlXlgjFZXFbKFc8YdzMejgxTuvvihGIpkyalbG6tuRYTwlQ2eSHTpBOv/KqixCA6uL G+IktQDStlZOo0EITgggcSkyCUxkw6mRMeHlptGQS0Byi4SmJG6DDWD/kyu9/ixYQdji n7K/AZsTDMgMhZ37lQGlQyx8QcESRMrxDJJ8w= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=WfCyRQaUGyTc12bkRHq6SMpAtorueLMvl38MqFUSCA/WpoBhmxmNYW+sXSAk9/2OVh VV5qi8y/7QUKHBlO+jm3SV5q4turWBIse8B4iVR+xsfoD6RcGbjvv1y8vjcAbJljdG6W gzBexzjz0lV6F11+Th2ls/xRg1hRnWPbbCqqk= MIME-Version: 1.0 Received: by 10.229.248.198 with SMTP id mh6mr5586523qcb.5.1296014449321; Tue, 25 Jan 2011 20:00:49 -0800 (PST) Received: by 10.229.188.200 with HTTP; Tue, 25 Jan 2011 20:00:49 -0800 (PST) In-Reply-To: <4D3F9012.5040607@mail-abuse.org> References: <20110122042917.83764.qmail@joyce.lan> <4D3DD87C.5040208@mail-abuse.org> <4D3DFF43.2020502@mail-abuse.org> <4D3E4281.8080700@mail-abuse.org> <20110125211457.GR68297@verdi> <21E51B60-9912-4410-BC98-B62596E28949@blighty.com> <4D3F9012.5040607@mail-abuse.org> Date: Tue, 25 Jan 2011 23:00:49 -0500 Message-ID: From: Dotzero To: Anti-Spam Research Group - IRTF Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2011 03:57:51 -0000 On Tue, Jan 25, 2011 at 10:08 PM, Douglas Otis wrote= : > On 1/25/11 5:49 PM, Dotzero wrote: >> >> =A0The triggering question was whether one can make decisions directly >> =A0based on SPF, DKIM or a combination of the two. Doug stated that one >> =A0could not. My position is that you generally can (combinations) for >> =A0domains that have good control over the mailstreams (particularly >> =A0abused brands such as financial). > > Mike, > > Correction. =A0To be clear, may I say one can use the position of stars i= n the > sky when making decisions. > And if you are navigating at night, that would be a useful thing to do - particularly if you have a sextant. > However, when making reasonable decisions about how well a provider is > managing their mail resource, SPF pass or fail represents a poor measure. All SPF does is provide a "link" between the IP address/host and what the domain (through a DNS record) indicates email for that domain is emitted. As I have previously said, it is an enabler. > =A0SPF pass offers little upon which an administrator can be judged. > =A0Malefactors are good at offering an SPF pass. =A0Providers that handle > messages where a customer's SPF record ends up failing also does not > necessarily mean the administrator has done anything wrong either. Context is everything. SPF pass means that the message (transport layer) is tied to the domain through the DNS record. I'm going to do a little flip flop here and say that if the behavior of the domain/IP address is known then then an SPF pass can and does mean something until the domain/IP address goes bad (the other discussion we have been having). As far as an SPF fail, it depends on the domain, the published record and the message itself to determine whether the administrator has done anything wrong. >Instead, > one must use identifiers that accurately track resources being managed by > the administrator when assessing their stewardship, not how clever their > customers are in deciding which SPF records to publish. I'm not clear on your distinction between administrators and customers. >=A0We both know > customers are likely to get this wrong. =A0Your statements about some str= ange > need to use "-all" suffixes in these records does not help the situation > either. > Either you have a confidence in what you are doing and publishing or you don't. I'll take a -all over a (naked) +all any day of the week. > Secondly, DKIM can be replayed and will not necessarily indicate the > intended recipient of the message. =A0Blindly using DKIM as a basis to as= sert > a signing domain is responsible for sending unsolicited bulk email may pr= ove > highly unfair when replayed by some malefactor. I agree that there are some potential issues with replay attacks (for example the double header issue). But as with your concerns about SPF and multiple lookup DOS attacks, we haven't seen them. Thee are much easier abuse vectors. >When mitigating SPAM, both > SPF and DKIM represent poor and often misused tools. I'm not so sure about the "poor" part but I'll grant you the misused. Is there any protocol on the internet that we can point to as not having examples of poor implementations? One slight digression, as we discuss DKIM, I am only commenting on 1st party signatures, not 3rd party signatures. >In addition, seldom > are both DKIM and SPF required for message acceptance, which leaves eithe= r > mechanism wide-open to abuse. =A0A requirement that both mechanisms pass = would > reduce the integrity of email delivery, and make recipients unhappy about > their lost messages. > Don't think of it as a requirement that both pass. For mailstreams that purport to be from a particular domain, what does a double pass tell you with regard to the likelyhood that a message with a double pass originated from the domain it purports to be (At both the transport and message level). > Use of cryptographic authentication of SMTP clients supported by DANE > resource records can operate within IPv6, and then offer truly effective > tools able to fairly aid in the mitigation of SPAM. =A0Use of third-party > authorization would also permit policies able to cope with how email is > normally used, without loosing track of the administrator's resource > management. > Let's just require everyone to use PKI and certs..... I'm not holding my br= eath. From vesely@tana.it Wed Jan 26 04:29:38 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 2FEA93A69B2 for ; Wed, 26 Jan 2011 04:29:38 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.103 X-Spam-Level: X-Spam-Status: No, score=-5.103 tagged_above=-999 required=5 tests=[AWL=-0.384, BAYES_00=-2.599, HELO_EQ_IT=0.635, HOST_EQ_IT=1.245, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KBluQxUUSrto for ; Wed, 26 Jan 2011 04:29:35 -0800 (PST) Received: from wmail.tana.it (www.tana.it [62.94.243.226]) by core3.amsl.com (Postfix) with ESMTP id 037613A685D for ; Wed, 26 Jan 2011 04:29:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tana.it; s=test; t=1296045150; bh=yLhMSSOdOphqIyFtx+QS5gTGA72rlsKuRL0efMOitJc=; l=1098; h=Message-ID:Date:From:MIME-Version:To:References:In-Reply-To: Content-Transfer-Encoding; b=Sy/BOg+0dRxeDZufwk/aSGTb0B2aESosO8xLHfTbErommPmbLEV3uU/0f8UTmS3gS bPKPLVxmmI32lVG+w+0MkoAfrHn8IDuBSWMQm5syX95pX1k9G9PDO2FlBAdQsdtu4Q Ux31PNoXWfJUr8ftL9w6huYLkCDSkuG1M++8EAKs= Received: from [172.25.197.158] (pcale.tana [172.25.197.158]) (AUTH: CRAM-MD5 515, TLS: TLS1.0,256bits,RSA_AES_256_CBC_SHA1) by wmail.tana.it with ESMTPSA; Wed, 26 Jan 2011 13:32:30 +0100 id 00000000005DC039.000000004D40145E.000030E3 Message-ID: <4D40145E.1090004@tana.it> Date: Wed, 26 Jan 2011 13:32:30 +0100 From: Alessandro Vesely User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101207 Lightning/1.0b2 Thunderbird/3.1.7 MIME-Version: 1.0 To: asrg@irtf.org References: <20110122042917.83764.qmail@joyce.lan> <4D3DD87C.5040208@mail-abuse.org> <4D3DFF43.2020502@mail-abuse.org> <4D3E4281.8080700@mail-abuse.org> <20110125211457.GR68297@verdi> In-Reply-To: <20110125211457.GR68297@verdi> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: Re: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2011 12:29:38 -0000 On 25/Jan/11 22:14, John Leslie wrote: > Dotzero wrote: >> Doug, could you share with us what reputation is? > > Reputation (as the name implies) is a prediction of the likelihood of > near-future behavior. Better yet, a prediction of the likelihood of a certain kind of response to complaints and abuse reports. Any host might be taken down, hijacked, circumvented, or otherwise abused for sending spam, what makes a difference is whether it has an abuse team who is willing and capable of countering that. _Reputation_ is a measure of such will and capacity. On 25/Jan/11 20:29, Douglas Otis wrote: > Reputation Definition: An assessment of an identifier that > accurately reflects the behavior of the resource administrator. > For example, these identifiers might be the address of an SMTP > client or the AS of the network provider. Let's not confuse authN and authZ: Identifiers are related to the former, assessments to the latter. Doug's example alludes to the problem of determining the worthier (more reputable) point in a hierarchy. From vesely@tana.it Wed Jan 26 04:35:27 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 5B2213A68AB for ; Wed, 26 Jan 2011 04:35:27 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.09 X-Spam-Level: X-Spam-Status: No, score=-5.09 tagged_above=-999 required=5 tests=[AWL=-0.371, BAYES_00=-2.599, HELO_EQ_IT=0.635, HOST_EQ_IT=1.245, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OTZkxZifpLEj for ; Wed, 26 Jan 2011 04:35:26 -0800 (PST) Received: from wmail.tana.it (wmail.tana.it [62.94.243.226]) by core3.amsl.com (Postfix) with ESMTP id 359FA3A685D for ; Wed, 26 Jan 2011 04:35:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tana.it; s=test; t=1296045506; bh=Zj8q8XM/22RjhKmqS+QlmeOgRy5eWjBYbLPhQjQswX8=; l=723; h=Message-ID:Date:From:MIME-Version:To:References:In-Reply-To: Content-Transfer-Encoding; b=LGEkubmkzHB241HwgErqvBIhZ2xwvroF1FRLIriZ+mXZv+vX6Pk4qCv59JriAXK5t U9FnbbyINIrohbHOFbLVmF8Qj+WkbtjvAa5MGZ1ygkTEf3fyis3sMn5VYbPrggQaXk STlTJkvorHyHiNBVJkt9wYW0LP0ULTVlps7YMWws= Received: from [172.25.197.158] (pcale.tana [172.25.197.158]) (AUTH: CRAM-MD5 515, TLS: TLS1.0,256bits,RSA_AES_256_CBC_SHA1) by wmail.tana.it with ESMTPSA; Wed, 26 Jan 2011 13:38:26 +0100 id 00000000005DC039.000000004D4015C2.00004D55 Message-ID: <4D4015C2.3090103@tana.it> Date: Wed, 26 Jan 2011 13:38:26 +0100 From: Alessandro Vesely User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101207 Lightning/1.0b2 Thunderbird/3.1.7 MIME-Version: 1.0 To: asrg@irtf.org References: <4D3DD87C.5040208@mail-abuse.org> <4D3DFF43.2020502@mail-abuse.org> <4D3E4281.8080700@mail-abuse.org> <20110125211457.GR68297@verdi> <20110126015631.GT68297@verdi> In-Reply-To: <20110126015631.GT68297@verdi> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: Re: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2011 12:35:27 -0000 On 26/Jan/11 02:56, John Leslie wrote: > Dotzero wrote: >> On Tue, Jan 25, 2011 at 4:16 PM, Paul Ferguson wrote: >>> On Tue, Jan 25, 2011 at 1:14 PM, John Leslie wrote: >>> >>>> Reputation (as the name implies) is a prediction of the likelihood of >>>> near-future behavior. >>> >>> ...based on previously observed behavior. >> >> So, what exactly does this mean when behavior suddenly changes? > > The reputation changes (almost as suddenly). No, it shouldn't. Although delays in fixing problems may affect it, reputation measures the willingness and the ability to deal with them, rather than the mere frequency at which problems occur. From graeme@graemef.net Wed Jan 26 04:54:39 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id D512B3A685D for ; Wed, 26 Jan 2011 04:54:39 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id s52R+5SyEy0J for ; Wed, 26 Jan 2011 04:54:39 -0800 (PST) Received: from boom.graemef.net (boom.graemef.net [82.113.154.29]) by core3.amsl.com (Postfix) with ESMTP id EA0C03A63C9 for ; Wed, 26 Jan 2011 04:54:38 -0800 (PST) Received: from cpc13-shep11-2-0-cust235.8-3.cable.virginmedia.com ([86.24.149.236] helo=[192.168.7.241]) by boom.graemef.net with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1Pi4wK-0005uZ-Q1 for asrg@irtf.org; Wed, 26 Jan 2011 12:57:38 +0000 From: Graeme Fowler To: Anti-Spam Research Group - IRTF In-Reply-To: <4D4015C2.3090103@tana.it> References: <4D3DD87C.5040208@mail-abuse.org> <4D3DFF43.2020502@mail-abuse.org> <4D3E4281.8080700@mail-abuse.org> <20110125211457.GR68297@verdi> <20110126015631.GT68297@verdi> <4D4015C2.3090103@tana.it> Content-Type: text/plain; charset="UTF-8" Date: Wed, 26 Jan 2011 12:57:32 +0000 Message-ID: <1296046652.3163.12.camel@ernie.internal.graemef.net> Mime-Version: 1.0 X-Mailer: Evolution 2.28.3 Content-Transfer-Encoding: 7bit Subject: Re: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2011 12:54:39 -0000 On Wed, 2011-01-26 at 13:38 +0100, Alessandro Vesely wrote: > No, it shouldn't. Although delays in fixing problems may affect it, > reputation measures the willingness and the ability to deal with them, > rather than the mere frequency at which problems occur. I (and no doubt several others) beg to differ. You may define "reputation" in that way, but that definition cannot be automated. At $workplace I use Exim to measure "reputation" - or "badness", or possibly even "goodness" - as defined by a function of the rate of invalid recipients, messages classified as spam, messages classified as malware, unverifiable sender domains (per DNS lookups, not callbacks), relay attempts, and a number of other indicators. If that function returns a value under a certain threshold as compared to the volume of "good" and "bad" mail arriving, that message is deferred. Currently we don't reject - but we could change the threshold, and do that instead. There's no way for me to factor in human values such as "willingness" or "ability" into that function; nor should I need to. If the data stream from a given source is rubbish, I don't want it - and yes, that can hurt legitimate messages (and has done so) but that's a chance we've been willing to take. So in my system, "reputation" is automated according to rates of hits on certain criteria. This is just as valid a system as a manual one, and should not be dismissed because it doesn't use human value judgements. Graeme From john@jlc.net Wed Jan 26 08:16:00 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id EB3833A67D6 for ; Wed, 26 Jan 2011 08:16:00 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.364 X-Spam-Level: X-Spam-Status: No, score=-106.364 tagged_above=-999 required=5 tests=[AWL=0.235, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qS0a3pKZE6xs for ; Wed, 26 Jan 2011 08:16:00 -0800 (PST) Received: from mailhost.jlc.net (mailhost.jlc.net [199.201.159.4]) by core3.amsl.com (Postfix) with ESMTP id 058003A67D2 for ; Wed, 26 Jan 2011 08:16:00 -0800 (PST) Received: by mailhost.jlc.net (Postfix, from userid 104) id A98A233C3C; Wed, 26 Jan 2011 11:19:00 -0500 (EST) Date: Wed, 26 Jan 2011 11:19:00 -0500 From: John Leslie To: Anti-Spam Research Group - IRTF Message-ID: <20110126161900.GW68297@verdi> References: <20110122042917.83764.qmail@joyce.lan> <4D3DD87C.5040208@mail-abuse.org> <4D3DFF43.2020502@mail-abuse.org> <4D3E4281.8080700@mail-abuse.org> <20110125211457.GR68297@verdi> <4D40145E.1090004@tana.it> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4D40145E.1090004@tana.it> User-Agent: Mutt/1.4.1i Subject: Re: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2011 16:16:01 -0000 Alessandro Vesely wrote: > On 25/Jan/11 22:14, John Leslie wrote: > >> Reputation (as the name implies) is a prediction of the likelihood of >> near-future behavior. > > Better yet, a prediction of the likelihood of a certain kind of > response to complaints and abuse reports. I am sympathetic to that, but that too can change on short notice; and many admins aren't volunteering to blindly accept the spam load until that runs its course. Bot-herders, if they know that timeframe, could easily adjust their spam-runs to finish before the deadline, leaving no actual useful action for the admins to take. (And the time-frame to actually _clear_ a botted computer is insanely long...) This is why I look to temporary errors as the appropriate action during an incident. > Any host might be taken down, hijacked, circumvented, or otherwise > abused for sending spam, what makes a difference is whether it has > an abuse team who is willing and capable of countering that. That does make _an_ important difference, but not one sufficient for most admins to blindly accept the spam load during an incident. > _Reputation_ is a measure of such will and capacity. "Your Network / Your Rules" applies, of course... But to me, "reputation" needs to suffer during the interim before corrective action is completed. OTOH, in the absence of a useful abuse-team, reputation shold _stay_ bad for a rather long time. IMHO, of course... Which is why I consider "vouching" services so important. -- John Leslie From dotzero@gmail.com Wed Jan 26 08:21:45 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 27B4B3A67D6 for ; Wed, 26 Jan 2011 08:21:45 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.428 X-Spam-Level: X-Spam-Status: No, score=-3.428 tagged_above=-999 required=5 tests=[AWL=0.171, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rptBJTZq8ihY for ; Wed, 26 Jan 2011 08:21:44 -0800 (PST) Received: from mail-qy0-f175.google.com (mail-qy0-f175.google.com [209.85.216.175]) by core3.amsl.com (Postfix) with ESMTP id E26EC3A67D2 for ; Wed, 26 Jan 2011 08:21:43 -0800 (PST) Received: by qyk8 with SMTP id 8so5146874qyk.13 for ; Wed, 26 Jan 2011 08:24:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type:content-transfer-encoding; bh=HpAi6eptKlR+hnNa1WrT5gEdASFtr7stMWiIy/OMIjc=; b=FlPmvYYZEcNbj/4taw7NCv8aWzbvFGrTY9CwO2WvUAMAy84jxwD+3+vsTCbAU6PydS e/hnZh2w0vMTjt9+D2TlVDwtC+uvuPwe4CFemLGhiSqRFfvA8n7Xct/wdMghr4JGB8OD 5At6IBMKV9KskN/NcXUjWkfKcnRa+Eh93qWFY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=VA1406yM78HLJ88ISrvCz0H+YLPlBXDV8X2tqGRw3ntKjTwqSG5GFX7OHASCnlDTy+ sJ+ffGHMXqMy4iEdUEwv2uDPBmKCA0xc0Ht4dIeUs1MkenRF7FtkprLdEREBzdAbOFiQ LhMaR4sRTzYAfIBWMv8mLfe4JIEkpBKxiBB/4= MIME-Version: 1.0 Received: by 10.229.231.21 with SMTP id jo21mr558107qcb.119.1296059084387; Wed, 26 Jan 2011 08:24:44 -0800 (PST) Received: by 10.229.188.200 with HTTP; Wed, 26 Jan 2011 08:24:44 -0800 (PST) In-Reply-To: <20110126015631.GT68297@verdi> References: <4D3DD87C.5040208@mail-abuse.org> <4D3DFF43.2020502@mail-abuse.org> <4D3E4281.8080700@mail-abuse.org> <20110125211457.GR68297@verdi> <20110126015631.GT68297@verdi> Date: Wed, 26 Jan 2011 11:24:44 -0500 Message-ID: From: Dotzero To: Anti-Spam Research Group - IRTF Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2011 16:21:45 -0000 On Tue, Jan 25, 2011 at 8:56 PM, John Leslie wrote: ! > >> But that isn't really reputation in the traditional sense of the word. > > =A0 I suppose when you use a word, you are entitled to mean "just what I > choose it to mean -- neither more nor less." Nonetheless, I contend > this is, exactly, "prediction of the likelihood of near-future behavior." > > =A0 In the Internet, reputation can change in a few milliseconds. > And this goes straight to one of the big issues with any discussion that uses the term "reputation". I'm going to disagree with you John when you argue that everyone gets to define what reputation is in their own way. If this were the case then any discussion of reputation is meaningless. We need a common definition. Do you get to choose what you mean when you say IP Address? What you or anyone else gets to define for yourself are the attributes that surround your use of the word reputation. That is, the implementation that utilizes reputation. From vesely@tana.it Wed Jan 26 08:30:17 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 5BA6C3A67DF for ; Wed, 26 Jan 2011 08:30:17 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.077 X-Spam-Level: X-Spam-Status: No, score=-5.077 tagged_above=-999 required=5 tests=[AWL=-0.358, BAYES_00=-2.599, HELO_EQ_IT=0.635, HOST_EQ_IT=1.245, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wAL8MsUhciwD for ; Wed, 26 Jan 2011 08:30:16 -0800 (PST) Received: from wmail.tana.it (mail.tana.it [62.94.243.226]) by core3.amsl.com (Postfix) with ESMTP id 1B6B83A67D6 for ; Wed, 26 Jan 2011 08:30:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tana.it; s=test; t=1296059596; bh=oh7Ss6k5LJNZJvKmX9obwXCCAPwBdvzrUEdzlCX8Om8=; l=1181; h=Message-ID:Date:From:MIME-Version:To:References:In-Reply-To: Content-Transfer-Encoding; b=C0Gmtupk62tPqY+MIMAPnizYtY4ncWUuIgFXTR8dYV5HY4MLkSVGjhdCyPpRgsTI2 xVqi6U4qNhCz/jhjnUUNljNZ5oYGVsdllCOmkkoXRNiQNqu1Q6klYPAUcvPTyHvaBj Rcefcr6XRaNKKoan9kNq9Ze6QX0u3NE3n5DGWG24= Received: from [172.25.197.158] (pcale.tana [172.25.197.158]) (AUTH: CRAM-MD5 515, TLS: TLS1.0,256bits,RSA_AES_256_CBC_SHA1) by wmail.tana.it with ESMTPSA; Wed, 26 Jan 2011 17:33:16 +0100 id 00000000005DC039.000000004D404CCC.0000108E Message-ID: <4D404CCB.8030304@tana.it> Date: Wed, 26 Jan 2011 17:33:15 +0100 From: Alessandro Vesely User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101207 Lightning/1.0b2 Thunderbird/3.1.7 MIME-Version: 1.0 To: asrg@irtf.org References: <4D3DD87C.5040208@mail-abuse.org> <4D3DFF43.2020502@mail-abuse.org> <4D3E4281.8080700@mail-abuse.org> <20110125211457.GR68297@verdi> <20110126015631.GT68297@verdi> <4D4015C2.3090103@tana.it> <1296046652.3163.12.camel@ernie.internal.graemef.net> In-Reply-To: <1296046652.3163.12.camel@ernie.internal.graemef.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: Re: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2011 16:30:17 -0000 On 26/Jan/11 13:57, Graeme Fowler wrote: > On Wed, 2011-01-26 at 13:38 +0100, Alessandro Vesely wrote: >> No, it shouldn't. Although delays in fixing problems may affect it, >> reputation measures the willingness and the ability to deal with them, >> rather than the mere frequency at which problems occur. > > I (and no doubt several others) beg to differ. > > You may define "reputation" in that way, but that definition cannot be > automated. Not easily, but possibly... > At $workplace I use Exim to measure "reputation" - or "badness", or > possibly even "goodness" - as defined by a function of [behavior. ...] > > So in my system, "reputation" is automated according to rates of hits on > certain criteria. This is just as valid a system as a manual one, and > should not be dismissed because it doesn't use human value judgements. I'm not proposing to dismiss that. However, I think "goodness" is a better term than "reputation" for such function. I propose to use the term "reputation" for the value upon which one decides whether to send an abuse report to the relevant entity. (Contrast that with deferring or rejecting mail from it.) From john@jlc.net Wed Jan 26 08:49:41 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 203123A680C for ; Wed, 26 Jan 2011 08:49:40 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.369 X-Spam-Level: X-Spam-Status: No, score=-106.369 tagged_above=-999 required=5 tests=[AWL=0.230, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id T+y+WhT+5J3P for ; Wed, 26 Jan 2011 08:49:38 -0800 (PST) Received: from mailhost.jlc.net (mailhost.jlc.net [199.201.159.4]) by core3.amsl.com (Postfix) with ESMTP id 2985F3A6801 for ; Wed, 26 Jan 2011 08:49:38 -0800 (PST) Received: by mailhost.jlc.net (Postfix, from userid 104) id 4B0BF33C4B; Wed, 26 Jan 2011 11:52:39 -0500 (EST) Date: Wed, 26 Jan 2011 11:52:39 -0500 From: John Leslie To: Anti-Spam Research Group - IRTF Message-ID: <20110126165239.GY68297@verdi> References: <4D3DFF43.2020502@mail-abuse.org> <4D3E4281.8080700@mail-abuse.org> <20110125211457.GR68297@verdi> <20110126015631.GT68297@verdi> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.1i Subject: Re: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2011 16:49:41 -0000 Dotzero wrote: > On Tue, Jan 25, 2011 at 8:56 PM, John Leslie wrote: >> >> In the Internet, reputation can change in a few milliseconds. > > And this goes straight to one of the big issues with any discussion > that uses the term "reputation". I'm going to disagree with you John > when you argue that everyone gets to define what reputation is in > their own way. I really don't know how to stop them... > If this were the case then any discussion of reputation is meaningless. If you believe that, I strongly urge you to stop trying. > We need a common definition. Do you get to choose what you mean when > you say IP Address? To the degree that some of us mean IPv4 only, yes. :^( > What you or anyone else gets to define for yourself are the attributes > that surround your use of the word reputation. That is, the > implementation that utilizes reputation. IMHO, any Reputation service _necessarily_ gets to define what it means by "reputation". (Otherwise, I see no reason to use any particular one.) -- John Leslie From john@jlc.net Wed Jan 26 08:58:24 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 457093A68D7 for ; Wed, 26 Jan 2011 08:58:24 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.374 X-Spam-Level: X-Spam-Status: No, score=-106.374 tagged_above=-999 required=5 tests=[AWL=0.225, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id g+UFc01fxM8U for ; Wed, 26 Jan 2011 08:58:23 -0800 (PST) Received: from mailhost.jlc.net (mailhost.jlc.net [199.201.159.4]) by core3.amsl.com (Postfix) with ESMTP id 617653A682D for ; Wed, 26 Jan 2011 08:58:23 -0800 (PST) Received: by mailhost.jlc.net (Postfix, from userid 104) id A11FE33C57; Wed, 26 Jan 2011 12:01:24 -0500 (EST) Date: Wed, 26 Jan 2011 12:01:24 -0500 From: John Leslie To: Anti-Spam Research Group - IRTF Message-ID: <20110126170124.GZ68297@verdi> References: <4D3E4281.8080700@mail-abuse.org> <20110125211457.GR68297@verdi> <20110126015631.GT68297@verdi> <4D4015C2.3090103@tana.it> <1296046652.3163.12.camel@ernie.internal.graemef.net> <4D404CCB.8030304@tana.it> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4D404CCB.8030304@tana.it> User-Agent: Mutt/1.4.1i Subject: Re: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2011 16:58:24 -0000 Alessandro Vesely wrote: > > I propose to use the term "reputation" for the value upon which one > decides whether to send an abuse report to the relevant entity. For many of us, that's the null set. (I propose that abuse reports are useful when sent to a reputation service by a customer of that service, and leave others to argue whether they have any value beyond that. Thus I'm going to hang onto the name "reputation service" for any such service.) Now, I'd be happy to use the term "vouching service" for something determining whether abuse reports would be helpful. My experience, BTW, shows them to hardly ever be helpful (except to a limited number of SMTP-sending domains, for which the occasion to send one is rather rare). > (Contrast that with deferring or rejecting mail from it.) I agree, it's a different animal. And I would agree, knowing when an abuse report will help _would_ be nice -- I'm just not holding my breath. -- John Leslie From vesely@tana.it Wed Jan 26 09:09:19 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id E437E3A684A for ; Wed, 26 Jan 2011 09:09:19 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.066 X-Spam-Level: X-Spam-Status: No, score=-5.066 tagged_above=-999 required=5 tests=[AWL=-0.347, BAYES_00=-2.599, HELO_EQ_IT=0.635, HOST_EQ_IT=1.245, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EUvV5GHUWaEB for ; Wed, 26 Jan 2011 09:09:18 -0800 (PST) Received: from wmail.tana.it (www.tana.it [62.94.243.226]) by core3.amsl.com (Postfix) with ESMTP id 397B43A6845 for ; Wed, 26 Jan 2011 09:09:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tana.it; s=test; t=1296061938; bh=TOF/6duBMli5zVxPQRMzLm/vtkDap15jWlZSZnQjF0k=; l=1440; h=Message-ID:Date:From:MIME-Version:To:References:In-Reply-To: Content-Transfer-Encoding; b=Y2XZ4B5/bPK7ORBOnYzzpGEKdnP41FGfq0jLWQznbq7l7XwonN3YiUNkEY7/MPwaA L7e7QNoI3PQfnK8I/bohym+YftS7MrDqzmettLPdEOBWR6wWniUNFqWiNDDTlfZHfA DWcfcZxKI9GMYMIFURkikx1jL8Or0k2eYDV0BfVA= Received: from [172.25.197.158] (pcale.tana [172.25.197.158]) (AUTH: CRAM-MD5 515, TLS: TLS1.0,256bits,RSA_AES_256_CBC_SHA1) by wmail.tana.it with ESMTPSA; Wed, 26 Jan 2011 18:12:18 +0100 id 00000000005DC042.000000004D4055F2.0000440F Message-ID: <4D4055F2.3040906@tana.it> Date: Wed, 26 Jan 2011 18:12:18 +0100 From: Alessandro Vesely User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101207 Lightning/1.0b2 Thunderbird/3.1.7 MIME-Version: 1.0 To: asrg@irtf.org References: <20110122042917.83764.qmail@joyce.lan> <4D3DD87C.5040208@mail-abuse.org> <4D3DFF43.2020502@mail-abuse.org> <4D3E4281.8080700@mail-abuse.org> <20110125211457.GR68297@verdi> <4D40145E.1090004@tana.it> <20110126161900.GW68297@verdi> In-Reply-To: <20110126161900.GW68297@verdi> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: Re: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2011 17:09:20 -0000 On 26/Jan/11 17:19, John Leslie wrote: > Alessandro Vesely wrote: >> Any host might be taken down, hijacked, circumvented, or otherwise >> abused for sending spam, what makes a difference is whether it has >> an abuse team who is willing and capable of countering that. > > That does make _an_ important difference, but not one sufficient > for most admins to blindly accept the spam load during an incident. Agreed. Whether to accept, filter, or reject should also depend on other factors. >> _Reputation_ is a measure of such will and capacity. > > "Your Network / Your Rules" applies, of course... Yes, for the behavior. "Reputation", as a term, does imply some agreement among the people who define its value for the relevant subject. This is a value that we want to share. > But to me, "reputation" needs to suffer during the interim before > corrective action is completed. Yes, if it takes too long. > OTOH, in the absence of a useful abuse-team, reputation shold _stay_ > bad for a rather long time. IMHO, of course... Yes, absolutely. > Which is why I consider "vouching" services so important. I too. However, I suddenly get struck dumb when I ask myself how would I manage in order to vouch for someone, what feedback would I need? Possibly, _all_ of the people who interact with an entity should contribute to the definition of its reputation. From CLEWIS@nortel.com Wed Jan 26 09:13:52 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id C6D573A6889 for ; Wed, 26 Jan 2011 09:13:52 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 884ybxZCVQEl for ; Wed, 26 Jan 2011 09:13:51 -0800 (PST) Received: from zrtps0kp.nortel.com (zrtps0kp.nortel.com [47.140.192.56]) by core3.amsl.com (Postfix) with ESMTP id 6CD4F3A681A for ; Wed, 26 Jan 2011 09:13:51 -0800 (PST) Received: from zrtphxs1.corp.nortel.com (zrtphxs1.corp.nortel.com [47.140.202.46]) by zrtps0kp.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id p0QHGnF28829 for ; Wed, 26 Jan 2011 17:16:49 GMT Received: from zrtphx5h0.corp.nortel.com ([47.140.202.65]) by zrtphxs1.corp.nortel.com with Microsoft SMTPSVC(6.0.3790.4675); Wed, 26 Jan 2011 12:16:47 -0500 Received: from [47.141.64.132] (47.141.64.132) by zrtphx5h0.corp.nortel.com (47.140.202.65) with Microsoft SMTP Server (TLS) id 8.1.340.0; Wed, 26 Jan 2011 12:16:47 -0500 Message-ID: <4D4056FE.7090806@nortel.com> Date: Wed, 26 Jan 2011 12:16:46 -0500 From: "Chris Lewis" Organization: Nortel User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.23) Gecko/20090812 Thunderbird/2.0.0.23 Mnenhy/0.7.6.666 MIME-Version: 1.0 To: References: <4D3DD87C.5040208@mail-abuse.org> <4D3DFF43.2020502@mail-abuse.org> <4D3E4281.8080700@mail-abuse.org> <20110125211457.GR68297@verdi> <20110126015631.GT68297@verdi> In-Reply-To: Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 26 Jan 2011 17:16:47.0779 (UTC) FILETIME=[D0424730:01CBBD7C] Subject: Re: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2011 17:13:52 -0000 On 1/26/2011 11:24 AM, Dotzero wrote: > On Tue, Jan 25, 2011 at 8:56 PM, John Leslie wrote: > ! >> >>> But that isn't really reputation in the traditional sense of the word. >> >> I suppose when you use a word, you are entitled to mean "just what I >> choose it to mean -- neither more nor less." Nonetheless, I contend >> this is, exactly, "prediction of the likelihood of near-future behavior." >> >> In the Internet, reputation can change in a few milliseconds. >> > > And this goes straight to one of the big issues with any discussion > that uses the term "reputation". I'm going to disagree with you John > when you argue that everyone gets to define what reputation is in > their own way. If this were the case then any discussion of reputation > is meaningless. We need a common definition. Do you get to choose what > you mean when you say IP Address? Everybody gets to define what a reputation means to them. It does in the non-online world, the on-line world should be no different. Other than that, you rely on the dictionary for what a reputation _is_, which leaves virtually infinite scope for details. Reputation: A vague notion of something that may imply whether you want a given email or not, which could be based on, but not limited to, goodness, badness, neutralness, desirability, lawsuit-happiness, subject matter, shoe size or hair color in perhaps more than one not-necessarily orthogonal dimension, and generally not necessarily conformant with anyone else's notions, criteria or implementation. > What you or anyone else gets to define for yourself are the attributes > that surround your use of the word reputation. That is, the > implementation that utilizes reputation. There's a confusion between the term "reputation" as embodied by what we normally mean by what a 3rd party could theoretically supply, and that potentially very broad combination of things that's used to derive a consolidated "reputation" at the time that a singular implementation making a real-time decision about a given email will use. My real-time decisions, for example, depend on a complex combination of 3rd party yes/nos, 3rd party "maybes/maybe nots", "what it looks like", corporate policy, phone calls from management, rumors and lies, and just plain curmudgeonly curmudgeonness. And we wouldn't have it any other way. There's no point in discussing the latter, because people will do what they gotta do/feel like. Nor should we constrain them. Secondly, I don't think there's much point in proscribing what the former is based upon/means, because there are as many ways as there are 3rd parties that it can be done and what a given "reputation" actually means precisely. It will always be up to the user (the receiver) to decide what secret (or not-so-secret) incantations, sacrificial goats and transformations of such "reputations" that will go into the final decision. The provider should explain what they're trying to impart (which may vary from one provider to another), but of necessity, the quality, coverage and semantics will differ between providers. Just as they do, for example, between the SBL and XBL and PBL. Each has different semantics, and recommended usage of the three _differ_. Do we want to standardize reputation service protocols? It'd be nice, but it may well be a complicated spec in order to preserve the semantic nuances of the provider's offerings. Better to define a generic protocol that provides for the user being able to parameterize what a given reputation stream means to them (hopefully with the guidance of the provider). You'll even need to be flexible in what the "entity this reputation is about" means. From vesely@tana.it Wed Jan 26 10:42:32 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id CAAAA3A6842 for ; Wed, 26 Jan 2011 10:42:32 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.755 X-Spam-Level: X-Spam-Status: No, score=-4.755 tagged_above=-999 required=5 tests=[AWL=-0.636, BAYES_00=-2.599, HELO_EQ_IT=0.635, HOST_EQ_IT=1.245, J_CHICKENPOX_54=0.6, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BUqMxCvQvHe5 for ; Wed, 26 Jan 2011 10:42:31 -0800 (PST) Received: from wmail.tana.it (wmail.tana.it [62.94.243.226]) by core3.amsl.com (Postfix) with ESMTP id 8ED3F3A6826 for ; Wed, 26 Jan 2011 10:42:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tana.it; s=test; t=1296067528; bh=PzoURCLmj+fykLvU2sCHW9aXzL66Vi4nEK2TowtweZc=; l=1788; h=Message-ID:Date:From:MIME-Version:To:References:In-Reply-To: Content-Transfer-Encoding; b=jPAk952CanTechV/9G3yu126W9c9L8TqlUZiQdAf5ljVyCaBqnbMXMAZ8GAU6HIoW yfC2MT4F9O1zRhLAQ1ID1RyqoFhSbokJMX/MTyonB8sle/f481x5huG+NjnRt90zS7 rUo2OduqL/Qxa/GpK+kT06+Y3qCS3vhvftSnwpRM= Received: from [172.25.197.158] (pcale.tana [172.25.197.158]) (AUTH: CRAM-MD5 515, TLS: TLS1.0,256bits,RSA_AES_256_CBC_SHA1) by wmail.tana.it with ESMTPSA; Wed, 26 Jan 2011 19:45:28 +0100 id 00000000005DC04A.000000004D406BC8.000072C9 Message-ID: <4D406BC8.5040101@tana.it> Date: Wed, 26 Jan 2011 19:45:28 +0100 From: Alessandro Vesely User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101207 Lightning/1.0b2 Thunderbird/3.1.7 MIME-Version: 1.0 To: asrg@irtf.org References: <4D3E4281.8080700@mail-abuse.org> <20110125211457.GR68297@verdi> <20110126015631.GT68297@verdi> <4D4015C2.3090103@tana.it> <1296046652.3163.12.camel@ernie.internal.graemef.net> <4D404CCB.8030304@tana.it> <20110126170124.GZ68297@verdi> In-Reply-To: <20110126170124.GZ68297@verdi> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: Re: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2011 18:42:32 -0000 On 26/Jan/11 18:01, John Leslie wrote: > Alessandro Vesely wrote: >> >> I propose to use the term "reputation" for the value upon which one >> decides whether to send an abuse report to the relevant entity. > > For many of us, that's the null set. Alas. > (I propose that abuse reports are useful when sent to a reputation > service by a customer of that service, and leave others to argue > whether they have any value beyond that. Thus I'm going to hang onto > the name "reputation service" for any such service.) That's a first step toward a constructive definition of "reputation". Your latter proposal can be changed by blurring the "customer of" membership, while enlarging a reputation service by adding more and more customers. For example, a site might host several virtual reputation services by assigning higher factors to reports sent by customers of many of them. But then, are we going to also call "reputation" the quality that one should have in order to become a customer of those services? Alternatively, we can imagine to send abuse reports to, say, abuse@arin.net, and have them forward them to the relevant ISP (or RIR) who, in turn, will forward them to the relevant ADMD, and so on. Then, we can imagine to optimize the resulting traffic by skipping some of the initial hops and directly addressing a target with a sufficiently high reputation. Both approaches should result in the same thing: a global reputation service. > Now, I'd be happy to use the term "vouching service" for something > determining whether abuse reports would be helpful. This may be a useful distinction. I'd still consider that vouch=none for a given domain should mean it has a pathologically low reputation. From dotzero@gmail.com Wed Jan 26 11:14:22 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 199BB3A6810 for ; Wed, 26 Jan 2011 11:14:22 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.847 X-Spam-Level: X-Spam-Status: No, score=-2.847 tagged_above=-999 required=5 tests=[AWL=-0.448, BAYES_00=-2.599, J_CHICKENPOX_52=0.6, J_CHICKENPOX_54=0.6, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GbKSE17apSzx for ; Wed, 26 Jan 2011 11:14:21 -0800 (PST) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by core3.amsl.com (Postfix) with ESMTP id 568FA3A680D for ; Wed, 26 Jan 2011 11:14:21 -0800 (PST) Received: by qwj9 with SMTP id 9so1279324qwj.13 for ; Wed, 26 Jan 2011 11:17:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type:content-transfer-encoding; bh=jMkJF5NxH3dWcjfoEx6z7qvGjCHzvX+y0YkrEmIrm20=; b=HpyOeDxB6Xix3WRW41Md2DDIq5dz0vDVPib9Y4PuNgtqxl/Va+KdOZUczyhpIKZCpv hKsyhd1ULruTFi/UdkrTHC6xp2ma6tvwi0/Hywpi7ADyvbH940fjihBJpVaN+z48gFNB HLi81kJdAsbgr+m9gVkJoVzPL23EiQvSPRcB8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=fD+RXzU8onzIXowIPpCrAJYR1CevIAlndhPIX8mj2Y+oBVKkxRRXkSQHAp2KjaAbY+ cJ/qVZN6s2rJPX42onwjyN4IPI4JCfIC2bs3cZ4ZGH476FdgMs4xPqrPewkLo0NbI6y6 XXSAPlEB+JUQOX6Xno8i5SMekbQzA/jT8n47I= MIME-Version: 1.0 Received: by 10.229.240.66 with SMTP id kz2mr678686qcb.233.1296069441990; Wed, 26 Jan 2011 11:17:21 -0800 (PST) Received: by 10.229.188.200 with HTTP; Wed, 26 Jan 2011 11:17:21 -0800 (PST) In-Reply-To: <4D406BC8.5040101@tana.it> References: <4D3E4281.8080700@mail-abuse.org> <20110125211457.GR68297@verdi> <20110126015631.GT68297@verdi> <4D4015C2.3090103@tana.it> <1296046652.3163.12.camel@ernie.internal.graemef.net> <4D404CCB.8030304@tana.it> <20110126170124.GZ68297@verdi> <4D406BC8.5040101@tana.it> Date: Wed, 26 Jan 2011 14:17:21 -0500 Message-ID: From: Dotzero To: Anti-Spam Research Group - IRTF Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2011 19:14:22 -0000 On Wed, Jan 26, 2011 at 1:45 PM, Alessandro Vesely wrote: > > This may be a useful distinction. =A0I'd still consider that vouch=3Dnone > for a given domain should mean it has a pathologically low reputation. vouch=3Dnone might more appropriately mean that vouching isn't available (for example, not enough data). vouch=3Dno might be more appropriate for a pathologically low reputation. From dotis@mail-abuse.org Wed Jan 26 14:53:16 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 41AE73A69F3 for ; Wed, 26 Jan 2011 14:53:16 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.533 X-Spam-Level: X-Spam-Status: No, score=-106.533 tagged_above=-999 required=5 tests=[AWL=0.066, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id R6B2xoa6oCHI for ; Wed, 26 Jan 2011 14:53:15 -0800 (PST) Received: from harry.mail-abuse.org (harry.mail-abuse.org [168.61.5.27]) by core3.amsl.com (Postfix) with ESMTP id 8AE1F3A69F0 for ; Wed, 26 Jan 2011 14:53:15 -0800 (PST) Received: from US-DOUGLASO-MB.local (gateway1.sjc.mail-abuse.org [168.61.5.81]) by harry.mail-abuse.org (Postfix) with ESMTP id C2CACA94457 for ; Wed, 26 Jan 2011 22:57:09 +0000 (UTC) Message-ID: <4D40A686.7090001@mail-abuse.org> Date: Wed, 26 Jan 2011 14:56:06 -0800 From: Douglas Otis User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: asrg@irtf.org References: <20110122042917.83764.qmail@joyce.lan> <4D3DD87C.5040208@mail-abuse.org> <4D3DFF43.2020502@mail-abuse.org> <4D3E4281.8080700@mail-abuse.org> <20110125211457.GR68297@verdi> <21E51B60-9912-4410-BC98-B62596E28949@blighty.com> <4D3F9012.5040607@mail-abuse.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2011 22:53:16 -0000 On 1/25/11 8:00 PM, Dotzero wrote: > Either you have a confidence in what you are doing and publishing or > you don't. I'll take a -all over a (naked) +all any day of the week. Mike, Actually, this is wrong. While the default modifier for a mechanism is '+', when no mechanism matches and there is no 'all' suffix, the default for a naked record is '?' or neutral. See section 4.7 of RFC4408. It is not a matter of having confidence when you publish, it is how email for the domain in question is used. As a general rule, asserting '-all' will result in the loss of some valid messages. Secondly, a pass result is not modified by any 'all' suffix either. -Doug From dotis@mail-abuse.org Wed Jan 26 14:53:16 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 4A7AB3A69F0 for ; Wed, 26 Jan 2011 14:53:16 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.535 X-Spam-Level: X-Spam-Status: No, score=-106.535 tagged_above=-999 required=5 tests=[AWL=0.064, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id APYyIoGRNSLd for ; Wed, 26 Jan 2011 14:53:15 -0800 (PST) Received: from harry.mail-abuse.org (harry.mail-abuse.org [168.61.5.27]) by core3.amsl.com (Postfix) with ESMTP id 8AF7B3A69F2 for ; Wed, 26 Jan 2011 14:53:15 -0800 (PST) Received: from US-DOUGLASO-MB.local (gateway1.sjc.mail-abuse.org [168.61.5.81]) by harry.mail-abuse.org (Postfix) with ESMTP id B82A1A94458 for ; Wed, 26 Jan 2011 22:57:12 +0000 (UTC) Message-ID: <4D40A68B.2040200@mail-abuse.org> Date: Wed, 26 Jan 2011 14:56:11 -0800 From: Douglas Otis User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: asrg@irtf.org References: <4D3DD87C.5040208@mail-abuse.org> <4D3DFF43.2020502@mail-abuse.org> <4D3E4281.8080700@mail-abuse.org> <20110125211457.GR68297@verdi> <20110126015631.GT68297@verdi> <4D4056FE.7090806@nortel.com> In-Reply-To: <4D4056FE.7090806@nortel.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2011 22:53:16 -0000 On 1/26/11 9:16 AM, Chris Lewis wrote: > > Everybody gets to define what a reputation means to them. It does in > the non-online world, the on-line world should be no different. Other > than that, you rely on the dictionary for what a reputation _is_, > which leaves virtually infinite scope for details. Chris, Allow me to disagree. As an industry, the goal should be to fairly assess identifiers that are tightly associated with accountable resource administrators. In the case of an SMTP client, this could be an IP address known to be statically assigned, or a cryptographically verified host name of the server providing the service. While it might not be practical to verify every connection using cryptography, the receiver can develop their own IP address ACLs based upon their view of connections from acceptable domains. At this time, it is not practical, nor desirable, to base reputations on individual users of a resource being evaluated. When there is abuse, administrators of that resource should be contacted, and be expected to correct the problem. Since there is virtually an unlimited supply of domain names and now IP addresses, there is now a need to vet introductions of new domains (vouching). IMHO, a good role for MAAWG and its members. -Doug From iane@sussex.ac.uk Thu Jan 27 02:04:19 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id F019328C0E3 for ; Thu, 27 Jan 2011 02:04:18 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.266 X-Spam-Level: X-Spam-Status: No, score=-3.266 tagged_above=-999 required=5 tests=[AWL=-3.333, BAYES_00=-2.599, DRUGS_ERECTILE=1, DRUGS_ERECTILE_OBFU=1.5, MANGLED_VIAGRA=2.5, RCVD_IN_DNSWL_MED=-4, SARE_OBFU_VIAGRA=1.666] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Mu9RJVqSgd7c for ; Thu, 27 Jan 2011 02:04:18 -0800 (PST) Received: from lynndie.uscs.susx.ac.uk (lynndie.uscs.susx.ac.uk [139.184.14.87]) by core3.amsl.com (Postfix) with ESMTP id 143D23A6AC7 for ; Thu, 27 Jan 2011 02:04:17 -0800 (PST) Received: from lewes.staff.uscs.susx.ac.uk ([139.184.135.133]:49895) by lynndie.uscs.susx.ac.uk with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id LFOESS-000FNG-JO for asrg@irtf.org; Thu, 27 Jan 2011 10:07:40 +0000 Date: Thu, 27 Jan 2011 10:07:15 +0000 From: Ian Eiloart Sender: iane@sussex.ac.uk To: Anti-Spam Research Group - IRTF Message-ID: <0CD2E96F96F8E52D9B1DEDCE@lewes.staff.uscs.susx.ac.uk> In-Reply-To: <20110126165239.GY68297@verdi> References: <4D3DFF43.2020502@mail-abuse.org> <4D3E4281.8080700@mail-abuse.org> <20110125211457.GR68297@verdi> <20110126015631.GT68297@verdi> <20110126165239.GY68297@verdi> Originator-Info: login-id=iane; server=mail.sussex.ac.uk; token_authority=http://www.sussex.ac.uk/its/help/support.php X-Mailer: Mulberry/4.0.8 (Mac OS X) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Sussex: true X-Sussex-transport: remote_smtp Subject: Re: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jan 2011 10:04:19 -0000 --On 26 January 2011 11:52:39 -0500 John Leslie wrote: >> And this goes straight to one of the big issues with any discussion >> that uses the term "reputation". I'm going to disagree with you John >> when you argue that everyone gets to define what reputation is in >> their own way. > > I really don't know how to stop them... It's a linguistic issue. All words are defined by loose convention, and "reputation" is one of them. Sure, people can use "reputation" to mean anything they like, but some of those definitions are useful for communicating with others and some aren't. But let's not forget that defining "reputation" means two things. First, the word "reputation" has a fairly tightly constrained meaning in the English language. Those who argue that reputation can mean anything probably aren't thinking of a new candidate entry for the Oxford English Dictionary. But, second, the method of determining the reputation score that you're going to publish for any given identifier. That method can vary widely. Some operators may just viscerally hate the products or services that are associated with a given domain (vi4gra.com or microsoft.com), whereas others may have a carefully considered metric based on (say) feedback from tens of thousands of sites around the world. Those arguing that "reputation" can mean anything probably are referring to the methodology used to arrive at a reputation score. We all know what a bad reputation *is*, but what we're really interested in is how a bad reputation *is acquired*. -- Ian Eiloart IT Services, University of Sussex 01273-873148 x3148 For new support requests, see http://www.sussex.ac.uk/its/help/ From johnl@iecc.com Thu Jan 27 13:48:45 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id A5D4B28C198 for ; Thu, 27 Jan 2011 13:48:45 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -111.073 X-Spam-Level: X-Spam-Status: No, score=-111.073 tagged_above=-999 required=5 tests=[AWL=0.126, BAYES_00=-2.599, HABEAS_ACCREDITED_SOI=-4.3, RCVD_IN_BSP_TRUSTED=-4.3, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bsuicrNUlhzX for ; Thu, 27 Jan 2011 13:48:44 -0800 (PST) Received: from gal.iecc.com (gal.iecc.com [64.57.183.53]) by core3.amsl.com (Postfix) with ESMTP id D930828C19C for ; Thu, 27 Jan 2011 13:48:43 -0800 (PST) Received: (qmail 71605 invoked from network); 27 Jan 2011 21:51:47 -0000 Received: from mail1.iecc.com (64.57.183.56) by mail1.iecc.com with QMQP; 27 Jan 2011 21:51:47 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=iecc.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=185f5.4d41e8f3.k1101; i=johnl@user.iecc.com; bh=HOaW5vBaDxYGieeWqhrxaxp2R8JPZ8jiLFKd5K4btw8=; b=P6tw+xYC2psardgc2ZNLmYhloe43AgRYzB/uO/babEU8PFehPPycseQ2n0uLREtqhMcm1qqAzfJNXHSsYmDaKEfrXJvO4WZuCRBg2Ek1PerAYHKJ8uHGbtulhBZl5IrCeZSaUdZsftdkxf5YMyZecCr26Za92rYHtVSov2qLywg= DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=taugh.com; h=date:message-id:from:to:subject:in-reply-to:cc:mime-version:content-type:content-transfer-encoding:vbr-info; s=185f5.4d41e8f3.k1101; olt=johnl@user.iecc.com; bh=HOaW5vBaDxYGieeWqhrxaxp2R8JPZ8jiLFKd5K4btw8=; b=FaZKrdSCGXHbNcimz7o/bn5scgS2mU9U+mPSbf8W9sbjbP7HZMGf7/WK/0asLtllKpPRL+lWpGwqhqtsGNdt3SVV6ZNysBk5FZbiSO7DdjKtG3zy0/SeGF3GQ97G/sUDtQhHM/hrZlqVlE4LB/2QKiRfCptkgroCzrlwUQD0eRU= VBR-Info: md=iecc.com; mc=all; mv=dwl.spamhaus.org Date: 27 Jan 2011 21:51:47 -0000 Message-ID: <20110127215147.99828.qmail@joyce.lan> From: John Levine To: asrg@irtf.org In-Reply-To: <0CD2E96F96F8E52D9B1DEDCE@lewes.staff.uscs.susx.ac.uk> Organization: Cc: X-Headerized: yes Mime-Version: 1.0 Content-type: text/plain; charset=utf-8 Content-transfer-encoding: 7bit Subject: Re: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jan 2011 21:48:45 -0000 >We all know what a bad reputation *is*, but what we're really interested in >is how a bad reputation *is acquired*. Agreed. I'm also interested in what one does with the info once you have it. R's, John From mike@mtcc.com Thu Jan 27 13:51:42 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 11E8628C179 for ; Thu, 27 Jan 2011 13:51:42 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.599 X-Spam-Level: X-Spam-Status: No, score=-4.599 tagged_above=-999 required=5 tests=[AWL=-2.000, BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id oC3phYN+EEr4 for ; Thu, 27 Jan 2011 13:51:40 -0800 (PST) Received: from takifugu.mtcc.com (mtcc.com [64.142.29.208]) by core3.amsl.com (Postfix) with ESMTP id 20BE23A69DE for ; Thu, 27 Jan 2011 13:51:40 -0800 (PST) Received: from takifugu.mtcc.com (fugu.mtcc.com [64.142.29.208]) (authenticated bits=0) by takifugu.mtcc.com (8.14.3/8.14.3) with ESMTP id p0RLshGc024104 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Thu, 27 Jan 2011 13:54:43 -0800 Message-ID: <4D41E9A3.9050803@mtcc.com> Date: Thu, 27 Jan 2011 13:54:43 -0800 From: Michael Thomas User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.8.1.22) Gecko/20090605 Thunderbird/2.0.0.22 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: Anti-Spam Research Group - IRTF References: <20110127215147.99828.qmail@joyce.lan> In-Reply-To: <20110127215147.99828.qmail@joyce.lan> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; l=336; t=1296165284; x=1297029284; c=relaxed/simple; s=thundersaddle.kirkwood; h=Content-Type:From:Subject:Content-Transfer-Encoding:MIME-Version; d=mtcc.com; i=mike@mtcc.com; z=From:=20Michael=20Thomas=20 |Subject:=20Re=3A=20[Asrg]=20What=20is=20Reputation=20Servi ce |Sender:=20 |To:=20Anti-Spam=20Research=20Group=20-=20IRTF=20 |Content-Type:=20text/plain=3B=20charset=3DISO-8859-1=3B=20 format=3Dflowed |Content-Transfer-Encoding:=207bit |MIME-Version:=201.0; bh=112nUZnIb8Efi5zXlu/obdkjpcrEOkNprOf3yeJmIAU=; b=qTUMLBzpJyxANK3ldiIkUTBwqsrl9bQPMSJ5IF19vRdXI23ajeN9oDiYDp 8vCw0bTAw/eiUpLb/uKxsg3ObmnoeYLJQIL5zTILgq3WFc8qiOGInA1NMWC9 5adycqD3VgDt6yoskljtiabnGWKEOHtu9t5shg50T3fnUWtEoBNmI=; Authentication-Results: ; v=0.1; dkim=pass header.i=mike@mtcc.com ( sig from mtcc.com/thundersaddle.kirkwood verified; ); dkim-asp=pass header.From=mike@mtcc.com Subject: Re: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jan 2011 21:51:42 -0000 On 01/27/2011 01:51 PM, John Levine wrote: >> We all know what a bad reputation *is*, but what we're really interested in >> is how a bad reputation *is acquired*. >> > Agreed. I'm also interested in what one does with the info once you > have it. > Equally important is how a bad reputation is unacquired. Mike From vesely@tana.it Fri Jan 28 01:14:15 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 205BC3A6955 for ; Fri, 28 Jan 2011 01:14:15 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.036 X-Spam-Level: X-Spam-Status: No, score=-5.036 tagged_above=-999 required=5 tests=[AWL=-0.317, BAYES_00=-2.599, HELO_EQ_IT=0.635, HOST_EQ_IT=1.245, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SzrNHVjIn3j8 for ; Fri, 28 Jan 2011 01:14:13 -0800 (PST) Received: from wmail.tana.it (www.tana.it [62.94.243.226]) by core3.amsl.com (Postfix) with ESMTP id 974673A6B09 for ; Fri, 28 Jan 2011 01:14:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tana.it; s=test; t=1296206237; bh=jBHmuI9iaEabEuyDyOyvkAyu4qV051jBBBGDa4ltNcM=; l=526; h=Message-ID:Date:From:MIME-Version:To:References:In-Reply-To: Content-Transfer-Encoding; b=Nmr9UInsIEiifw4YkOSDxW7EDczS8FJmccpd/R0yNcoHo06Thq1bqStROMNJcfTGo Ib6exPqJrWZ/Ip5+466EJFAWKk9fIzylBxCL8mqg/AU3goTYeGOx4r2MoUBIGGZzvQ 4rHHQC7xpRtEtb2X4YpanZQlH4i8c4HAK8Mt9Lfw= Received: from [172.25.197.158] (pcale.tana [172.25.197.158]) (AUTH: CRAM-MD5 515, TLS: TLS1.0,256bits,RSA_AES_256_CBC_SHA1) by wmail.tana.it with ESMTPSA; Fri, 28 Jan 2011 10:17:17 +0100 id 00000000005DC04A.000000004D42899D.00001DDB Message-ID: <4D42899E.5080805@tana.it> Date: Fri, 28 Jan 2011 10:17:18 +0100 From: Alessandro Vesely User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101207 Lightning/1.0b2 Thunderbird/3.1.7 MIME-Version: 1.0 To: asrg@irtf.org References: <20110127215147.99828.qmail@joyce.lan> <4D41E9A3.9050803@mtcc.com> In-Reply-To: <4D41E9A3.9050803@mtcc.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: Re: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2011 09:14:15 -0000 On 27/Jan/11 22:54, Michael Thomas wrote: > On 01/27/2011 01:51 PM, John Levine wrote: >>> We all know what a bad reputation *is*, but what we're really >>> interested in is how a bad reputation *is acquired*. >> >> Agreed. I'm also interested in what one does with the info once you >> have it. We can do two sorts of things: share it, and use it for decision making. > Equally important is how a bad reputation is unacquired. Isn't that the same as acquiring good reputation? But whom from, in each case? From Jose-Marcio.Martins@mines-paristech.fr Fri Jan 28 01:37:13 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 1364C3A6BFD for ; Fri, 28 Jan 2011 01:37:13 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.249 X-Spam-Level: X-Spam-Status: No, score=-2.249 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_FR=0.35] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UmOmmLnrCCMa for ; Fri, 28 Jan 2011 01:37:12 -0800 (PST) Received: from boipeva.ensmp.fr (smtp-1.ensmp.fr [194.214.158.101]) by core3.amsl.com (Postfix) with ESMTP id F3C7828C0F3 for ; Fri, 28 Jan 2011 01:37:10 -0800 (PST) Received: from localhost.localdomain (joe.j-chkmail.org [88.168.143.55]) (authenticated bits=0) by boipeva.ensmp.fr (8.14.3/8.14.3/JMMC-11/Mar/2010) with ESMTP id p0S9eFQ4028829 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 28 Jan 2011 10:40:15 +0100 (MET) Message-ID: <4D428ED7.8070008@mines-paristech.fr> Date: Fri, 28 Jan 2011 10:39:35 +0100 From: Jose-Marcio Martins da Cruz User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.16) Gecko/20110104 Fedora/2.0.11-2.fc14 SeaMonkey/2.0.11 MIME-Version: 1.0 To: Anti-Spam Research Group - IRTF References: <20110127215147.99828.qmail@joyce.lan> <4D41E9A3.9050803@mtcc.com> <4D42899E.5080805@tana.it> In-Reply-To: <4D42899E.5080805@tana.it> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Miltered: at boipeva with ID 4D428EFF.000 by Joe's j-chkmail (http : // j-chkmail dot ensmp dot fr)! X-j-chkmail-Enveloppe: 4D428EFF.000 from joe.j-chkmail.org/joe.j-chkmail.org/88.168.143.55/localhost.localdomain/ X-Scores-Stats: 4D428EFF.000 B=0.00493 L=0.04841 G=0.00025 Agree X-j-chkmail-Score: MSGID : 4D428EFF.000 on boipeva.ensmp.fr : j-chkmail score : . : R=. U=. O=# B=0.000 -> S=0.000 X-j-chkmail-Status: Ham Subject: Re: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Jose-Marcio.Martins@mines-paristech.fr, Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2011 09:37:13 -0000 Alessandro Vesely wrote: > On 27/Jan/11 22:54, Michael Thomas wrote: >> On 01/27/2011 01:51 PM, John Levine wrote: >>>> We all know what a bad reputation *is*, but what we're really >>>> interested in is how a bad reputation *is acquired*. >>> >>> Agreed. I'm also interested in what one does with the info once you >>> have it. > > We can do two sorts of things: share it, and use it for decision making. I think an additionnal value could be attached to reputation - let me call it a confidence value. It's the same reasonning that in statistics - you can estimate a value of same variable but you should also estimate its confidence : how well your "estimation" represents the "real" value of that variable. And this may be important in decision making. Confidence value is something intuitive. One reason, *but not the only one*, is well known for all of us : some RBLs are more reliable than others. Another reason is the amount of data collected before assigning a bad reputation to someone. > >> Equally important is how a bad reputation is unacquired. > > Isn't that the same as acquiring good reputation? But whom from, in > each case? No. Unacquaring a bad reputation may meant acquiring a neutral reputation. From iane@sussex.ac.uk Fri Jan 28 03:08:46 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id BD9143A67B2 for ; Fri, 28 Jan 2011 03:08:46 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.229 X-Spam-Level: X-Spam-Status: No, score=-6.229 tagged_above=-999 required=5 tests=[AWL=0.370, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kjGPVSimIBkH for ; Fri, 28 Jan 2011 03:08:45 -0800 (PST) Received: from sivits.uscs.susx.ac.uk (sivits.uscs.susx.ac.uk [139.184.14.88]) by core3.amsl.com (Postfix) with ESMTP id 728D63A67AA for ; Fri, 28 Jan 2011 03:08:45 -0800 (PST) Received: from lewes.staff.uscs.susx.ac.uk ([139.184.135.133]:64762) by sivits.uscs.susx.ac.uk with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id LFQCGA-000IBY-F8 for asrg@irtf.org; Fri, 28 Jan 2011 11:12:10 +0000 Date: Fri, 28 Jan 2011 11:11:50 +0000 From: Ian Eiloart Sender: iane@sussex.ac.uk To: Anti-Spam Research Group - IRTF Message-ID: In-Reply-To: <20110127215147.99828.qmail@joyce.lan> References: <20110127215147.99828.qmail@joyce.lan> Originator-Info: login-id=iane; server=mail.sussex.ac.uk; token_authority=http://www.sussex.ac.uk/its/help/support.php X-Mailer: Mulberry/4.0.8 (Mac OS X) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Sussex: true X-Sussex-transport: remote_smtp Subject: Re: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2011 11:08:46 -0000 --On 27 January 2011 21:51:47 +0000 John Levine wrote: >> We all know what a bad reputation *is*, but what we're really interested >> in is how a bad reputation *is acquired*. > > Agreed. I'm also interested in what one does with the info once you > have it. Well, yes. But the communication between a reputation service provider, and a reputation service consumer doesn't need to discuss that. The provider might give some hints about how they imagine the information being useful, but in the end it's up to the consumer to decide what to do with the information. For example, I think the "RFC ignorant" lists were supposed to be used as blacklists, but some people ended up using them as whitelists! The key thing is that the consumer knows enough about the service that they can apply it to their own benefit. > R's, > John > _______________________________________________ > Asrg mailing list > Asrg@irtf.org > http://www.irtf.org/mailman/listinfo/asrg -- Ian Eiloart IT Services, University of Sussex 01273-873148 x3148 For new support requests, see http://www.sussex.ac.uk/its/help/ From iane@sussex.ac.uk Fri Jan 28 03:14:32 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id BD8F63A67B2 for ; Fri, 28 Jan 2011 03:14:32 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.266 X-Spam-Level: X-Spam-Status: No, score=-6.266 tagged_above=-999 required=5 tests=[AWL=0.333, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qaFdqrAGuhfs for ; Fri, 28 Jan 2011 03:14:31 -0800 (PST) Received: from sivits.uscs.susx.ac.uk (sivits.uscs.susx.ac.uk [139.184.14.88]) by core3.amsl.com (Postfix) with ESMTP id 9CC6C3A63D2 for ; Fri, 28 Jan 2011 03:14:31 -0800 (PST) Received: from lewes.staff.uscs.susx.ac.uk ([139.184.135.133]:64825) by sivits.uscs.susx.ac.uk with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id LFQCPX-000KZA-G5 for asrg@irtf.org; Fri, 28 Jan 2011 11:17:57 +0000 Date: Fri, 28 Jan 2011 11:17:37 +0000 From: Ian Eiloart Sender: iane@sussex.ac.uk To: Anti-Spam Research Group - IRTF Message-ID: <845FE748023A768D06F9FB08@lewes.staff.uscs.susx.ac.uk> In-Reply-To: <4D42899E.5080805@tana.it> References: <20110127215147.99828.qmail@joyce.lan> <4D41E9A3.9050803@mtcc.com> <4D42899E.5080805@tana.it> Originator-Info: login-id=iane; server=mail.sussex.ac.uk; token_authority=http://www.sussex.ac.uk/its/help/support.php X-Mailer: Mulberry/4.0.8 (Mac OS X) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Sussex: true X-Sussex-transport: remote_smtp Subject: Re: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2011 11:14:32 -0000 --On 28 January 2011 10:17:18 +0100 Alessandro Vesely wrote: > >> Equally important is how a bad reputation is unacquired. > > Isn't that the same as acquiring good reputation? But whom from, in > each case? > ____________ It's not quite the same. Reputation scores can be positive, negative or zero. And, they might not be simply trinary, but graded - say as percentages. Unaquiring bad reputation might simply be a matter of going back to neutral reputation, at least for a probationary period. And, it could be gradual. For example, you might imagine that a domain's reputation would be badly hit by a system compromise, but gradually recovers over a period of a few days. -- Ian Eiloart IT Services, University of Sussex 01273-873148 x3148 For new support requests, see http://www.sussex.ac.uk/its/help/ From mike@mtcc.com Fri Jan 28 07:20:13 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 7109B3A68A4 for ; Fri, 28 Jan 2011 07:20:13 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.599 X-Spam-Level: X-Spam-Status: No, score=-3.599 tagged_above=-999 required=5 tests=[AWL=-1.000, BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id O4x5kbLVKEPt for ; Fri, 28 Jan 2011 07:20:12 -0800 (PST) Received: from takifugu.mtcc.com (mtcc.com [64.142.29.208]) by core3.amsl.com (Postfix) with ESMTP id 66E913A687D for ; Fri, 28 Jan 2011 07:20:11 -0800 (PST) Received: from takifugu.mtcc.com (fugu.mtcc.com [64.142.29.208]) (authenticated bits=0) by takifugu.mtcc.com (8.14.3/8.14.3) with ESMTP id p0SFNGlh001704 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Fri, 28 Jan 2011 07:23:16 -0800 Message-ID: <4D42DF64.1090904@mtcc.com> Date: Fri, 28 Jan 2011 07:23:16 -0800 From: Michael Thomas User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.8.1.22) Gecko/20090605 Thunderbird/2.0.0.22 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: Anti-Spam Research Group - IRTF References: <20110127215147.99828.qmail@joyce.lan> <4D41E9A3.9050803@mtcc.com> <4D42899E.5080805@tana.it> In-Reply-To: <4D42899E.5080805@tana.it> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; l=1686; t=1296228197; x=1297092197; c=relaxed/simple; s=thundersaddle.kirkwood; h=Content-Type:From:Subject:Content-Transfer-Encoding:MIME-Version; d=mtcc.com; i=mike@mtcc.com; z=From:=20Michael=20Thomas=20 |Subject:=20Re=3A=20[Asrg]=20What=20is=20Reputation=20Servi ce |Sender:=20 |To:=20Anti-Spam=20Research=20Group=20-=20IRTF=20 |Content-Type:=20text/plain=3B=20charset=3DISO-8859-1=3B=20 format=3Dflowed |Content-Transfer-Encoding:=207bit |MIME-Version:=201.0; bh=wSjqO5YmvbyX63c1C3IKVh49y3mRP21tM9W7izwV6H4=; b=LLy0SUahSxmui+OJFwy8KlmHIKky4az3iYYl1sJaCMdxj+eHcNyjVvp1Iz dqf8kPwmRA3ADdV2/AgeFdrFKvuIqe712wmdSCvQKZCHatMHFV8m3IhL8/RK nW20/j/QiCcwW29Pb2ncn4uPFF0zHqRSUa6Ht0gHDoL/Hl+y/siRE=; Authentication-Results: ; v=0.1; dkim=pass header.i=mike@mtcc.com ( sig from mtcc.com/thundersaddle.kirkwood verified; ); dkim-asp=pass header.From=mike@mtcc.com Subject: Re: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2011 15:20:13 -0000 On 01/28/2011 01:17 AM, Alessandro Vesely wrote: > On 27/Jan/11 22:54, Michael Thomas wrote: > >> On 01/27/2011 01:51 PM, John Levine wrote: >> >>>> We all know what a bad reputation *is*, but what we're really >>>> interested in is how a bad reputation *is acquired*. >>>> >>> Agreed. I'm also interested in what one does with the info once you >>> have it. >>> > We can do two sorts of things: share it, and use it for decision making. > > >> Equally important is how a bad reputation is unacquired. >> > Isn't that the same as acquiring good reputation? But whom from, in > each case? > > From what I've seen, they seem very asymmetric. Whereas acquiring a bad reputation is automated, unacquiring a 'bad' reputation seems to generally require human intervention, pleading, and in some cases what amount to bribes. Considering that many of these reputation systems are hair triggered out of necessity, the amount of false positives is bound to be pretty high. So how to get back out of a blackhole is a whole lot more arcane than falling into one. Worse is that in some cases you simply cannot because the black hole's operators are uninterested, unavailable, unable, or in some cases just crazy. Remember that it takes nothing to set up shop as a purveyor of blackholes. So there are zillions of them each vying to be the super massive blackhole. Some of them are really good, some of them are really shady. And from what I can tell, there are a lot of mail operators who don't seem to realize or care that there are quality differences. Maybe we need a blackhole of blackholes. Mike From esa@winteri.org Fri Jan 28 07:51:16 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id E7F3F3A680E for ; Fri, 28 Jan 2011 07:51:16 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.976 X-Spam-Level: X-Spam-Status: No, score=-2.976 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RefDNlXtYhB7 for ; Fri, 28 Jan 2011 07:50:52 -0800 (PST) Received: from mail-gx0-f182.google.com (mail-gx0-f182.google.com [209.85.161.182]) by core3.amsl.com (Postfix) with ESMTP id DE1743A67A8 for ; Fri, 28 Jan 2011 07:50:51 -0800 (PST) Received: by gxk8 with SMTP id 8so1194509gxk.13 for ; Fri, 28 Jan 2011 07:53:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=winteri.org; s=google; h=domainkey-signature:mime-version:sender:x-originating-ip :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:content-type; bh=Cc8rMWXnj1AuVCydMxEsP13dx4EHIIKXJg5rgE2DmYM=; b=e7ykcyQlMwnnH4Z9+rC5FAwNW8fB3VyFXz1Oofjpy6w8py6tTyb0RrJCsfGk6YXn6H JyKaTtTMokjX36KFpe0aEqq1b7VuEg6kIPrkdisgyAd3A2BA7tlfL3uysrTMP9VSXzqY 1QgiIcFRoL2gfBrVLlhZ/q8ExUMOziddHocP4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=winteri.org; s=google; h=mime-version:sender:x-originating-ip:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type; b=H0SRM//5RxJ2SgjeML5k7C4pwFPxmk0BdOgJjP9pK2rnituJFebjSYK+cVxIPsRQuP YvAVti4XHN3A5/GK7YECiz2mLYlZ5d7cN7S29ZcTyOfr8+tmLAV79O8jALSR8jsajvlT Yakeplis1ViW+nDaAMk2qEtvMGNs7BLobC6NU= MIME-Version: 1.0 Received: by 10.151.105.11 with SMTP id h11mr4639715ybm.231.1296230037422; Fri, 28 Jan 2011 07:53:57 -0800 (PST) Sender: esa@winteri.org Received: by 10.151.113.7 with HTTP; Fri, 28 Jan 2011 07:53:57 -0800 (PST) X-Originating-IP: [91.199.43.40] In-Reply-To: <4D42DF64.1090904@mtcc.com> References: <20110127215147.99828.qmail@joyce.lan> <4D41E9A3.9050803@mtcc.com> <4D42899E.5080805@tana.it> <4D42DF64.1090904@mtcc.com> Date: Fri, 28 Jan 2011 16:53:57 +0100 X-Google-Sender-Auth: ubxuVN4lajaRdBI7fLXtAORuJ0M Message-ID: From: Esa Laitinen To: Anti-Spam Research Group - IRTF Content-Type: multipart/alternative; boundary=00151751154eed8424049aea1221 Subject: Re: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2011 15:51:17 -0000 --00151751154eed8424049aea1221 Content-Type: text/plain; charset=ISO-8859-1 On Fri, Jan 28, 2011 at 4:23 PM, Michael Thomas wrote: > From what I've seen, they seem very asymmetric. Whereas acquiring a bad > reputation is automated, unacquiring a 'bad' reputation seems to generally > require human intervention, pleading, and in some cases what > amount to bribes. Considering that many of these reputation systems are > hair triggered out of necessity, the amount of false positives is bound to > be pretty high. So how to get back out of a blackhole is a whole lot > more arcane than falling into one. > True. This actually should motivate the mail senders to take care in their e-mail hygiene so that they won't fall into that hole in the first place. Not necessarily a bad thing, IMHO. And this is due to the fact that the offenders don't change their behaviour very often, or if they do, it doesn't often last very long. > > Worse is that in some cases you simply cannot because the black hole's > operators are uninterested, unavailable, unable, or in some cases just > crazy. Remember that it takes nothing to set up shop as a purveyor of > blackholes. So there are zillions of them each vying to be the super massive > blackhole. Some of them are really good, some of them are > really shady. And from what I can tell, there are a lot of mail operators > who don't seem to realize or care that there are quality differences. > > It doesn't really hurt much if somebody is in "Crazy Al's left-handers block list": badly managed services will lose their users pretty quickly. -- Mr. Esa Laitinen Tel. +41 76 200 2870 skype/yahoo: reunaesa Blog: http://happiloppuuahistaa.blogspot.com --00151751154eed8424049aea1221 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
On Fri, Jan 28, 2011 at 4:23 PM, Michael Tho= mas <mike@mtcc.com> wrote:
>From what I've seen, they seem very asymmetric. Whereas acquiring a bad= reputation is automated, unacquiring a 'bad' reputation seems to g= enerally require human intervention, pleading, and in some cases what
amount to bribes. Considering that many of these reputation systems are hai= r triggered out of necessity, the amount of false positives is bound to be = pretty high. So how to get back out of a blackhole is a whole lot
more arcane than falling into one.

True. This actu= ally should motivate the mail senders to take care in their e-mail hygiene = so that they won't fall into that hole in the first place. Not necessar= ily a bad thing, IMHO.

And this is due to the fact that the offenders don't change their b= ehaviour very often, or if they do, it doesn't often last very long.
=A0

Worse is that in some cases you simply cannot because the black hole's = operators are uninterested, unavailable, unable, or in some cases just craz= y. =A0Remember that it takes nothing to set up shop as a purveyor of blackh= oles. So there are zillions of them each vying to be the super massive blac= khole. Some of them are really good, some of them are
really shady. And from what I can tell, there are a lot of mail operators who don't seem to realize or care that there are quality differences.

It doesn't really hurt much if somebody is = in "Crazy Al's left-handers block list": badly managed servic= es will lose their users pretty quickly.



--
Mr. Esa Laitinen
Tel. +41 7= 6 200 2870
skype/yahoo: reunaesa
Blog: http://happiloppuuahistaa.blogspot.c= om

--00151751154eed8424049aea1221-- From mike@mtcc.com Fri Jan 28 08:15:48 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 46CCB3A6830 for ; Fri, 28 Jan 2011 08:15:48 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.266 X-Spam-Level: X-Spam-Status: No, score=-3.266 tagged_above=-999 required=5 tests=[AWL=-0.667, BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HQ838IVu1qrZ for ; Fri, 28 Jan 2011 08:15:47 -0800 (PST) Received: from takifugu.mtcc.com (mtcc.com [64.142.29.208]) by core3.amsl.com (Postfix) with ESMTP id 0C9023A67C3 for ; Fri, 28 Jan 2011 08:15:46 -0800 (PST) Received: from takifugu.mtcc.com (fugu.mtcc.com [64.142.29.208]) (authenticated bits=0) by takifugu.mtcc.com (8.14.3/8.14.3) with ESMTP id p0SGIqq5024082 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Fri, 28 Jan 2011 08:18:52 -0800 Message-ID: <4D42EC6C.5090606@mtcc.com> Date: Fri, 28 Jan 2011 08:18:52 -0800 From: Michael Thomas User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.8.1.22) Gecko/20090605 Thunderbird/2.0.0.22 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: Anti-Spam Research Group - IRTF References: <20110127215147.99828.qmail@joyce.lan> <4D41E9A3.9050803@mtcc.com> <4D42899E.5080805@tana.it> <4D42DF64.1090904@mtcc.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; l=2354; t=1296231533; x=1297095533; c=relaxed/simple; s=thundersaddle.kirkwood; h=Content-Type:From:Subject:Content-Transfer-Encoding:MIME-Version; d=mtcc.com; i=mike@mtcc.com; z=From:=20Michael=20Thomas=20 |Subject:=20Re=3A=20[Asrg]=20What=20is=20Reputation=20Servi ce |Sender:=20 |To:=20Anti-Spam=20Research=20Group=20-=20IRTF=20 |Content-Type:=20text/plain=3B=20charset=3DISO-8859-1=3B=20 format=3Dflowed |Content-Transfer-Encoding:=207bit |MIME-Version:=201.0; bh=3ii0rDXNAWc7AAbOzF8Ys5heHXB6DdD1YfNxcZEbKYE=; b=EGvxpCaR8rRdlcr4E+J1uzDNc4VAVpXCk3ALS9IHwF1YQC80rhubTwKhDl 2guOnMl7gcJqsSDtiPtDE7wGCZSIvtefRY3JnuDfrTyCo+B0CS2qhfDwHSZk NjJo7seVD2MupJHNLMW1SVw9VhQxvh0alr0fReax3yybxgjFJsHds=; Authentication-Results: ; v=0.1; dkim=pass header.i=mike@mtcc.com ( sig from mtcc.com/thundersaddle.kirkwood verified; ); dkim-asp=pass header.From=mike@mtcc.com Subject: Re: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2011 16:15:48 -0000 On 01/28/2011 07:53 AM, Esa Laitinen wrote: > > On Fri, Jan 28, 2011 at 4:23 PM, Michael Thomas > wrote: > > >From what I've seen, they seem very asymmetric. Whereas acquiring > a bad reputation is automated, unacquiring a 'bad' reputation > seems to generally require human intervention, pleading, and in > some cases what > amount to bribes. Considering that many of these reputation > systems are hair triggered out of necessity, the amount of false > positives is bound to be pretty high. So how to get back out of a > blackhole is a whole lot > more arcane than falling into one. > > > True. This actually should motivate the mail senders to take care in > their e-mail hygiene so that they won't fall into that hole in the > first place. Not necessarily a bad thing, IMHO. While mail hygiene is a good thing, you can get sucked into blackholes for perfectly innocent reasons. The easiest is that you inherit address space that was in a blackhole. Getting out of the blackholes of the usual suspects isn't that hard, but like i say it's definitely not automated and can be very shady itself. > > It doesn't really hurt much if somebody is in "Crazy Al's left-handers > block list": badly managed services will lose their users pretty quickly. But not quickly enough. All it takes is a few clueless companies or isp's to use badly run blackholes to make them a nuisance. Worst of all, is when they claim you're on some blackhole but don't tell you which one, and it's not one of the ones that the various blackhole aggregator/test sites knows about. I've been having problems with some Swiss ISP that does this. But it's the whole process that lacks any standardization or best common practices as far as I can tell. If you're a legitimate sender, you really don't want to waste your time sending stuff that readers don't want to see. But the feedback loop is either nonexistent, very murky or hasn't scaled down to where medium to small (read: have day jobs doing other things than email delivery) can wire them up. It seems to me that improving that situation would dramatically help the false positive rate as well as the problem of accidentally falling into blackholes of the clueful blackhole operators. Mike From CLEWIS@nortel.com Fri Jan 28 09:10:56 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 21E283A68D2 for ; Fri, 28 Jan 2011 09:10:56 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.522 X-Spam-Level: X-Spam-Status: No, score=-2.522 tagged_above=-999 required=5 tests=[AWL=-4.078, BAYES_05=-1.11, DRUGS_ERECTILE=1, DRUGS_ERECTILE_OBFU=1.5, MANGLED_VIAGRA=2.5, RCVD_IN_DNSWL_MED=-4, SARE_OBFU_VIAGRA=1.666] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cy0njooeMCWM for ; Fri, 28 Jan 2011 09:10:55 -0800 (PST) Received: from zrtps0kp.nortel.com (zrtps0kp.nortel.com [47.140.192.56]) by core3.amsl.com (Postfix) with ESMTP id A2EE83A67B0 for ; Fri, 28 Jan 2011 09:10:54 -0800 (PST) Received: from zrtphxs1.corp.nortel.com (zrtphxs1.corp.nortel.com [47.140.202.46]) by zrtps0kp.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id p0SHDvu28994 for ; Fri, 28 Jan 2011 17:13:57 GMT Received: from zrtphx5h0.corp.nortel.com ([47.140.202.65]) by zrtphxs1.corp.nortel.com with Microsoft SMTPSVC(6.0.3790.4675); Fri, 28 Jan 2011 12:13:55 -0500 Received: from [47.141.64.156] (47.141.64.156) by zrtphx5h0.corp.nortel.com (47.140.202.65) with Microsoft SMTP Server (TLS) id 8.1.340.0; Fri, 28 Jan 2011 12:13:54 -0500 Message-ID: <4D42F951.9060102@nortel.com> Date: Fri, 28 Jan 2011 12:13:53 -0500 From: "Chris Lewis" Organization: Nortel User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.23) Gecko/20090812 Thunderbird/2.0.0.23 Mnenhy/0.7.6.666 MIME-Version: 1.0 To: References: <4D3DFF43.2020502@mail-abuse.org> <4D3E4281.8080700@mail-abuse.org> <20110125211457.GR68297@verdi> <20110126015631.GT68297@verdi> <20110126165239.GY68297@verdi> <0CD2E96F96F8E52D9B1DEDCE@lewes.staff.uscs.susx.ac.uk> In-Reply-To: <0CD2E96F96F8E52D9B1DEDCE@lewes.staff.uscs.susx.ac.uk> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-OriginalArrivalTime: 28 Jan 2011 17:13:55.0492 (UTC) FILETIME=[BE64DE40:01CBBF0E] Subject: Re: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2011 17:10:56 -0000 On 1/27/2011 5:07 AM, Ian Eiloart wrote: > But let's not forget that defining "reputation" means two things. First, > the word "reputation" has a fairly tightly constrained meaning in the > English language. Those who argue that reputation can mean anything > probably aren't thinking of a new candidate entry for the Oxford English > Dictionary. No, the existing dictionary definition may be "constrained" in one sense, but not necessarily constrained well enough for communication via a computer protocol. Computers don't find human language easy after all. Some write as if it can be adequately be represented as a single "good", "bad" or "neutral" indicator, or some scale between the three. But can it? Even the dictionary indicates it can't. Here's one dictionary definition: reputation [ˌrɛpjʊˈteɪʃən] n 1. the estimation in which a person or thing is generally held; opinion 2. a high opinion generally held about a person or thing; esteem 3. notoriety or fame, esp for some specified characteristic have a reputation to be known or notorious, esp for promiscuity, excessive drinking, or the like Note the "specified characteristic". The reputation may not necessarily be in the good-bad dimension. Or the good-bad may depend on the context in which the question is asked. Eg: could you rely on that same good-bad in the context of children versus adults? Do we _really_ believe that there's a single reputation dimension in our context, or at least, that we can "get by" with just one? I don't think so. > But, second, the method of determining the reputation score that you're > going to publish for any given identifier. That method can vary widely. > Some operators may just viscerally hate the products or services that are > associated with a given domain (vi4gra.com or microsoft.com), whereas > others may have a carefully considered metric based on (say) feedback from > tens of thousands of sites around the world. Those arguing that > "reputation" can mean anything probably are referring to the methodology > used to arrive at a reputation score. That speaks more to reputation of reputation instantiations in some sense - which is probably out of scope. But it also does imply that a reputation service may well be measuring different but still valid things, and it would be best that the reputation service be able to tell the questioner what it actually means. > We all know what a bad reputation *is*, Kinda like porn, "we know it when we see it"? ;-) Seriously, but do we? Entity A and entity B may both have validly "bad" reputations, but for entirely different reasons, even within the same reputation service, in such a way that they need to be interpreted differently. A reputation service needs to be able to impart at least some notion of context/reason so that a receiver can make an intelligent choice to respect/disregard/amplify/deprecate or whatever. I don't think a simple yes/no/dunno or even a scale is adequate. Spamhaus Zen, for example, is multi-dimensional. Even its grossly simplistic expressive power is greater than I see the road we're treading here. From rob@invaluement.com Fri Jan 28 10:13:50 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 659B93A68FA for ; Fri, 28 Jan 2011 10:13:50 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.11 X-Spam-Level: X-Spam-Status: No, score=-1.11 tagged_above=-999 required=5 tests=[BAYES_05=-1.11] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id L6JoeMZwTyN4 for ; Fri, 28 Jan 2011 10:13:49 -0800 (PST) Received: from mail.powerviewmail.com (mail.powerviewmail.com [204.9.77.40]) by core3.amsl.com (Postfix) with ESMTP id 39DA23A68F5 for ; Fri, 28 Jan 2011 10:13:49 -0800 (PST) Received: from [204.9.77.40] ([204.9.77.40]) by mail.powerviewmail.com (IceWarp 9.4.1) with ASMTP id LTS70255 for ; Fri, 28 Jan 2011 13:16:55 -0500 Message-ID: <4D430817.2010400@invaluement.com> Date: Fri, 28 Jan 2011 13:16:55 -0500 From: Rob McEwen User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: Anti-Spam Research Group - IRTF X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-PVSYS-SMTPAUTH: Y X-PVSYS-SECONDS: 0 Subject: [Asrg] IPv6 "Spam Example" Challenge X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2011 18:13:50 -0000 RE: IPv6 "Spam Example" Challenge (cross posted on SPAM-L) I'd love for someone to point out a real-world example (just one!) of a spam sent via an IPv6 ip address that meets the following criteria: (1) Has *previously* been spotted sending spam at least one time (to be sure... this would have to be in a different SMTP session, and probably at least 10+ minutes earlier) (2) The spam was NOT sent from a botnet IP --OR-- alternatively, the spam was able to get past greylisting via a retry that happened at least 10 seconds after the original attempt (so this would exclude "woodpecker" botnets that get past some greylisting measures by doing a retry less than a few seconds after the initial attempt). (3) The IP that the spam was sent from is NOT an ISP's MTA where legit mail is also sent from that IP. Likewise, it should NOT be an IP that is shared between spammers and legitimate senders. It should instead be an IP that is delegated to the sender (at the time the spam was sent...and as far as anyone can reasonably determine). (4) There should be at least one "caught red-handed" piece of evidence that the sender was sending UBE--such as sending to a harvest address, ROSKO spammer, obfuscation that only spammers would use, hidden whois records, use of domain blacklisted by surbl/uribl/etc, etc. Please include all such factors. (5) If possible, please include a copy of the spam, with headers. Feel free to obfuscate the recipients identity as well as obfuscating any alphanumeric codes that a spammer could use to trace the message back to the sender. (preferably, put it in a pastebin or text file on a server) I'm extremely curious as to whether this will be fast and easy... with numerous examples provided quickly... or whether it might be months/years before anyone can find a single such example! (maybe somewhere in between?) Any takers? -- Rob McEwen http://dnsbl.invaluement.com/ rob@invaluement.com +1 (478) 475-9032 From rob@invaluement.com Fri Jan 28 10:18:37 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id CEDF03A6918 for ; Fri, 28 Jan 2011 10:18:37 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.855 X-Spam-Level: X-Spam-Status: No, score=-1.855 tagged_above=-999 required=5 tests=[AWL=0.745, BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id beOY7YKtQi1V for ; Fri, 28 Jan 2011 10:18:36 -0800 (PST) Received: from mail.powerviewmail.com (mail.powerviewmail.com [204.9.77.40]) by core3.amsl.com (Postfix) with ESMTP id B97F33A681A for ; Fri, 28 Jan 2011 10:18:36 -0800 (PST) Received: from [204.9.77.40] ([204.9.77.40]) by mail.powerviewmail.com (IceWarp 9.4.1) with ASMTP id LTX70243 for ; Fri, 28 Jan 2011 13:21:43 -0500 Message-ID: <4D430937.8090005@invaluement.com> Date: Fri, 28 Jan 2011 13:21:43 -0500 From: Rob McEwen User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: Anti-Spam Research Group - IRTF References: <4D430817.2010400@invaluement.com> In-Reply-To: <4D430817.2010400@invaluement.com> X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-PVSYS-SMTPAUTH: Y X-PVSYS-SECONDS: 0 Subject: Re: [Asrg] IPv6 "Spam Example" Challenge X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2011 18:18:38 -0000 On 1/28/2011 1:16 PM, Rob McEwen wrote: > as well as obfuscating any > alphanumeric codes that a spammer could use to trace the message back to > the sender oops. typo. I meant "back to the original recipient" -- Rob McEwen http://dnsbl.invaluement.com/ rob@invaluement.com +1 (478) 475-9032 From vesely@tana.it Sat Jan 29 03:46:56 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 10EC83A6452 for ; Sat, 29 Jan 2011 03:46:56 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.819 X-Spam-Level: X-Spam-Status: No, score=-3.819 tagged_above=-999 required=5 tests=[AWL=-1.514, BAYES_40=-0.185, HELO_EQ_IT=0.635, HOST_EQ_IT=1.245, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gjIuivteX1PL for ; Sat, 29 Jan 2011 03:46:55 -0800 (PST) Received: from wmail.tana.it (www.tana.it [62.94.243.226]) by core3.amsl.com (Postfix) with ESMTP id DD01E3A635F for ; Sat, 29 Jan 2011 03:46:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tana.it; s=test; t=1296301798; bh=bTWfcydSmtEVID3H25L445BEkJxpMuNvPbT59z5Q1iE=; l=836; h=Message-ID:Date:From:MIME-Version:To:References:In-Reply-To: Content-Transfer-Encoding; b=JNRxW3d0wGZci70vW7+kWAq7DjJK5uuMQsxOXbCESrZdlGQdXSu64DBeuRZHJnAs6 CCK49amszAeOKufuKq/g+fGNdThCjrd3AXfEUI/SprKlYfjncOnLbbNoFgyoY0CF4Y /CpMX4S5RmffFuqbZhwPX3bhWqv859kCn9jN15JU= Received: from [172.25.197.158] (pcale.tana [172.25.197.158]) (AUTH: CRAM-MD5 515, TLS: TLS1.0,256bits,RSA_AES_256_CBC_SHA1) by wmail.tana.it with ESMTPSA; Sat, 29 Jan 2011 12:49:58 +0100 id 00000000005DC04B.000000004D43FEE6.0000788C Message-ID: <4D43FEE6.7020807@tana.it> Date: Sat, 29 Jan 2011 12:49:58 +0100 From: Alessandro Vesely User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101207 Lightning/1.0b2 Thunderbird/3.1.7 MIME-Version: 1.0 To: asrg@irtf.org References: <4D3DFF43.2020502@mail-abuse.org> <4D3E4281.8080700@mail-abuse.org> <20110125211457.GR68297@verdi> <20110126015631.GT68297@verdi> <20110126165239.GY68297@verdi> <0CD2E96F96F8E52D9B1DEDCE@lewes.staff.uscs.susx.ac.uk> <4D42F951.9060102@nortel.com> In-Reply-To: <4D42F951.9060102@nortel.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Subject: Re: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Jan 2011 11:46:56 -0000 On 28/Jan/11 18:13, Chris Lewis wrote: > Here's one dictionary definition: Let me recall there's a proposed corollary to Godwin's law http://www.irtf.org/mail-archive/web/asrg/current/msg15449.html > reputation [ˌrɛpjʊˈteɪʃən] > n > 1. the estimation in which a person or thing is generally held; opinion > 2. a high opinion generally held about a person or thing; esteem > 3. notoriety or fame, esp for some specified characteristic > have a reputation to be known or notorious, esp for promiscuity, > excessive drinking, or the like At any rate, please note the "generally held" part, which seems quite essential to me. In facts, in the off-line world, people say phrases like "despite her reputation, IME she is ". Thus, reputation would connote the shared facets of the experience. From vesely@tana.it Sat Jan 29 04:27:56 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 9CA963A67B5 for ; Sat, 29 Jan 2011 04:27:56 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.983 X-Spam-Level: X-Spam-Status: No, score=-4.983 tagged_above=-999 required=5 tests=[AWL=-0.264, BAYES_00=-2.599, HELO_EQ_IT=0.635, HOST_EQ_IT=1.245, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KWK1zMUjC2+t for ; Sat, 29 Jan 2011 04:27:54 -0800 (PST) Received: from wmail.tana.it (wmail.tana.it [62.94.243.226]) by core3.amsl.com (Postfix) with ESMTP id 73A473A677C for ; Sat, 29 Jan 2011 04:27:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tana.it; s=test; t=1296304261; bh=P4+GjTfA51PEEzd0Kr78CldWnJ5Px788HjRxGpWAuGY=; l=1852; h=Message-ID:Date:From:MIME-Version:To:References:In-Reply-To: Content-Transfer-Encoding; b=GRjDm8eD1pG/Q4LodTBhMc9w8IpGj/t6xaLaGXLN7pNoUmgOmDiZISjiruyancQFt OIvL/qjaNNDoSEC5hGTsFSgL1lpvPHmVsN0EhdRR80l3rRD4GrwY6SRBQfABL6H4ci if79gveygurKRZGaq2qoFy7a/2wosyTME4+BzBZ4= Received: from [172.25.197.158] (pcale.tana [172.25.197.158]) (AUTH: CRAM-MD5 515, TLS: TLS1.0,256bits,RSA_AES_256_CBC_SHA1) by wmail.tana.it with ESMTPSA; Sat, 29 Jan 2011 13:31:01 +0100 id 00000000005DC04C.000000004D440885.000037E8 Message-ID: <4D440885.1060104@tana.it> Date: Sat, 29 Jan 2011 13:31:01 +0100 From: Alessandro Vesely User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101207 Lightning/1.0b2 Thunderbird/3.1.7 MIME-Version: 1.0 To: asrg@irtf.org References: <20110127215147.99828.qmail@joyce.lan> <4D41E9A3.9050803@mtcc.com> <4D42899E.5080805@tana.it> <4D42DF64.1090904@mtcc.com> <4D42EC6C.5090606@mtcc.com> In-Reply-To: <4D42EC6C.5090606@mtcc.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: Re: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Jan 2011 12:27:56 -0000 On 28/Jan/11 17:18, Michael Thomas wrote: > On 01/28/2011 07:53 AM, Esa Laitinen wrote: >> It doesn't really hurt much if somebody is in "Crazy Al's >> left-handers block list": badly managed services will lose their >> users pretty quickly. > > But not quickly enough. All it takes is a few clueless companies > or isp's to use badly run blackholes to make them a nuisance. Worst > of all, is when they claim you're on some blackhole but don't tell you > which one, and it's not one of the ones that the various blackhole > aggregator/test sites knows about. I've been having problems with > some Swiss ISP that does this. We (improperly) call reputation the judgment of self appointed block list operators. Spamhaus has a fame of being very good at it, but people who've been erroneously bitten don't agree. > But it's the whole process that lacks any standardization or best > common practices as far as I can tell. If you're a legitimate sender, > you really don't want to waste your time sending stuff that readers > don't want to see. But the feedback loop is either nonexistent, very > murky or hasn't scaled down to where medium to small (read: have > day jobs doing other things than email delivery) can wire them up. Hm.. Actually, there are some papers about how to run DNSBLs. However, the semantics of a list is what it is: the unappealable opinion of its operators. Would it be better if it were the result of a poll, say, or the average opinion of thousands of mailboxes operators? > It seems to me that improving that situation would dramatically > help the false positive rate as well as the problem of accidentally > falling into blackholes of the clueful blackhole operators. That would imply a radical change in attitude, from individual survival tactics to free cooperative strategies. From john@jlc.net Sat Jan 29 05:34:02 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 36DB63A67D7 for ; Sat, 29 Jan 2011 05:34:02 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.384 X-Spam-Level: X-Spam-Status: No, score=-106.384 tagged_above=-999 required=5 tests=[AWL=0.215, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id E5ZBam276Jw3 for ; Sat, 29 Jan 2011 05:34:01 -0800 (PST) Received: from mailhost.jlc.net (mailhost.jlc.net [199.201.159.4]) by core3.amsl.com (Postfix) with ESMTP id CEAE93A67A1 for ; Sat, 29 Jan 2011 05:34:00 -0800 (PST) Received: by mailhost.jlc.net (Postfix, from userid 104) id 0D4BE33C36; Sat, 29 Jan 2011 08:37:09 -0500 (EST) Date: Sat, 29 Jan 2011 08:37:09 -0500 From: John Leslie To: Anti-Spam Research Group - IRTF Message-ID: <20110129133708.GE68297@verdi> References: <20110125211457.GR68297@verdi> <20110126015631.GT68297@verdi> <20110126165239.GY68297@verdi> <0CD2E96F96F8E52D9B1DEDCE@lewes.staff.uscs.susx.ac.uk> <4D42F951.9060102@nortel.com> <4D43FEE6.7020807@tana.it> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4D43FEE6.7020807@tana.it> User-Agent: Mutt/1.4.1i Subject: Re: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Jan 2011 13:34:02 -0000 Alessandro Vesely wrote: > > Let me recall there's a proposed corollary to Godwin's law > http://www.irtf.org/mail-archive/web/asrg/current/msg15449.html ] ] I think that as soon as you start quoting the dictionary, you've lost ] the argument. I thoroughly disagree. Perhaps quoting the dictionary "starts a different" argument, but citing a scholar's opinion on derivation and usage gives us something to focus on. IMHO, this aids the process of convergence. ] In any technical field, one needs jargon as a shorthand to refer to ] agreed concepts. Perhaps there is such a need during development -- to refer to a concept succinctly -- but it's fatal to standards-setting to use poorly-defined words, and IMHO it's at best "sloppy" in documenting research. ] When you start quoting the dictionary, you're arguing that some word ] has only its conventional meaning That would be _misquoting_ a dictionary. I presume competent writers to not need to "misquote". ] Words like responsible, accredit, certify, are evolving into jargon, ] but we haven't yet come to consensus on what if any jargon meaning ] they have. If there's _no_ consensus, using them to describe research is a bad idea. (In standard and law-writing, it's OK to include your own definitions. In describing research, such re-definitions confuse the reader.) ] The dictionary doesn't help, so leave it out of this discussion, please. I thoroughly disagree! >> reputation [??r??pj????te??????n] >> n >> 1. the estimation in which a person or thing is generally held; opinion >> 2. a high opinion generally held about a person or thing; esteem >> 3. notoriety or fame, esp for some specified characteristic >> have a reputation to be known or notorious, esp for promiscuity, >> excessive drinking, or the like For starters, that's three rather different meanings... I'd argue that the third is the one closest to the concept we're discussing. > At any rate, please note the "generally held" part, which seems quite > essential to me... ... but it goes with the first definition, not the third... > Thus, reputation would connote the shared facets of the experience. I can't think of an existent "reputation service" which aims for that. Instead, they aim at "notoriety", and try to make early calls of that. ==== Obviously, there are folks posting here who may be fixed on the first or second defitions above. I believe quoting the dictionary may be the best way to clarify the confusion of what we're actually discussing... -- John Leslie From CLEWIS@nortel.com Sat Jan 29 08:11:24 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 434B33A681E for ; Sat, 29 Jan 2011 08:11:24 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.56 X-Spam-Level: X-Spam-Status: No, score=-4.56 tagged_above=-999 required=5 tests=[AWL=2.039, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id G1TckkxDA5S5 for ; Sat, 29 Jan 2011 08:11:23 -0800 (PST) Received: from zrtps0kp.nortel.com (zrtps0kp.nortel.com [47.140.192.56]) by core3.amsl.com (Postfix) with ESMTP id EFE813A6814 for ; Sat, 29 Jan 2011 08:11:22 -0800 (PST) Received: from zrtphxs1.corp.nortel.com (casmtp.ca.nortel.com [47.140.202.46]) by zrtps0kp.nortel.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id p0TGETT14673 for ; Sat, 29 Jan 2011 16:14:29 GMT Received: from zrtphx5h0.corp.nortel.com ([47.140.202.65]) by zrtphxs1.corp.nortel.com with Microsoft SMTPSVC(6.0.3790.4675); Sat, 29 Jan 2011 11:14:27 -0500 Received: from [47.141.64.53] (47.141.64.53) by zrtphx5h0.corp.nortel.com (47.140.202.65) with Microsoft SMTP Server (TLS) id 8.1.340.0; Sat, 29 Jan 2011 11:14:27 -0500 Message-ID: <4D443CE1.8090009@nortel.com> Date: Sat, 29 Jan 2011 11:14:25 -0500 From: "Chris Lewis" Organization: Nortel User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.23) Gecko/20090812 Thunderbird/2.0.0.23 Mnenhy/0.7.6.666 MIME-Version: 1.0 To: References: <20110125211457.GR68297@verdi> <20110126015631.GT68297@verdi> <20110126165239.GY68297@verdi> <0CD2E96F96F8E52D9B1DEDCE@lewes.staff.uscs.susx.ac.uk> <4D42F951.9060102@nortel.com> <4D43FEE6.7020807@tana.it> <20110129133708.GE68297@verdi> In-Reply-To: <20110129133708.GE68297@verdi> Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 29 Jan 2011 16:14:27.0820 (UTC) FILETIME=[9A4F12C0:01CBBFCF] Subject: Re: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Jan 2011 16:11:24 -0000 On 1/29/2011 8:37 AM, John Leslie wrote: > Alessandro Vesely wrote: >> >> Let me recall there's a proposed corollary to Godwin's law >> http://www.irtf.org/mail-archive/web/asrg/current/msg15449.html > ] > ] I think that as soon as you start quoting the dictionary, you've lost > ] the argument. > > I thoroughly disagree. Perhaps quoting the dictionary "starts a > different" argument, but citing a scholar's opinion on derivation and > usage gives us something to focus on. IMHO, this aids the process of > convergence. I'd ordinarily agree with Alessandro, which is why I don't recall ever quoting dictionaries before in decades of discussions like this, but he misses the point that I was directly responding to a comment that "Reputation" has a single dictionary meaning that we should rely upon for our work here, but as I attempted and you also pointed out explicitly, there are _three_. Indeed, before we can start using a word as technical jargon, we have to understand what the jargon _means_. I'm not sure we do yet. And secondly, more to the point I elaborated on somewhat more, the third meaning is, I think, more in tune with what I think we want to do. The point being that reputation isn't a single-dimensional "good-neutral-bad" trinary or scalar thing and we should be considering whether it's usefully multi-dimensional. That makes a huge difference in what such a thing needs to be able to express. From vesely@tana.it Sun Jan 30 06:22:51 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id CB29F3A67D6 for ; Sun, 30 Jan 2011 06:22:51 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.976 X-Spam-Level: X-Spam-Status: No, score=-4.976 tagged_above=-999 required=5 tests=[AWL=-0.257, BAYES_00=-2.599, HELO_EQ_IT=0.635, HOST_EQ_IT=1.245, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mbhEhbd0Z0GI for ; Sun, 30 Jan 2011 06:22:50 -0800 (PST) Received: from wmail.tana.it (www.tana.it [62.94.243.226]) by core3.amsl.com (Postfix) with ESMTP id 8798B3A6767 for ; Sun, 30 Jan 2011 06:22:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tana.it; s=test; t=1296397560; bh=nQjbz/sIpBbZn4SkWXj/GWM+EnbkUpOwZrDOxd/b3Es=; l=2107; h=Message-ID:Date:From:MIME-Version:To:References:In-Reply-To: Content-Transfer-Encoding; b=ZTLCYHXFFeBGM2mOvcohrWuf1DfrhimLrsaDgrKKjyLyt8oGfpLbkZzClJZ1r3JZx KC3St5P9X55+pjNc2BRXVtDLLgVgCCFcAOqoLn1Nwp1jhPDTyc1OJ8V0Y3AYYpUl8p gv1BOdnF76GnNhEARub6/QVNq+y0R3KYb6QmuLa4= Received: from [172.25.197.158] (pcale.tana [172.25.197.158]) (AUTH: CRAM-MD5 515, TLS: TLS1.0,256bits,RSA_AES_256_CBC_SHA1) by wmail.tana.it with ESMTPSA; Sun, 30 Jan 2011 15:26:00 +0100 id 00000000005DC042.000000004D4574F8.00002EE1 Message-ID: <4D4574F8.8030405@tana.it> Date: Sun, 30 Jan 2011 15:26:00 +0100 From: Alessandro Vesely User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101207 Lightning/1.0b2 Thunderbird/3.1.7 MIME-Version: 1.0 To: asrg@irtf.org References: <20110125211457.GR68297@verdi> <20110126015631.GT68297@verdi> <20110126165239.GY68297@verdi> <0CD2E96F96F8E52D9B1DEDCE@lewes.staff.uscs.susx.ac.uk> <4D42F951.9060102@nortel.com> <4D43FEE6.7020807@tana.it> <20110129133708.GE68297@verdi> In-Reply-To: <20110129133708.GE68297@verdi> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: Re: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Jan 2011 14:22:52 -0000 On 29/Jan/11 14:37, John Leslie wrote: > Alessandro Vesely wrote: >>> reputation [??r??pj????te??????n] >>> n >>> 1. the estimation in which a person or thing is generally held; opinion >>> 2. a high opinion generally held about a person or thing; esteem >>> 3. notoriety or fame, esp for some specified characteristic >>> have a reputation to be known or notorious, esp for promiscuity, >>> excessive drinking, or the like > > For starters, that's three rather different meanings... I'd argue > that the third is the one closest to the concept we're discussing. > >> At any rate, please note the "generally held" part, which seems quite >> essential to me... > > ... but it goes with the first definition, not the third... The first two of them, actually. "Known or notorious" is obviously implicit in "generally held", so the third definition seems to just drop the requirement that the general public should be willing to hold the underlying judgment. (Compare this with off-line world's news services.) I don't see a real difference among those definitions, because even if we take 1 or 2, we want reputation records to be usable also by people who never had direct experience with the relevant domain/IP. For such users, definition 3 has to apply. >> Thus, reputation would connote the shared facets of the experience. > > I can't think of an existent "reputation service" which aims for that. > Instead, they aim at "notoriety", and try to make early calls of that. I have difficulties thinking of an existent "reputation service" at all :-) Indeed, much of the confusion on this term may stem from the fact that no impeccably designed reputation services exist yet. By example, are DNSBLs reputation services? Is Spamhaus DWL a reputation service? Is Spamassassin AutoWhitelist, running at a given MTA, a reputation service for the users of such MTA? If there are well established uses of the term "reputation" we cannot change its technical meaning. We may use adjectives to disambiguate, when needed. From iane@sussex.ac.uk Mon Jan 31 07:09:02 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 5D02B3A6C01 for ; Mon, 31 Jan 2011 07:09:02 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.296 X-Spam-Level: X-Spam-Status: No, score=-6.296 tagged_above=-999 required=5 tests=[AWL=0.303, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id A-cvWZz2YEQv for ; Mon, 31 Jan 2011 07:09:01 -0800 (PST) Received: from lynndie.uscs.susx.ac.uk (lynndie.uscs.susx.ac.uk [139.184.14.87]) by core3.amsl.com (Postfix) with ESMTP id 03A193A6C21 for ; Mon, 31 Jan 2011 07:08:59 -0800 (PST) Received: from lewes.staff.uscs.susx.ac.uk ([139.184.135.133]:63466) by lynndie.uscs.susx.ac.uk with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id LFW7LO-0007Z7-L3 for asrg@irtf.org; Mon, 31 Jan 2011 15:13:00 +0000 Date: Mon, 31 Jan 2011 15:12:08 +0000 From: Ian Eiloart Sender: iane@sussex.ac.uk To: Anti-Spam Research Group - IRTF Message-ID: <43F25682CB76DD2C912C3244@lewes.staff.uscs.susx.ac.uk> In-Reply-To: <4D42F951.9060102@nortel.com> References: <4D3DFF43.2020502@mail-abuse.org> <4D3E4281.8080700@mail-abuse.org> <20110125211457.GR68297@verdi> <20110126015631.GT68297@verdi> <20110126165239.GY68297@verdi> <0CD2E96F96F8E52D9B1DEDCE@lewes.staff.uscs.susx.ac.uk> <4D42F951.9060102@nortel.com> Originator-Info: login-id=iane; server=mail.sussex.ac.uk; token_authority=http://www.sussex.ac.uk/its/help/support.php X-Mailer: Mulberry/4.0.8 (Mac OS X) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Sussex: true X-Sussex-transport: remote_smtp Subject: Re: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Jan 2011 15:09:02 -0000 --On 28 January 2011 12:13:53 -0500 Chris Lewis wrote: > On 1/27/2011 5:07 AM, Ian Eiloart wrote: > >> But let's not forget that defining "reputation" means two things. First, >> the word "reputation" has a fairly tightly constrained meaning in the >> English language. Those who argue that reputation can mean anything >> probably aren't thinking of a new candidate entry for the Oxford English >> Dictionary. > > No, the existing dictionary definition may be "constrained" in one sense, > but not necessarily constrained well enough for communication via a > computer protocol. Computers don't find human language easy after all. True, but that's not the point. What matters is how one aqcuires reputation. That is, the listing policy for the reputation service. That's what needs to be communicated when someone is considering whether to subscribe to a reputation service. Computer communication isn't relevant here. > Some write as if it can be adequately be represented as a single "good", > "bad" or "neutral" indicator, or some scale between the three. That depends on what you mean by adequate. And that depends on what you're doing with the listing. If you're using it in -say- spamassassin, then you want to apply a score. A trinary result may be helpful, and the consumer of the service will weight that score accordingly. More useful would be -say- a positive or negative score out of 10 or 100 (more granualarity probably would not be meaningful). Again, the consumer would want to weight that score according to their judgement of the reliablity of the provider. > >> We all know what a bad reputation *is*, > > Kinda like porn, "we know it when we see it"? ;-) > > Seriously, but do we? Yes, but different people, as you say, will differ as to whether it's deserved. > Entity A and entity B may both have validly "bad" > reputations, but for entirely different reasons, even within the same > reputation service, in such a way that they need to be interpreted > differently. A reputation service needs to be able to impart at least > some notion of context/reason so that a receiver can make an intelligent > choice to respect/disregard/amplify/deprecate or whatever. > > I don't think a simple yes/no/dunno or even a scale is adequate. > > Spamhaus Zen, for example, is multi-dimensional. Even its grossly > simplistic expressive power is greater than I see the road we're treading > here. > _______________________________________________ > Asrg mailing list > Asrg@irtf.org > http://www.irtf.org/mailman/listinfo/asrg -- Ian Eiloart IT Services, University of Sussex 01273-873148 x3148 For new support requests, see http://www.sussex.ac.uk/its/help/ From dotis@mail-abuse.org Mon Jan 31 10:52:24 2011 Return-Path: X-Original-To: asrg@core3.amsl.com Delivered-To: asrg@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 388FC3A6A6F for ; Mon, 31 Jan 2011 10:52:24 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.536 X-Spam-Level: X-Spam-Status: No, score=-106.536 tagged_above=-999 required=5 tests=[AWL=0.063, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id B0ygIAWMWQk9 for ; Mon, 31 Jan 2011 10:52:21 -0800 (PST) Received: from harry.mail-abuse.org (harry.mail-abuse.org [168.61.5.27]) by core3.amsl.com (Postfix) with ESMTP id 84EA13A6965 for ; Mon, 31 Jan 2011 10:52:21 -0800 (PST) Received: from US-DOUGLASO-MB.local (gateway1.sjc.mail-abuse.org [168.61.5.81]) by harry.mail-abuse.org (Postfix) with ESMTP id BE4C8A9443B for ; Mon, 31 Jan 2011 18:56:33 +0000 (UTC) Message-ID: <4D4705A4.2060304@mail-abuse.org> Date: Mon, 31 Jan 2011 10:55:32 -0800 From: Douglas Otis User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: asrg@irtf.org References: <20110125211457.GR68297@verdi> <20110126015631.GT68297@verdi> <20110126165239.GY68297@verdi> <0CD2E96F96F8E52D9B1DEDCE@lewes.staff.uscs.susx.ac.uk> <4D42F951.9060102@nortel.com> <4D43FEE6.7020807@tana.it> <20110129133708.GE68297@verdi> <4D4574F8.8030405@tana.it> In-Reply-To: <4D4574F8.8030405@tana.it> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Asrg] What is Reputation Service X-BeenThere: asrg@irtf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Anti-Spam Research Group - IRTF List-Id: Anti-Spam Research Group - IRTF List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Jan 2011 18:52:24 -0000 On 1/30/11 6:26 AM, Alessandro Vesely wrote: > I have difficulties thinking of an existent "reputation service" at > all :-) Indeed, much of the confusion on this term may stem from the > fact that no impeccably designed reputation services exist yet. > > By example, are DNSBLs reputation services? An important aspect of mitigation services is being missed. This would be the IDENTIFIER being assessed, and whether it directly associates with an administrator of a service being rated for stewardship. After all, momentary abuse events are often forgiven when the administrator can be observed responding. Only administrators are able to impact the level of abuse seen on the Internet. Without administrator's crucial efforts, mitigation services would also fail, in addition to the services they defend. -Doug