
Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n6TCY4tu055819 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 29 Jul 2009 05:34:04 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n6TCY4eQ055818; Wed, 29 Jul 2009 05:34:04 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from mail-px0-f192.google.com (mail-px0-f192.google.com [209.85.216.192]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n6TCXqPr055803 for <ietf-imapext@imc.org>; Wed, 29 Jul 2009 05:34:02 -0700 (MST) (envelope-from barryleiba.mailing.lists@gmail.com)
Received: by pxi30 with SMTP id 30so479921pxi.16 for <ietf-imapext@imc.org>; Wed, 29 Jul 2009 05:33:51 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:reply-to:date:message-id :subject:from:to:content-type:content-transfer-encoding; bh=EOb+oYZ8KhCkjSo1khF/aLBdQ7t3H84eVvgpjrMdwJk=; b=Ch69Fp5FOttWTYYTfze1V2RnAgK3pMrFnTLhXctms7LuN6UKJstU2cwpckZYx1abkN guPYrau8Z3W3z3BVB4kIl4PivFB1UHcGjaCjLvhZdI3rb+y5tmVyveH1tui8ZFeVGAxW 9woLzgqt40hJHVdHhcMkIx1d3d1CQsavMwdSE=
DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:reply-to:date:message-id:subject:from:to:content-type :content-transfer-encoding; b=QYnPEzlBzxd86fMtQD8awGZGDv6Fg13zdZ5A3+1no6c0ZeHK8Ix4kTWtH/prI2Ckbp QHuvLa7LVjjb/y0n/GIWvjVqsMHYYr7p5uXMjFr8HfLn2n5Jkx56bM1ccZnbgM96FKVJ D2/XsYA8yB899i0MIKGHBUx6kVbcYCuK+N2WA=
MIME-Version: 1.0
Received: by 10.114.180.16 with SMTP id c16mr13654081waf.57.1248870831894;  Wed, 29 Jul 2009 05:33:51 -0700 (PDT)
Reply-To: barryleiba@computer.org
Date: Wed, 29 Jul 2009 08:33:51 -0400
Message-ID: <6c9fcc2a0907290533v4b96ea0co76e4cb6b0b24ca34@mail.gmail.com>
Subject: "Friends of IMAP" dinner in Stockholm on IETF Friday
From: Barry Leiba <barryleiba.mailing.lists@gmail.com>
To: morg@ietf.org, IMAP Extensions WG <ietf-imapext@imc.org>, ima@ietf.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

We'll be gathering for the traditional "Friends of IMAP" dinner, after
the IETF meeting, in Stockholm.  Because the only time we can get most
of the people together is Friday, we'll do it on Friday evening, 31
July.  We'll meet in the entrance lobby of the Clarion Sign Hotel at
19:00.

No plan, yet, about where to do.

Barry



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n6LLnU00071181 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 21 Jul 2009 14:49:31 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n6LLnU2h071180; Tue, 21 Jul 2009 14:49:30 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from orthanc.ca (orthanc.ca [208.86.224.138]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n6LLnJT7071159 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for <ietf-imapext@imc.org>; Tue, 21 Jul 2009 14:49:30 -0700 (MST) (envelope-from lyndon@orthanc.ca)
Received: from [192.168.0.42] (S01060011242eafe2.cg.shawcable.net [68.145.131.177]) (authenticated bits=0) by orthanc.ca (8.14.3/8.14.3) with ESMTP id n6LLnGSE063020 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 21 Jul 2009 15:49:17 -0600 (MDT) (envelope-from lyndon@orthanc.ca)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=orthanc.ca; s=2009-1; t=1248212958; bh=NrmKwo6Y6GZtjjabrv6/ebzejzrOkRT6W/HZakv7JOk=; h=Date:From:To:Subject:In-Reply-To:Message-ID:References: MIME-Version:Content-Type; b=N5GQepe7FeixBXyiAFnlazZ4YJxkBsdlqkEnprHM6xFF33VniUhFH/py026wJ8v9R 4YwYIo/AohIXQdsobPXsrANy7vr+KQtzkFYoDUSvHiftSE/vc1et7ntm72eqSzkyXd zOo7zMrPF3HyqTWg1KvuZayZ88KAx1BT9+8ZKd2j61+niYjNTgt+hmSW7X++TqYZxy UlFtQ5TxhoxpkFHrAyjUkyG8GKzpAyIGI/B7a5EmAuTKf6kWfpVy1huTLqzNXiOb6e DV1VCN+Xp3q7/RUBXuiivFBzYbBecRoiBnPpE9JqNPUpBryCcZH4bF57ML6/gP6XD7 GW3WlzaYl+Vaw==
Date: Tue, 21 Jul 2009 15:49:11 -0600 (MDT)
From: Lyndon Nerenberg <lyndon@orthanc.ca>
To: imap-protocol@u.washington.edu, ietf-imapext@imc.org
Subject: Re: A survey of Spam-related keyword usage by IMAP clients.
In-Reply-To: <01NB1I98RDS4001ML6@mauve.mrochek.com>
Message-ID: <alpine.BSF.2.00.0907211541390.25656@legolas.orthanc.ca>
References: <1247001507.8602.109.camel@legolas.orthanc.ca> <01NB1I98RDS4001ML6@mauve.mrochek.com>
User-Agent: Alpine 2.00 (BSF 1167 2008-08-23)
Organization: The Frobozz Magic Homing Pigeon Company
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII
X-Spam-Status: No, score=0.3 required=5.0 tests=AWL,BAYES_00,DKIM_SIGNED, DKIM_VERIFIED,RCVD_IN_PBL,RCVD_IN_SORBS_DUL autolearn=no version=3.2.5
X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on orthanc.ca
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

The response to the keyword survey has been ... underwhelming. Surely 
there is more than one client represented on the mailing lists?!?

Once again, the information we're looking for is described below. Please 
forward this to every client writer you know and ask them to respond. 
We're never going to solve this problem if these people don't start 
participating.

Please send responses directly to me.

>> It's time to get a current view of how IMAP clients are using keywords
>> to manage spam. If you are a client author I would appreciate it if you
>> could take a moment to send me a note describing which keywords your
>> client uses for spam-related processing, and what each of the keywords
>> signifies.

> I'd also really like to know if the client has "this is spam" or "this 
> is not spam" buttons in the UI, and if it does how those buttons 
> interact with the use of keywords.

> I ask this because users of any client that has such buttons are clearly 
> going to use those buttons preferentially over any other labelling 
> mechanism. And if a significant number of clients have such buttons that 
> do a particular thing - regardless of what that thing is - that's pretty 
> much going to be determinative in finding the most easily deployed 
> approach.

>> If your client does not use keywords for this purpose, that's
>> valuable information, too, as is a breakdown of functionality by
>> software version number if there are significant differences. (If you
>> use non-keyword facilities to manage spam feel free to include a short
>> description of your methods if you think the info might be relevant.)
>
> Yes please. I'm especially interested in how many clients do this by 
> resending the message to a special address.
>
>> I'm primarily interested in getting authoritative data from the client
>> authors, but others should feel free to chime in with deployment-based
>> experience of related keyword usage, but please indicate the source of
>> your comments (e.g. deployment experience, source code examination).



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n692ob3f093436 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 8 Jul 2009 19:50:37 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n692obeK093435; Wed, 8 Jul 2009 19:50:37 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from daboo.name (daboo.name [151.201.22.177]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n692oQtQ093411 for <ietf-imapext@imc.org>; Wed, 8 Jul 2009 19:50:36 -0700 (MST) (envelope-from cyrus@daboo.name)
Received: from localhost (localhost [127.0.0.1]) by daboo.name (Postfix) with ESMTP id 036E81691025; Wed,  8 Jul 2009 22:50:25 -0400 (EDT)
X-Virus-Scanned: amavisd-new at daboo.name
Received: from daboo.name ([127.0.0.1]) by localhost (chewy.mulberrymail.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1mRu4KuRy9ZF; Wed,  8 Jul 2009 22:50:23 -0400 (EDT)
Received: from [10.0.1.8] (unknown [10.0.1.1]) by daboo.name (Postfix) with ESMTP id 5D0B7169101A; Wed,  8 Jul 2009 22:50:23 -0400 (EDT)
Date: Wed, 08 Jul 2009 22:50:22 -0400
From: Cyrus Daboo <cyrus@daboo.name>
To: Ned Freed <ned.freed@mrochek.com>, George Michaelson <ggm@pobox.com>
cc: ietf-imapext@imc.org, general discussion of application-layer protocols <discuss@ietf.org>
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
Message-ID: <20A9EE8D49D98D53C36F6F48@socrates.local>
In-Reply-To: <01NB338GXYBS00007A@mauve.mrochek.com>
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <28941.1246868682.560969@puncture> <4A51D1A1.5040507@isode.com>	<01NAZVQAYWH4001DLH@mauve.mrochek.com> <47CB230E-B713-4098-82D4-318744434C46@pobox.com> <01NB0BOZYUPY001ML6@mauve.mrochek.com> <6DECD560-B257-4E6E-80E7-A99A69D297C6@pobox.com> <01NB14JAPS8E001ML6@mauve.mrochek.com> <F4A90756-9251-4CB8-9776-77F3202617A5@pobox.com> <01NB1TV04NJS001ML6@mauve.mrochek.com> <4A25E43A-83E3-4A59-9C0B-C334820275BF@pobox.com> <01NB338GXYBS00007A@mauve.mrochek.com>
X-Mailer: Mulberry/4.1.0a1 (Mac OS X)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline; size=675
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

Hi Ned,

--On July 8, 2009 5:30:06 PM -0700 Ned Freed <ned.freed@mrochek.com> wrote:

> I note, however, that now that CALDAV is complete we see Apple,
> Thunderbird, and gmail all either supporting it or working on supporting
> it.

For a more comprehensive list of implementations see: 
<http://caldav.calconnect.org/implementations.html>.

> It's a little much to expect Outlook to shift from it's currrent
> proprietary way of doing all this, but even Microsoft may eventually have
> no choice but to support the emerging standards in this area.

In the meantime the folks at <http://zideone.com> have a plugin for Outlook 
that does CalDAV and CardDAV.

-- 
Cyrus Daboo



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n690oUYQ085187 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 8 Jul 2009 17:50:30 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n690oUZD085186; Wed, 8 Jul 2009 17:50:30 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from mauve.mrochek.com (mauve.mrochek.com [66.59.230.40]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n690oIJr085155 for <ietf-imapext@imc.org>; Wed, 8 Jul 2009 17:50:28 -0700 (MST) (envelope-from ned.freed@mrochek.com)
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NB338IGJXS00QEO9@mauve.mrochek.com> for ietf-imapext@imc.org; Wed, 8 Jul 2009 17:50:16 -0700 (PDT)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NB32ESTD2800007A@mauve.mrochek.com>; Wed, 08 Jul 2009 17:50:13 -0700 (PDT)
Date: Wed, 08 Jul 2009 17:30:06 -0700 (PDT)
From: Ned Freed <ned.freed@mrochek.com>
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
In-reply-to: "Your message dated Thu, 09 Jul 2009 08:27:39 +1000" <4A25E43A-83E3-4A59-9C0B-C334820275BF@pobox.com>
To: George Michaelson <ggm@pobox.com>
Cc: Ned Freed <ned.freed@mrochek.com>, ietf-imapext@imc.org, general discussion of application-layer protocols <discuss@ietf.org>
Message-id: <01NB338GXYBS00007A@mauve.mrochek.com>
MIME-version: 1.0
Content-type: TEXT/PLAIN; format=flowed; delsp=yes
Content-transfer-encoding: 7BIT
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <28941.1246868682.560969@puncture> <4A51D1A1.5040507@isode.com> <01NAZVQAYWH4001DLH@mauve.mrochek.com> <47CB230E-B713-4098-82D4-318744434C46@pobox.com> <01NB0BOZYUPY001ML6@mauve.mrochek.com> <6DECD560-B257-4E6E-80E7-A99A69D297C6@pobox.com> <01NB14JAPS8E001ML6@mauve.mrochek.com> <F4A90756-9251-4CB8-9776-77F3202617A5@pobox.com> <01NB1TV04NJS001ML6@mauve.mrochek.com> <4A25E43A-83E3-4A59-9C0B-C334820275BF@pobox.com>
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

> >>
> >
> > The place where I'd like to see white and black listing done is in
> > CARDDAV, not
> > IMAP.
> >
> > 				Ned

> so there's only one real question: if it was standardized, would the
> likes of google implement it? How about the client side?

This is actually one of the reasons why CARDDAV is a better choice to piggyback
on.

RIght now proprietary environments like Exchange/Outlook and Lotus Notes
have a significant advantage over standards-based solutions: They provide
a fully integrated mail/calendar/addressbook client experience.

One reason standards-based solutions have a problem with this has been the lack
of open protocols for accessing and manipulating calendar and addressbook
information.

But this is changing. CALDAV addresses the calendar access issue, CARDDAV does
the same for address books. Once all the pieces are in place, and assuming we
get the details right, this will allow standards-based clients and servers all
to interoperate.

Uptake isn't going to be fast because these are complex protocols and
getting the integration right isn't easy. But for once the incentives
appear to be woring for us, not against us.

> because, unless its in gmail AND in the Apple Mail client,
> Thunderbird, Outlook, its not useful.

That may be true now, but at some point given sufficient adoption things flip
around to  where not having support for this stuff makes your client or server
not useful.

I note, however, that now that CALDAV is complete we see Apple, Thunderbird,
and gmail all either supporting it or working on supporting it. It's a little
much to expect Outlook to shift from it's currrent proprietary way of doing
all this, but even Microsoft may eventually have no choice but to support
the emerging standards in this area.

				Ned



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n68MS1Vi076663 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 8 Jul 2009 15:28:01 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n68MS1PQ076662; Wed, 8 Jul 2009 15:28:01 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from ey-out-2122.google.com (ey-out-2122.google.com [74.125.78.25]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n68MRnxn076647 for <ietf-imapext@imc.org>; Wed, 8 Jul 2009 15:28:00 -0700 (MST) (envelope-from geeohgeegeeoh@gmail.com)
Received: by ey-out-2122.google.com with SMTP id 4so1337311eyf.9 for <ietf-imapext@imc.org>; Wed, 08 Jul 2009 15:27:48 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:cc:message-id:from:to :in-reply-to:content-type:content-transfer-encoding:mime-version :subject:date:references:x-mailer; bh=TXGOqBBjbdG9nXZAFR79a2n2MsazOtTdRvdhoCvIooU=; b=NQk57XltAke0iE7kWCRT7H0wqFsmRkobkUn5XDMgS21BgacIhWqog3xlYn1RdmP6RI YLrgX6bpp9J/1f112knqqr9xnrhCrMb7raCi4GpQvvwOZZtAjv41E19s0NzKC/mQMeby eVn36eHb3AzFZyHIs/nScOpeGowqWF3ZuGX2k=
DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:cc:message-id:from:to:in-reply-to:content-type :content-transfer-encoding:mime-version:subject:date:references :x-mailer; b=ZwEzaNEjT+YsIf+t5klQYJEkYjykNSrF5W2HdSnz8c5SXePXIu6zu4YOOVKxBJCKmf ai/4sMfp3DF6ylc03AQWZTEMDR93uudksCQ9x3okpwUnv3Fmt9rFXvElZCPV2O4VRPHg Nn+b5V/J+DWWVxERWoF2yAmccVjbmIYdsO9Yg=
Received: by 10.210.109.19 with SMTP id h19mr8612445ebc.56.1247092068279; Wed, 08 Jul 2009 15:27:48 -0700 (PDT)
Received: from 192-168-1-5.tpgi.com.au (60-240-104-100-nsw-pppoe.tpgi.com.au [60.240.104.100]) by mx.google.com with ESMTPS id 7sm1176374eyg.19.2009.07.08.15.27.44 (version=TLSv1/SSLv3 cipher=RC4-MD5); Wed, 08 Jul 2009 15:27:47 -0700 (PDT)
Cc: ietf-imapext@imc.org, general discussion of application-layer protocols <discuss@ietf.org>
Message-Id: <4A25E43A-83E3-4A59-9C0B-C334820275BF@pobox.com>
From: George Michaelson <ggm@pobox.com>
To: Ned Freed <ned.freed@mrochek.com>
In-Reply-To: <01NB1TV04NJS001ML6@mauve.mrochek.com>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
Content-Transfer-Encoding: 7bit
Mime-Version: 1.0 (Apple Message framework v935.3)
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
Date: Thu, 9 Jul 2009 08:27:39 +1000
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <28941.1246868682.560969@puncture> <4A51D1A1.5040507@isode.com> <01NAZVQAYWH4001DLH@mauve.mrochek.com> <47CB230E-B713-4098-82D4-318744434C46@pobox.com> <01NB0BOZYUPY001ML6@mauve.mrochek.com> <6DECD560-B257-4E6E-80E7-A99A69D297C6@pobox.com> <01NB14JAPS8E001ML6@mauve.mrochek.com> <F4A90756-9251-4CB8-9776-77F3202617A5@pobox.com> <01NB1TV04NJS001ML6@mauve.mrochek.com>
X-Mailer: Apple Mail (2.935.3)
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

>>
>
> The place where I'd like to see white and black listing done is in  
> CARDDAV, not
> IMAP.
>
> 				Ned

so there's only one real question: if it was standardized, would the  
likes of google implement it? How about the client side?

because, unless its in gmail AND in the Apple Mail client,  
Thunderbird, Outlook, its not useful.

-G



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n68J9Bcd063429 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 8 Jul 2009 12:09:11 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n68J9BbU063428; Wed, 8 Jul 2009 12:09:11 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from mauve.mrochek.com (mauve.mrochek.com [66.59.230.40]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n68J902N063391 for <ietf-imapext@imc.org>; Wed, 8 Jul 2009 12:09:10 -0700 (MST) (envelope-from ned.freed@mrochek.com)
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NB2RBB0E4G015OWJ@mauve.mrochek.com> for ietf-imapext@imc.org; Wed, 8 Jul 2009 12:08:55 -0700 (PDT)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NB2NV4DYVK001ML6@mauve.mrochek.com>; Wed, 08 Jul 2009 12:08:52 -0700 (PDT)
Date: Wed, 08 Jul 2009 11:54:05 -0700 (PDT)
From: Ned Freed <ned.freed@mrochek.com>
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
In-reply-to: "Your message dated Wed, 08 Jul 2009 11:08:28 +0100" <4A54701C.6080107@isode.com>
To: Alexey Melnikov <alexey.melnikov@isode.com>
Cc: George Michaelson <ggm@pobox.com>, Ned Freed <ned.freed@mrochek.com>, ietf-imapext@imc.org, general discussion of application-layer protocols <discuss@ietf.org>
Message-id: <01NB2RB8UTA4001ML6@mauve.mrochek.com>
MIME-version: 1.0
Content-type: TEXT/PLAIN; format=flowed
Content-transfer-encoding: 7BIT
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <28941.1246868682.560969@puncture> <4A51D1A1.5040507@isode.com> <01NAZVQAYWH4001DLH@mauve.mrochek.com> <47CB230E-B713-4098-82D4-318744434C46@pobox.com> <01NB0BOZYUPY001ML6@mauve.mrochek.com> <6DECD560-B257-4E6E-80E7-A99A69D297C6@pobox.com> <01NB14JAPS8E001ML6@mauve.mrochek.com> <F4A90756-9251-4CB8-9776-77F3202617A5@pobox.com> <01NB1TV04NJS001ML6@mauve.mrochek.com> <4A54701C.6080107@isode.com>
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

> Ned Freed wrote:

> >> I have no desire to argue against my own proposal! sorry if I
> >> misunderstood the tenor of some comments to this suggestion.
> >
> >> The original posting asked about two things
> >
> >> 1) spam/ham
> >
> >> 2) white/black
> >
> >> so far, people seem to be focussing on 1). I acknowledge problems with
> >> 1), I appreciate that people are trying to address 1) as a possibility.
> >
> >> I asked about 2). I asked you if, ignoring 1), it was possible or
> >> plausible to send white/black information about sender, if 1) was hard
> >> or intractable.
> >
> > Of course it's possible, but this has no business being in IMAP. IMAP
> > is a
> > message access protocol, not an address handling protocol. It is also
> > already
> > quite complex, and extending its function to include what amountss to a
> > specialized address book capability is a really bad idea IMO.
> >
> > In fact a pretty good case can be made that it's inappropriate to use
> > IMAP for spam/ham classification, but at the spam/ham classification is
> > intrinsicly tied to messages in the store. The same cannot be said for
> > address white and blacklisting.

> +1

> >> Does that make more sense? I do realize white/black listing is a
> >> different PROBLEM to spam/ham marking, but neither of them currently
> >> have support directly in imap, and I imagine both of them would use
> >> IMAP extensions to pass information client-server.
> >
> > The place where I'd like to see white and black listing done is in
> > CARDDAV, not
> > IMAP.

> Or LDAP :-).

That's actually how we do it now. We're hoping to move to CARDDAV in the
future.

> There is also a Sieve extension for making use of whitelists/blacklists
> during mail delivery. See draft-melnikov-sieve-external-lists-02.txt.
> It can work with CARDDAV, LDAP and other things.

Interestingly, this extension in its current form is not compatible with
at least one of the proposals that was made here. The issue is one I've
raised before - the fact that :list is separate argument and not a match type.

Where there is clearly some utility in being able to say stuff along the
lines of:

    header :contains :list "subject" "tag:dirty-word-list"

The problem is that in order for this to work the list has to be enumerable.
Not all lists are enumerable, and even some that are are so large even though 
oyu can enumerate them in theory you can't afford to do so in practice.

One use case where this matters is when the list is a set of hashed values. The
way you find if something is "on the list" is to hash it and see if that value
appears. And even if the input string is short enough that you can enumerate
all the unique substrings, how about :matches and :regex? Good luck with
those.

Hashed lists have in fact been proposed in the present discussion as a means of
avoiding giving your address whitelist to the mail server. I happen not to
think this is a useful thing to do for a variety of reasons, mostly having to
do with address canonicalization (or lack thereof), but there are other
use cases where hashed lists make more sense.

So, although it reduces functionality, I believe :list should be a match type
and the underlying comparison type that's done should be a property of the list
itself.

				Ned



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n68A9TeU021174 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 8 Jul 2009 03:09:29 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n68A9TUq021173; Wed, 8 Jul 2009 03:09:29 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from rufus.isode.com (rufus.isode.com [62.3.217.251]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n68A9Guv021150 for <ietf-imapext@imc.org>; Wed, 8 Jul 2009 03:09:27 -0700 (MST) (envelope-from alexey.melnikov@isode.com)
Received: from [94.197.220.252] (94.197.220.252.threembb.co.uk [94.197.220.252])  by rufus.isode.com (submission channel) via TCP with ESMTPA  id <SlRwSQBV9BFk@rufus.isode.com>; Wed, 8 Jul 2009 11:09:14 +0100
Message-ID: <4A54701C.6080107@isode.com>
Date: Wed, 08 Jul 2009 11:08:28 +0100
From: Alexey Melnikov <alexey.melnikov@isode.com>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/20050915
X-Accept-Language: en-us, en
To: George Michaelson <ggm@pobox.com>
CC: Ned Freed <ned.freed@mrochek.com>, ietf-imapext@imc.org, general discussion of application-layer protocols <discuss@ietf.org>
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <28941.1246868682.560969@puncture> <4A51D1A1.5040507@isode.com> <01NAZVQAYWH4001DLH@mauve.mrochek.com> <47CB230E-B713-4098-82D4-318744434C46@pobox.com> <01NB0BOZYUPY001ML6@mauve.mrochek.com> <6DECD560-B257-4E6E-80E7-A99A69D297C6@pobox.com> <01NB14JAPS8E001ML6@mauve.mrochek.com> <F4A90756-9251-4CB8-9776-77F3202617A5@pobox.com> <01NB1TV04NJS001ML6@mauve.mrochek.com>
In-Reply-To: <01NB1TV04NJS001ML6@mauve.mrochek.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

Ned Freed wrote:

>> I have no desire to argue against my own proposal! sorry if I
>> misunderstood the tenor of some comments to this suggestion.
>
>> The original posting asked about two things
>
>> 1) spam/ham
>
>> 2) white/black
>
>> so far, people seem to be focussing on 1). I acknowledge problems with
>> 1), I appreciate that people are trying to address 1) as a possibility.
>
>> I asked about 2). I asked you if, ignoring 1), it was possible or
>> plausible to send white/black information about sender, if 1) was hard
>> or intractable.
>
> Of course it's possible, but this has no business being in IMAP. IMAP 
> is a
> message access protocol, not an address handling protocol. It is also 
> already
> quite complex, and extending its function to include what amountss to a
> specialized address book capability is a really bad idea IMO.
>
> In fact a pretty good case can be made that it's inappropriate to use 
> IMAP for spam/ham classification, but at the spam/ham classification is
> intrinsicly tied to messages in the store. The same cannot be said for
> address white and blacklisting.

+1

>> Does that make more sense? I do realize white/black listing is a
>> different PROBLEM to spam/ham marking, but neither of them currently
>> have support directly in imap, and I imagine both of them would use
>> IMAP extensions to pass information client-server.
>
> The place where I'd like to see white and black listing done is in 
> CARDDAV, not
> IMAP.

Or LDAP :-).

There is also a Sieve extension for making use of whitelists/blacklists 
during mail delivery. See draft-melnikov-sieve-external-lists-02.txt.
It can work with CARDDAV, LDAP and other things.



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n686RNP5005850 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 7 Jul 2009 23:27:23 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n686RNSC005849; Tue, 7 Jul 2009 23:27:23 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from mauve.mrochek.com (mauve.mrochek.com [66.59.230.40]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n686RC8w005831 for <ietf-imapext@imc.org>; Tue, 7 Jul 2009 23:27:22 -0700 (MST) (envelope-from ned.freed@mrochek.com)
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NB20PU9RC0018VKL@mauve.mrochek.com> for ietf-imapext@imc.org; Tue, 7 Jul 2009 23:27:09 -0700 (PDT)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NB09YSB36O001ML6@mauve.mrochek.com>; Tue, 07 Jul 2009 23:27:07 -0700 (PDT)
Date: Tue, 07 Jul 2009 23:19:32 -0700 (PDT)
From: Ned Freed <ned.freed@mrochek.com>
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
In-reply-to: "Your message dated Wed, 08 Jul 2009 08:08:05 +0200" <B33373D2-8602-409F-82D7-9E728808E6DD@muada.com>
To: Iljitsch van Beijnum <iljitsch@muada.com>
Cc: Ned Freed <ned.freed@mrochek.com>, George Michaelson <ggm@pobox.com>, ietf-imapext@imc.org, general discussion of application-layer protocols <discuss@ietf.org>
Message-id: <01NB20PSV3XE001ML6@mauve.mrochek.com>
MIME-version: 1.0
Content-type: TEXT/PLAIN; format=flowed; delsp=yes
Content-transfer-encoding: 7BIT
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <28941.1246868682.560969@puncture> <4A51D1A1.5040507@isode.com> <01NAZVQAYWH4001DLH@mauve.mrochek.com> <47CB230E-B713-4098-82D4-318744434C46@pobox.com> <01NB0BOZYUPY001ML6@mauve.mrochek.com> <6DECD560-B257-4E6E-80E7-A99A69D297C6@pobox.com> <01NB14JAPS8E001ML6@mauve.mrochek.com> <F4A90756-9251-4CB8-9776-77F3202617A5@pobox.com> <01NB1TV04NJS001ML6@mauve.mrochek.com> <B33373D2-8602-409F-82D7-9E728808E6DD@muada.com>
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

> On 8 jul 2009, at 5:02, Ned Freed wrote:

> > The place where I'd like to see white and black listing done is in
> > CARDDAV, not
> > IMAP.

> I never felt comfortable uploading all my contact info to Google. (May
> be somewhat strange since they do have all the mail sent to one of my
> email addresses.)

That's fine, but this doesn't translate into a need to put an addressbook
capability in IMAP.

> So I think white/blacklists shouldn't be complete contact info, but
> rather, just lists of hashes of email addresses.

So use CARDDAV to only upload the stuff that's needed to Google. Problem
solved.

I'll also note that addressbook programs have had facilities for exporting
subsets of the information they contain pretty much from the start. So doing
this  is hardly entering new territory.

And most users want an authoritative address book with all their contact
information in one place. I know I do - it's much too confusing otherwise.

				Ned



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n6868wS6004690 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 7 Jul 2009 23:08:58 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n6868wTD004689; Tue, 7 Jul 2009 23:08:58 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n6868jhu004670 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for <ietf-imapext@imc.org>; Tue, 7 Jul 2009 23:08:57 -0700 (MST) (envelope-from iljitsch@muada.com)
Received: from [192.168.0.193] (static-167-138-7-89.ipcom.comunitel.net [89.7.138.167] (may be forged)) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n686896f066815 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Wed, 8 Jul 2009 08:08:12 +0200 (CEST) (envelope-from iljitsch@muada.com)
Cc: George Michaelson <ggm@pobox.com>, ietf-imapext@imc.org, general discussion of application-layer protocols <discuss@ietf.org>
Message-Id: <B33373D2-8602-409F-82D7-9E728808E6DD@muada.com>
From: Iljitsch van Beijnum <iljitsch@muada.com>
To: Ned Freed <ned.freed@mrochek.com>
In-Reply-To: <01NB1TV04NJS001ML6@mauve.mrochek.com>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
Content-Transfer-Encoding: 7bit
Mime-Version: 1.0 (Apple Message framework v930.3)
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
Date: Wed, 8 Jul 2009 08:08:05 +0200
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <28941.1246868682.560969@puncture> <4A51D1A1.5040507@isode.com> <01NAZVQAYWH4001DLH@mauve.mrochek.com> <47CB230E-B713-4098-82D4-318744434C46@pobox.com> <01NB0BOZYUPY001ML6@mauve.mrochek.com> <6DECD560-B257-4E6E-80E7-A99A69D297C6@pobox.com> <01NB14JAPS8E001ML6@mauve.mrochek.com> <F4A90756-9251-4CB8-9776-77F3202617A5@pobox.com> <01NB1TV04NJS001ML6@mauve.mrochek.com>
X-Mailer: Apple Mail (2.930.3)
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

On 8 jul 2009, at 5:02, Ned Freed wrote:

> The place where I'd like to see white and black listing done is in  
> CARDDAV, not
> IMAP.

I never felt comfortable uploading all my contact info to Google. (May  
be somewhat strange since they do have all the mail sent to one of my  
email addresses.)

So I think white/blacklists shouldn't be complete contact info, but  
rather, just lists of hashes of email addresses.



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n683owwp097426 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 7 Jul 2009 20:50:58 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n683ow9d097425; Tue, 7 Jul 2009 20:50:58 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from sca-es-mail-1.sun.com (sca-es-mail-1.Sun.COM [192.18.43.132]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n683oliB097403 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO) for <ietf-imapext@imc.org>; Tue, 7 Jul 2009 20:50:57 -0700 (MST) (envelope-from Anil.Srivastava@Sun.COM)
Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n683okp8013077 for <ietf-imapext@imc.org>; Tue, 7 Jul 2009 20:50:46 -0700 (PDT)
MIME-version: 1.0
Content-transfer-encoding: 7BIT
Content-type: TEXT/PLAIN; CHARSET=US-ASCII
Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KMG001002MKDM00@fe-sfbay-09.sun.com> for ietf-imapext@imc.org; Tue, 07 Jul 2009 20:50:46 -0700 (PDT)
Received: from [192.168.132.10] ([unknown] [129.150.24.16]) by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KMG00M0Z2OLJYD0@fe-sfbay-09.sun.com>; Tue, 07 Jul 2009 20:50:46 -0700 (PDT)
Date: Tue, 07 Jul 2009 20:50:40 -0700 (PDT)
From: Anil SRIVASTAVA <Anil.Srivastava@Sun.COM>
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
In-reply-to: <01NB1TV04NJS001ML6@mauve.mrochek.com>
To: Ned Freed <ned.freed@mrochek.com>
Cc: George Michaelson <ggm@pobox.com>, ietf-imapext@imc.org, general discussion of application-layer protocols <discuss@ietf.org>
Reply-to: Anil.Srivastava@Sun.COM
Message-id:  <alpine.OSX.2.00.0907072050170.346@dhcp-usca15-127-42.red.iplanet.com>
Organization: Sun Microsystems [http://www.sun.com]
Accept-Language: English; en
X-Message-Class: normal
X-Endorsement: Message brought to you by Sun Java Communication Suite 6
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <28941.1246868682.560969@puncture> <4A51D1A1.5040507@isode.com> <01NAZVQAYWH4001DLH@mauve.mrochek.com> <47CB230E-B713-4098-82D4-318744434C46@pobox.com> <01NB0BOZYUPY001ML6@mauve.mrochek.com> <6DECD560-B257-4E6E-80E7-A99A69D297C6@pobox.com> <01NB14JAPS8E001ML6@mauve.mrochek.com> <F4A90756-9251-4CB8-9776-77F3202617A5@pobox.com> <01NB1TV04NJS001ML6@mauve.mrochek.com>
User-Agent: Alpine 2.00 (OSX 1167 2008-08-23)
X-Message-flag: Outlook: the best virus distribution system around
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

On 2009-07-07/20:02 [-0700], ned.freed@mrochek.com [Ned Freed] wrote:

> 
> > I have no desire to argue against my own proposal! sorry if I
> > misunderstood the tenor of some comments to this suggestion.
> 
> > The original posting asked about two things
> 
> > 1) spam/ham
> 
> > 2) white/black
> 
> > so far, people seem to be focussing on 1). I acknowledge problems with
> > 1), I appreciate that people are trying to address 1) as a possibility.
> 
> > I asked about 2). I asked you if, ignoring 1), it was possible or
> > plausible to send white/black information about sender, if 1) was hard
> > or intractable.
> 
> Of course it's possible, but this has no business being in IMAP. IMAP is 
> a message access protocol, not an address handling protocol. It is also 
> already quite complex, and extending its function to include what 
> amountss to a specialized address book capability is a really bad idea 
> IMO.
> 
> In fact a pretty good case can be made that it's inappropriate to 
> useIMAP for spam/ham classification, but at the spam/ham classification 
> is intrinsicly tied to messages in the store. The same cannot be said 
> for address white and blacklisting.
> 
> > Does that make more sense? I do realize white/black listing is a 
> > different PROBLEM to spam/ham marking, but neither of them currently 
> > have support directly in imap, and I imagine both of them would use 
> > IMAP extensions to pass information client-server.
> 
> The place where I'd like to see white and black listing done is in 
> CARDDAV, not IMAP.
>

+1 

Anil
 
> 				Ned
> 

_______________
Anil SRIVASTAVA  |  anil.srivastava@Sun.COM  | Sun Microsystems, Inc



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n683BX2I095423 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 7 Jul 2009 20:11:33 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n683BXTh095422; Tue, 7 Jul 2009 20:11:33 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from mauve.mrochek.com (mauve.mrochek.com [66.59.230.40]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n683BLQf095395 for <ietf-imapext@imc.org>; Tue, 7 Jul 2009 20:11:32 -0700 (MST) (envelope-from ned.freed@mrochek.com)
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NB1TV23J0G00SKEL@mauve.mrochek.com> for ietf-imapext@imc.org; Tue, 7 Jul 2009 20:11:19 -0700 (PDT)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NB09YSB36O001ML6@mauve.mrochek.com>; Tue, 07 Jul 2009 20:11:16 -0700 (PDT)
Date: Tue, 07 Jul 2009 20:02:51 -0700 (PDT)
From: Ned Freed <ned.freed@mrochek.com>
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
In-reply-to: "Your message dated Wed, 08 Jul 2009 09:42:58 +1000" <F4A90756-9251-4CB8-9776-77F3202617A5@pobox.com>
To: George Michaelson <ggm@pobox.com>
Cc: Ned Freed <ned.freed@mrochek.com>, ietf-imapext@imc.org, general discussion of application-layer protocols <discuss@ietf.org>
Message-id: <01NB1TV04NJS001ML6@mauve.mrochek.com>
MIME-version: 1.0
Content-type: TEXT/PLAIN; format=flowed; delsp=yes
Content-transfer-encoding: 7BIT
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <28941.1246868682.560969@puncture> <4A51D1A1.5040507@isode.com> <01NAZVQAYWH4001DLH@mauve.mrochek.com> <47CB230E-B713-4098-82D4-318744434C46@pobox.com> <01NB0BOZYUPY001ML6@mauve.mrochek.com> <6DECD560-B257-4E6E-80E7-A99A69D297C6@pobox.com> <01NB14JAPS8E001ML6@mauve.mrochek.com> <F4A90756-9251-4CB8-9776-77F3202617A5@pobox.com>
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

> I have no desire to argue against my own proposal! sorry if I
> misunderstood the tenor of some comments to this suggestion.

> The original posting asked about two things

> 1) spam/ham

> 2) white/black

> so far, people seem to be focussing on 1). I acknowledge problems with
> 1), I appreciate that people are trying to address 1) as a possibility.

> I asked about 2). I asked you if, ignoring 1), it was possible or
> plausible to send white/black information about sender, if 1) was hard
> or intractable.

Of course it's possible, but this has no business being in IMAP. IMAP is a
message access protocol, not an address handling protocol. It is also already
quite complex, and extending its function to include what amountss to a
specialized address book capability is a really bad idea IMO.

In fact a pretty good case can be made that it's inappropriate to use 
IMAP for spam/ham classification, but at the spam/ham classification is
intrinsicly tied to messages in the store. The same cannot be said for
address white and blacklisting.

> Does that make more sense? I do realize white/black listing is a
> different PROBLEM to spam/ham marking, but neither of them currently
> have support directly in imap, and I imagine both of them would use
> IMAP extensions to pass information client-server.

The place where I'd like to see white and black listing done is in CARDDAV, not
IMAP.

				Ned



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n67Lce9D076644 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 7 Jul 2009 14:38:40 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n67LceGt076643; Tue, 7 Jul 2009 14:38:40 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from mauve.mrochek.com (mauve.mrochek.com [66.59.230.40]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n67LcTPl076626 for <ietf-imapext@imc.org>; Tue, 7 Jul 2009 14:38:39 -0700 (MST) (envelope-from ned.freed@mrochek.com)
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NB1I9BBPC0005UZP@mauve.mrochek.com> for ietf-imapext@imc.org; Tue, 7 Jul 2009 14:38:26 -0700 (PDT)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NB09YSB36O001ML6@mauve.mrochek.com>; Tue, 07 Jul 2009 14:38:22 -0700 (PDT)
Date: Tue, 07 Jul 2009 14:32:14 -0700 (PDT)
From: Ned Freed <ned.freed@mrochek.com>
Subject: Re: A survey of Spam-related keyword usage by IMAP clients.
In-reply-to: "Your message dated Tue, 07 Jul 2009 15:18:27 -0600" <1247001507.8602.109.camel@legolas.orthanc.ca>
To: Lyndon Nerenberg <lyndon@orthanc.ca>
Cc: imap-protocol@u.washington.edu, ietf-imapext@imc.org
Message-id: <01NB1I98RDS4001ML6@mauve.mrochek.com>
MIME-version: 1.0
Content-type: TEXT/PLAIN
Content-transfer-encoding: 7BIT
References: <1247001507.8602.109.camel@legolas.orthanc.ca>
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

I was about to post something very much like this but you beat me to it.

> It's time to get a current view of how IMAP clients are using keywords
> to manage spam. If you are a client author I would appreciate it if you
> could take a moment to send me a note describing which keywords your
> client uses for spam-related processing, and what each of the keywords
> signifies.

I'd also really like to know if the client has "this is spam" or "this is not
spam" buttons in the UI, and if it does how those buttons interact with the use
of keywords.

I ask this because users of any client that has such buttons are clearly going
to use those buttons preferentially over any other labelling mechanism. And if
a significant number of clients have such buttons that do a particular thing -
regardless of what that thing is - that's pretty much going to be determinative
in finding the most easily deployed approach.

> If your client does not use keywords for this purpose, that's
> valuable information, too, as is a breakdown of functionality by
> software version number if there are significant differences. (If you
> use non-keyword facilities to manage spam feel free to include a short
> description of your methods if you think the info might be relevant.)

Yes please. I'm especially interested in how many clients do this by resending
the message to a special address.

> I'm primarily interested in getting authoritative data from the client
> authors, but others should feel free to chime in with deployment-based
> experience of related keyword usage, but please indicate the source of
> your comments (e.g. deployment experience, source code examination).

> I will summarize the responses to the imap-ext list in a couple of weeks
> (or sooner if the response rate dictates.)

Only other thing I'd add is that I'd like to hear from webmail authors in
addition to standalone client authors.

				Ned



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n67LIkIZ075729 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 7 Jul 2009 14:18:46 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n67LIkBq075728; Tue, 7 Jul 2009 14:18:46 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from orthanc.ca (orthanc.ca [208.86.224.138]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n67LIZf6075715 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for <ietf-imapext@imc.org>; Tue, 7 Jul 2009 14:18:46 -0700 (MST) (envelope-from lyndon@orthanc.ca)
Received: from [192.168.0.42] (S01060011242eafe2.cg.shawcable.net [68.145.131.177]) (authenticated bits=0) by orthanc.ca (8.14.3/8.14.3) with ESMTP id n67LIXI0035352; Tue, 7 Jul 2009 15:18:34 -0600 (MDT) (envelope-from lyndon@orthanc.ca)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=orthanc.ca; s=2009-1; t=1247001514; bh=c4PxB9qIGUMk/XQCTpIvPlRhSPz2J2pr9i0QDZKASzA=; h=Subject:From:Reply-To:To:Content-Type:Date:Message-Id: Mime-Version:Content-Transfer-Encoding; b=BaiYAGjFDN6csFVJ7hFbuVOL1ObbRcaXp28cc+XlM6MFJ0Y1mjJ6hxhVEN0SsIunb MJVcLLOzqOEXQ/zjcIPOkjgKeXS9iIh+4oRM4pxo2I5kXdHqMbt+vgQbsXW6IBakRu dFoPgrurC0jurUvTomSpdYqLT1jfc9zzrbgpu3BlMmFXkbA8+4DGSJSMT9STB2g2x7 yYdsF87Tv9bbjaWD+HEFoeOXHVZFhBxPzrPt67JbTptYU5PiWm8MtUscM0uE19t8SG WGGrNo021kUR8uAMcv9t1PrpPQJNeya5smTcxj0i0k2zor9BEtzdVYef3FqskL/t5f l7l+islxxktPw==
Subject: A survey of Spam-related keyword usage by IMAP clients.
From: Lyndon Nerenberg <lyndon@orthanc.ca>
Reply-To: lyndon+imapspamkeywordsurvey@orthanc.ca
To: imap-protocol@u.washington.edu, ietf-imapext@imc.org
Content-Type: text/plain
Date: Tue, 07 Jul 2009 15:18:27 -0600
Message-Id: <1247001507.8602.109.camel@legolas.orthanc.ca>
Mime-Version: 1.0
X-Mailer: Evolution 2.26.2 FreeBSD GNOME Team Port 
Content-Transfer-Encoding: 7bit
X-Spam-Status: No, score=2.1 required=5.0 tests=DKIM_SIGNED,DKIM_VERIFIED, RCVD_IN_PBL,RCVD_IN_SORBS_DUL autolearn=no version=3.2.5
X-Spam-Level: **
X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on orthanc.ca
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

It's time to get a current view of how IMAP clients are using keywords
to manage spam. If you are a client author I would appreciate it if you
could take a moment to send me a note describing which keywords your
client uses for spam-related processing, and what each of the keywords
signifies. If your client does not use keywords for this purpose, that's
valuable information, too, as is a breakdown of functionality by
software version number if there are significant differences. (If you
use non-keyword facilities to manage spam feel free to include a short
description of your methods if you think the info might be relevant.)

I'm primarily interested in getting authoritative data from the client
authors, but others should feel free to chime in with deployment-based
experience of related keyword usage, but please indicate the source of
your comments (e.g. deployment experience, source code examination).

I will summarize the responses to the imap-ext list in a couple of weeks
(or sooner if the response rate dictates.)

--lyndon



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n67HW0g5061835 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 7 Jul 2009 10:32:00 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n67HW0iT061834; Tue, 7 Jul 2009 10:32:00 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from orthanc.ca (orthanc.ca [208.86.224.138]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n67HVmxZ061800 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for <ietf-imapext@imc.org>; Tue, 7 Jul 2009 10:31:59 -0700 (MST) (envelope-from lyndon@orthanc.ca)
Received: from [192.168.0.42] (S01060011242eafe2.cg.shawcable.net [68.145.131.177]) (authenticated bits=0) by orthanc.ca (8.14.3/8.14.3) with ESMTP id n67HVJe6030816; Tue, 7 Jul 2009 11:31:20 -0600 (MDT) (envelope-from lyndon@orthanc.ca)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=orthanc.ca; s=2009-1; t=1246987881; bh=dqQG1R61W3fm+QI72VYQMJEiJ/ert5IkyFlRyhkB2Q0=; h=Subject:From:To:Cc:In-Reply-To:References:Content-Type:Date: Message-Id:Mime-Version:Content-Transfer-Encoding; b=Teu5CztYJbN9luvvex3j2ymkN17mq/1iXMk1anpY1/j8DxZfn4a/XHg5Cg6JLeF5z TL9mrgBINQE/bFmUP8M6IyE9Y9yd+XCBn8zGqvNoqniN3VI4uFWX1jOcVTeplLDMj+ fdjCbtx0NB+q2A6iYhvgz/yi87heqHncOJ9m1sSCSiSU8xfC/Hwr1ARTonjCuRVt+f QXZfz5u+0giKTGpvmo6pCbG+njBtMjjfu9Ql3DfzwbUndDt/IYpltzE9PaaIrNJZRM uIog/qABIwS5SzIc41Jm/b3Gti4FNPXfPXJM3kgiLS8ldJBs/gt9Jx4C1YXVtK6XRN D/I7pLQ9Kumgw==
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
From: Lyndon Nerenberg <lyndon@orthanc.ca>
To: Iljitsch van Beijnum <iljitsch@muada.com>
Cc: ietf-imapext@imc.org, discuss@ietf.org
In-Reply-To: <E6275AB7-B916-47F8-9135-B1BEFFE11814@muada.com>
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <01NAZUST62HQ001DLH@mauve.mrochek.com> <1246902821.8602.70.camel@legolas.orthanc.ca> <E6275AB7-B916-47F8-9135-B1BEFFE11814@muada.com>
Content-Type: text/plain
Date: Tue, 07 Jul 2009 11:31:13 -0600
Message-Id: <1246987873.8602.92.camel@legolas.orthanc.ca>
Mime-Version: 1.0
X-Mailer: Evolution 2.26.2 FreeBSD GNOME Team Port 
Content-Transfer-Encoding: 7bit
X-Spam-Status: No, score=2.1 required=5.0 tests=DKIM_SIGNED,DKIM_VERIFIED, RCVD_IN_PBL,RCVD_IN_SORBS_DUL autolearn=no version=3.2.5
X-Spam-Level: **
X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on orthanc.ca
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

On Tue, 2009-07-07 at 09:21 +0200, Iljitsch van Beijnum wrote:
> 3. manually flagged as spam
> 4. manually flagged as non-spam
> 
> Where 1 and 2 / 3 and 4 are mutually exclusive but 1 + 4 and 2 + 3  
> should trigger retraining of baysian filters etc.

The keywords I'm describing are intended to tell the server to process
the flagged message through its filters (i.e. train on the message). It
isn't relevant to the proposal whether a human sets them, just what the
server's behaviour is in their presence.

--lyndon



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n67G22Al055589 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 7 Jul 2009 09:02:02 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n67G22WK055588; Tue, 7 Jul 2009 09:02:02 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from mauve.mrochek.com (mauve.mrochek.com [66.59.230.40]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n67G21Qs055581 for <ietf-imapext@imc.org>; Tue, 7 Jul 2009 09:02:01 -0700 (MST) (envelope-from ned.freed@mrochek.com)
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NB16I3I5TS0017B6@mauve.mrochek.com> for ietf-imapext@imc.org; Tue, 7 Jul 2009 09:01:56 -0700 (PDT)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NB09YSB36O001ML6@mauve.mrochek.com>; Tue, 07 Jul 2009 09:01:52 -0700 (PDT)
Date: Tue, 07 Jul 2009 08:43:46 -0700 (PDT)
From: Ned Freed <ned.freed@mrochek.com>
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
In-reply-to: "Your message dated Tue, 07 Jul 2009 11:15:04 -0400" <F02388CBAFB7CC46475A5F66@caldav.corp.apple.com>
To: Cyrus Daboo <cyrus@daboo.name>
Cc: Ned Freed <ned.freed@mrochek.com>, Lyndon Nerenberg <lyndon@orthanc.ca>, ietf-imapext@imc.org, discuss@ietf.org
Message-id: <01NB16I1ENDC001ML6@mauve.mrochek.com>
MIME-version: 1.0
Content-type: TEXT/PLAIN; format=flowed
Content-transfer-encoding: 7BIT
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <01NAZUST62HQ001DLH@mauve.mrochek.com> <1246902821.8602.70.camel@legolas.orthanc.ca> <01NAZWHC66ZU000078@mauve.mrochek.com> <1246904761.8602.86.camel@legolas.orthanc.ca> <01NB13KIAQHQ001ML6@mauve.mrochek.com> <F02388CBAFB7CC46475A5F66@caldav.corp.apple.com>
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

> Hi Ned,

> --On July 7, 2009 7:22:42 AM -0700 Ned Freed <ned.freed@mrochek.com> wrote:

> > Now suppose I implement this with folder annotations. I'm in business
> > immediately. Things just work no matter what client people use.

> Actually you need to consider what the user experience is likely to be on
> the client side. There are several things to consider: for spam messages
> users will typically want to delete them from their regular mailbox. With
> ham messages they will typically want to leave the message in place (or
> file them away after processing). In each case I certainly would not want
> the spam messages or an extra copy of the ham messages to reside on my
> server consuming quota or other resources, or being indexed as part of a
> server-wide search etc.

> With that in mind lets look at how each solution might work client-side:

> 1) Keywords: for a spam message the user sets the spam keyword, then sets
> the deleted flag (and then may explicitly expunge if the client has that as
> an option as opposed to hide-deleted-messages with lazy expunge). For a ham
> message the user just sets the ham keyword. i.e. two user operations for
> spam, one for ham.

> 2) Mailbox (I am going to assume that the client has both a move and copy
> command available - the former implicitly setting the delete flag after the
> IMAP COPY command): for a spam message the user moves the message to the
> spam mailbox. For a ham message the user copies the message to the ham
> mailbox. NB both of these assume that the server does not actually store
> the spam/ham messages in the spam/ham mailboxes (or perhaps it garbage
> collects them).

> Things to note: typically clients don't offer shortcuts "move to mailbox X"
> or "copy to mailbox X" so the mailbox operations will involve some drag and
> drop or menu list selection operation. The same may be true for setting
> arbitrary keywords (though Mail.app and I suspect Thunderbird do have
> buttons for this). Frankly the best user experience would be for the client
> to have spam/ham buttons and commands that would allow the user to do the
> appropriate actions in one click or keyboard shortcut operation - that is
> certainly important if the level of spam is moderate to high.

The presence of spam/not-spam buttons in existing clients is an important
factor I hadn't really considered, but I think it has implications that go a
lot further than you let on here.

Simply put, if enough popular clients already have such buttons, and those
buttons do different things in different clients, I think we're pretty much
screwed. Given the presence of such buttons you're not going to be able to get
users to do something different to deal with spam, whether it's adding a
keyword or annotation or performing a folder move.

Our only recourse with such clients in the mix is to write a specification for
what such buttons should do in terms of signalling the server and then hope
that clients actually adopt whatever that specification says. And frankly, if
there's some degree of commonality in what those buttons do in existing clients
and it can be made to work, we probably should adopt that approach no matter
what it happens to be.

> So at the end of the day, for the best (and frankly simplest) user
> experience clients do have to change for both the keyword and move to
> mailbox approaches. Now for keywords that change does have to be immediate
> if arbitrary keyword support is not already present in the client, whereas
> for move to mailbox, or clients already supporting keywords, that change
> can be done over time - though as Ned points out client changes tend to
> happen on geological time-scales!

I agree completely that a keyword-based approach has the potential to offer
a better user experience. The question is whether the possibility of offering
a better user experience at some distant point down the road is worth giving
up on a more immediately deployable solution.

But as I indicated above, now that you've brought up the important point of
existing clients having spam classification buttons, that's the real gorilla in
the room we have to deal with.

> So other approaches to consider:

> 1) An extension with commands SPAM and HAM to explicitly tag one or more
> messages.

> 2) A "forward to spam/ham" option that specifies special email addresses
> for the user where spam/ham can be forwarded to trigger training.

I believe this is what the buttons I've seen on at least a couple of
clients already do. If that's indeed the case, it's worth considering.

> 3) An IMAP URL handed off to the SPAM system with URLAUTH to allow that to
> pull details of the message for training. (This could be tied to (2) in
> that forward-without-download could be used in (2) with URLAUTH etc).

> These of course all require new client/server behaviors and thus are
> probably non-starters, but they should not be dismissed entirely.

I'm actually not sure (2) does with all clients, but I agree regarding (1) and
(3).

				Ned



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n67FFNmk052661 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 7 Jul 2009 08:15:23 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n67FFNxo052660; Tue, 7 Jul 2009 08:15:23 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from daboo.name (daboo.name [151.201.22.177]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n67FFBck052647 for <ietf-imapext@imc.org>; Tue, 7 Jul 2009 08:15:22 -0700 (MST) (envelope-from cyrus@daboo.name)
Received: from localhost (localhost [127.0.0.1]) by daboo.name (Postfix) with ESMTP id 6B24B1680AD5; Tue,  7 Jul 2009 11:15:11 -0400 (EDT)
X-Virus-Scanned: amavisd-new at daboo.name
Received: from daboo.name ([127.0.0.1]) by localhost (chewy.mulberrymail.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jPdNDLFU4hpp; Tue,  7 Jul 2009 11:15:11 -0400 (EDT)
Received: from caldav.corp.apple.com (unknown [17.101.32.44]) by daboo.name (Postfix) with ESMTP id A54A91680ACE; Tue,  7 Jul 2009 11:15:08 -0400 (EDT)
Date: Tue, 07 Jul 2009 11:15:04 -0400
From: Cyrus Daboo <cyrus@daboo.name>
To: Ned Freed <ned.freed@mrochek.com>, Lyndon Nerenberg <lyndon@orthanc.ca>
cc: ietf-imapext@imc.org, discuss@ietf.org
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
Message-ID: <F02388CBAFB7CC46475A5F66@caldav.corp.apple.com>
In-Reply-To: <01NB13KIAQHQ001ML6@mauve.mrochek.com>
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <01NAZUST62HQ001DLH@mauve.mrochek.com> <1246902821.8602.70.camel@legolas.orthanc.ca> <01NAZWHC66ZU000078@mauve.mrochek.com> <1246904761.8602.86.camel@legolas.orthanc.ca> <01NB13KIAQHQ001ML6@mauve.mrochek.com>
X-Mailer: Mulberry/4.1.0a1 (Mac OS X)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline; size=3318
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

Hi Ned,

--On July 7, 2009 7:22:42 AM -0700 Ned Freed <ned.freed@mrochek.com> wrote:

> Now suppose I implement this with folder annotations. I'm in business
> immediately. Things just work no matter what client people use.

Actually you need to consider what the user experience is likely to be on 
the client side. There are several things to consider: for spam messages 
users will typically want to delete them from their regular mailbox. With 
ham messages they will typically want to leave the message in place (or 
file them away after processing). In each case I certainly would not want 
the spam messages or an extra copy of the ham messages to reside on my 
server consuming quota or other resources, or being indexed as part of a 
server-wide search etc.

With that in mind lets look at how each solution might work client-side:

1) Keywords: for a spam message the user sets the spam keyword, then sets 
the deleted flag (and then may explicitly expunge if the client has that as 
an option as opposed to hide-deleted-messages with lazy expunge). For a ham 
message the user just sets the ham keyword. i.e. two user operations for 
spam, one for ham.

2) Mailbox (I am going to assume that the client has both a move and copy 
command available - the former implicitly setting the delete flag after the 
IMAP COPY command): for a spam message the user moves the message to the 
spam mailbox. For a ham message the user copies the message to the ham 
mailbox. NB both of these assume that the server does not actually store 
the spam/ham messages in the spam/ham mailboxes (or perhaps it garbage 
collects them).

Things to note: typically clients don't offer shortcuts "move to mailbox X" 
or "copy to mailbox X" so the mailbox operations will involve some drag and 
drop or menu list selection operation. The same may be true for setting 
arbitrary keywords (though Mail.app and I suspect Thunderbird do have 
buttons for this). Frankly the best user experience would be for the client 
to have spam/ham buttons and commands that would allow the user to do the 
appropriate actions in one click or keyboard shortcut operation - that is 
certainly important if the level of spam is moderate to high.

So at the end of the day, for the best (and frankly simplest) user 
experience clients do have to change for both the keyword and move to 
mailbox approaches. Now for keywords that change does have to be immediate 
if arbitrary keyword support is not already present in the client, whereas 
for move to mailbox, or clients already supporting keywords, that change 
can be done over time - though as Ned points out client changes tend to 
happen on geological time-scales!

So other approaches to consider:

1) An extension with commands SPAM and HAM to explicitly tag one or more 
messages.
2) A "forward to spam/ham" option that specifies special email addresses 
for the user where spam/ham can be forwarded to trigger training.
3) An IMAP URL handed off to the SPAM system with URLAUTH to allow that to 
pull details of the message for training. (This could be tied to (2) in 
that forward-without-download could be used in (2) with URLAUTH etc).

These of course all require new client/server behaviors and thus are 
probably non-starters, but they should not be dismissed entirely.

-- 
Cyrus Daboo



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n67F5kjL051887 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 7 Jul 2009 08:05:46 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n67F5j4X051886; Tue, 7 Jul 2009 08:05:45 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from mauve.mrochek.com (mauve.mrochek.com [66.59.230.40]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n67F5hkv051879 for <ietf-imapext@imc.org>; Tue, 7 Jul 2009 08:05:44 -0700 (MST) (envelope-from ned.freed@mrochek.com)
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NB14JDJETS00HWDR@mauve.mrochek.com> for ietf-imapext@imc.org; Tue, 7 Jul 2009 08:05:41 -0700 (PDT)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NB09YSB36O001ML6@mauve.mrochek.com>; Tue, 07 Jul 2009 08:05:37 -0700 (PDT)
Date: Tue, 07 Jul 2009 07:42:42 -0700 (PDT)
From: Ned Freed <ned.freed@mrochek.com>
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
In-reply-to: "Your message dated Tue, 07 Jul 2009 11:28:53 +1000" <6DECD560-B257-4E6E-80E7-A99A69D297C6@pobox.com>
To: George Michaelson <ggm@pobox.com>
Cc: Ned Freed <ned.freed@mrochek.com>, ietf-imapext@imc.org, general discussion of application-layer protocols <discuss@ietf.org>
Message-id: <01NB14JAPS8E001ML6@mauve.mrochek.com>
MIME-version: 1.0
Content-type: TEXT/PLAIN; format=flowed; delsp=yes
Content-transfer-encoding: 7BIT
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <28941.1246868682.560969@puncture> <4A51D1A1.5040507@isode.com> <01NAZVQAYWH4001DLH@mauve.mrochek.com> <47CB230E-B713-4098-82D4-318744434C46@pobox.com> <01NB0BOZYUPY001ML6@mauve.mrochek.com> <6DECD560-B257-4E6E-80E7-A99A69D297C6@pobox.com>
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

> hang on.

> The MS has the message. It presumably has some held state of it,
> specific to its implementation.

Actually, that's highly unlikely. There is some amounts of state IMAP requires
that's associated with the message - flags/keywords, annotations, some remnants
of the envelope. But beyond that there's little point in keeping a bunch of
implementation-specific state. Why? Because there's no way to expose such state
through the IMAP protocol. So when messages are FETCHed the state stays behind.
And more to the point, when a client moves a message, implementation-specific
state tends to get lost, because the client doesn't know how to access it in
order to move it.

The result is that in most csaes IMAP servers don't store much beyond the
things the standard requires them to store. The differences between servers lie
in how the standard stuff is stored and to what extent metainformation, like
body structured, is cached instead of being recomputed.

> I don't want deep knowledge of that
> model, I really just want to be able to override it with my explicit
> knowledge.

> I simply want to pass the toggle:
	
> 	If you class this as spam, please class as ham
> or
> 	if you class this as ham, please class as spam

This is exactly what we're trying to accomplish. The question is what mechanism
to use to do it.

> what it means, semantically, is of course MS specific. I don't care
> how its held, or what the effect on the weighting is, in the re-class
> as spam. In the class as ham, I care quite a lot that its sender is
> removed from hard blocks of course.

I'm afraid you cannot count on that happening either. The problem is that
such blocks are often outside the control of the service provider to change.
In many cases the provider has subscribed to some sort of spam filtering
service.

Whitelisting may be an alternative in such cases, but you have to be careful
lest the amount of whitelisting information grow to the point where it is
unmanageable. (I've seen this happen.)

> Yes, I see that moving this state between servers requires the entire
> message flow. But, if we forget that for now, and just consider that I
> have some mail in my local client cache, which I feel is now worthy of
> a state change, why do I have to pass full-state baysian model
> information?

You appear to be arguing against a straw man of your own construction here,
because AFAIK nobody has suggested such a thing.

> I just want to tell an MS "change your own flags to match
> my flags" in the hard sense: yes-or-no, not some qualitative weighting
> model.

The only "weight" people have mentioned in any of this is storing a spam score
rather than a yes/no indicator. I've already responded that while there is a
place for such information in spam handling, it makes no sense in this
particular context.

> How it acts, is up to it. I don't see why I need a complex flag
> model to say what I think, as the consumer of the mail.

Again, you appear to be arguing against a proposal nobody has made. (Perhaps
some proposal was made in an earlier thread not on the imapext list.) I'll also
point out that flags are binary by definition in IMAP - you cannot implement
even a single-valued spam score using them. If you want to store a reviseable
per-message score, you have no choice but to use message-level annotations.

> If I moved off SPAM/HAM to white/black on sender, would that be easier
> for you to consider?

Whitelisting and blacklisting is actually  a fairly different problem.

				Ned



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n67EcSSu050175 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 7 Jul 2009 07:38:28 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n67EcSmM050174; Tue, 7 Jul 2009 07:38:28 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from mauve.mrochek.com (mauve.mrochek.com [66.59.230.40]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n67EcRDP050168 for <ietf-imapext@imc.org>; Tue, 7 Jul 2009 07:38:28 -0700 (MST) (envelope-from ned.freed@mrochek.com)
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NB13KKILF4001742@mauve.mrochek.com> for ietf-imapext@imc.org; Tue, 7 Jul 2009 07:38:24 -0700 (PDT)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NB09YSB36O001ML6@mauve.mrochek.com>; Tue, 07 Jul 2009 07:38:20 -0700 (PDT)
Date: Tue, 07 Jul 2009 07:22:42 -0700 (PDT)
From: Ned Freed <ned.freed@mrochek.com>
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
In-reply-to: "Your message dated Mon, 06 Jul 2009 12:26:01 -0600" <1246904761.8602.86.camel@legolas.orthanc.ca>
To: Lyndon Nerenberg <lyndon@orthanc.ca>
Cc: Ned Freed <ned.freed@mrochek.com>, ietf-imapext@imc.org, discuss@ietf.org
Message-id: <01NB13KIAQHQ001ML6@mauve.mrochek.com>
MIME-version: 1.0
Content-type: TEXT/PLAIN
Content-transfer-encoding: 7BIT
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <01NAZUST62HQ001DLH@mauve.mrochek.com> <1246902821.8602.70.camel@legolas.orthanc.ca> <01NAZWHC66ZU000078@mauve.mrochek.com> <1246904761.8602.86.camel@legolas.orthanc.ca>
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

> On Mon, 2009-07-06 at 11:00 -0700, Ned Freed wrote:
> > If there's a reason multiple folders can't be marked this way I sure
> > don't
> > know what it is.

> You're right -- I didn't parse what you were saying correctly the first
> time around.

> But if your concern is adoption rates, a keyword-only solution will get
> wider take-up than something that requires support for annotations. (And
> from a purely engineering standpoint, the simplest solution wins.)

My concern isn't adoption rate. Rather, my concern is, as someone who provides
server software, how I can offer a better user experience in regards to spam.

Suppose I implement spam filter retraining based on setting of flags. how well
this works, or whether it works at all, depends entirely on the client people
use. And past experience on client uptake of new features like this is best
described as "lousy".

Now suppose I implement this with folder annotations. I'm in business
immediately. Things just work no matter what client people use.

There is, of course, the issue of setting the annotation on the proper folder
and  having a client to do that. But this isn't as much of a problem as it
sounds. Since the annotation only has to be set once, it can be done through a
simple web page, or better still, make it an option in the webmail interface.
(Everyone has a webmail interface these days.) And in many cases even this is
unnecessary, because many service providers predesignate a folder as the spam
folder.

The only way I could get behind a flag-based solution is if there really was an
common set of ad-hoc flag choices already in use by all the major clients - not
a couple, all. But I don't believe that's the case.

We've had to choose between client and server side solutions many times in the
past, and in almost every case the conclusion has been server side solutions
are a deployment win. I really wish this weren't true - I wish clients would
pick up new capabilities in a more timely fashion - but that's just not how
things work.

				Ned



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n67E1J39047551 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 7 Jul 2009 07:01:19 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n67E1J5p047550; Tue, 7 Jul 2009 07:01:19 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from mauve.mrochek.com (mauve.mrochek.com [66.59.230.40]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n67E18tv047537 for <ietf-imapext@imc.org>; Tue, 7 Jul 2009 07:01:19 -0700 (MST) (envelope-from ned.freed@mrochek.com)
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NB12AAY2E8006J0V@mauve.mrochek.com> for ietf-imapext@imc.org; Tue, 7 Jul 2009 07:01:06 -0700 (PDT)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NB09YSB36O001ML6@mauve.mrochek.com>; Tue, 07 Jul 2009 07:01:04 -0700 (PDT)
Date: Tue, 07 Jul 2009 06:54:49 -0700 (PDT)
From: Ned Freed <ned.freed@mrochek.com>
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
In-reply-to: "Your message dated Tue, 07 Jul 2009 09:21:11 +0200" <E6275AB7-B916-47F8-9135-B1BEFFE11814@muada.com>
To: Iljitsch van Beijnum <iljitsch@muada.com>
Cc: Lyndon Nerenberg <lyndon@orthanc.ca>, Ned Freed <ned.freed@mrochek.com>, ietf-imapext@imc.org, discuss@ietf.org
Message-id: <01NB12A9GJE8001ML6@mauve.mrochek.com>
MIME-version: 1.0
Content-type: TEXT/PLAIN; format=flowed; delsp=yes
Content-transfer-encoding: 7BIT
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <01NAZUST62HQ001DLH@mauve.mrochek.com> <1246902821.8602.70.camel@legolas.orthanc.ca> <E6275AB7-B916-47F8-9135-B1BEFFE11814@muada.com>
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

> On 6 jul 2009, at 19:53, Lyndon Nerenberg wrote:

> > A message's 'state of spamminess' is an attribute of the
> > message, properly denoted by keywords that stick to (and with) the
> > message.

> That's nice in the abstract, but why do we have servers classify
> messages as spam to begin with? In order to not have to look at them,
> or to be able to look at them when it suits us to clean out the spam
> rather than in between mail that we actually wanted to receive.

Actually, the primary reason servers classify messages as spam is so they
can avoid having to deal with them. If the overhead could be ignored no
server operator would bother - spam filteringh is expensive and difficult.
The reason it's done is the alternate of receiving and dealing with the
those spam messages, with all the ramificaations, is even more expensive
and difficult.

This more than anything is why a score and not a yes/no verdict is important. A
score lets you do stuff like reject messages clearly found to be spam at the
SMTP level, while letting likely but less certain to be spam messages through
to specially designated spam folders or whatever.

> So moving spam to a separate folder is not something that happens
> because of lack of a better mechanism, but the thing that we want to
> happen 98% of the time anyway.

Sometimes yes, sometimes no.

> > $spam and $nospam

> There are more states than that:

> 1. automatically flagged as spam
> 2. automatically flagged as non-spam
> 3. manually flagged as spam
> 4. manually flagged as non-spam

> Where 1 and 2 / 3 and 4 are mutually exclusive but 1 + 4 and 2 + 3
> should trigger retraining of baysian filters etc.

It's not the flag state of the message that triggers retraining,  it's
the change of flag state (or folder locaation). Your proposal is unnecessatily
complicated.

				Ned



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n677NmeN018693 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 7 Jul 2009 00:23:48 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n677Nmem018692; Tue, 7 Jul 2009 00:23:48 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n677NZPA018671 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for <ietf-imapext@imc.org>; Tue, 7 Jul 2009 00:23:46 -0700 (MST) (envelope-from iljitsch@muada.com)
Received: from claw.it.uc3m.es (claw.it.uc3m.es [163.117.139.55]) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n677LFxk057589 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Tue, 7 Jul 2009 09:21:16 +0200 (CEST) (envelope-from iljitsch@muada.com)
Cc: Ned Freed <ned.freed@mrochek.com>, ietf-imapext@imc.org, discuss@ietf.org
Message-Id: <E6275AB7-B916-47F8-9135-B1BEFFE11814@muada.com>
From: Iljitsch van Beijnum <iljitsch@muada.com>
To: Lyndon Nerenberg <lyndon@orthanc.ca>
In-Reply-To: <1246902821.8602.70.camel@legolas.orthanc.ca>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
Content-Transfer-Encoding: 7bit
Mime-Version: 1.0 (Apple Message framework v935.3)
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
Date: Tue, 7 Jul 2009 09:21:11 +0200
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <01NAZUST62HQ001DLH@mauve.mrochek.com> <1246902821.8602.70.camel@legolas.orthanc.ca>
X-Mailer: Apple Mail (2.935.3)
X-Spam-Status: No, score=-2.6 required=3.4 tests=BAYES_00 autolearn=ham  version=3.0.2
X-Spam-Checker-Version: SpamAssassin 3.0.2 (2004-11-16) on sequoia.muada.com
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

On 6 jul 2009, at 19:53, Lyndon Nerenberg wrote:

> A message's 'state of spamminess' is an attribute of the
> message, properly denoted by keywords that stick to (and with) the
> message.

That's nice in the abstract, but why do we have servers classify  
messages as spam to begin with? In order to not have to look at them,  
or to be able to look at them when it suits us to clean out the spam  
rather than in between mail that we actually wanted to receive.

So moving spam to a separate folder is not something that happens  
because of lack of a better mechanism, but the thing that we want to  
happen 98% of the time anyway.

> $spam and $nospam

There are more states than that:

1. automatically flagged as spam
2. automatically flagged as non-spam
3. manually flagged as spam
4. manually flagged as non-spam

Where 1 and 2 / 3 and 4 are mutually exclusive but 1 + 4 and 2 + 3  
should trigger retraining of baysian filters etc.



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n671KT0n000916 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 18:20:29 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n671KT7P000915; Mon, 6 Jul 2009 18:20:29 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from mauve.mrochek.com (mauve.mrochek.com [66.59.230.40]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n671KI1r000900 for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 18:20:28 -0700 (MST) (envelope-from ned.freed@mrochek.com)
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NB0BP0Z28000QPIT@mauve.mrochek.com> for ietf-imapext@imc.org; Mon, 6 Jul 2009 18:20:16 -0700 (PDT)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NB09YSB36O001ML6@mauve.mrochek.com>; Mon, 06 Jul 2009 18:20:14 -0700 (PDT)
Date: Mon, 06 Jul 2009 17:42:58 -0700 (PDT)
From: Ned Freed <ned.freed@mrochek.com>
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
In-reply-to: "Your message dated Tue, 07 Jul 2009 08:51:10 +1000" <47CB230E-B713-4098-82D4-318744434C46@pobox.com>
To: George Michaelson <ggm@pobox.com>
Cc: Ned Freed <ned.freed@mrochek.com>, ietf-imapext@imc.org, general discussion of application-layer protocols <discuss@ietf.org>
Message-id: <01NB0BOZYUPY001ML6@mauve.mrochek.com>
MIME-version: 1.0
Content-type: TEXT/PLAIN; format=flowed; delsp=yes
Content-transfer-encoding: 7BIT
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <28941.1246868682.560969@puncture> <4A51D1A1.5040507@isode.com> <01NAZVQAYWH4001DLH@mauve.mrochek.com> <47CB230E-B713-4098-82D4-318744434C46@pobox.com>
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

> >
> >
> > This is why I think folder annotations are a better bet. You mark
> > the spam
> > folder and use that marking to attach meaning to the movement of
> > messages into
> > and out of that folder. It's not perfect, but it has the key
> > advantage that it
> > works with any client (OK, any client that can use folders) once the
> > annotation
> > is in place.
> >
> > 				Ned

> I have two IMAP servers, one google, one not.

That's a problem in and of itself, because last time I looked gmail is rather
limited in its support of IMAP.

> how do I synchronize state between them.

By "state" you appear to be referring to spam filtering state. If so, the short
answer is you can't possibly do it.

> After all, If I (and I do)
> sometimes have to send from one, to reply to mails from the other
> (mail clients are bad at selecting sender account) it would be useful
> to have at least white and blacklist sender synchronization between
> them.

Sorry, that's simply not realistic. The number of different spam filtering
mechanisms in use is much too large, and the lack of semantic equivalents
across them all makes this an intractable problem.

> Or, I could be sharing explicit spam tagging information to
> improve BOTH of their tracking. Do I really have to pass the entire
> messages between imap servers?

If you want to train the server that doesn't have the message that you do,
or don't regard, it as spam, moving the entire message is the only option.

I will note that moving messages from one IMAP server to another without
having to download and upload them would be possible by relaxing the
relative URL restriction in the CATENATE extension defined in RFC 4469.

But sending misclassified messages to multiple servers is not going to get them
synchronized. Again, there are a myriad of approaches to spam filtering with
hugely different semantics. In fact it is entirely possible that you'll cause
their behavior to diverge instead of converging.

> An explicit extension/flag/keyword would be unquestionably more
> efficient use of the network.

This makes no sense at all. Setting a flag associated with a message stored
on one server is not going to be visible to the other in any way. So the
flagging approach offers no solution to the multiple server problem at all.

The folder annotation approach doesn't work any better, of course.

> And, removing the mails from the spam folder in your methodology
> implies saying 'its not spam' -so the server I move them FROM thinks
> they've been un-tagged.

No, not really. Since there's no facility in IMAP to move a message between
servers, the way you have to do it is to FETCH the message from one server and
APPEND it to the other. In constrast moving a message from one folder to
another on a single server is done with a COPY/SETFLAG sequence. Entirely
different.

> Again, I say what I said to Ijllitch: I don't like depending on side-
> effects. I think explicit messaging is better.

I don't especially like depending on side effects either, but when the choice
is between a mechanism that works with the vast array of currently deployed
clients and one that requires deployment of new client software everywhere to
stand even a chance of wortking, I'm strongly inclined to pursue the former.

Again, had we standardized flags for marking messages as spam 10 years ago, I'd
be fully in favor of using them for this additional purpose because there would
be plenty of deployed client support by now. But we didn't, there isn't, and I
don't hold out a lot of hope for _anything_ deploying with reasonable speed in
the IMAP client space right now.

				Ned



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66LRRPf087626 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 14:27:27 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n66LRRTx087625; Mon, 6 Jul 2009 14:27:27 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from sca-ea-mail-2.sun.com (sca-ea-mail-2.Sun.COM [192.18.43.25]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66LRFwU087616 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO) for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 14:27:26 -0700 (MST) (envelope-from Nicolas.Williams@sun.com)
Received: from dm-central-02.central.sun.com ([129.147.62.5]) by sca-ea-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n66LRFnJ017468 for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 21:27:15 GMT
Received: from binky.Central.Sun.COM (binky.Central.Sun.COM [129.153.128.104]) by dm-central-02.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n66LRFcP000553 for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 15:27:15 -0600 (MDT)
Received: from binky.Central.Sun.COM (localhost [127.0.0.1]) by binky.Central.Sun.COM (8.14.3+Sun/8.14.3) with ESMTP id n66KW0YG028138; Mon, 6 Jul 2009 15:32:00 -0500 (CDT)
Received: (from nw141292@localhost) by binky.Central.Sun.COM (8.14.3+Sun/8.14.3/Submit) id n66KVhZo028137; Mon, 6 Jul 2009 15:31:43 -0500 (CDT)
X-Authentication-Warning: binky.Central.Sun.COM: nw141292 set sender to Nicolas.Williams@sun.com using -f
Date: Mon, 6 Jul 2009 15:31:43 -0500
From: Nicolas Williams <Nicolas.Williams@sun.com>
To: Tony Finch <dot@dotat.at>
Cc: Michael Kirkham <mikek@muonics.com>, ietf-imapext@imc.org, Ned Freed <ned.freed@mrochek.com>, George Michaelson <ggm@pobox.com>, discuss@ietf.org
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
Message-ID: <20090706203143.GR15302@Sun.COM>
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <01NAZUST62HQ001DLH@mauve.mrochek.com> <20090706100834.R49379@eng-bsd.muonics.com> <alpine.LSU.2.00.0907062124240.30197@hermes-2.csi.cam.ac.uk>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <alpine.LSU.2.00.0907062124240.30197@hermes-2.csi.cam.ac.uk>
User-Agent: Mutt/1.5.7i
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

On Mon, Jul 06, 2009 at 09:25:32PM +0100, Tony Finch wrote:
> On Mon, 6 Jul 2009, Michael Kirkham wrote:
> > On Sun, 5 Jul 2009, Ned Freed wrote:
> >
> > > Please explain how moving a message to a random, arbtirarily named folder
> > > tells the server anything useful.
> >
> > Seems to me this is up to the server configuration and software running on the
> > server.  I don't see why you need a specific contract between IMAP client and
> > server for flagging spam
> 
> So that clients can have user interfaces that expose the functionality
> in a more friendly manner.

More specifically: so that users and their clients don't need to know
specific details of the server's configuration.



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66L7bn3086261 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 14:07:37 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n66L7bUX086260; Mon, 6 Jul 2009 14:07:37 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from rufus.isode.com (rufus.isode.com [62.3.217.251]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66L7ZFJ086252 for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 14:07:36 -0700 (MST) (envelope-from alexey.melnikov@isode.com)
Received: from [92.40.43.85] (92.40.43.85.sub.mbb.three.co.uk [92.40.43.85])  by rufus.isode.com (submission channel) via TCP with ESMTPA  id <SlJnlQBV9D0r@rufus.isode.com>; Mon, 6 Jul 2009 22:07:34 +0100
Message-ID: <4A526763.1020202@isode.com>
Date: Mon, 06 Jul 2009 22:06:43 +0100
From: Alexey Melnikov <alexey.melnikov@isode.com>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/20050915
X-Accept-Language: en-us, en
To: Iljitsch van Beijnum <iljitsch@muada.com>
CC: ietf-imapext@imc.org, discuss@ietf.org
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <01NAZUST62HQ001DLH@mauve.mrochek.com> <20090706100834.R49379@eng-bsd.muonics.com> <01NAZXHZ47O8000078@mauve.mrochek.com> <60154352-AE34-4FDD-A286-A2AF63C07903@muada.com>
In-Reply-To: <60154352-AE34-4FDD-A286-A2AF63C07903@muada.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

Iljitsch van Beijnum wrote:
 [...]

> About folders for spam: Apple's Mail lets the user select "use this  
> folder for... {sent, drafts, junk, trash}, which is nice if you used 
> a  different client before that uses a different folder naming scheme 
> for  these things. Works well enough although a little more 
> automation  wouldn't hurt.
>
> The point that working with folders can be done with existing 
> software  (I later train spamassassin with what Mail put in the "junk" 
> folder,  even though SA doesn't know about IMAP) is an important one. 
> Requiring  keywords that are hardcoded in clients and servers means 
> deployment  will be extremely slow, if it ever happens.

I don't think your argument in favour of folders (mailboxes) is valid. 
An IMAP server don't need to treat $Junk/$NotJunk any specially, it just 
needs to be able to store arbitrary IMAP keywords. Most servers already do.
A recent Thunderbird can also be configured to set arbitrary IMAP 
keywords. (It also recognized $Junk/$NonJunk natively, but this is 
besides my point).




Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66KtxQN085525 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 13:55:59 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n66KtxmJ085524; Mon, 6 Jul 2009 13:55:59 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from kalyani.oryx.com (kalyani.oryx.com [195.30.37.30]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66KtlJL085512 for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 13:55:58 -0700 (MST) (envelope-from arnt@gulbrandsen.priv.no)
Received: from kalyani.oryx.com (localhost [127.0.0.1]) by kalyani.oryx.com (Postfix) with ESMTP id C7F212E2A2; Mon,  6 Jul 2009 22:54:17 +0200 (CEST)
Received: from arnt@oryx.com (HELO lochnagar.oryx.com) by kalyani.oryx.com (Archiveopteryx 3.1.0) with esmtp id 1246913657-80172-80171/6/65 (5 recipients); Mon, 6 Jul 2009 22:54:17 +0200
Message-Id: <QlppcXA5Naz0BxqARBkNPQ.md5@lochnagar.oryx.com>
Date: Mon, 6 Jul 2009 22:56:19 +0200
From: Arnt Gulbrandsen <arnt@gulbrandsen.priv.no>
To: ietf-imapext@imc.org
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
Cc: Lyndon Nerenberg <lyndon@orthanc.ca>, Ned Freed <ned.freed@mrochek.com>, discuss@ietf.org
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <01NAZUST62HQ001DLH@mauve.mrochek.com> <1246902821.8602.70.camel@legolas.orthanc.ca> <01NAZWHC66ZU000078@mauve.mrochek.com> <1246904761.8602.86.camel@legolas.orthanc.ca>
In-Reply-To: <1246904761.8602.86.camel@legolas.orthanc.ca>
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset=iso-8859-1; format=flowed
Mime-Version: 1.0
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

Lyndon Nerenberg writes:
> But if your concern is adoption rates, a keyword-only solution will=20
> get wider take-up than something that requires support for=20
> annotations.

Given that two significant clients (say two of the top five) use the=20
keyword approach today, I think it's safe to say =ABhas received wider=20
take-up=BB, not =ABwill get wider=BB.

Arnt



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66KPrZQ083864 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 13:25:53 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n66KPrdC083863; Mon, 6 Jul 2009 13:25:53 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from ppsw-6.csi.cam.ac.uk (ppsw-6.csi.cam.ac.uk [131.111.8.136]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66KPfG0083850 for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 13:25:52 -0700 (MST) (envelope-from fanf2@hermes.cam.ac.uk)
X-Cam-AntiVirus: no malware found
X-Cam-SpamDetails: not scanned
X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/
Received: from hermes-2.csi.cam.ac.uk ([131.111.8.54]:54795) by ppsw-6.csi.cam.ac.uk (smtp.hermes.cam.ac.uk [131.111.8.156]:25) with esmtpa (EXTERNAL:fanf2) id 1MNukm-0001XD-JY (Exim 4.70) (return-path <fanf2@hermes.cam.ac.uk>); Mon, 06 Jul 2009 21:25:32 +0100
Received: from fanf2 (helo=localhost) by hermes-2.csi.cam.ac.uk (hermes.cam.ac.uk) with local-esmtp id 1MNukm-0003j2-1E (Exim 4.67) (return-path <fanf2@hermes.cam.ac.uk>); Mon, 06 Jul 2009 21:25:32 +0100
Date: Mon, 6 Jul 2009 21:25:32 +0100
From: Tony Finch <dot@dotat.at>
X-X-Sender: fanf2@hermes-2.csi.cam.ac.uk
To: Michael Kirkham <mikek@muonics.com>
cc: Ned Freed <ned.freed@mrochek.com>, ietf-imapext@imc.org, George Michaelson <ggm@pobox.com>, discuss@ietf.org
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
In-Reply-To: <20090706100834.R49379@eng-bsd.muonics.com>
Message-ID: <alpine.LSU.2.00.0907062124240.30197@hermes-2.csi.cam.ac.uk>
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <01NAZUST62HQ001DLH@mauve.mrochek.com> <20090706100834.R49379@eng-bsd.muonics.com>
User-Agent: Alpine 2.00 (LSU 1167 2008-08-23)
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

On Mon, 6 Jul 2009, Michael Kirkham wrote:
> On Sun, 5 Jul 2009, Ned Freed wrote:
>
> > Please explain how moving a message to a random, arbtirarily named folder
> > tells the server anything useful.
>
> Seems to me this is up to the server configuration and software running on the
> server.  I don't see why you need a specific contract between IMAP client and
> server for flagging spam

So that clients can have user interfaces that expose the functionality
in a more friendly manner.

Tony.
-- 
f.anthony.n.finch  <dot@dotat.at>  http://dotat.at/
GERMAN BIGHT HUMBER: SOUTHWEST 5 TO 7. MODERATE OR ROUGH. SQUALLY SHOWERS.
MODERATE OR GOOD.



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66KGvEm083408 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 13:16:57 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n66KGvix083407; Mon, 6 Jul 2009 13:16:57 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from peirce.dave.cridland.net (peirce.dave.cridland.net [217.155.137.61]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66KGiZS083386 for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 13:16:56 -0700 (MST) (envelope-from dave@cridland.net)
Received: from puncture ((unknown) [217.155.137.60]) by peirce.dave.cridland.net (submission) via TCP with ESMTPA id <SlJbqQAd5Hve@peirce.dave.cridland.net>; Mon, 6 Jul 2009 21:16:42 +0100
X-SMTP-Protocol-Errors: NORDNS
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <01NAZUST62HQ001DLH@mauve.mrochek.com> <1246902821.8602.70.camel@legolas.orthanc.ca>
In-Reply-To: <1246902821.8602.70.camel@legolas.orthanc.ca>
MIME-Version: 1.0
Message-Id: <13196.1246911390.279383@puncture>
Date: Mon, 06 Jul 2009 21:16:30 +0100
From: Dave Cridland <dave@cridland.net>
To: Lyndon Nerenberg <lyndon@orthanc.ca>, <ietf-imapext@imc.org>, general discussion of application-layer protocols <discuss@ietf.org>, Ned Freed <ned.freed@mrochek.com>
Content-Type: text/plain; delsp="yes"; charset="us-ascii"; format="flowed"
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

On Mon Jul  6 18:53:41 2009, Lyndon Nerenberg wrote:
> On Sun, 2009-07-05 at 17:40 -0700, Ned Freed wrote:
> > The obvious way to make this work in IMAP is with a standardized
> > folder
> > annotation saying "messages put in this folder were flagged as  
> spam".
> > Then
> > moves of messages to the folder with this annotation provide the
> > server with
> > the necessary information that this message is considered to be  
> spam
> > by the
> > user. And by the same token, moving a message from this folder  
> can be
> > construed
> > as the message not being considered spam after all.
> 
> This is just trying to turn a folder into a keyword, but without  
> full
> keyword semantics.

+1.

It's fundamentally no different from clients trying to replace the  
\Deleted keyword with a Trash mailbox, and has all the related  
issues, which we presumably all know. (Or else can find the attempt  
at documenting them I once tried).

This is compounded by the state being a tristate rather than binary -  
messages are "Ham", or "Spam", or else "The user has not expressed  
any decision". Should I move all my messages into another folder to  
mark them as good ham?

Dave.
-- 
Dave Cridland - mailto:dave@cridland.net - xmpp:dwd@dave.cridland.net
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66Jvcjm082027 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 12:57:38 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n66Jvca1082026; Mon, 6 Jul 2009 12:57:38 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66JvPCe082013 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 12:57:37 -0700 (MST) (envelope-from iljitsch@muada.com)
Received: from [192.168.2.4] (static-167-138-7-89.ipcom.comunitel.net [89.7.138.167] (may be forged)) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n66JthxZ053208 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Mon, 6 Jul 2009 21:55:44 +0200 (CEST) (envelope-from iljitsch@muada.com)
Cc: Michael Kirkham <mikek@muonics.com>, ietf-imapext@imc.org, George Michaelson <ggm@pobox.com>, discuss@ietf.org
Message-Id: <60154352-AE34-4FDD-A286-A2AF63C07903@muada.com>
From: Iljitsch van Beijnum <iljitsch@muada.com>
To: Ned Freed <ned.freed@mrochek.com>
In-Reply-To: <01NAZXHZ47O8000078@mauve.mrochek.com>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
Content-Transfer-Encoding: 7bit
Mime-Version: 1.0 (Apple Message framework v935.3)
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
Date: Mon, 6 Jul 2009 21:55:39 +0200
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <01NAZUST62HQ001DLH@mauve.mrochek.com> <20090706100834.R49379@eng-bsd.muonics.com> <01NAZXHZ47O8000078@mauve.mrochek.com>
X-Mailer: Apple Mail (2.935.3)
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

On 6 jul 2009, at 20:07, Ned Freed wrote:

>> it was pretty painless for me to set up cron jobs to suck in
>> whatever messages I save to certain folders (that could have been  
>> named
>> anything) for training my spam filter.

> Are you serious? Cron jobs? Do you have any idea how minute a  
> fraction of the
> users of email even know what a cron job is, let alone know how to  
> set one up?

Hey, everyone needs a hobby.

I have my procmail set up to check whether I have an IMAP session with  
the server. If so, I let the client do the mail filtering, if not, the  
server does it. That way, the unread counters for folders get updated  
when the client is connected, but the client doesn't have to move  
large volume mailing list traffic to the correct folder, costing me  
expensive wireless data traffic and/or slowing things down.

About folders for spam: Apple's Mail lets the user select "use this  
folder for... {sent, drafts, junk, trash}, which is nice if you used a  
different client before that uses a different folder naming scheme for  
these things. Works well enough although a little more automation  
wouldn't hurt.

The point that working with folders can be done with existing software  
(I later train spamassassin with what Mail put in the "junk" folder,  
even though SA doesn't know about IMAP) is an important one. Requiring  
keywords that are hardcoded in clients and servers means deployment  
will be extremely slow, if it ever happens.



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66IdsKm076543 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 11:39:54 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n66IdsZL076542; Mon, 6 Jul 2009 11:39:54 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from mauve.mrochek.com (mauve.mrochek.com [66.59.230.40]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66IdrNX076536 for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 11:39:53 -0700 (MST) (envelope-from ned.freed@mrochek.com)
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NAZXPKJNKG00SFNX@mauve.mrochek.com> for ietf-imapext@imc.org; Mon, 6 Jul 2009 11:39:50 -0700 (PDT)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NAZUBB7I4G000078@mauve.mrochek.com>; Mon, 06 Jul 2009 11:39:46 -0700 (PDT)
Date: Mon, 06 Jul 2009 11:35:17 -0700 (PDT)
From: Ned Freed <ned.freed@mrochek.com>
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
In-reply-to: "Your message dated Mon, 06 Jul 2009 12:02:40 -0600" <1246903360.8602.78.camel@legolas.orthanc.ca>
To: Lyndon Nerenberg <lyndon@orthanc.ca>
Cc: Ned Freed <ned.freed@mrochek.com>, ietf-imapext@imc.org
Message-id: <01NAZXPHSOKI000078@mauve.mrochek.com>
MIME-version: 1.0
Content-type: TEXT/PLAIN
Content-transfer-encoding: 7BIT
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <28941.1246868682.560969@puncture> <4A51D1A1.5040507@isode.com> <01NAZVQAYWH4001DLH@mauve.mrochek.com> <1246903360.8602.78.camel@legolas.orthanc.ca>
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

> On Mon, 2009-07-06 at 10:39 -0700, Ned Freed wrote:
> > The problem with using keywords for this is that you are now
> > dependence on your
> > client supporting that particular set of keywords and their associated
> > semantics. Had we standardize such a keyword set 10 or so years ago
> > this
> > approach miight have had a chance, but the client base is now too
> > large and
> > too difficult to update.

> Is it really too late? We're making an request to process a particular
> message through the site's "learning" filters, i.e. this is explicit
> request to have the server take action on the message. I don't that
> behaviour is implied by any of the current schemes floating around.

Well, you have two choices: Try to extend an existing ad-hoc keyword
usage scheme that has some deployment but doesn't have quite these semaantics
currently, or start from scratch with a fresh keyword set that has zero
deployment and is unlikely to displace any of the ad-hoc schemes.

This choice is why I view using keywords as at a minimum suboptimal.

> I don't see how it could hurt to standardize a pair of new keywords (as
> I suggested in my previous message) with this behaviour documented. It
> lets the user request specific identified behaviour from the server; the
> existing keywords are all private switches for client behaviour, in my
> experience, so the new keyword behaviour would be complimentary to
> what's already out there.

I on't think it would hurt either, the question is whether it will actually
accomplish something.

				Ned



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66IXp3l076243 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 11:33:51 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n66IXpf4076242; Mon, 6 Jul 2009 11:33:51 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from mauve.mrochek.com (mauve.mrochek.com [66.59.230.40]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66IXoMR076235 for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 11:33:51 -0700 (MST) (envelope-from ned.freed@mrochek.com)
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NAZXI19NJK00SFNX@mauve.mrochek.com> for ietf-imapext@imc.org; Mon, 6 Jul 2009 11:33:46 -0700 (PDT)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NAZUBB7I4G000078@mauve.mrochek.com>; Mon, 06 Jul 2009 11:33:43 -0700 (PDT)
Date: Mon, 06 Jul 2009 11:07:18 -0700 (PDT)
From: Ned Freed <ned.freed@mrochek.com>
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
In-reply-to: "Your message dated Mon, 06 Jul 2009 10:26:10 -0700 (PDT)" <20090706100834.R49379@eng-bsd.muonics.com>
To: Michael Kirkham <mikek@muonics.com>
Cc: Ned Freed <ned.freed@mrochek.com>, Iljitsch van Beijnum <iljitsch@muada.com>, ietf-imapext@imc.org, George Michaelson <ggm@pobox.com>, discuss@ietf.org
Message-id: <01NAZXHZ47O8000078@mauve.mrochek.com>
MIME-version: 1.0
Content-type: TEXT/PLAIN; format=flowed
Content-transfer-encoding: 7BIT
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <01NAZUST62HQ001DLH@mauve.mrochek.com> <20090706100834.R49379@eng-bsd.muonics.com>
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

> On Sun, 5 Jul 2009, Ned Freed wrote:

> > Please explain how moving a message to a random, arbtirarily named
> > folder tells the server anything useful.

> Seems to me this is up to the server configuration and software running on
> the server.  I don't see why you need a specific contract between IMAP
> client and server for flagging spam,

The primary reason is so that the server can use the information to
automaticallly refine it's filtering policies on a per-user basis.

Other uses include, but are not limited to, setting expiration and arhival
policies.

> and it's of limited usefulness.

On the contrary, such a mechanism would be HUGELY useful. We have any number of
customers begging for it, as a matter of fact.

> For
> one thing, it was pretty painless for me to set up cron jobs to suck in
> whatever messages I save to certain folders (that could have been named
> anything) for training my spam filter.

Are you serious? Cron jobs? Do you have any idea how minute a fraction of the
users of email even know what a cron job is, let alone know how to set one up?

The minute you start talking about setting up your own cron jobs, it  pretty
much goes without saying that you're operating entirely outside the space where
99+% of the use of IMAP takes place.

> I also have other cron jobs set up
> to do other things with email completely unrelated to spam that also
> simply involve saving to a particular folder (e.g. a tickler file for
> email--I move messages to folders named for a month or day, and cron jobs
> automatiacally reinsert them back in my inbox later).  That kind of thing
> isn't addressed by having something spam-specific, but as far as my email
> client is concerned, they're flagged the same way.

And no doubt you also tweak SpamAssasin rules, set up milters, etc. etc. This
is far beyond the capability of most users.

				Ned



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66IUei8076008 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 11:30:40 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n66IUe9J076007; Mon, 6 Jul 2009 11:30:40 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from rufus.isode.com (rufus.isode.com [62.3.217.251]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66IUc3x076001 for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 11:30:39 -0700 (MST) (envelope-from alexey.melnikov@isode.com)
Received: from [92.40.185.222] (92.40.185.222.sub.mbb.three.co.uk [92.40.185.222])  by rufus.isode.com (submission channel) via TCP with ESMTPA  id <SlJCzABV9Cxd@rufus.isode.com>; Mon, 6 Jul 2009 19:30:37 +0100
Message-ID: <4A524280.9000106@isode.com>
Date: Mon, 06 Jul 2009 19:29:20 +0100
From: Alexey Melnikov <alexey.melnikov@isode.com>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/20050915
X-Accept-Language: en-us, en
To: Lyndon Nerenberg <lyndon@orthanc.ca>
CC: Ned Freed <ned.freed@mrochek.com>, ietf-imapext@imc.org, discuss@ietf.org
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <01NAZUST62HQ001DLH@mauve.mrochek.com> <1246902821.8602.70.camel@legolas.orthanc.ca> <01NAZWHC66ZU000078@mauve.mrochek.com> <1246904761.8602.86.camel@legolas.orthanc.ca>
In-Reply-To: <1246904761.8602.86.camel@legolas.orthanc.ca>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

Lyndon Nerenberg wrote:

>But if your concern is adoption rates, a keyword-only solution will get
>wider take-up than something that requires support for annotations. (And
>from a purely engineering standpoint, the simplest solution wins.)
>  
>
+1.



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66IQLrS075707 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 11:26:21 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n66IQL2x075706; Mon, 6 Jul 2009 11:26:21 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from orthanc.ca (orthanc.ca [208.86.224.138]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66IQK0p075700 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 11:26:21 -0700 (MST) (envelope-from lyndon@orthanc.ca)
Received: from [192.168.0.42] (S01060011242eafe2.cg.shawcable.net [68.145.131.177]) (authenticated bits=0) by orthanc.ca (8.14.3/8.14.3) with ESMTP id n66IQ7lm095621; Mon, 6 Jul 2009 12:26:08 -0600 (MDT) (envelope-from lyndon@orthanc.ca)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=orthanc.ca; s=2009-1; t=1246904768; bh=WwrMcL9AQ6/lx87Ocpx19pPgFGa77TTioXFJrUQ6PG0=; h=Subject:From:To:Cc:In-Reply-To:References:Content-Type:Date: Message-Id:Mime-Version:Content-Transfer-Encoding; b=UpIRbLSvwCIbBPMKoSj32racmtHCxgrbXDONA7nswDpkJvUQBkRfgEKOzqw9ehBSX YqeF4VAVWmpI7RWOIjmTAI2ZPbGUTdryjtWGZE602ZKCP9EZ9hrIJ9Qtr+GQg6GHMa lMllguVaer1Sv+6qwOzmPhrWvCjFsB3G14Ss4qZmLMAieFrdu7Gv5Ak7bUoT71w/XN zcSGgCEzIAq0IddWJXtOZcAKiIkrAwDGtz3tXN0q0wdB4wSn4uCM0RBfucYRP0e+nY Lef60eIg9cG+xn15i8e9WRCjqp2JLP0tub0DmBDo/XyWz7DdCXRxDgJYNDclblVKa8 5pEpPz6qEeLag==
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
From: Lyndon Nerenberg <lyndon@orthanc.ca>
To: Ned Freed <ned.freed@mrochek.com>
Cc: ietf-imapext@imc.org, discuss@ietf.org
In-Reply-To: <01NAZWHC66ZU000078@mauve.mrochek.com>
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <01NAZUST62HQ001DLH@mauve.mrochek.com> <1246902821.8602.70.camel@legolas.orthanc.ca> <01NAZWHC66ZU000078@mauve.mrochek.com>
Content-Type: text/plain
Date: Mon, 06 Jul 2009 12:26:01 -0600
Message-Id: <1246904761.8602.86.camel@legolas.orthanc.ca>
Mime-Version: 1.0
X-Mailer: Evolution 2.26.2 FreeBSD GNOME Team Port 
Content-Transfer-Encoding: 7bit
X-Spam-Status: No, score=2.1 required=5.0 tests=DKIM_SIGNED,DKIM_VERIFIED, RCVD_IN_PBL,RCVD_IN_SORBS_DUL autolearn=no version=3.2.5
X-Spam-Level: **
X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on orthanc.ca
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

On Mon, 2009-07-06 at 11:00 -0700, Ned Freed wrote:
> If there's a reason multiple folders can't be marked this way I sure
> don't
> know what it is.

You're right -- I didn't parse what you were saying correctly the first
time around.

But if your concern is adoption rates, a keyword-only solution will get
wider take-up than something that requires support for annotations. (And
from a purely engineering standpoint, the simplest solution wins.)

--lyndon



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66IGNeL075045 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 11:16:23 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n66IGN2k075044; Mon, 6 Jul 2009 11:16:23 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from rufus.isode.com (rufus.isode.com [62.3.217.251]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66IGBM6075027 for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 11:16:22 -0700 (MST) (envelope-from alexey.melnikov@isode.com)
Received: from [172.16.2.141] (shiny.isode.com [62.3.217.250])  by rufus.isode.com (submission channel) via TCP with ESMTPA  id <SlI=ZQBV9Lui@rufus.isode.com>; Mon, 6 Jul 2009 19:16:09 +0100
Message-ID: <4A523F0E.5090706@isode.com>
Date: Mon, 06 Jul 2009 19:14:38 +0100
From: Alexey Melnikov <alexey.melnikov@isode.com>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/20050915
X-Accept-Language: en-us, en
To: Arnt Gulbrandsen <arnt@gulbrandsen.priv.no>
CC: general discussion of application-layer protocols <discuss@ietf.org>, ietf-imapext@imc.org, Ned Freed <ned.freed@mrochek.com>, Iljitsch van Beijnum <iljitsch@muada.com>, George Michaelson <ggm@pobox.com>
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <01NAZUST62HQ001DLH@mauve.mrochek.com> <UP3Anyei29+Pc7cGSi2Sdg.md5@lochnagar.oryx.com>
In-Reply-To: <UP3Anyei29+Pc7cGSi2Sdg.md5@lochnagar.oryx.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

Arnt Gulbrandsen wrote:

> Ned Freed writes:
>
>> The obvious way to make this work in IMAP is with a standardized 
>> folder annotation saying "messages put in this folder were flagged as 
>> spam".
>
> I think what Thunderbird and Apple Mail do is to set specific message 
> flags on messages in whatever folders they happen to be in: $Junk if 
> the user said a particular message is spam, $autojunk if tb 
> autonomously decided that message is spam, $nonjunk and $autononjunk 
> for the opposite. (I may be wrong about the exact flag names.)

Something like that, yes.

> This was specified in some expired draft,

Which was mine.

> which IMO now has been frozen by deployment. Apple Mail and 
> Thunderbird add up to >10% for me. Time for an informational RFC, 
> perhaps. "Widely-used IMAP flag names"?

Yes. I think even more important to create the IANA registry for keywords.
Do you want to help finish my draft?



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66I4Fwi074004 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 11:04:15 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n66I4FMU074003; Mon, 6 Jul 2009 11:04:15 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from mauve.mrochek.com (mauve.mrochek.com [66.59.230.40]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66I4E7T073996 for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 11:04:14 -0700 (MST) (envelope-from ned.freed@mrochek.com)
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NAZWHDBNTS005LCB@mauve.mrochek.com> for ietf-imapext@imc.org; Mon, 6 Jul 2009 11:04:12 -0700 (PDT)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NAZUBB7I4G000078@mauve.mrochek.com>; Mon, 06 Jul 2009 11:04:10 -0700 (PDT)
Date: Mon, 06 Jul 2009 11:00:24 -0700 (PDT)
From: Ned Freed <ned.freed@mrochek.com>
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
In-reply-to: "Your message dated Mon, 06 Jul 2009 11:53:41 -0600" <1246902821.8602.70.camel@legolas.orthanc.ca>
To: Lyndon Nerenberg <lyndon@orthanc.ca>
Cc: Ned Freed <ned.freed@mrochek.com>, ietf-imapext@imc.org, discuss@ietf.org
Message-id: <01NAZWHC66ZU000078@mauve.mrochek.com>
MIME-version: 1.0
Content-type: TEXT/PLAIN
Content-transfer-encoding: 7BIT
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <01NAZUST62HQ001DLH@mauve.mrochek.com> <1246902821.8602.70.camel@legolas.orthanc.ca>
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

> On Sun, 2009-07-05 at 17:40 -0700, Ned Freed wrote:
> > The obvious way to make this work in IMAP is with a standardized
> > folder
> > annotation saying "messages put in this folder were flagged as spam".
> > Then
> > moves of messages to the folder with this annotation provide the
> > server with
> > the necessary information that this message is considered to be spam
> > by the
> > user. And by the same token, moving a message from this folder can be
> > construed
> > as the message not being considered spam after all.

> This is just trying to turn a folder into a keyword, but without full
> keyword semantics.

Actually, it's a hack t work around the lack of client support, which is why
I didn't propose a keyword-based mechanism.

> The most obvious flaw is that it constrains the
> client to storing all the spam in one bucket.

If there's a reason multiple folders can't be marked this way I sure don't
know what it is.

> It doesn't allow for (say)
> sorting junk by type into different folders (perhaps on different
> servers). A message's 'state of spamminess' is an attribute of the
> message, properly denoted by keywords that stick to (and with) the
> message.

I see no reason why this couldn't be done, if you really want to.

> If you want this functionality (and I think it's a useful service for
> servers to provide) you need to implement two keywords: $spam and
> $nospam (these are just example names). Setting $spam on a message would
> instruct the server to pass the message off to the filtering component
> as spam; setting $nospam would instruct the server to pass the message
> off to the filtering component as valid (non-spam). The two keywords are
> exclusive, and it would be up to the server to decide what action to
> take if the client tries to light up both at once (e.g. unset the other
> flag and take the corresponding action, reject the command, or silently
> ignore the request).

Again, the problem with using keywords is now you have to get a significant
fraction of clients on board before it's useful. I'd be willing to go along
with a keyword-based approach if I could convinced that's going to happen, but
everything recentlts has supported exactly the opposite conclusion.

				Ned



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66I2nHe073879 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 11:02:49 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n66I2nuu073878; Mon, 6 Jul 2009 11:02:49 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from orthanc.ca (orthanc.ca [208.86.224.138]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66I2lok073870 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 11:02:48 -0700 (MST) (envelope-from lyndon@orthanc.ca)
Received: from [192.168.0.42] (S01060011242eafe2.cg.shawcable.net [68.145.131.177]) (authenticated bits=0) by orthanc.ca (8.14.3/8.14.3) with ESMTP id n66I2jxa095178; Mon, 6 Jul 2009 12:02:46 -0600 (MDT) (envelope-from lyndon@orthanc.ca)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=orthanc.ca; s=2009-1; t=1246903366; bh=IaIoBTuR8KNs1Prrx1DWRJHGYAhCgKsDSqZgGFvSJU8=; h=Subject:From:To:Cc:In-Reply-To:References:Content-Type:Date: Message-Id:Mime-Version:Content-Transfer-Encoding; b=DWAl8lPHNm+tcsPjH8YhV1CW28atPInh4lE7MFgls71WXpZ5ZwPZqfPVQNAbeMEKO bKWOuk+eg9enx41+z1NavXTfNT76KPjPNsMcD+6mQQWk8DD42mEi9ArK1axgjP8Ksl pP+wvE/Pg9Xhl5u3EWfcJZmMzSf1J0UywQ5dtsvNoUTZEJ0hudctPD+TJozZa1VqYy 8B9Q42QE304Da8Dtoxj5c4E4NrXWD9Ad/1lt8RU8bHtWMQM5MYafwozoCB1/sIco7a jzwh2UNEPJpcI4O0fb8FVKzxL290Fp+xbOmshBD1zFtkd8JHVftc0tlpvGySPu3A9V 0WPA/tyv6SebA==
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
From: Lyndon Nerenberg <lyndon@orthanc.ca>
To: Ned Freed <ned.freed@mrochek.com>
Cc: ietf-imapext@imc.org
In-Reply-To: <01NAZVQAYWH4001DLH@mauve.mrochek.com>
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <28941.1246868682.560969@puncture> <4A51D1A1.5040507@isode.com> <01NAZVQAYWH4001DLH@mauve.mrochek.com>
Content-Type: text/plain
Date: Mon, 06 Jul 2009 12:02:40 -0600
Message-Id: <1246903360.8602.78.camel@legolas.orthanc.ca>
Mime-Version: 1.0
X-Mailer: Evolution 2.26.2 FreeBSD GNOME Team Port 
Content-Transfer-Encoding: 7bit
X-Spam-Status: No, score=2.1 required=5.0 tests=DKIM_SIGNED,DKIM_VERIFIED, RCVD_IN_PBL,RCVD_IN_SORBS_DUL autolearn=no version=3.2.5
X-Spam-Level: **
X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on orthanc.ca
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

On Mon, 2009-07-06 at 10:39 -0700, Ned Freed wrote:
> The problem with using keywords for this is that you are now
> dependence on your
> client supporting that particular set of keywords and their associated
> semantics. Had we standardize such a keyword set 10 or so years ago
> this
> approach miight have had a chance, but the client base is now too
> large and
> too difficult to update.

Is it really too late? We're making an request to process a particular
message through the site's "learning" filters, i.e. this is explicit
request to have the server take action on the message. I don't that
behaviour is implied by any of the current schemes floating around.

I don't see how it could hurt to standardize a pair of new keywords (as
I suggested in my previous message) with this behaviour documented. It
lets the user request specific identified behaviour from the server; the
existing keywords are all private switches for client behaviour, in my
experience, so the new keyword behaviour would be complimentary to
what's already out there.

--lyndon



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66I2Ss8073854 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 11:02:29 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n66I2SPw073853; Mon, 6 Jul 2009 11:02:28 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from sca-ea-mail-2.sun.com (sca-ea-mail-2.Sun.COM [192.18.43.25]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66I25Sn073823 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO) for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 11:02:16 -0700 (MST) (envelope-from Nicolas.Williams@sun.com)
Received: from dm-central-02.central.sun.com ([129.147.62.5]) by sca-ea-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n66I24VB028574 for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 18:02:05 GMT
Received: from binky.Central.Sun.COM (binky.Central.Sun.COM [129.153.128.104]) by dm-central-02.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n66I246j054013 for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 12:02:04 -0600 (MDT)
Received: from binky.Central.Sun.COM (localhost [127.0.0.1]) by binky.Central.Sun.COM (8.14.3+Sun/8.14.3) with ESMTP id n66HiKfZ019363; Mon, 6 Jul 2009 12:44:20 -0500 (CDT)
Received: (from nw141292@localhost) by binky.Central.Sun.COM (8.14.3+Sun/8.14.3/Submit) id n66HiEk1019362; Mon, 6 Jul 2009 12:44:14 -0500 (CDT)
X-Authentication-Warning: binky.Central.Sun.COM: nw141292 set sender to Nicolas.Williams@sun.com using -f
Date: Mon, 6 Jul 2009 12:44:14 -0500
From: Nicolas Williams <Nicolas.Williams@sun.com>
To: Ned Freed <ned.freed@mrochek.com>
Cc: Iljitsch van Beijnum <iljitsch@muada.com>, ietf-imapext@imc.org, George Michaelson <ggm@pobox.com>, discuss@ietf.org
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
Message-ID: <20090706174414.GM15302@Sun.COM>
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <01NAZUST62HQ001DLH@mauve.mrochek.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <01NAZUST62HQ001DLH@mauve.mrochek.com>
User-Agent: Mutt/1.5.7i
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

On Sun, Jul 05, 2009 at 05:40:31PM -0700, Ned Freed wrote:
> 
> >On 3 jul 2009, at 1:05, George Michaelson wrote:
> 
> >> As an example, at the moment if I wish to inform google that I have
> >> known spam in local folders, I have to go to the web interface and
> >> manually tag. If there was an IMAP extension, I could review my
> >> local baysian junk folder, remove all non-spam (and flag the senders
> >> as white-listed if need be), and request the rest to be flagged as
> >> spam back on the IMAP backed MS.
> 
> >Doesn't moving the spam messages to the spam folder accomplish this
> >already? That's what my client does.
> 
> Please explain how moving a message to a random, arbtirarily named
> folder tells the server anything useful. People use all sorts of
> different naming schemes for folders, and it's common for clients to
> provide lots of flexibility in this area.

In particular the word 'spam' is really an English-language word.  L10N
would demand that the name of this folder not be 'spam' in all locales,
or at least that clients know how to map that folder name to the user's
language(s).  But even if we go for the latter, having special folder
names means denying or complicating the use of those names in other
languages for other purposes; it's bad enough we have one well-known
folder name (INBOX), let's have no more.

> The obvious way to make this work in IMAP is with a standardized
> folder annotation saying "messages put in this folder were flagged as
> spam". Then moves of messages to the folder with this annotation
> provide the server with the necessary information that this message is
> considered to be spam by the user. And by the same token, moving a
> message from this folder can be construed as the message not being
> considered spam after all.

+1

Or even just a message flag/annotation.  I understand the "move junk
mail to the junk mail folder" metaphor for flagging junk mail as such,
but one might want to flag junk mail and still move it to an "archived
mail" folder.  I don't find my argument for a per-message flag terribly
convincing, so I won't push it.

> It isn't perfect, but as long as this is taken as an indicator, not as
> an absolute statement, it should get the job done.
> 
> >But if you want this, I'd say that it needs to be a fractional thing,
> >not a binary spam/no spam indication. For instance, the server could
> >give something a spam score of 2 and the client also 2 and together
> >that would be 4 so the message is presumed to be spam (assuming the
> >spam threshold is 3), but in a binary system no spam OR no spam = no
> >spam.
> 
> There's a place for fractional scoring in spam filtering, but this isn't
> it. A user doesn't think "this message is 90% likely to be spam", they
> either think it's spam or it isn't.

+1

Fractional scoring by the server, or by software running on the client
can be useful, but the users themselves will generally make binary spam/
not-spam decisions.

Nico
-- 



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66Hx0oW073500 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 10:59:00 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n66Hx066073499; Mon, 6 Jul 2009 10:59:00 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from orthanc.ca (orthanc.ca [208.86.224.138]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66HwnKx073488 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 10:59:00 -0700 (MST) (envelope-from lyndon@orthanc.ca)
Received: from [192.168.0.42] (S01060011242eafe2.cg.shawcable.net [68.145.131.177]) (authenticated bits=0) by orthanc.ca (8.14.3/8.14.3) with ESMTP id n66HrlfS094952; Mon, 6 Jul 2009 11:53:47 -0600 (MDT) (envelope-from lyndon@orthanc.ca)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=orthanc.ca; s=2009-1; t=1246902828; bh=43yaytwJNSzxZ5RcTMNBAjKIwGNjzZ4YtRQZoliwL+M=; h=Subject:From:To:Cc:In-Reply-To:References:Content-Type:Date: Message-Id:Mime-Version:Content-Transfer-Encoding; b=SiB2C1/Pib6nDPH+BvnP1LLBdtfZGOBmv1Ifkl7CFQsN8W6leE2GPuD2pWLLmEq7p m8JIt+jG6YoCjg9osnORaz1AaXz5EDGObCkkfc12rK1tPWFybSFIcbDeRs6JMXtEQI HLujERdLnxCngkOVdrcrfCNS/8fKe0eecVM2SkEFtRx6tYgrFarb6Zfh/JOH+hIvKj o8ftiLby6kxMszh0mviaX+WDdehBTSllN7GJ7HajGs6OlCgZgglBDGwZpc6hIfAqBc QzLGt5QhcI0cbqleDaZxkrEK5UawakmhEZKqnNbwLl/cvz48ph2SlAG6sXnjkpTIQC tsbWon+QvYcLQ==
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
From: Lyndon Nerenberg <lyndon@orthanc.ca>
To: Ned Freed <ned.freed@mrochek.com>
Cc: ietf-imapext@imc.org, discuss@ietf.org
In-Reply-To: <01NAZUST62HQ001DLH@mauve.mrochek.com>
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <01NAZUST62HQ001DLH@mauve.mrochek.com>
Content-Type: text/plain
Date: Mon, 06 Jul 2009 11:53:41 -0600
Message-Id: <1246902821.8602.70.camel@legolas.orthanc.ca>
Mime-Version: 1.0
X-Mailer: Evolution 2.26.2 FreeBSD GNOME Team Port 
Content-Transfer-Encoding: 7bit
X-Spam-Status: No, score=2.1 required=5.0 tests=DKIM_SIGNED,DKIM_VERIFIED, RCVD_IN_PBL,RCVD_IN_SORBS_DUL autolearn=no version=3.2.5
X-Spam-Level: **
X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on orthanc.ca
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

On Sun, 2009-07-05 at 17:40 -0700, Ned Freed wrote:
> The obvious way to make this work in IMAP is with a standardized
> folder
> annotation saying "messages put in this folder were flagged as spam".
> Then
> moves of messages to the folder with this annotation provide the
> server with
> the necessary information that this message is considered to be spam
> by the
> user. And by the same token, moving a message from this folder can be
> construed
> as the message not being considered spam after all.

This is just trying to turn a folder into a keyword, but without full
keyword semantics. The most obvious flaw is that it constrains the
client to storing all the spam in one bucket. It doesn't allow for (say)
sorting junk by type into different folders (perhaps on different
servers). A message's 'state of spamminess' is an attribute of the
message, properly denoted by keywords that stick to (and with) the
message.

If you want this functionality (and I think it's a useful service for
servers to provide) you need to implement two keywords: $spam and
$nospam (these are just example names). Setting $spam on a message would
instruct the server to pass the message off to the filtering component
as spam; setting $nospam would instruct the server to pass the message
off to the filtering component as valid (non-spam). The two keywords are
exclusive, and it would be up to the server to decide what action to
take if the client tries to light up both at once (e.g. unset the other
flag and take the corresponding action, reject the command, or silently
ignore the request).



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66HnmSm072964 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 10:49:48 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n66Hnm4M072963; Mon, 6 Jul 2009 10:49:48 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from kalyani.oryx.com (kalyani.oryx.com [195.30.37.30]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66HnaXR072932 for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 10:49:47 -0700 (MST) (envelope-from arnt@gulbrandsen.priv.no)
Received: from kalyani.oryx.com (localhost [127.0.0.1]) by kalyani.oryx.com (Postfix) with ESMTP id D25AD2E2F5; Mon,  6 Jul 2009 19:48:06 +0200 (CEST)
Received: from arnt@oryx.com (HELO lochnagar.oryx.com) by kalyani.oryx.com (Archiveopteryx 3.1.0) with esmtp id 1246902486-80172-80171/6/64 (6 recipients); Mon, 6 Jul 2009 19:48:06 +0200
Message-Id: <UP3Anyei29+Pc7cGSi2Sdg.md5@lochnagar.oryx.com>
Date: Mon, 6 Jul 2009 19:50:08 +0200
From: Arnt Gulbrandsen <arnt@gulbrandsen.priv.no>
To: general discussion of application-layer protocols <discuss@ietf.org>, ietf-imapext@imc.org
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
Cc: Ned Freed <ned.freed@mrochek.com>, Iljitsch van Beijnum <iljitsch@muada.com>, George Michaelson <ggm@pobox.com>
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <01NAZUST62HQ001DLH@mauve.mrochek.com>
In-Reply-To: <01NAZUST62HQ001DLH@mauve.mrochek.com>
Content-Type: text/plain; format=flowed
Mime-Version: 1.0
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

Ned Freed writes:
> The obvious way to make this work in IMAP is with a standardized 
> folder annotation saying "messages put in this folder were flagged as 
> spam".

I think what Thunderbird and Apple Mail do is to set specific message 
flags on messages in whatever folders they happen to be in: $Junk if 
the user said a particular message is spam, $autojunk if tb 
autonomously decided that message is spam, $nonjunk and $autononjunk 
for the opposite. (I may be wrong about the exact flag names.)

This was specified in some expired draft, which IMO now has been frozen 
by deployment. Apple Mail and Thunderbird add up to >10% for me. Time 
for an informational RFC, perhaps. "Widely-used IMAP flag names"?

Arnt



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66HhEDH072587 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 10:43:14 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n66HhDT0072586; Mon, 6 Jul 2009 10:43:13 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from mauve.mrochek.com (mauve.mrochek.com [66.59.230.40]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66HhDn1072580 for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 10:43:13 -0700 (MST) (envelope-from ned.freed@mrochek.com)
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NAZVQBTCS0002DWJ@mauve.mrochek.com> for ietf-imapext@imc.org; Mon, 6 Jul 2009 10:43:11 -0700 (PDT)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NAUEIO1ZRK001DLH@mauve.mrochek.com>; Mon, 06 Jul 2009 10:43:09 -0700 (PDT)
Date: Mon, 06 Jul 2009 10:39:40 -0700 (PDT)
From: Ned Freed <ned.freed@mrochek.com>
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
In-reply-to: "Your message dated Mon, 06 Jul 2009 11:27:45 +0100" <4A51D1A1.5040507@isode.com>
To: Alexey Melnikov <alexey.melnikov@isode.com>
Cc: Iljitsch van Beijnum <iljitsch@muada.com>, Dave Cridland <dave@cridland.net>, ietf-imapext@imc.org, general discussion of application-layer protocols <discuss@ietf.org>, George Michaelson <ggm@pobox.com>
Message-id: <01NAZVQAYWH4001DLH@mauve.mrochek.com>
MIME-version: 1.0
Content-type: TEXT/PLAIN; format=flowed
Content-transfer-encoding: 7BIT
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <28941.1246868682.560969@puncture> <4A51D1A1.5040507@isode.com>
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

> Dave Cridland wrote:

> > On Sun Jul  5 20:03:30 2009, Iljitsch van Beijnum wrote:
> >
> >> On 3 jul 2009, at 1:05, George Michaelson wrote:
> >>
> >>> As an example, at the moment if I wish to inform google that I
> >>> have  known spam in local folders, I have to go to the web
> >>> interface and  manually tag. If there was an IMAP extension, I
> >>> could review my  local baysian junk folder, remove all non-spam
> >>> (and flag the senders  as white-listed if need be), and request  the
> >>> rest to be flagged as  spam back on the IMAP backed MS.
> >>
> >> Doesn't moving the spam messages to the spam folder accomplish this
> >>  already? That's what my client does.
> >
> > There's also a proposal (expired?) to use keywords to signal  spamminess.

> To expand on this: I believe the proposal is implemented by several
> major email clients (Thunderbird is one of them).

> [For people not familiar with IMAP: IMAP keywords is a basic IMAP
> feature implemented by majority of IMAP servers I've seen.]

The problem with using keywords for this is that you are now dependence on your
client supporting that particular set of keywords and their associated
semantics. Had we standardize such a keyword set 10 or so years ago this
approach miight have had a chance, but the client base is now too large and
too difficult to update.

This is why I think folder annotations are a better bet. You mark the spam
folder and use that marking to attach meaning to the movement of messages into
and out of that folder. It's not perfect, but it has the key advantage that it
works with any client (OK, any client that can use folders) once the annotation
is in place.

				Ned



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66HGOHH070959 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 10:16:24 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n66HGOq9070958; Mon, 6 Jul 2009 10:16:24 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from mauve.mrochek.com (mauve.mrochek.com [66.59.230.40]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66HGC4s070942 for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 10:16:22 -0700 (MST) (envelope-from ned.freed@mrochek.com)
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NAZUSTSTW000S1PD@mauve.mrochek.com> for ietf-imapext@imc.org; Mon, 6 Jul 2009 10:16:10 -0700 (PDT)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NAUEIO1ZRK001DLH@mauve.mrochek.com>; Mon, 06 Jul 2009 10:16:09 -0700 (PDT)
Date: Sun, 05 Jul 2009 17:40:31 -0700 (PDT)
From: Ned Freed <ned.freed@mrochek.com>
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
In-reply-to: "Your message dated Sun, 05 Jul 2009 21:03:30 +0200" <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com>
To: Iljitsch van Beijnum <iljitsch@muada.com>
Cc: George Michaelson <ggm@pobox.com>, ietf-imapext@imc.org, discuss@ietf.org
Message-id: <01NAZUST62HQ001DLH@mauve.mrochek.com>
MIME-version: 1.0
Content-type: TEXT/PLAIN; format=flowed; delsp=yes
Content-transfer-encoding: 7BIT
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com>
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

> On 3 jul 2009, at 1:05, George Michaelson wrote:

> > As an example, at the moment if I wish to inform google that I have
> > known spam in local folders, I have to go to the web interface and
> > manually tag. If there was an IMAP extension, I could review my
> > local baysian junk folder, remove all non-spam (and flag the senders
> > as white-listed if need be), and request the rest to be flagged as
> > spam back on the IMAP backed MS.

> Doesn't moving the spam messages to the spam folder accomplish this
> already? That's what my client does.

Please explain how moving a message to a random, arbtirarily named folder tells
the server anything useful. People use all sorts of different naming schemes
for folders, and it's common for clients to provide lots of flexibility in this
area.

The obvious way to make this work in IMAP is with a standardized folder
annotation saying "messages put in this folder were flagged as spam". Then
moves of messages to the folder with this annotation provide the server with
the necessary information that this message is considered to be spam by the
user. And by the same token, moving a message from this folder can be construed
as the message not being considered spam after all.

It isn't perfect, but as long as this is taken as an indicator, not as an
absolute statement, it should get the job done.

> But if you want this, I'd say that it needs to be a fractional thing,
> not a binary spam/no spam indication. For instance, the server could
> give something a spam score of 2 and the client also 2 and together
> that would be 4 so the message is presumed to be spam (assuming the
> spam threshold is 3), but in a binary system no spam OR no spam = no
> spam.

There's a place for fractional scoring in spam filtering, but this isn't
it. A user doesn't think "this message is 90% likely to be spam", they
either think it's spam or it isn't.

				Ned



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66ASkjD042325 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 03:28:46 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n66ASk49042324; Mon, 6 Jul 2009 03:28:46 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from rufus.isode.com (rufus.isode.com [62.3.217.251]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66ASY2Z042311 for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 03:28:45 -0700 (MST) (envelope-from alexey.melnikov@isode.com)
Received: from [92.40.185.222] (92.40.185.222.sub.mbb.three.co.uk [92.40.185.222])  by rufus.isode.com (submission channel) via TCP with ESMTPA  id <SlHRzwBV9F-C@rufus.isode.com>; Mon, 6 Jul 2009 11:28:33 +0100
Message-ID: <4A51D1A1.5040507@isode.com>
Date: Mon, 06 Jul 2009 11:27:45 +0100
From: Alexey Melnikov <alexey.melnikov@isode.com>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/20050915
X-Accept-Language: en-us, en
To: Iljitsch van Beijnum <iljitsch@muada.com>
CC: Dave Cridland <dave@cridland.net>, ietf-imapext@imc.org, general discussion of application-layer protocols <discuss@ietf.org>, George Michaelson <ggm@pobox.com>
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <28941.1246868682.560969@puncture>
In-Reply-To: <28941.1246868682.560969@puncture>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

Dave Cridland wrote:

> On Sun Jul  5 20:03:30 2009, Iljitsch van Beijnum wrote:
>
>> On 3 jul 2009, at 1:05, George Michaelson wrote:
>>
>>> As an example, at the moment if I wish to inform google that I  
>>> have  known spam in local folders, I have to go to the web  
>>> interface and  manually tag. If there was an IMAP extension, I  
>>> could review my  local baysian junk folder, remove all non-spam  
>>> (and flag the senders  as white-listed if need be), and request  the 
>>> rest to be flagged as  spam back on the IMAP backed MS.
>>
>> Doesn't moving the spam messages to the spam folder accomplish this  
>>  already? That's what my client does.
>
> There's also a proposal (expired?) to use keywords to signal  spamminess.

To expand on this: I believe the proposal is implemented by several 
major email clients (Thunderbird is one of them).

[For people not familiar with IMAP: IMAP keywords is a basic IMAP 
feature implemented by majority of IMAP servers I've seen.]



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n6695u1m036140 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 02:05:56 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n6695uci036139; Mon, 6 Jul 2009 02:05:56 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from peirce.dave.cridland.net (peirce.dave.cridland.net [217.155.137.61]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n6695ssS036133 for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 02:05:55 -0700 (MST) (envelope-from dave@cridland.net)
Received: from puncture ((unknown) [217.155.137.60]) by peirce.dave.cridland.net (submission) via TCP with ESMTPA id <SlG-bQAd5CTE@peirce.dave.cridland.net>; Mon, 6 Jul 2009 10:05:49 +0100
X-SMTP-Protocol-Errors: NORDNS
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <28941.1246868682.560969@puncture> <CEE69A99-B772-414A-9BE6-61C841BEDC29@muada.com>
In-Reply-To: <CEE69A99-B772-414A-9BE6-61C841BEDC29@muada.com>
MIME-Version: 1.0
Message-Id: <28941.1246871141.850024@puncture>
Date: Mon, 06 Jul 2009 10:05:41 +0100
From: Dave Cridland <dave@cridland.net>
To: Iljitsch van Beijnum <iljitsch@muada.com>, <ietf-imapext@imc.org>, general discussion of application-layer protocols <discuss@ietf.org>, George Michaelson <ggm@pobox.com>
Content-Type: text/plain; delsp="yes"; charset="iso-8859-1"; format="flowed"
Content-Transfer-Encoding: quoted-printable
X-Encoded: Changed encoding from 8Bit for 7bit transmission
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

On Mon Jul  6 09:37:06 2009, Iljitsch van Beijnum wrote:
> =

> On 6 jul 2009, at 10:24, Dave Cridland wrote:
> =

>>> But if you want this, I'd say that it needs to be a fractional   =

>>> thing,  not a binary spam/no spam indication. For instance, the   =

>>> server could  give something a spam score of 2 and the client  =

>>> also  2 and together  that would be 4 so the message is presumed  =

>>> to be  spam (assuming the  spam threshold is 3), but in a binary  =

>>> system no  spam OR no spam =3D no  spam.
> =

>> The notion of using two spam filters in concert to attempt to make  =

>>  an overall decision is basically flawed, because it fits into one  =

>> or  other case above - if you do go to the effort of having a  =

>> range  based spamminess from one, you'd have to use it as mere  =

>> input into  the other's decision process, since combining them  =

>> na=EFvely would  produce poorly weighted results.
> =

> Right. The filtering would have to be complimentary. So perhaps  =

> there  should be an indication of the type of filtering done, too.  =

> The reason  to combine server and client side approaches is that  =

> servers can  easily do blacklists, and running any type of  =

> filtering on the server  gives it the opportunity to reject the  =

> message during the SMTP  session, creating a good error message for  =

> real senders but no useless  bounces to innocent third parties in  =

> the case of spoofed senders. On  the client is useful because the  =

> client can typically better afford to  run CPU intensive stuff like  =

> baysian filters and the client probably  has a better list of  =

> previous correspondents that it can use to  whitelist.

Whitelists should, in principle, be easily comminicable to the server  =

(and easily processed).

Bayesian filters on the client are only really useful if the same  =

client is always used - this is often not the case (and, in my  =

experience, is becoming rarer). Otherwise the quality of your spam  =

filtering radically changes depending on the client you happen to be  =

using at the time.

And basically, any filtering at all that can be done on the server  =

saves a vast amount of bandwidth, and makes use of email on smaller  =

devices (like mobile phones) instantly more useful.

The one massive advantage that a client has is direct contact with  =

the user, and the eyeball remains the best spam filtering technology  =

we have.

I'm not suggesting that clients must not do spam filtering -  =

obviously they do, and it's often useful - but I think we should aim  =

to make server-side filtering the best we can. A very simple uniform  =

feedback mechanism of whether the user thinks a message is spam or  =

not makes this very much easier.

Dave.
-- =

Dave Cridland - mailto:dave@cridland.net - xmpp:dwd@dave.cridland.net
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n668cJVi034324 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 01:38:19 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n668cJNr034323; Mon, 6 Jul 2009 01:38:19 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n668c6R0034298 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 01:38:18 -0700 (MST) (envelope-from iljitsch@muada.com)
Received: from claw.it.uc3m.es (claw.it.uc3m.es [163.117.139.55] (may be forged)) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n668bAXQ048131 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Mon, 6 Jul 2009 10:37:11 +0200 (CEST) (envelope-from iljitsch@muada.com)
Cc: <ietf-imapext@imc.org>, general discussion of application-layer protocols <discuss@ietf.org>, George Michaelson <ggm@pobox.com>
Message-Id: <CEE69A99-B772-414A-9BE6-61C841BEDC29@muada.com>
From: Iljitsch van Beijnum <iljitsch@muada.com>
To: Dave Cridland <dave@cridland.net>
In-Reply-To: <28941.1246868682.560969@puncture>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed; delsp=yes
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (Apple Message framework v935.3)
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
Date: Mon, 6 Jul 2009 10:37:06 +0200
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <28941.1246868682.560969@puncture>
X-Mailer: Apple Mail (2.935.3)
X-Spam-Status: No, score=-2.6 required=3.4 tests=BAYES_00 autolearn=ham  version=3.0.2
X-Spam-Checker-Version: SpamAssassin 3.0.2 (2004-11-16) on sequoia.muada.com
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

On 6 jul 2009, at 10:24, Dave Cridland wrote:

>> But if you want this, I'd say that it needs to be a fractional =20
>> thing,  not a binary spam/no spam indication. For instance, the =20
>> server could  give something a spam score of 2 and the client also =20=

>> 2 and together  that would be 4 so the message is presumed to be =20
>> spam (assuming the  spam threshold is 3), but in a binary system no =20=

>> spam OR no spam =3D no  spam.

> The notion of using two spam filters in concert to attempt to make =20
> an overall decision is basically flawed, because it fits into one or =20=

> other case above - if you do go to the effort of having a range =20
> based spamminess from one, you'd have to use it as mere input into =20
> the other's decision process, since combining them na=EFvely would =20
> produce poorly weighted results.

Right. The filtering would have to be complimentary. So perhaps there =20=

should be an indication of the type of filtering done, too. The reason =20=

to combine server and client side approaches is that servers can =20
easily do blacklists, and running any type of filtering on the server =20=

gives it the opportunity to reject the message during the SMTP =20
session, creating a good error message for real senders but no useless =20=

bounces to innocent third parties in the case of spoofed senders. On =20
the client is useful because the client can typically better afford to =20=

run CPU intensive stuff like baysian filters and the client probably =20
has a better list of previous correspondents that it can use to =20
whitelist.=



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n668PBfu033253 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 01:25:11 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n668PB3C033252; Mon, 6 Jul 2009 01:25:11 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from peirce.dave.cridland.net (peirce.dave.cridland.net [217.155.137.61]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n668Ow8R033229 for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 01:25:09 -0700 (MST) (envelope-from dave@cridland.net)
Received: from puncture ((unknown) [217.155.137.60]) by peirce.dave.cridland.net (submission) via TCP with ESMTPA id <SlG00gAd5Jez@peirce.dave.cridland.net>; Mon, 6 Jul 2009 09:24:51 +0100
X-SMTP-Protocol-Errors: NORDNS
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com>
In-Reply-To: <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com>
MIME-Version: 1.0
Message-Id: <28941.1246868682.560969@puncture>
Date: Mon, 06 Jul 2009 09:24:42 +0100
From: Dave Cridland <dave@cridland.net>
To: Iljitsch van Beijnum <iljitsch@muada.com>, <ietf-imapext@imc.org>, general discussion of application-layer protocols <discuss@ietf.org>, George Michaelson <ggm@pobox.com>
Content-Type: text/plain; delsp="yes"; charset="iso-8859-1"; format="flowed"
Content-Transfer-Encoding: quoted-printable
X-Encoded: Changed encoding from 8Bit for 7bit transmission
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

On Sun Jul  5 20:03:30 2009, Iljitsch van Beijnum wrote:
> On 3 jul 2009, at 1:05, George Michaelson wrote:
> =

>> As an example, at the moment if I wish to inform google that I  =

>> have  known spam in local folders, I have to go to the web  =

>> interface and  manually tag. If there was an IMAP extension, I  =

>> could review my  local baysian junk folder, remove all non-spam  =

>> (and flag the senders  as white-listed if need be), and request  =

>> the rest to be flagged as  spam back on the IMAP backed MS.
> =

> Doesn't moving the spam messages to the spam folder accomplish this  =

>  already? That's what my client does.
> =

> =

There's also a proposal (expired?) to use keywords to signal  =

spamminess.

> But if you want this, I'd say that it needs to be a fractional  =

> thing,  not a binary spam/no spam indication. For instance, the  =

> server could  give something a spam score of 2 and the client also  =

> 2 and together  that would be 4 so the message is presumed to be  =

> spam (assuming the  spam threshold is 3), but in a binary system no  =

> spam OR no spam =3D no  spam.

Of course, if both client and server use precisely the same criteria,  =

you've simply halved your threshold.

There's two cases:

1) The server has some feedback-based spam detection mechanism, like  =

a bayesian filter. You want to teach the server's filter about your  =

explicit decisions.

2) Your client has a spam filter (or some sort), you want the server  =

to tell your client about it's preliminary findings.

The notion of using two spam filters in concert to attempt to make an  =

overall decision is basically flawed, because it fits into one or  =

other case above - if you do go to the effort of having a range based  =

spamminess from one, you'd have to use it as mere input into the  =

other's decision process, since combining them na=EFvely would produce  =

poorly weighted results.

A simple keyword approach isn't quite as good as ranges, but it does  =

have the useful property that nearly every service already offers  =

arbitrary keywords, and so is quite likely to already offer it.

Ranges, on the other hand, require the use of annotations, which is a  =

rather more complex area.

Dave.
-- =

Dave Cridland - mailto:dave@cridland.net - xmpp:dwd@dave.cridland.net
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n65J4A4u086920 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 5 Jul 2009 12:04:10 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n65J4AKm086919; Sun, 5 Jul 2009 12:04:10 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n65J3w6a086896 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for <ietf-imapext@imc.org>; Sun, 5 Jul 2009 12:04:09 -0700 (MST) (envelope-from iljitsch@muada.com)
Received: from [192.168.2.4] (static-167-138-7-89.ipcom.comunitel.net [89.7.138.167] (may be forged)) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n65J3Z8C043962 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Sun, 5 Jul 2009 21:03:38 +0200 (CEST) (envelope-from iljitsch@muada.com)
Cc: ietf-imapext@imc.org, discuss@ietf.org
Message-Id: <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com>
From: Iljitsch van Beijnum <iljitsch@muada.com>
To: George Michaelson <ggm@pobox.com>
In-Reply-To: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
Content-Transfer-Encoding: 7bit
Mime-Version: 1.0 (Apple Message framework v935.3)
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
Date: Sun, 5 Jul 2009 21:03:30 +0200
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com>
X-Mailer: Apple Mail (2.935.3)
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

On 3 jul 2009, at 1:05, George Michaelson wrote:

> As an example, at the moment if I wish to inform google that I have  
> known spam in local folders, I have to go to the web interface and  
> manually tag. If there was an IMAP extension, I could review my  
> local baysian junk folder, remove all non-spam (and flag the senders  
> as white-listed if need be), and request the rest to be flagged as  
> spam back on the IMAP backed MS.

Doesn't moving the spam messages to the spam folder accomplish this  
already? That's what my client does.

But if you want this, I'd say that it needs to be a fractional thing,  
not a binary spam/no spam indication. For instance, the server could  
give something a spam score of 2 and the client also 2 and together  
that would be 4 so the message is presumed to be spam (assuming the  
spam threshold is 3), but in a binary system no spam OR no spam = no  
spam.



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n6TCY4tu055819 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 29 Jul 2009 05:34:04 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n6TCY4eQ055818; Wed, 29 Jul 2009 05:34:04 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from mail-px0-f192.google.com (mail-px0-f192.google.com [209.85.216.192]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n6TCXqPr055803 for <ietf-imapext@imc.org>; Wed, 29 Jul 2009 05:34:02 -0700 (MST) (envelope-from barryleiba.mailing.lists@gmail.com)
Received: by pxi30 with SMTP id 30so479921pxi.16 for <ietf-imapext@imc.org>; Wed, 29 Jul 2009 05:33:51 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:reply-to:date:message-id :subject:from:to:content-type:content-transfer-encoding; bh=EOb+oYZ8KhCkjSo1khF/aLBdQ7t3H84eVvgpjrMdwJk=; b=Ch69Fp5FOttWTYYTfze1V2RnAgK3pMrFnTLhXctms7LuN6UKJstU2cwpckZYx1abkN guPYrau8Z3W3z3BVB4kIl4PivFB1UHcGjaCjLvhZdI3rb+y5tmVyveH1tui8ZFeVGAxW 9woLzgqt40hJHVdHhcMkIx1d3d1CQsavMwdSE=
DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:reply-to:date:message-id:subject:from:to:content-type :content-transfer-encoding; b=QYnPEzlBzxd86fMtQD8awGZGDv6Fg13zdZ5A3+1no6c0ZeHK8Ix4kTWtH/prI2Ckbp QHuvLa7LVjjb/y0n/GIWvjVqsMHYYr7p5uXMjFr8HfLn2n5Jkx56bM1ccZnbgM96FKVJ D2/XsYA8yB899i0MIKGHBUx6kVbcYCuK+N2WA=
MIME-Version: 1.0
Received: by 10.114.180.16 with SMTP id c16mr13654081waf.57.1248870831894;  Wed, 29 Jul 2009 05:33:51 -0700 (PDT)
Reply-To: barryleiba@computer.org
Date: Wed, 29 Jul 2009 08:33:51 -0400
Message-ID: <6c9fcc2a0907290533v4b96ea0co76e4cb6b0b24ca34@mail.gmail.com>
Subject: "Friends of IMAP" dinner in Stockholm on IETF Friday
From: Barry Leiba <barryleiba.mailing.lists@gmail.com>
To: morg@ietf.org, IMAP Extensions WG <ietf-imapext@imc.org>, ima@ietf.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

We'll be gathering for the traditional "Friends of IMAP" dinner, after
the IETF meeting, in Stockholm.  Because the only time we can get most
of the people together is Friday, we'll do it on Friday evening, 31
July.  We'll meet in the entrance lobby of the Clarion Sign Hotel at
19:00.

No plan, yet, about where to do.

Barry



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n6LLnU00071181 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 21 Jul 2009 14:49:31 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n6LLnU2h071180; Tue, 21 Jul 2009 14:49:30 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from orthanc.ca (orthanc.ca [208.86.224.138]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n6LLnJT7071159 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for <ietf-imapext@imc.org>; Tue, 21 Jul 2009 14:49:30 -0700 (MST) (envelope-from lyndon@orthanc.ca)
Received: from [192.168.0.42] (S01060011242eafe2.cg.shawcable.net [68.145.131.177]) (authenticated bits=0) by orthanc.ca (8.14.3/8.14.3) with ESMTP id n6LLnGSE063020 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 21 Jul 2009 15:49:17 -0600 (MDT) (envelope-from lyndon@orthanc.ca)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=orthanc.ca; s=2009-1; t=1248212958; bh=NrmKwo6Y6GZtjjabrv6/ebzejzrOkRT6W/HZakv7JOk=; h=Date:From:To:Subject:In-Reply-To:Message-ID:References: MIME-Version:Content-Type; b=N5GQepe7FeixBXyiAFnlazZ4YJxkBsdlqkEnprHM6xFF33VniUhFH/py026wJ8v9R 4YwYIo/AohIXQdsobPXsrANy7vr+KQtzkFYoDUSvHiftSE/vc1et7ntm72eqSzkyXd zOo7zMrPF3HyqTWg1KvuZayZ88KAx1BT9+8ZKd2j61+niYjNTgt+hmSW7X++TqYZxy UlFtQ5TxhoxpkFHrAyjUkyG8GKzpAyIGI/B7a5EmAuTKf6kWfpVy1huTLqzNXiOb6e DV1VCN+Xp3q7/RUBXuiivFBzYbBecRoiBnPpE9JqNPUpBryCcZH4bF57ML6/gP6XD7 GW3WlzaYl+Vaw==
Date: Tue, 21 Jul 2009 15:49:11 -0600 (MDT)
From: Lyndon Nerenberg <lyndon@orthanc.ca>
To: imap-protocol@u.washington.edu, ietf-imapext@imc.org
Subject: Re: A survey of Spam-related keyword usage by IMAP clients.
In-Reply-To: <01NB1I98RDS4001ML6@mauve.mrochek.com>
Message-ID: <alpine.BSF.2.00.0907211541390.25656@legolas.orthanc.ca>
References: <1247001507.8602.109.camel@legolas.orthanc.ca> <01NB1I98RDS4001ML6@mauve.mrochek.com>
User-Agent: Alpine 2.00 (BSF 1167 2008-08-23)
Organization: The Frobozz Magic Homing Pigeon Company
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII
X-Spam-Status: No, score=0.3 required=5.0 tests=AWL,BAYES_00,DKIM_SIGNED, DKIM_VERIFIED,RCVD_IN_PBL,RCVD_IN_SORBS_DUL autolearn=no version=3.2.5
X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on orthanc.ca
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

The response to the keyword survey has been ... underwhelming. Surely 
there is more than one client represented on the mailing lists?!?

Once again, the information we're looking for is described below. Please 
forward this to every client writer you know and ask them to respond. 
We're never going to solve this problem if these people don't start 
participating.

Please send responses directly to me.

>> It's time to get a current view of how IMAP clients are using keywords
>> to manage spam. If you are a client author I would appreciate it if you
>> could take a moment to send me a note describing which keywords your
>> client uses for spam-related processing, and what each of the keywords
>> signifies.

> I'd also really like to know if the client has "this is spam" or "this 
> is not spam" buttons in the UI, and if it does how those buttons 
> interact with the use of keywords.

> I ask this because users of any client that has such buttons are clearly 
> going to use those buttons preferentially over any other labelling 
> mechanism. And if a significant number of clients have such buttons that 
> do a particular thing - regardless of what that thing is - that's pretty 
> much going to be determinative in finding the most easily deployed 
> approach.

>> If your client does not use keywords for this purpose, that's
>> valuable information, too, as is a breakdown of functionality by
>> software version number if there are significant differences. (If you
>> use non-keyword facilities to manage spam feel free to include a short
>> description of your methods if you think the info might be relevant.)
>
> Yes please. I'm especially interested in how many clients do this by 
> resending the message to a special address.
>
>> I'm primarily interested in getting authoritative data from the client
>> authors, but others should feel free to chime in with deployment-based
>> experience of related keyword usage, but please indicate the source of
>> your comments (e.g. deployment experience, source code examination).



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n692ob3f093436 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 8 Jul 2009 19:50:37 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n692obeK093435; Wed, 8 Jul 2009 19:50:37 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from daboo.name (daboo.name [151.201.22.177]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n692oQtQ093411 for <ietf-imapext@imc.org>; Wed, 8 Jul 2009 19:50:36 -0700 (MST) (envelope-from cyrus@daboo.name)
Received: from localhost (localhost [127.0.0.1]) by daboo.name (Postfix) with ESMTP id 036E81691025; Wed,  8 Jul 2009 22:50:25 -0400 (EDT)
X-Virus-Scanned: amavisd-new at daboo.name
Received: from daboo.name ([127.0.0.1]) by localhost (chewy.mulberrymail.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1mRu4KuRy9ZF; Wed,  8 Jul 2009 22:50:23 -0400 (EDT)
Received: from [10.0.1.8] (unknown [10.0.1.1]) by daboo.name (Postfix) with ESMTP id 5D0B7169101A; Wed,  8 Jul 2009 22:50:23 -0400 (EDT)
Date: Wed, 08 Jul 2009 22:50:22 -0400
From: Cyrus Daboo <cyrus@daboo.name>
To: Ned Freed <ned.freed@mrochek.com>, George Michaelson <ggm@pobox.com>
cc: ietf-imapext@imc.org, general discussion of application-layer protocols <discuss@ietf.org>
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
Message-ID: <20A9EE8D49D98D53C36F6F48@socrates.local>
In-Reply-To: <01NB338GXYBS00007A@mauve.mrochek.com>
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <28941.1246868682.560969@puncture> <4A51D1A1.5040507@isode.com>	<01NAZVQAYWH4001DLH@mauve.mrochek.com> <47CB230E-B713-4098-82D4-318744434C46@pobox.com> <01NB0BOZYUPY001ML6@mauve.mrochek.com> <6DECD560-B257-4E6E-80E7-A99A69D297C6@pobox.com> <01NB14JAPS8E001ML6@mauve.mrochek.com> <F4A90756-9251-4CB8-9776-77F3202617A5@pobox.com> <01NB1TV04NJS001ML6@mauve.mrochek.com> <4A25E43A-83E3-4A59-9C0B-C334820275BF@pobox.com> <01NB338GXYBS00007A@mauve.mrochek.com>
X-Mailer: Mulberry/4.1.0a1 (Mac OS X)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline; size=675
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

Hi Ned,

--On July 8, 2009 5:30:06 PM -0700 Ned Freed <ned.freed@mrochek.com> wrote:

> I note, however, that now that CALDAV is complete we see Apple,
> Thunderbird, and gmail all either supporting it or working on supporting
> it.

For a more comprehensive list of implementations see: 
<http://caldav.calconnect.org/implementations.html>.

> It's a little much to expect Outlook to shift from it's currrent
> proprietary way of doing all this, but even Microsoft may eventually have
> no choice but to support the emerging standards in this area.

In the meantime the folks at <http://zideone.com> have a plugin for Outlook 
that does CalDAV and CardDAV.

-- 
Cyrus Daboo



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n690oUYQ085187 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 8 Jul 2009 17:50:30 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n690oUZD085186; Wed, 8 Jul 2009 17:50:30 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from mauve.mrochek.com (mauve.mrochek.com [66.59.230.40]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n690oIJr085155 for <ietf-imapext@imc.org>; Wed, 8 Jul 2009 17:50:28 -0700 (MST) (envelope-from ned.freed@mrochek.com)
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NB338IGJXS00QEO9@mauve.mrochek.com> for ietf-imapext@imc.org; Wed, 8 Jul 2009 17:50:16 -0700 (PDT)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NB32ESTD2800007A@mauve.mrochek.com>; Wed, 08 Jul 2009 17:50:13 -0700 (PDT)
Date: Wed, 08 Jul 2009 17:30:06 -0700 (PDT)
From: Ned Freed <ned.freed@mrochek.com>
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
In-reply-to: "Your message dated Thu, 09 Jul 2009 08:27:39 +1000" <4A25E43A-83E3-4A59-9C0B-C334820275BF@pobox.com>
To: George Michaelson <ggm@pobox.com>
Cc: Ned Freed <ned.freed@mrochek.com>, ietf-imapext@imc.org, general discussion of application-layer protocols <discuss@ietf.org>
Message-id: <01NB338GXYBS00007A@mauve.mrochek.com>
MIME-version: 1.0
Content-type: TEXT/PLAIN; format=flowed; delsp=yes
Content-transfer-encoding: 7BIT
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <28941.1246868682.560969@puncture> <4A51D1A1.5040507@isode.com> <01NAZVQAYWH4001DLH@mauve.mrochek.com> <47CB230E-B713-4098-82D4-318744434C46@pobox.com> <01NB0BOZYUPY001ML6@mauve.mrochek.com> <6DECD560-B257-4E6E-80E7-A99A69D297C6@pobox.com> <01NB14JAPS8E001ML6@mauve.mrochek.com> <F4A90756-9251-4CB8-9776-77F3202617A5@pobox.com> <01NB1TV04NJS001ML6@mauve.mrochek.com> <4A25E43A-83E3-4A59-9C0B-C334820275BF@pobox.com>
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

> >>
> >
> > The place where I'd like to see white and black listing done is in
> > CARDDAV, not
> > IMAP.
> >
> > 				Ned

> so there's only one real question: if it was standardized, would the
> likes of google implement it? How about the client side?

This is actually one of the reasons why CARDDAV is a better choice to piggyback
on.

RIght now proprietary environments like Exchange/Outlook and Lotus Notes
have a significant advantage over standards-based solutions: They provide
a fully integrated mail/calendar/addressbook client experience.

One reason standards-based solutions have a problem with this has been the lack
of open protocols for accessing and manipulating calendar and addressbook
information.

But this is changing. CALDAV addresses the calendar access issue, CARDDAV does
the same for address books. Once all the pieces are in place, and assuming we
get the details right, this will allow standards-based clients and servers all
to interoperate.

Uptake isn't going to be fast because these are complex protocols and
getting the integration right isn't easy. But for once the incentives
appear to be woring for us, not against us.

> because, unless its in gmail AND in the Apple Mail client,
> Thunderbird, Outlook, its not useful.

That may be true now, but at some point given sufficient adoption things flip
around to  where not having support for this stuff makes your client or server
not useful.

I note, however, that now that CALDAV is complete we see Apple, Thunderbird,
and gmail all either supporting it or working on supporting it. It's a little
much to expect Outlook to shift from it's currrent proprietary way of doing
all this, but even Microsoft may eventually have no choice but to support
the emerging standards in this area.

				Ned



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n68MS1Vi076663 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 8 Jul 2009 15:28:01 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n68MS1PQ076662; Wed, 8 Jul 2009 15:28:01 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from ey-out-2122.google.com (ey-out-2122.google.com [74.125.78.25]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n68MRnxn076647 for <ietf-imapext@imc.org>; Wed, 8 Jul 2009 15:28:00 -0700 (MST) (envelope-from geeohgeegeeoh@gmail.com)
Received: by ey-out-2122.google.com with SMTP id 4so1337311eyf.9 for <ietf-imapext@imc.org>; Wed, 08 Jul 2009 15:27:48 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:cc:message-id:from:to :in-reply-to:content-type:content-transfer-encoding:mime-version :subject:date:references:x-mailer; bh=TXGOqBBjbdG9nXZAFR79a2n2MsazOtTdRvdhoCvIooU=; b=NQk57XltAke0iE7kWCRT7H0wqFsmRkobkUn5XDMgS21BgacIhWqog3xlYn1RdmP6RI YLrgX6bpp9J/1f112knqqr9xnrhCrMb7raCi4GpQvvwOZZtAjv41E19s0NzKC/mQMeby eVn36eHb3AzFZyHIs/nScOpeGowqWF3ZuGX2k=
DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:cc:message-id:from:to:in-reply-to:content-type :content-transfer-encoding:mime-version:subject:date:references :x-mailer; b=ZwEzaNEjT+YsIf+t5klQYJEkYjykNSrF5W2HdSnz8c5SXePXIu6zu4YOOVKxBJCKmf ai/4sMfp3DF6ylc03AQWZTEMDR93uudksCQ9x3okpwUnv3Fmt9rFXvElZCPV2O4VRPHg Nn+b5V/J+DWWVxERWoF2yAmccVjbmIYdsO9Yg=
Received: by 10.210.109.19 with SMTP id h19mr8612445ebc.56.1247092068279; Wed, 08 Jul 2009 15:27:48 -0700 (PDT)
Received: from 192-168-1-5.tpgi.com.au (60-240-104-100-nsw-pppoe.tpgi.com.au [60.240.104.100]) by mx.google.com with ESMTPS id 7sm1176374eyg.19.2009.07.08.15.27.44 (version=TLSv1/SSLv3 cipher=RC4-MD5); Wed, 08 Jul 2009 15:27:47 -0700 (PDT)
Cc: ietf-imapext@imc.org, general discussion of application-layer protocols <discuss@ietf.org>
Message-Id: <4A25E43A-83E3-4A59-9C0B-C334820275BF@pobox.com>
From: George Michaelson <ggm@pobox.com>
To: Ned Freed <ned.freed@mrochek.com>
In-Reply-To: <01NB1TV04NJS001ML6@mauve.mrochek.com>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
Content-Transfer-Encoding: 7bit
Mime-Version: 1.0 (Apple Message framework v935.3)
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
Date: Thu, 9 Jul 2009 08:27:39 +1000
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <28941.1246868682.560969@puncture> <4A51D1A1.5040507@isode.com> <01NAZVQAYWH4001DLH@mauve.mrochek.com> <47CB230E-B713-4098-82D4-318744434C46@pobox.com> <01NB0BOZYUPY001ML6@mauve.mrochek.com> <6DECD560-B257-4E6E-80E7-A99A69D297C6@pobox.com> <01NB14JAPS8E001ML6@mauve.mrochek.com> <F4A90756-9251-4CB8-9776-77F3202617A5@pobox.com> <01NB1TV04NJS001ML6@mauve.mrochek.com>
X-Mailer: Apple Mail (2.935.3)
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

>>
>
> The place where I'd like to see white and black listing done is in  
> CARDDAV, not
> IMAP.
>
> 				Ned

so there's only one real question: if it was standardized, would the  
likes of google implement it? How about the client side?

because, unless its in gmail AND in the Apple Mail client,  
Thunderbird, Outlook, its not useful.

-G



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n68J9Bcd063429 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 8 Jul 2009 12:09:11 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n68J9BbU063428; Wed, 8 Jul 2009 12:09:11 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from mauve.mrochek.com (mauve.mrochek.com [66.59.230.40]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n68J902N063391 for <ietf-imapext@imc.org>; Wed, 8 Jul 2009 12:09:10 -0700 (MST) (envelope-from ned.freed@mrochek.com)
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NB2RBB0E4G015OWJ@mauve.mrochek.com> for ietf-imapext@imc.org; Wed, 8 Jul 2009 12:08:55 -0700 (PDT)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NB2NV4DYVK001ML6@mauve.mrochek.com>; Wed, 08 Jul 2009 12:08:52 -0700 (PDT)
Date: Wed, 08 Jul 2009 11:54:05 -0700 (PDT)
From: Ned Freed <ned.freed@mrochek.com>
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
In-reply-to: "Your message dated Wed, 08 Jul 2009 11:08:28 +0100" <4A54701C.6080107@isode.com>
To: Alexey Melnikov <alexey.melnikov@isode.com>
Cc: George Michaelson <ggm@pobox.com>, Ned Freed <ned.freed@mrochek.com>, ietf-imapext@imc.org, general discussion of application-layer protocols <discuss@ietf.org>
Message-id: <01NB2RB8UTA4001ML6@mauve.mrochek.com>
MIME-version: 1.0
Content-type: TEXT/PLAIN; format=flowed
Content-transfer-encoding: 7BIT
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <28941.1246868682.560969@puncture> <4A51D1A1.5040507@isode.com> <01NAZVQAYWH4001DLH@mauve.mrochek.com> <47CB230E-B713-4098-82D4-318744434C46@pobox.com> <01NB0BOZYUPY001ML6@mauve.mrochek.com> <6DECD560-B257-4E6E-80E7-A99A69D297C6@pobox.com> <01NB14JAPS8E001ML6@mauve.mrochek.com> <F4A90756-9251-4CB8-9776-77F3202617A5@pobox.com> <01NB1TV04NJS001ML6@mauve.mrochek.com> <4A54701C.6080107@isode.com>
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

> Ned Freed wrote:

> >> I have no desire to argue against my own proposal! sorry if I
> >> misunderstood the tenor of some comments to this suggestion.
> >
> >> The original posting asked about two things
> >
> >> 1) spam/ham
> >
> >> 2) white/black
> >
> >> so far, people seem to be focussing on 1). I acknowledge problems with
> >> 1), I appreciate that people are trying to address 1) as a possibility.
> >
> >> I asked about 2). I asked you if, ignoring 1), it was possible or
> >> plausible to send white/black information about sender, if 1) was hard
> >> or intractable.
> >
> > Of course it's possible, but this has no business being in IMAP. IMAP
> > is a
> > message access protocol, not an address handling protocol. It is also
> > already
> > quite complex, and extending its function to include what amountss to a
> > specialized address book capability is a really bad idea IMO.
> >
> > In fact a pretty good case can be made that it's inappropriate to use
> > IMAP for spam/ham classification, but at the spam/ham classification is
> > intrinsicly tied to messages in the store. The same cannot be said for
> > address white and blacklisting.

> +1

> >> Does that make more sense? I do realize white/black listing is a
> >> different PROBLEM to spam/ham marking, but neither of them currently
> >> have support directly in imap, and I imagine both of them would use
> >> IMAP extensions to pass information client-server.
> >
> > The place where I'd like to see white and black listing done is in
> > CARDDAV, not
> > IMAP.

> Or LDAP :-).

That's actually how we do it now. We're hoping to move to CARDDAV in the
future.

> There is also a Sieve extension for making use of whitelists/blacklists
> during mail delivery. See draft-melnikov-sieve-external-lists-02.txt.
> It can work with CARDDAV, LDAP and other things.

Interestingly, this extension in its current form is not compatible with
at least one of the proposals that was made here. The issue is one I've
raised before - the fact that :list is separate argument and not a match type.

Where there is clearly some utility in being able to say stuff along the
lines of:

    header :contains :list "subject" "tag:dirty-word-list"

The problem is that in order for this to work the list has to be enumerable.
Not all lists are enumerable, and even some that are are so large even though 
oyu can enumerate them in theory you can't afford to do so in practice.

One use case where this matters is when the list is a set of hashed values. The
way you find if something is "on the list" is to hash it and see if that value
appears. And even if the input string is short enough that you can enumerate
all the unique substrings, how about :matches and :regex? Good luck with
those.

Hashed lists have in fact been proposed in the present discussion as a means of
avoiding giving your address whitelist to the mail server. I happen not to
think this is a useful thing to do for a variety of reasons, mostly having to
do with address canonicalization (or lack thereof), but there are other
use cases where hashed lists make more sense.

So, although it reduces functionality, I believe :list should be a match type
and the underlying comparison type that's done should be a property of the list
itself.

				Ned



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n68A9TeU021174 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 8 Jul 2009 03:09:29 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n68A9TUq021173; Wed, 8 Jul 2009 03:09:29 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from rufus.isode.com (rufus.isode.com [62.3.217.251]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n68A9Guv021150 for <ietf-imapext@imc.org>; Wed, 8 Jul 2009 03:09:27 -0700 (MST) (envelope-from alexey.melnikov@isode.com)
Received: from [94.197.220.252] (94.197.220.252.threembb.co.uk [94.197.220.252])  by rufus.isode.com (submission channel) via TCP with ESMTPA  id <SlRwSQBV9BFk@rufus.isode.com>; Wed, 8 Jul 2009 11:09:14 +0100
Message-ID: <4A54701C.6080107@isode.com>
Date: Wed, 08 Jul 2009 11:08:28 +0100
From: Alexey Melnikov <alexey.melnikov@isode.com>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/20050915
X-Accept-Language: en-us, en
To: George Michaelson <ggm@pobox.com>
CC: Ned Freed <ned.freed@mrochek.com>, ietf-imapext@imc.org, general discussion of application-layer protocols <discuss@ietf.org>
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <28941.1246868682.560969@puncture> <4A51D1A1.5040507@isode.com> <01NAZVQAYWH4001DLH@mauve.mrochek.com> <47CB230E-B713-4098-82D4-318744434C46@pobox.com> <01NB0BOZYUPY001ML6@mauve.mrochek.com> <6DECD560-B257-4E6E-80E7-A99A69D297C6@pobox.com> <01NB14JAPS8E001ML6@mauve.mrochek.com> <F4A90756-9251-4CB8-9776-77F3202617A5@pobox.com> <01NB1TV04NJS001ML6@mauve.mrochek.com>
In-Reply-To: <01NB1TV04NJS001ML6@mauve.mrochek.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

Ned Freed wrote:

>> I have no desire to argue against my own proposal! sorry if I
>> misunderstood the tenor of some comments to this suggestion.
>
>> The original posting asked about two things
>
>> 1) spam/ham
>
>> 2) white/black
>
>> so far, people seem to be focussing on 1). I acknowledge problems with
>> 1), I appreciate that people are trying to address 1) as a possibility.
>
>> I asked about 2). I asked you if, ignoring 1), it was possible or
>> plausible to send white/black information about sender, if 1) was hard
>> or intractable.
>
> Of course it's possible, but this has no business being in IMAP. IMAP 
> is a
> message access protocol, not an address handling protocol. It is also 
> already
> quite complex, and extending its function to include what amountss to a
> specialized address book capability is a really bad idea IMO.
>
> In fact a pretty good case can be made that it's inappropriate to use 
> IMAP for spam/ham classification, but at the spam/ham classification is
> intrinsicly tied to messages in the store. The same cannot be said for
> address white and blacklisting.

+1

>> Does that make more sense? I do realize white/black listing is a
>> different PROBLEM to spam/ham marking, but neither of them currently
>> have support directly in imap, and I imagine both of them would use
>> IMAP extensions to pass information client-server.
>
> The place where I'd like to see white and black listing done is in 
> CARDDAV, not
> IMAP.

Or LDAP :-).

There is also a Sieve extension for making use of whitelists/blacklists 
during mail delivery. See draft-melnikov-sieve-external-lists-02.txt.
It can work with CARDDAV, LDAP and other things.



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n686RNP5005850 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 7 Jul 2009 23:27:23 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n686RNSC005849; Tue, 7 Jul 2009 23:27:23 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from mauve.mrochek.com (mauve.mrochek.com [66.59.230.40]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n686RC8w005831 for <ietf-imapext@imc.org>; Tue, 7 Jul 2009 23:27:22 -0700 (MST) (envelope-from ned.freed@mrochek.com)
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NB20PU9RC0018VKL@mauve.mrochek.com> for ietf-imapext@imc.org; Tue, 7 Jul 2009 23:27:09 -0700 (PDT)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NB09YSB36O001ML6@mauve.mrochek.com>; Tue, 07 Jul 2009 23:27:07 -0700 (PDT)
Date: Tue, 07 Jul 2009 23:19:32 -0700 (PDT)
From: Ned Freed <ned.freed@mrochek.com>
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
In-reply-to: "Your message dated Wed, 08 Jul 2009 08:08:05 +0200" <B33373D2-8602-409F-82D7-9E728808E6DD@muada.com>
To: Iljitsch van Beijnum <iljitsch@muada.com>
Cc: Ned Freed <ned.freed@mrochek.com>, George Michaelson <ggm@pobox.com>, ietf-imapext@imc.org, general discussion of application-layer protocols <discuss@ietf.org>
Message-id: <01NB20PSV3XE001ML6@mauve.mrochek.com>
MIME-version: 1.0
Content-type: TEXT/PLAIN; format=flowed; delsp=yes
Content-transfer-encoding: 7BIT
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <28941.1246868682.560969@puncture> <4A51D1A1.5040507@isode.com> <01NAZVQAYWH4001DLH@mauve.mrochek.com> <47CB230E-B713-4098-82D4-318744434C46@pobox.com> <01NB0BOZYUPY001ML6@mauve.mrochek.com> <6DECD560-B257-4E6E-80E7-A99A69D297C6@pobox.com> <01NB14JAPS8E001ML6@mauve.mrochek.com> <F4A90756-9251-4CB8-9776-77F3202617A5@pobox.com> <01NB1TV04NJS001ML6@mauve.mrochek.com> <B33373D2-8602-409F-82D7-9E728808E6DD@muada.com>
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

> On 8 jul 2009, at 5:02, Ned Freed wrote:

> > The place where I'd like to see white and black listing done is in
> > CARDDAV, not
> > IMAP.

> I never felt comfortable uploading all my contact info to Google. (May
> be somewhat strange since they do have all the mail sent to one of my
> email addresses.)

That's fine, but this doesn't translate into a need to put an addressbook
capability in IMAP.

> So I think white/blacklists shouldn't be complete contact info, but
> rather, just lists of hashes of email addresses.

So use CARDDAV to only upload the stuff that's needed to Google. Problem
solved.

I'll also note that addressbook programs have had facilities for exporting
subsets of the information they contain pretty much from the start. So doing
this  is hardly entering new territory.

And most users want an authoritative address book with all their contact
information in one place. I know I do - it's much too confusing otherwise.

				Ned



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n6868wS6004690 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 7 Jul 2009 23:08:58 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n6868wTD004689; Tue, 7 Jul 2009 23:08:58 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n6868jhu004670 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for <ietf-imapext@imc.org>; Tue, 7 Jul 2009 23:08:57 -0700 (MST) (envelope-from iljitsch@muada.com)
Received: from [192.168.0.193] (static-167-138-7-89.ipcom.comunitel.net [89.7.138.167] (may be forged)) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n686896f066815 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Wed, 8 Jul 2009 08:08:12 +0200 (CEST) (envelope-from iljitsch@muada.com)
Cc: George Michaelson <ggm@pobox.com>, ietf-imapext@imc.org, general discussion of application-layer protocols <discuss@ietf.org>
Message-Id: <B33373D2-8602-409F-82D7-9E728808E6DD@muada.com>
From: Iljitsch van Beijnum <iljitsch@muada.com>
To: Ned Freed <ned.freed@mrochek.com>
In-Reply-To: <01NB1TV04NJS001ML6@mauve.mrochek.com>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
Content-Transfer-Encoding: 7bit
Mime-Version: 1.0 (Apple Message framework v930.3)
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
Date: Wed, 8 Jul 2009 08:08:05 +0200
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <28941.1246868682.560969@puncture> <4A51D1A1.5040507@isode.com> <01NAZVQAYWH4001DLH@mauve.mrochek.com> <47CB230E-B713-4098-82D4-318744434C46@pobox.com> <01NB0BOZYUPY001ML6@mauve.mrochek.com> <6DECD560-B257-4E6E-80E7-A99A69D297C6@pobox.com> <01NB14JAPS8E001ML6@mauve.mrochek.com> <F4A90756-9251-4CB8-9776-77F3202617A5@pobox.com> <01NB1TV04NJS001ML6@mauve.mrochek.com>
X-Mailer: Apple Mail (2.930.3)
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

On 8 jul 2009, at 5:02, Ned Freed wrote:

> The place where I'd like to see white and black listing done is in  
> CARDDAV, not
> IMAP.

I never felt comfortable uploading all my contact info to Google. (May  
be somewhat strange since they do have all the mail sent to one of my  
email addresses.)

So I think white/blacklists shouldn't be complete contact info, but  
rather, just lists of hashes of email addresses.



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n683owwp097426 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 7 Jul 2009 20:50:58 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n683ow9d097425; Tue, 7 Jul 2009 20:50:58 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from sca-es-mail-1.sun.com (sca-es-mail-1.Sun.COM [192.18.43.132]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n683oliB097403 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO) for <ietf-imapext@imc.org>; Tue, 7 Jul 2009 20:50:57 -0700 (MST) (envelope-from Anil.Srivastava@Sun.COM)
Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n683okp8013077 for <ietf-imapext@imc.org>; Tue, 7 Jul 2009 20:50:46 -0700 (PDT)
MIME-version: 1.0
Content-transfer-encoding: 7BIT
Content-type: TEXT/PLAIN; CHARSET=US-ASCII
Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KMG001002MKDM00@fe-sfbay-09.sun.com> for ietf-imapext@imc.org; Tue, 07 Jul 2009 20:50:46 -0700 (PDT)
Received: from [192.168.132.10] ([unknown] [129.150.24.16]) by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KMG00M0Z2OLJYD0@fe-sfbay-09.sun.com>; Tue, 07 Jul 2009 20:50:46 -0700 (PDT)
Date: Tue, 07 Jul 2009 20:50:40 -0700 (PDT)
From: Anil SRIVASTAVA <Anil.Srivastava@Sun.COM>
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
In-reply-to: <01NB1TV04NJS001ML6@mauve.mrochek.com>
To: Ned Freed <ned.freed@mrochek.com>
Cc: George Michaelson <ggm@pobox.com>, ietf-imapext@imc.org, general discussion of application-layer protocols <discuss@ietf.org>
Reply-to: Anil.Srivastava@Sun.COM
Message-id:  <alpine.OSX.2.00.0907072050170.346@dhcp-usca15-127-42.red.iplanet.com>
Organization: Sun Microsystems [http://www.sun.com]
Accept-Language: English; en
X-Message-Class: normal
X-Endorsement: Message brought to you by Sun Java Communication Suite 6
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <28941.1246868682.560969@puncture> <4A51D1A1.5040507@isode.com> <01NAZVQAYWH4001DLH@mauve.mrochek.com> <47CB230E-B713-4098-82D4-318744434C46@pobox.com> <01NB0BOZYUPY001ML6@mauve.mrochek.com> <6DECD560-B257-4E6E-80E7-A99A69D297C6@pobox.com> <01NB14JAPS8E001ML6@mauve.mrochek.com> <F4A90756-9251-4CB8-9776-77F3202617A5@pobox.com> <01NB1TV04NJS001ML6@mauve.mrochek.com>
User-Agent: Alpine 2.00 (OSX 1167 2008-08-23)
X-Message-flag: Outlook: the best virus distribution system around
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

On 2009-07-07/20:02 [-0700], ned.freed@mrochek.com [Ned Freed] wrote:

> 
> > I have no desire to argue against my own proposal! sorry if I
> > misunderstood the tenor of some comments to this suggestion.
> 
> > The original posting asked about two things
> 
> > 1) spam/ham
> 
> > 2) white/black
> 
> > so far, people seem to be focussing on 1). I acknowledge problems with
> > 1), I appreciate that people are trying to address 1) as a possibility.
> 
> > I asked about 2). I asked you if, ignoring 1), it was possible or
> > plausible to send white/black information about sender, if 1) was hard
> > or intractable.
> 
> Of course it's possible, but this has no business being in IMAP. IMAP is 
> a message access protocol, not an address handling protocol. It is also 
> already quite complex, and extending its function to include what 
> amountss to a specialized address book capability is a really bad idea 
> IMO.
> 
> In fact a pretty good case can be made that it's inappropriate to 
> useIMAP for spam/ham classification, but at the spam/ham classification 
> is intrinsicly tied to messages in the store. The same cannot be said 
> for address white and blacklisting.
> 
> > Does that make more sense? I do realize white/black listing is a 
> > different PROBLEM to spam/ham marking, but neither of them currently 
> > have support directly in imap, and I imagine both of them would use 
> > IMAP extensions to pass information client-server.
> 
> The place where I'd like to see white and black listing done is in 
> CARDDAV, not IMAP.
>

+1 

Anil
 
> 				Ned
> 

_______________
Anil SRIVASTAVA  |  anil.srivastava@Sun.COM  | Sun Microsystems, Inc



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n683BX2I095423 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 7 Jul 2009 20:11:33 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n683BXTh095422; Tue, 7 Jul 2009 20:11:33 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from mauve.mrochek.com (mauve.mrochek.com [66.59.230.40]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n683BLQf095395 for <ietf-imapext@imc.org>; Tue, 7 Jul 2009 20:11:32 -0700 (MST) (envelope-from ned.freed@mrochek.com)
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NB1TV23J0G00SKEL@mauve.mrochek.com> for ietf-imapext@imc.org; Tue, 7 Jul 2009 20:11:19 -0700 (PDT)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NB09YSB36O001ML6@mauve.mrochek.com>; Tue, 07 Jul 2009 20:11:16 -0700 (PDT)
Date: Tue, 07 Jul 2009 20:02:51 -0700 (PDT)
From: Ned Freed <ned.freed@mrochek.com>
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
In-reply-to: "Your message dated Wed, 08 Jul 2009 09:42:58 +1000" <F4A90756-9251-4CB8-9776-77F3202617A5@pobox.com>
To: George Michaelson <ggm@pobox.com>
Cc: Ned Freed <ned.freed@mrochek.com>, ietf-imapext@imc.org, general discussion of application-layer protocols <discuss@ietf.org>
Message-id: <01NB1TV04NJS001ML6@mauve.mrochek.com>
MIME-version: 1.0
Content-type: TEXT/PLAIN; format=flowed; delsp=yes
Content-transfer-encoding: 7BIT
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <28941.1246868682.560969@puncture> <4A51D1A1.5040507@isode.com> <01NAZVQAYWH4001DLH@mauve.mrochek.com> <47CB230E-B713-4098-82D4-318744434C46@pobox.com> <01NB0BOZYUPY001ML6@mauve.mrochek.com> <6DECD560-B257-4E6E-80E7-A99A69D297C6@pobox.com> <01NB14JAPS8E001ML6@mauve.mrochek.com> <F4A90756-9251-4CB8-9776-77F3202617A5@pobox.com>
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

> I have no desire to argue against my own proposal! sorry if I
> misunderstood the tenor of some comments to this suggestion.

> The original posting asked about two things

> 1) spam/ham

> 2) white/black

> so far, people seem to be focussing on 1). I acknowledge problems with
> 1), I appreciate that people are trying to address 1) as a possibility.

> I asked about 2). I asked you if, ignoring 1), it was possible or
> plausible to send white/black information about sender, if 1) was hard
> or intractable.

Of course it's possible, but this has no business being in IMAP. IMAP is a
message access protocol, not an address handling protocol. It is also already
quite complex, and extending its function to include what amountss to a
specialized address book capability is a really bad idea IMO.

In fact a pretty good case can be made that it's inappropriate to use 
IMAP for spam/ham classification, but at the spam/ham classification is
intrinsicly tied to messages in the store. The same cannot be said for
address white and blacklisting.

> Does that make more sense? I do realize white/black listing is a
> different PROBLEM to spam/ham marking, but neither of them currently
> have support directly in imap, and I imagine both of them would use
> IMAP extensions to pass information client-server.

The place where I'd like to see white and black listing done is in CARDDAV, not
IMAP.

				Ned



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n67Lce9D076644 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 7 Jul 2009 14:38:40 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n67LceGt076643; Tue, 7 Jul 2009 14:38:40 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from mauve.mrochek.com (mauve.mrochek.com [66.59.230.40]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n67LcTPl076626 for <ietf-imapext@imc.org>; Tue, 7 Jul 2009 14:38:39 -0700 (MST) (envelope-from ned.freed@mrochek.com)
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NB1I9BBPC0005UZP@mauve.mrochek.com> for ietf-imapext@imc.org; Tue, 7 Jul 2009 14:38:26 -0700 (PDT)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NB09YSB36O001ML6@mauve.mrochek.com>; Tue, 07 Jul 2009 14:38:22 -0700 (PDT)
Date: Tue, 07 Jul 2009 14:32:14 -0700 (PDT)
From: Ned Freed <ned.freed@mrochek.com>
Subject: Re: A survey of Spam-related keyword usage by IMAP clients.
In-reply-to: "Your message dated Tue, 07 Jul 2009 15:18:27 -0600" <1247001507.8602.109.camel@legolas.orthanc.ca>
To: Lyndon Nerenberg <lyndon@orthanc.ca>
Cc: imap-protocol@u.washington.edu, ietf-imapext@imc.org
Message-id: <01NB1I98RDS4001ML6@mauve.mrochek.com>
MIME-version: 1.0
Content-type: TEXT/PLAIN
Content-transfer-encoding: 7BIT
References: <1247001507.8602.109.camel@legolas.orthanc.ca>
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

I was about to post something very much like this but you beat me to it.

> It's time to get a current view of how IMAP clients are using keywords
> to manage spam. If you are a client author I would appreciate it if you
> could take a moment to send me a note describing which keywords your
> client uses for spam-related processing, and what each of the keywords
> signifies.

I'd also really like to know if the client has "this is spam" or "this is not
spam" buttons in the UI, and if it does how those buttons interact with the use
of keywords.

I ask this because users of any client that has such buttons are clearly going
to use those buttons preferentially over any other labelling mechanism. And if
a significant number of clients have such buttons that do a particular thing -
regardless of what that thing is - that's pretty much going to be determinative
in finding the most easily deployed approach.

> If your client does not use keywords for this purpose, that's
> valuable information, too, as is a breakdown of functionality by
> software version number if there are significant differences. (If you
> use non-keyword facilities to manage spam feel free to include a short
> description of your methods if you think the info might be relevant.)

Yes please. I'm especially interested in how many clients do this by resending
the message to a special address.

> I'm primarily interested in getting authoritative data from the client
> authors, but others should feel free to chime in with deployment-based
> experience of related keyword usage, but please indicate the source of
> your comments (e.g. deployment experience, source code examination).

> I will summarize the responses to the imap-ext list in a couple of weeks
> (or sooner if the response rate dictates.)

Only other thing I'd add is that I'd like to hear from webmail authors in
addition to standalone client authors.

				Ned



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n67LIkIZ075729 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 7 Jul 2009 14:18:46 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n67LIkBq075728; Tue, 7 Jul 2009 14:18:46 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from orthanc.ca (orthanc.ca [208.86.224.138]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n67LIZf6075715 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for <ietf-imapext@imc.org>; Tue, 7 Jul 2009 14:18:46 -0700 (MST) (envelope-from lyndon@orthanc.ca)
Received: from [192.168.0.42] (S01060011242eafe2.cg.shawcable.net [68.145.131.177]) (authenticated bits=0) by orthanc.ca (8.14.3/8.14.3) with ESMTP id n67LIXI0035352; Tue, 7 Jul 2009 15:18:34 -0600 (MDT) (envelope-from lyndon@orthanc.ca)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=orthanc.ca; s=2009-1; t=1247001514; bh=c4PxB9qIGUMk/XQCTpIvPlRhSPz2J2pr9i0QDZKASzA=; h=Subject:From:Reply-To:To:Content-Type:Date:Message-Id: Mime-Version:Content-Transfer-Encoding; b=BaiYAGjFDN6csFVJ7hFbuVOL1ObbRcaXp28cc+XlM6MFJ0Y1mjJ6hxhVEN0SsIunb MJVcLLOzqOEXQ/zjcIPOkjgKeXS9iIh+4oRM4pxo2I5kXdHqMbt+vgQbsXW6IBakRu dFoPgrurC0jurUvTomSpdYqLT1jfc9zzrbgpu3BlMmFXkbA8+4DGSJSMT9STB2g2x7 yYdsF87Tv9bbjaWD+HEFoeOXHVZFhBxPzrPt67JbTptYU5PiWm8MtUscM0uE19t8SG WGGrNo021kUR8uAMcv9t1PrpPQJNeya5smTcxj0i0k2zor9BEtzdVYef3FqskL/t5f l7l+islxxktPw==
Subject: A survey of Spam-related keyword usage by IMAP clients.
From: Lyndon Nerenberg <lyndon@orthanc.ca>
Reply-To: lyndon+imapspamkeywordsurvey@orthanc.ca
To: imap-protocol@u.washington.edu, ietf-imapext@imc.org
Content-Type: text/plain
Date: Tue, 07 Jul 2009 15:18:27 -0600
Message-Id: <1247001507.8602.109.camel@legolas.orthanc.ca>
Mime-Version: 1.0
X-Mailer: Evolution 2.26.2 FreeBSD GNOME Team Port 
Content-Transfer-Encoding: 7bit
X-Spam-Status: No, score=2.1 required=5.0 tests=DKIM_SIGNED,DKIM_VERIFIED, RCVD_IN_PBL,RCVD_IN_SORBS_DUL autolearn=no version=3.2.5
X-Spam-Level: **
X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on orthanc.ca
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

It's time to get a current view of how IMAP clients are using keywords
to manage spam. If you are a client author I would appreciate it if you
could take a moment to send me a note describing which keywords your
client uses for spam-related processing, and what each of the keywords
signifies. If your client does not use keywords for this purpose, that's
valuable information, too, as is a breakdown of functionality by
software version number if there are significant differences. (If you
use non-keyword facilities to manage spam feel free to include a short
description of your methods if you think the info might be relevant.)

I'm primarily interested in getting authoritative data from the client
authors, but others should feel free to chime in with deployment-based
experience of related keyword usage, but please indicate the source of
your comments (e.g. deployment experience, source code examination).

I will summarize the responses to the imap-ext list in a couple of weeks
(or sooner if the response rate dictates.)

--lyndon



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n67HW0g5061835 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 7 Jul 2009 10:32:00 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n67HW0iT061834; Tue, 7 Jul 2009 10:32:00 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from orthanc.ca (orthanc.ca [208.86.224.138]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n67HVmxZ061800 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for <ietf-imapext@imc.org>; Tue, 7 Jul 2009 10:31:59 -0700 (MST) (envelope-from lyndon@orthanc.ca)
Received: from [192.168.0.42] (S01060011242eafe2.cg.shawcable.net [68.145.131.177]) (authenticated bits=0) by orthanc.ca (8.14.3/8.14.3) with ESMTP id n67HVJe6030816; Tue, 7 Jul 2009 11:31:20 -0600 (MDT) (envelope-from lyndon@orthanc.ca)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=orthanc.ca; s=2009-1; t=1246987881; bh=dqQG1R61W3fm+QI72VYQMJEiJ/ert5IkyFlRyhkB2Q0=; h=Subject:From:To:Cc:In-Reply-To:References:Content-Type:Date: Message-Id:Mime-Version:Content-Transfer-Encoding; b=Teu5CztYJbN9luvvex3j2ymkN17mq/1iXMk1anpY1/j8DxZfn4a/XHg5Cg6JLeF5z TL9mrgBINQE/bFmUP8M6IyE9Y9yd+XCBn8zGqvNoqniN3VI4uFWX1jOcVTeplLDMj+ fdjCbtx0NB+q2A6iYhvgz/yi87heqHncOJ9m1sSCSiSU8xfC/Hwr1ARTonjCuRVt+f QXZfz5u+0giKTGpvmo6pCbG+njBtMjjfu9Ql3DfzwbUndDt/IYpltzE9PaaIrNJZRM uIog/qABIwS5SzIc41Jm/b3Gti4FNPXfPXJM3kgiLS8ldJBs/gt9Jx4C1YXVtK6XRN D/I7pLQ9Kumgw==
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
From: Lyndon Nerenberg <lyndon@orthanc.ca>
To: Iljitsch van Beijnum <iljitsch@muada.com>
Cc: ietf-imapext@imc.org, discuss@ietf.org
In-Reply-To: <E6275AB7-B916-47F8-9135-B1BEFFE11814@muada.com>
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <01NAZUST62HQ001DLH@mauve.mrochek.com> <1246902821.8602.70.camel@legolas.orthanc.ca> <E6275AB7-B916-47F8-9135-B1BEFFE11814@muada.com>
Content-Type: text/plain
Date: Tue, 07 Jul 2009 11:31:13 -0600
Message-Id: <1246987873.8602.92.camel@legolas.orthanc.ca>
Mime-Version: 1.0
X-Mailer: Evolution 2.26.2 FreeBSD GNOME Team Port 
Content-Transfer-Encoding: 7bit
X-Spam-Status: No, score=2.1 required=5.0 tests=DKIM_SIGNED,DKIM_VERIFIED, RCVD_IN_PBL,RCVD_IN_SORBS_DUL autolearn=no version=3.2.5
X-Spam-Level: **
X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on orthanc.ca
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

On Tue, 2009-07-07 at 09:21 +0200, Iljitsch van Beijnum wrote:
> 3. manually flagged as spam
> 4. manually flagged as non-spam
> 
> Where 1 and 2 / 3 and 4 are mutually exclusive but 1 + 4 and 2 + 3  
> should trigger retraining of baysian filters etc.

The keywords I'm describing are intended to tell the server to process
the flagged message through its filters (i.e. train on the message). It
isn't relevant to the proposal whether a human sets them, just what the
server's behaviour is in their presence.

--lyndon



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n67G22Al055589 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 7 Jul 2009 09:02:02 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n67G22WK055588; Tue, 7 Jul 2009 09:02:02 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from mauve.mrochek.com (mauve.mrochek.com [66.59.230.40]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n67G21Qs055581 for <ietf-imapext@imc.org>; Tue, 7 Jul 2009 09:02:01 -0700 (MST) (envelope-from ned.freed@mrochek.com)
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NB16I3I5TS0017B6@mauve.mrochek.com> for ietf-imapext@imc.org; Tue, 7 Jul 2009 09:01:56 -0700 (PDT)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NB09YSB36O001ML6@mauve.mrochek.com>; Tue, 07 Jul 2009 09:01:52 -0700 (PDT)
Date: Tue, 07 Jul 2009 08:43:46 -0700 (PDT)
From: Ned Freed <ned.freed@mrochek.com>
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
In-reply-to: "Your message dated Tue, 07 Jul 2009 11:15:04 -0400" <F02388CBAFB7CC46475A5F66@caldav.corp.apple.com>
To: Cyrus Daboo <cyrus@daboo.name>
Cc: Ned Freed <ned.freed@mrochek.com>, Lyndon Nerenberg <lyndon@orthanc.ca>, ietf-imapext@imc.org, discuss@ietf.org
Message-id: <01NB16I1ENDC001ML6@mauve.mrochek.com>
MIME-version: 1.0
Content-type: TEXT/PLAIN; format=flowed
Content-transfer-encoding: 7BIT
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <01NAZUST62HQ001DLH@mauve.mrochek.com> <1246902821.8602.70.camel@legolas.orthanc.ca> <01NAZWHC66ZU000078@mauve.mrochek.com> <1246904761.8602.86.camel@legolas.orthanc.ca> <01NB13KIAQHQ001ML6@mauve.mrochek.com> <F02388CBAFB7CC46475A5F66@caldav.corp.apple.com>
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

> Hi Ned,

> --On July 7, 2009 7:22:42 AM -0700 Ned Freed <ned.freed@mrochek.com> wrote:

> > Now suppose I implement this with folder annotations. I'm in business
> > immediately. Things just work no matter what client people use.

> Actually you need to consider what the user experience is likely to be on
> the client side. There are several things to consider: for spam messages
> users will typically want to delete them from their regular mailbox. With
> ham messages they will typically want to leave the message in place (or
> file them away after processing). In each case I certainly would not want
> the spam messages or an extra copy of the ham messages to reside on my
> server consuming quota or other resources, or being indexed as part of a
> server-wide search etc.

> With that in mind lets look at how each solution might work client-side:

> 1) Keywords: for a spam message the user sets the spam keyword, then sets
> the deleted flag (and then may explicitly expunge if the client has that as
> an option as opposed to hide-deleted-messages with lazy expunge). For a ham
> message the user just sets the ham keyword. i.e. two user operations for
> spam, one for ham.

> 2) Mailbox (I am going to assume that the client has both a move and copy
> command available - the former implicitly setting the delete flag after the
> IMAP COPY command): for a spam message the user moves the message to the
> spam mailbox. For a ham message the user copies the message to the ham
> mailbox. NB both of these assume that the server does not actually store
> the spam/ham messages in the spam/ham mailboxes (or perhaps it garbage
> collects them).

> Things to note: typically clients don't offer shortcuts "move to mailbox X"
> or "copy to mailbox X" so the mailbox operations will involve some drag and
> drop or menu list selection operation. The same may be true for setting
> arbitrary keywords (though Mail.app and I suspect Thunderbird do have
> buttons for this). Frankly the best user experience would be for the client
> to have spam/ham buttons and commands that would allow the user to do the
> appropriate actions in one click or keyboard shortcut operation - that is
> certainly important if the level of spam is moderate to high.

The presence of spam/not-spam buttons in existing clients is an important
factor I hadn't really considered, but I think it has implications that go a
lot further than you let on here.

Simply put, if enough popular clients already have such buttons, and those
buttons do different things in different clients, I think we're pretty much
screwed. Given the presence of such buttons you're not going to be able to get
users to do something different to deal with spam, whether it's adding a
keyword or annotation or performing a folder move.

Our only recourse with such clients in the mix is to write a specification for
what such buttons should do in terms of signalling the server and then hope
that clients actually adopt whatever that specification says. And frankly, if
there's some degree of commonality in what those buttons do in existing clients
and it can be made to work, we probably should adopt that approach no matter
what it happens to be.

> So at the end of the day, for the best (and frankly simplest) user
> experience clients do have to change for both the keyword and move to
> mailbox approaches. Now for keywords that change does have to be immediate
> if arbitrary keyword support is not already present in the client, whereas
> for move to mailbox, or clients already supporting keywords, that change
> can be done over time - though as Ned points out client changes tend to
> happen on geological time-scales!

I agree completely that a keyword-based approach has the potential to offer
a better user experience. The question is whether the possibility of offering
a better user experience at some distant point down the road is worth giving
up on a more immediately deployable solution.

But as I indicated above, now that you've brought up the important point of
existing clients having spam classification buttons, that's the real gorilla in
the room we have to deal with.

> So other approaches to consider:

> 1) An extension with commands SPAM and HAM to explicitly tag one or more
> messages.

> 2) A "forward to spam/ham" option that specifies special email addresses
> for the user where spam/ham can be forwarded to trigger training.

I believe this is what the buttons I've seen on at least a couple of
clients already do. If that's indeed the case, it's worth considering.

> 3) An IMAP URL handed off to the SPAM system with URLAUTH to allow that to
> pull details of the message for training. (This could be tied to (2) in
> that forward-without-download could be used in (2) with URLAUTH etc).

> These of course all require new client/server behaviors and thus are
> probably non-starters, but they should not be dismissed entirely.

I'm actually not sure (2) does with all clients, but I agree regarding (1) and
(3).

				Ned



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n67FFNmk052661 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 7 Jul 2009 08:15:23 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n67FFNxo052660; Tue, 7 Jul 2009 08:15:23 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from daboo.name (daboo.name [151.201.22.177]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n67FFBck052647 for <ietf-imapext@imc.org>; Tue, 7 Jul 2009 08:15:22 -0700 (MST) (envelope-from cyrus@daboo.name)
Received: from localhost (localhost [127.0.0.1]) by daboo.name (Postfix) with ESMTP id 6B24B1680AD5; Tue,  7 Jul 2009 11:15:11 -0400 (EDT)
X-Virus-Scanned: amavisd-new at daboo.name
Received: from daboo.name ([127.0.0.1]) by localhost (chewy.mulberrymail.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jPdNDLFU4hpp; Tue,  7 Jul 2009 11:15:11 -0400 (EDT)
Received: from caldav.corp.apple.com (unknown [17.101.32.44]) by daboo.name (Postfix) with ESMTP id A54A91680ACE; Tue,  7 Jul 2009 11:15:08 -0400 (EDT)
Date: Tue, 07 Jul 2009 11:15:04 -0400
From: Cyrus Daboo <cyrus@daboo.name>
To: Ned Freed <ned.freed@mrochek.com>, Lyndon Nerenberg <lyndon@orthanc.ca>
cc: ietf-imapext@imc.org, discuss@ietf.org
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
Message-ID: <F02388CBAFB7CC46475A5F66@caldav.corp.apple.com>
In-Reply-To: <01NB13KIAQHQ001ML6@mauve.mrochek.com>
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <01NAZUST62HQ001DLH@mauve.mrochek.com> <1246902821.8602.70.camel@legolas.orthanc.ca> <01NAZWHC66ZU000078@mauve.mrochek.com> <1246904761.8602.86.camel@legolas.orthanc.ca> <01NB13KIAQHQ001ML6@mauve.mrochek.com>
X-Mailer: Mulberry/4.1.0a1 (Mac OS X)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline; size=3318
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

Hi Ned,

--On July 7, 2009 7:22:42 AM -0700 Ned Freed <ned.freed@mrochek.com> wrote:

> Now suppose I implement this with folder annotations. I'm in business
> immediately. Things just work no matter what client people use.

Actually you need to consider what the user experience is likely to be on 
the client side. There are several things to consider: for spam messages 
users will typically want to delete them from their regular mailbox. With 
ham messages they will typically want to leave the message in place (or 
file them away after processing). In each case I certainly would not want 
the spam messages or an extra copy of the ham messages to reside on my 
server consuming quota or other resources, or being indexed as part of a 
server-wide search etc.

With that in mind lets look at how each solution might work client-side:

1) Keywords: for a spam message the user sets the spam keyword, then sets 
the deleted flag (and then may explicitly expunge if the client has that as 
an option as opposed to hide-deleted-messages with lazy expunge). For a ham 
message the user just sets the ham keyword. i.e. two user operations for 
spam, one for ham.

2) Mailbox (I am going to assume that the client has both a move and copy 
command available - the former implicitly setting the delete flag after the 
IMAP COPY command): for a spam message the user moves the message to the 
spam mailbox. For a ham message the user copies the message to the ham 
mailbox. NB both of these assume that the server does not actually store 
the spam/ham messages in the spam/ham mailboxes (or perhaps it garbage 
collects them).

Things to note: typically clients don't offer shortcuts "move to mailbox X" 
or "copy to mailbox X" so the mailbox operations will involve some drag and 
drop or menu list selection operation. The same may be true for setting 
arbitrary keywords (though Mail.app and I suspect Thunderbird do have 
buttons for this). Frankly the best user experience would be for the client 
to have spam/ham buttons and commands that would allow the user to do the 
appropriate actions in one click or keyboard shortcut operation - that is 
certainly important if the level of spam is moderate to high.

So at the end of the day, for the best (and frankly simplest) user 
experience clients do have to change for both the keyword and move to 
mailbox approaches. Now for keywords that change does have to be immediate 
if arbitrary keyword support is not already present in the client, whereas 
for move to mailbox, or clients already supporting keywords, that change 
can be done over time - though as Ned points out client changes tend to 
happen on geological time-scales!

So other approaches to consider:

1) An extension with commands SPAM and HAM to explicitly tag one or more 
messages.
2) A "forward to spam/ham" option that specifies special email addresses 
for the user where spam/ham can be forwarded to trigger training.
3) An IMAP URL handed off to the SPAM system with URLAUTH to allow that to 
pull details of the message for training. (This could be tied to (2) in 
that forward-without-download could be used in (2) with URLAUTH etc).

These of course all require new client/server behaviors and thus are 
probably non-starters, but they should not be dismissed entirely.

-- 
Cyrus Daboo



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n67F5kjL051887 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 7 Jul 2009 08:05:46 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n67F5j4X051886; Tue, 7 Jul 2009 08:05:45 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from mauve.mrochek.com (mauve.mrochek.com [66.59.230.40]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n67F5hkv051879 for <ietf-imapext@imc.org>; Tue, 7 Jul 2009 08:05:44 -0700 (MST) (envelope-from ned.freed@mrochek.com)
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NB14JDJETS00HWDR@mauve.mrochek.com> for ietf-imapext@imc.org; Tue, 7 Jul 2009 08:05:41 -0700 (PDT)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NB09YSB36O001ML6@mauve.mrochek.com>; Tue, 07 Jul 2009 08:05:37 -0700 (PDT)
Date: Tue, 07 Jul 2009 07:42:42 -0700 (PDT)
From: Ned Freed <ned.freed@mrochek.com>
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
In-reply-to: "Your message dated Tue, 07 Jul 2009 11:28:53 +1000" <6DECD560-B257-4E6E-80E7-A99A69D297C6@pobox.com>
To: George Michaelson <ggm@pobox.com>
Cc: Ned Freed <ned.freed@mrochek.com>, ietf-imapext@imc.org, general discussion of application-layer protocols <discuss@ietf.org>
Message-id: <01NB14JAPS8E001ML6@mauve.mrochek.com>
MIME-version: 1.0
Content-type: TEXT/PLAIN; format=flowed; delsp=yes
Content-transfer-encoding: 7BIT
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <28941.1246868682.560969@puncture> <4A51D1A1.5040507@isode.com> <01NAZVQAYWH4001DLH@mauve.mrochek.com> <47CB230E-B713-4098-82D4-318744434C46@pobox.com> <01NB0BOZYUPY001ML6@mauve.mrochek.com> <6DECD560-B257-4E6E-80E7-A99A69D297C6@pobox.com>
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

> hang on.

> The MS has the message. It presumably has some held state of it,
> specific to its implementation.

Actually, that's highly unlikely. There is some amounts of state IMAP requires
that's associated with the message - flags/keywords, annotations, some remnants
of the envelope. But beyond that there's little point in keeping a bunch of
implementation-specific state. Why? Because there's no way to expose such state
through the IMAP protocol. So when messages are FETCHed the state stays behind.
And more to the point, when a client moves a message, implementation-specific
state tends to get lost, because the client doesn't know how to access it in
order to move it.

The result is that in most csaes IMAP servers don't store much beyond the
things the standard requires them to store. The differences between servers lie
in how the standard stuff is stored and to what extent metainformation, like
body structured, is cached instead of being recomputed.

> I don't want deep knowledge of that
> model, I really just want to be able to override it with my explicit
> knowledge.

> I simply want to pass the toggle:
	
> 	If you class this as spam, please class as ham
> or
> 	if you class this as ham, please class as spam

This is exactly what we're trying to accomplish. The question is what mechanism
to use to do it.

> what it means, semantically, is of course MS specific. I don't care
> how its held, or what the effect on the weighting is, in the re-class
> as spam. In the class as ham, I care quite a lot that its sender is
> removed from hard blocks of course.

I'm afraid you cannot count on that happening either. The problem is that
such blocks are often outside the control of the service provider to change.
In many cases the provider has subscribed to some sort of spam filtering
service.

Whitelisting may be an alternative in such cases, but you have to be careful
lest the amount of whitelisting information grow to the point where it is
unmanageable. (I've seen this happen.)

> Yes, I see that moving this state between servers requires the entire
> message flow. But, if we forget that for now, and just consider that I
> have some mail in my local client cache, which I feel is now worthy of
> a state change, why do I have to pass full-state baysian model
> information?

You appear to be arguing against a straw man of your own construction here,
because AFAIK nobody has suggested such a thing.

> I just want to tell an MS "change your own flags to match
> my flags" in the hard sense: yes-or-no, not some qualitative weighting
> model.

The only "weight" people have mentioned in any of this is storing a spam score
rather than a yes/no indicator. I've already responded that while there is a
place for such information in spam handling, it makes no sense in this
particular context.

> How it acts, is up to it. I don't see why I need a complex flag
> model to say what I think, as the consumer of the mail.

Again, you appear to be arguing against a proposal nobody has made. (Perhaps
some proposal was made in an earlier thread not on the imapext list.) I'll also
point out that flags are binary by definition in IMAP - you cannot implement
even a single-valued spam score using them. If you want to store a reviseable
per-message score, you have no choice but to use message-level annotations.

> If I moved off SPAM/HAM to white/black on sender, would that be easier
> for you to consider?

Whitelisting and blacklisting is actually  a fairly different problem.

				Ned



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n67EcSSu050175 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 7 Jul 2009 07:38:28 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n67EcSmM050174; Tue, 7 Jul 2009 07:38:28 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from mauve.mrochek.com (mauve.mrochek.com [66.59.230.40]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n67EcRDP050168 for <ietf-imapext@imc.org>; Tue, 7 Jul 2009 07:38:28 -0700 (MST) (envelope-from ned.freed@mrochek.com)
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NB13KKILF4001742@mauve.mrochek.com> for ietf-imapext@imc.org; Tue, 7 Jul 2009 07:38:24 -0700 (PDT)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NB09YSB36O001ML6@mauve.mrochek.com>; Tue, 07 Jul 2009 07:38:20 -0700 (PDT)
Date: Tue, 07 Jul 2009 07:22:42 -0700 (PDT)
From: Ned Freed <ned.freed@mrochek.com>
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
In-reply-to: "Your message dated Mon, 06 Jul 2009 12:26:01 -0600" <1246904761.8602.86.camel@legolas.orthanc.ca>
To: Lyndon Nerenberg <lyndon@orthanc.ca>
Cc: Ned Freed <ned.freed@mrochek.com>, ietf-imapext@imc.org, discuss@ietf.org
Message-id: <01NB13KIAQHQ001ML6@mauve.mrochek.com>
MIME-version: 1.0
Content-type: TEXT/PLAIN
Content-transfer-encoding: 7BIT
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <01NAZUST62HQ001DLH@mauve.mrochek.com> <1246902821.8602.70.camel@legolas.orthanc.ca> <01NAZWHC66ZU000078@mauve.mrochek.com> <1246904761.8602.86.camel@legolas.orthanc.ca>
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

> On Mon, 2009-07-06 at 11:00 -0700, Ned Freed wrote:
> > If there's a reason multiple folders can't be marked this way I sure
> > don't
> > know what it is.

> You're right -- I didn't parse what you were saying correctly the first
> time around.

> But if your concern is adoption rates, a keyword-only solution will get
> wider take-up than something that requires support for annotations. (And
> from a purely engineering standpoint, the simplest solution wins.)

My concern isn't adoption rate. Rather, my concern is, as someone who provides
server software, how I can offer a better user experience in regards to spam.

Suppose I implement spam filter retraining based on setting of flags. how well
this works, or whether it works at all, depends entirely on the client people
use. And past experience on client uptake of new features like this is best
described as "lousy".

Now suppose I implement this with folder annotations. I'm in business
immediately. Things just work no matter what client people use.

There is, of course, the issue of setting the annotation on the proper folder
and  having a client to do that. But this isn't as much of a problem as it
sounds. Since the annotation only has to be set once, it can be done through a
simple web page, or better still, make it an option in the webmail interface.
(Everyone has a webmail interface these days.) And in many cases even this is
unnecessary, because many service providers predesignate a folder as the spam
folder.

The only way I could get behind a flag-based solution is if there really was an
common set of ad-hoc flag choices already in use by all the major clients - not
a couple, all. But I don't believe that's the case.

We've had to choose between client and server side solutions many times in the
past, and in almost every case the conclusion has been server side solutions
are a deployment win. I really wish this weren't true - I wish clients would
pick up new capabilities in a more timely fashion - but that's just not how
things work.

				Ned



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n67E1J39047551 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 7 Jul 2009 07:01:19 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n67E1J5p047550; Tue, 7 Jul 2009 07:01:19 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from mauve.mrochek.com (mauve.mrochek.com [66.59.230.40]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n67E18tv047537 for <ietf-imapext@imc.org>; Tue, 7 Jul 2009 07:01:19 -0700 (MST) (envelope-from ned.freed@mrochek.com)
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NB12AAY2E8006J0V@mauve.mrochek.com> for ietf-imapext@imc.org; Tue, 7 Jul 2009 07:01:06 -0700 (PDT)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NB09YSB36O001ML6@mauve.mrochek.com>; Tue, 07 Jul 2009 07:01:04 -0700 (PDT)
Date: Tue, 07 Jul 2009 06:54:49 -0700 (PDT)
From: Ned Freed <ned.freed@mrochek.com>
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
In-reply-to: "Your message dated Tue, 07 Jul 2009 09:21:11 +0200" <E6275AB7-B916-47F8-9135-B1BEFFE11814@muada.com>
To: Iljitsch van Beijnum <iljitsch@muada.com>
Cc: Lyndon Nerenberg <lyndon@orthanc.ca>, Ned Freed <ned.freed@mrochek.com>, ietf-imapext@imc.org, discuss@ietf.org
Message-id: <01NB12A9GJE8001ML6@mauve.mrochek.com>
MIME-version: 1.0
Content-type: TEXT/PLAIN; format=flowed; delsp=yes
Content-transfer-encoding: 7BIT
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <01NAZUST62HQ001DLH@mauve.mrochek.com> <1246902821.8602.70.camel@legolas.orthanc.ca> <E6275AB7-B916-47F8-9135-B1BEFFE11814@muada.com>
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

> On 6 jul 2009, at 19:53, Lyndon Nerenberg wrote:

> > A message's 'state of spamminess' is an attribute of the
> > message, properly denoted by keywords that stick to (and with) the
> > message.

> That's nice in the abstract, but why do we have servers classify
> messages as spam to begin with? In order to not have to look at them,
> or to be able to look at them when it suits us to clean out the spam
> rather than in between mail that we actually wanted to receive.

Actually, the primary reason servers classify messages as spam is so they
can avoid having to deal with them. If the overhead could be ignored no
server operator would bother - spam filteringh is expensive and difficult.
The reason it's done is the alternate of receiving and dealing with the
those spam messages, with all the ramificaations, is even more expensive
and difficult.

This more than anything is why a score and not a yes/no verdict is important. A
score lets you do stuff like reject messages clearly found to be spam at the
SMTP level, while letting likely but less certain to be spam messages through
to specially designated spam folders or whatever.

> So moving spam to a separate folder is not something that happens
> because of lack of a better mechanism, but the thing that we want to
> happen 98% of the time anyway.

Sometimes yes, sometimes no.

> > $spam and $nospam

> There are more states than that:

> 1. automatically flagged as spam
> 2. automatically flagged as non-spam
> 3. manually flagged as spam
> 4. manually flagged as non-spam

> Where 1 and 2 / 3 and 4 are mutually exclusive but 1 + 4 and 2 + 3
> should trigger retraining of baysian filters etc.

It's not the flag state of the message that triggers retraining,  it's
the change of flag state (or folder locaation). Your proposal is unnecessatily
complicated.

				Ned



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n677NmeN018693 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 7 Jul 2009 00:23:48 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n677Nmem018692; Tue, 7 Jul 2009 00:23:48 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n677NZPA018671 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for <ietf-imapext@imc.org>; Tue, 7 Jul 2009 00:23:46 -0700 (MST) (envelope-from iljitsch@muada.com)
Received: from claw.it.uc3m.es (claw.it.uc3m.es [163.117.139.55]) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n677LFxk057589 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Tue, 7 Jul 2009 09:21:16 +0200 (CEST) (envelope-from iljitsch@muada.com)
Cc: Ned Freed <ned.freed@mrochek.com>, ietf-imapext@imc.org, discuss@ietf.org
Message-Id: <E6275AB7-B916-47F8-9135-B1BEFFE11814@muada.com>
From: Iljitsch van Beijnum <iljitsch@muada.com>
To: Lyndon Nerenberg <lyndon@orthanc.ca>
In-Reply-To: <1246902821.8602.70.camel@legolas.orthanc.ca>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
Content-Transfer-Encoding: 7bit
Mime-Version: 1.0 (Apple Message framework v935.3)
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
Date: Tue, 7 Jul 2009 09:21:11 +0200
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <01NAZUST62HQ001DLH@mauve.mrochek.com> <1246902821.8602.70.camel@legolas.orthanc.ca>
X-Mailer: Apple Mail (2.935.3)
X-Spam-Status: No, score=-2.6 required=3.4 tests=BAYES_00 autolearn=ham  version=3.0.2
X-Spam-Checker-Version: SpamAssassin 3.0.2 (2004-11-16) on sequoia.muada.com
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

On 6 jul 2009, at 19:53, Lyndon Nerenberg wrote:

> A message's 'state of spamminess' is an attribute of the
> message, properly denoted by keywords that stick to (and with) the
> message.

That's nice in the abstract, but why do we have servers classify  
messages as spam to begin with? In order to not have to look at them,  
or to be able to look at them when it suits us to clean out the spam  
rather than in between mail that we actually wanted to receive.

So moving spam to a separate folder is not something that happens  
because of lack of a better mechanism, but the thing that we want to  
happen 98% of the time anyway.

> $spam and $nospam

There are more states than that:

1. automatically flagged as spam
2. automatically flagged as non-spam
3. manually flagged as spam
4. manually flagged as non-spam

Where 1 and 2 / 3 and 4 are mutually exclusive but 1 + 4 and 2 + 3  
should trigger retraining of baysian filters etc.



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n671KT0n000916 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 18:20:29 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n671KT7P000915; Mon, 6 Jul 2009 18:20:29 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from mauve.mrochek.com (mauve.mrochek.com [66.59.230.40]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n671KI1r000900 for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 18:20:28 -0700 (MST) (envelope-from ned.freed@mrochek.com)
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NB0BP0Z28000QPIT@mauve.mrochek.com> for ietf-imapext@imc.org; Mon, 6 Jul 2009 18:20:16 -0700 (PDT)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NB09YSB36O001ML6@mauve.mrochek.com>; Mon, 06 Jul 2009 18:20:14 -0700 (PDT)
Date: Mon, 06 Jul 2009 17:42:58 -0700 (PDT)
From: Ned Freed <ned.freed@mrochek.com>
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
In-reply-to: "Your message dated Tue, 07 Jul 2009 08:51:10 +1000" <47CB230E-B713-4098-82D4-318744434C46@pobox.com>
To: George Michaelson <ggm@pobox.com>
Cc: Ned Freed <ned.freed@mrochek.com>, ietf-imapext@imc.org, general discussion of application-layer protocols <discuss@ietf.org>
Message-id: <01NB0BOZYUPY001ML6@mauve.mrochek.com>
MIME-version: 1.0
Content-type: TEXT/PLAIN; format=flowed; delsp=yes
Content-transfer-encoding: 7BIT
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <28941.1246868682.560969@puncture> <4A51D1A1.5040507@isode.com> <01NAZVQAYWH4001DLH@mauve.mrochek.com> <47CB230E-B713-4098-82D4-318744434C46@pobox.com>
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

> >
> >
> > This is why I think folder annotations are a better bet. You mark
> > the spam
> > folder and use that marking to attach meaning to the movement of
> > messages into
> > and out of that folder. It's not perfect, but it has the key
> > advantage that it
> > works with any client (OK, any client that can use folders) once the
> > annotation
> > is in place.
> >
> > 				Ned

> I have two IMAP servers, one google, one not.

That's a problem in and of itself, because last time I looked gmail is rather
limited in its support of IMAP.

> how do I synchronize state between them.

By "state" you appear to be referring to spam filtering state. If so, the short
answer is you can't possibly do it.

> After all, If I (and I do)
> sometimes have to send from one, to reply to mails from the other
> (mail clients are bad at selecting sender account) it would be useful
> to have at least white and blacklist sender synchronization between
> them.

Sorry, that's simply not realistic. The number of different spam filtering
mechanisms in use is much too large, and the lack of semantic equivalents
across them all makes this an intractable problem.

> Or, I could be sharing explicit spam tagging information to
> improve BOTH of their tracking. Do I really have to pass the entire
> messages between imap servers?

If you want to train the server that doesn't have the message that you do,
or don't regard, it as spam, moving the entire message is the only option.

I will note that moving messages from one IMAP server to another without
having to download and upload them would be possible by relaxing the
relative URL restriction in the CATENATE extension defined in RFC 4469.

But sending misclassified messages to multiple servers is not going to get them
synchronized. Again, there are a myriad of approaches to spam filtering with
hugely different semantics. In fact it is entirely possible that you'll cause
their behavior to diverge instead of converging.

> An explicit extension/flag/keyword would be unquestionably more
> efficient use of the network.

This makes no sense at all. Setting a flag associated with a message stored
on one server is not going to be visible to the other in any way. So the
flagging approach offers no solution to the multiple server problem at all.

The folder annotation approach doesn't work any better, of course.

> And, removing the mails from the spam folder in your methodology
> implies saying 'its not spam' -so the server I move them FROM thinks
> they've been un-tagged.

No, not really. Since there's no facility in IMAP to move a message between
servers, the way you have to do it is to FETCH the message from one server and
APPEND it to the other. In constrast moving a message from one folder to
another on a single server is done with a COPY/SETFLAG sequence. Entirely
different.

> Again, I say what I said to Ijllitch: I don't like depending on side-
> effects. I think explicit messaging is better.

I don't especially like depending on side effects either, but when the choice
is between a mechanism that works with the vast array of currently deployed
clients and one that requires deployment of new client software everywhere to
stand even a chance of wortking, I'm strongly inclined to pursue the former.

Again, had we standardized flags for marking messages as spam 10 years ago, I'd
be fully in favor of using them for this additional purpose because there would
be plenty of deployed client support by now. But we didn't, there isn't, and I
don't hold out a lot of hope for _anything_ deploying with reasonable speed in
the IMAP client space right now.

				Ned



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66LRRPf087626 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 14:27:27 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n66LRRTx087625; Mon, 6 Jul 2009 14:27:27 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from sca-ea-mail-2.sun.com (sca-ea-mail-2.Sun.COM [192.18.43.25]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66LRFwU087616 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO) for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 14:27:26 -0700 (MST) (envelope-from Nicolas.Williams@sun.com)
Received: from dm-central-02.central.sun.com ([129.147.62.5]) by sca-ea-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n66LRFnJ017468 for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 21:27:15 GMT
Received: from binky.Central.Sun.COM (binky.Central.Sun.COM [129.153.128.104]) by dm-central-02.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n66LRFcP000553 for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 15:27:15 -0600 (MDT)
Received: from binky.Central.Sun.COM (localhost [127.0.0.1]) by binky.Central.Sun.COM (8.14.3+Sun/8.14.3) with ESMTP id n66KW0YG028138; Mon, 6 Jul 2009 15:32:00 -0500 (CDT)
Received: (from nw141292@localhost) by binky.Central.Sun.COM (8.14.3+Sun/8.14.3/Submit) id n66KVhZo028137; Mon, 6 Jul 2009 15:31:43 -0500 (CDT)
X-Authentication-Warning: binky.Central.Sun.COM: nw141292 set sender to Nicolas.Williams@sun.com using -f
Date: Mon, 6 Jul 2009 15:31:43 -0500
From: Nicolas Williams <Nicolas.Williams@sun.com>
To: Tony Finch <dot@dotat.at>
Cc: Michael Kirkham <mikek@muonics.com>, ietf-imapext@imc.org, Ned Freed <ned.freed@mrochek.com>, George Michaelson <ggm@pobox.com>, discuss@ietf.org
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
Message-ID: <20090706203143.GR15302@Sun.COM>
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <01NAZUST62HQ001DLH@mauve.mrochek.com> <20090706100834.R49379@eng-bsd.muonics.com> <alpine.LSU.2.00.0907062124240.30197@hermes-2.csi.cam.ac.uk>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <alpine.LSU.2.00.0907062124240.30197@hermes-2.csi.cam.ac.uk>
User-Agent: Mutt/1.5.7i
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

On Mon, Jul 06, 2009 at 09:25:32PM +0100, Tony Finch wrote:
> On Mon, 6 Jul 2009, Michael Kirkham wrote:
> > On Sun, 5 Jul 2009, Ned Freed wrote:
> >
> > > Please explain how moving a message to a random, arbtirarily named folder
> > > tells the server anything useful.
> >
> > Seems to me this is up to the server configuration and software running on the
> > server.  I don't see why you need a specific contract between IMAP client and
> > server for flagging spam
> 
> So that clients can have user interfaces that expose the functionality
> in a more friendly manner.

More specifically: so that users and their clients don't need to know
specific details of the server's configuration.



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66L7bn3086261 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 14:07:37 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n66L7bUX086260; Mon, 6 Jul 2009 14:07:37 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from rufus.isode.com (rufus.isode.com [62.3.217.251]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66L7ZFJ086252 for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 14:07:36 -0700 (MST) (envelope-from alexey.melnikov@isode.com)
Received: from [92.40.43.85] (92.40.43.85.sub.mbb.three.co.uk [92.40.43.85])  by rufus.isode.com (submission channel) via TCP with ESMTPA  id <SlJnlQBV9D0r@rufus.isode.com>; Mon, 6 Jul 2009 22:07:34 +0100
Message-ID: <4A526763.1020202@isode.com>
Date: Mon, 06 Jul 2009 22:06:43 +0100
From: Alexey Melnikov <alexey.melnikov@isode.com>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/20050915
X-Accept-Language: en-us, en
To: Iljitsch van Beijnum <iljitsch@muada.com>
CC: ietf-imapext@imc.org, discuss@ietf.org
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <01NAZUST62HQ001DLH@mauve.mrochek.com> <20090706100834.R49379@eng-bsd.muonics.com> <01NAZXHZ47O8000078@mauve.mrochek.com> <60154352-AE34-4FDD-A286-A2AF63C07903@muada.com>
In-Reply-To: <60154352-AE34-4FDD-A286-A2AF63C07903@muada.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

Iljitsch van Beijnum wrote:
 [...]

> About folders for spam: Apple's Mail lets the user select "use this  
> folder for... {sent, drafts, junk, trash}, which is nice if you used 
> a  different client before that uses a different folder naming scheme 
> for  these things. Works well enough although a little more 
> automation  wouldn't hurt.
>
> The point that working with folders can be done with existing 
> software  (I later train spamassassin with what Mail put in the "junk" 
> folder,  even though SA doesn't know about IMAP) is an important one. 
> Requiring  keywords that are hardcoded in clients and servers means 
> deployment  will be extremely slow, if it ever happens.

I don't think your argument in favour of folders (mailboxes) is valid. 
An IMAP server don't need to treat $Junk/$NotJunk any specially, it just 
needs to be able to store arbitrary IMAP keywords. Most servers already do.
A recent Thunderbird can also be configured to set arbitrary IMAP 
keywords. (It also recognized $Junk/$NonJunk natively, but this is 
besides my point).




Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66KtxQN085525 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 13:55:59 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n66KtxmJ085524; Mon, 6 Jul 2009 13:55:59 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from kalyani.oryx.com (kalyani.oryx.com [195.30.37.30]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66KtlJL085512 for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 13:55:58 -0700 (MST) (envelope-from arnt@gulbrandsen.priv.no)
Received: from kalyani.oryx.com (localhost [127.0.0.1]) by kalyani.oryx.com (Postfix) with ESMTP id C7F212E2A2; Mon,  6 Jul 2009 22:54:17 +0200 (CEST)
Received: from arnt@oryx.com (HELO lochnagar.oryx.com) by kalyani.oryx.com (Archiveopteryx 3.1.0) with esmtp id 1246913657-80172-80171/6/65 (5 recipients); Mon, 6 Jul 2009 22:54:17 +0200
Message-Id: <QlppcXA5Naz0BxqARBkNPQ.md5@lochnagar.oryx.com>
Date: Mon, 6 Jul 2009 22:56:19 +0200
From: Arnt Gulbrandsen <arnt@gulbrandsen.priv.no>
To: ietf-imapext@imc.org
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
Cc: Lyndon Nerenberg <lyndon@orthanc.ca>, Ned Freed <ned.freed@mrochek.com>, discuss@ietf.org
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <01NAZUST62HQ001DLH@mauve.mrochek.com> <1246902821.8602.70.camel@legolas.orthanc.ca> <01NAZWHC66ZU000078@mauve.mrochek.com> <1246904761.8602.86.camel@legolas.orthanc.ca>
In-Reply-To: <1246904761.8602.86.camel@legolas.orthanc.ca>
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset=iso-8859-1; format=flowed
Mime-Version: 1.0
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

Lyndon Nerenberg writes:
> But if your concern is adoption rates, a keyword-only solution will=20
> get wider take-up than something that requires support for=20
> annotations.

Given that two significant clients (say two of the top five) use the=20
keyword approach today, I think it's safe to say =ABhas received wider=20
take-up=BB, not =ABwill get wider=BB.

Arnt



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66KPrZQ083864 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 13:25:53 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n66KPrdC083863; Mon, 6 Jul 2009 13:25:53 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from ppsw-6.csi.cam.ac.uk (ppsw-6.csi.cam.ac.uk [131.111.8.136]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66KPfG0083850 for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 13:25:52 -0700 (MST) (envelope-from fanf2@hermes.cam.ac.uk)
X-Cam-AntiVirus: no malware found
X-Cam-SpamDetails: not scanned
X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/
Received: from hermes-2.csi.cam.ac.uk ([131.111.8.54]:54795) by ppsw-6.csi.cam.ac.uk (smtp.hermes.cam.ac.uk [131.111.8.156]:25) with esmtpa (EXTERNAL:fanf2) id 1MNukm-0001XD-JY (Exim 4.70) (return-path <fanf2@hermes.cam.ac.uk>); Mon, 06 Jul 2009 21:25:32 +0100
Received: from fanf2 (helo=localhost) by hermes-2.csi.cam.ac.uk (hermes.cam.ac.uk) with local-esmtp id 1MNukm-0003j2-1E (Exim 4.67) (return-path <fanf2@hermes.cam.ac.uk>); Mon, 06 Jul 2009 21:25:32 +0100
Date: Mon, 6 Jul 2009 21:25:32 +0100
From: Tony Finch <dot@dotat.at>
X-X-Sender: fanf2@hermes-2.csi.cam.ac.uk
To: Michael Kirkham <mikek@muonics.com>
cc: Ned Freed <ned.freed@mrochek.com>, ietf-imapext@imc.org, George Michaelson <ggm@pobox.com>, discuss@ietf.org
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
In-Reply-To: <20090706100834.R49379@eng-bsd.muonics.com>
Message-ID: <alpine.LSU.2.00.0907062124240.30197@hermes-2.csi.cam.ac.uk>
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <01NAZUST62HQ001DLH@mauve.mrochek.com> <20090706100834.R49379@eng-bsd.muonics.com>
User-Agent: Alpine 2.00 (LSU 1167 2008-08-23)
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

On Mon, 6 Jul 2009, Michael Kirkham wrote:
> On Sun, 5 Jul 2009, Ned Freed wrote:
>
> > Please explain how moving a message to a random, arbtirarily named folder
> > tells the server anything useful.
>
> Seems to me this is up to the server configuration and software running on the
> server.  I don't see why you need a specific contract between IMAP client and
> server for flagging spam

So that clients can have user interfaces that expose the functionality
in a more friendly manner.

Tony.
-- 
f.anthony.n.finch  <dot@dotat.at>  http://dotat.at/
GERMAN BIGHT HUMBER: SOUTHWEST 5 TO 7. MODERATE OR ROUGH. SQUALLY SHOWERS.
MODERATE OR GOOD.



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66KGvEm083408 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 13:16:57 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n66KGvix083407; Mon, 6 Jul 2009 13:16:57 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from peirce.dave.cridland.net (peirce.dave.cridland.net [217.155.137.61]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66KGiZS083386 for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 13:16:56 -0700 (MST) (envelope-from dave@cridland.net)
Received: from puncture ((unknown) [217.155.137.60]) by peirce.dave.cridland.net (submission) via TCP with ESMTPA id <SlJbqQAd5Hve@peirce.dave.cridland.net>; Mon, 6 Jul 2009 21:16:42 +0100
X-SMTP-Protocol-Errors: NORDNS
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <01NAZUST62HQ001DLH@mauve.mrochek.com> <1246902821.8602.70.camel@legolas.orthanc.ca>
In-Reply-To: <1246902821.8602.70.camel@legolas.orthanc.ca>
MIME-Version: 1.0
Message-Id: <13196.1246911390.279383@puncture>
Date: Mon, 06 Jul 2009 21:16:30 +0100
From: Dave Cridland <dave@cridland.net>
To: Lyndon Nerenberg <lyndon@orthanc.ca>, <ietf-imapext@imc.org>, general discussion of application-layer protocols <discuss@ietf.org>, Ned Freed <ned.freed@mrochek.com>
Content-Type: text/plain; delsp="yes"; charset="us-ascii"; format="flowed"
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

On Mon Jul  6 18:53:41 2009, Lyndon Nerenberg wrote:
> On Sun, 2009-07-05 at 17:40 -0700, Ned Freed wrote:
> > The obvious way to make this work in IMAP is with a standardized
> > folder
> > annotation saying "messages put in this folder were flagged as  
> spam".
> > Then
> > moves of messages to the folder with this annotation provide the
> > server with
> > the necessary information that this message is considered to be  
> spam
> > by the
> > user. And by the same token, moving a message from this folder  
> can be
> > construed
> > as the message not being considered spam after all.
> 
> This is just trying to turn a folder into a keyword, but without  
> full
> keyword semantics.

+1.

It's fundamentally no different from clients trying to replace the  
\Deleted keyword with a Trash mailbox, and has all the related  
issues, which we presumably all know. (Or else can find the attempt  
at documenting them I once tried).

This is compounded by the state being a tristate rather than binary -  
messages are "Ham", or "Spam", or else "The user has not expressed  
any decision". Should I move all my messages into another folder to  
mark them as good ham?

Dave.
-- 
Dave Cridland - mailto:dave@cridland.net - xmpp:dwd@dave.cridland.net
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66Jvcjm082027 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 12:57:38 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n66Jvca1082026; Mon, 6 Jul 2009 12:57:38 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66JvPCe082013 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 12:57:37 -0700 (MST) (envelope-from iljitsch@muada.com)
Received: from [192.168.2.4] (static-167-138-7-89.ipcom.comunitel.net [89.7.138.167] (may be forged)) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n66JthxZ053208 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Mon, 6 Jul 2009 21:55:44 +0200 (CEST) (envelope-from iljitsch@muada.com)
Cc: Michael Kirkham <mikek@muonics.com>, ietf-imapext@imc.org, George Michaelson <ggm@pobox.com>, discuss@ietf.org
Message-Id: <60154352-AE34-4FDD-A286-A2AF63C07903@muada.com>
From: Iljitsch van Beijnum <iljitsch@muada.com>
To: Ned Freed <ned.freed@mrochek.com>
In-Reply-To: <01NAZXHZ47O8000078@mauve.mrochek.com>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
Content-Transfer-Encoding: 7bit
Mime-Version: 1.0 (Apple Message framework v935.3)
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
Date: Mon, 6 Jul 2009 21:55:39 +0200
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <01NAZUST62HQ001DLH@mauve.mrochek.com> <20090706100834.R49379@eng-bsd.muonics.com> <01NAZXHZ47O8000078@mauve.mrochek.com>
X-Mailer: Apple Mail (2.935.3)
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

On 6 jul 2009, at 20:07, Ned Freed wrote:

>> it was pretty painless for me to set up cron jobs to suck in
>> whatever messages I save to certain folders (that could have been  
>> named
>> anything) for training my spam filter.

> Are you serious? Cron jobs? Do you have any idea how minute a  
> fraction of the
> users of email even know what a cron job is, let alone know how to  
> set one up?

Hey, everyone needs a hobby.

I have my procmail set up to check whether I have an IMAP session with  
the server. If so, I let the client do the mail filtering, if not, the  
server does it. That way, the unread counters for folders get updated  
when the client is connected, but the client doesn't have to move  
large volume mailing list traffic to the correct folder, costing me  
expensive wireless data traffic and/or slowing things down.

About folders for spam: Apple's Mail lets the user select "use this  
folder for... {sent, drafts, junk, trash}, which is nice if you used a  
different client before that uses a different folder naming scheme for  
these things. Works well enough although a little more automation  
wouldn't hurt.

The point that working with folders can be done with existing software  
(I later train spamassassin with what Mail put in the "junk" folder,  
even though SA doesn't know about IMAP) is an important one. Requiring  
keywords that are hardcoded in clients and servers means deployment  
will be extremely slow, if it ever happens.



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66IdsKm076543 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 11:39:54 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n66IdsZL076542; Mon, 6 Jul 2009 11:39:54 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from mauve.mrochek.com (mauve.mrochek.com [66.59.230.40]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66IdrNX076536 for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 11:39:53 -0700 (MST) (envelope-from ned.freed@mrochek.com)
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NAZXPKJNKG00SFNX@mauve.mrochek.com> for ietf-imapext@imc.org; Mon, 6 Jul 2009 11:39:50 -0700 (PDT)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NAZUBB7I4G000078@mauve.mrochek.com>; Mon, 06 Jul 2009 11:39:46 -0700 (PDT)
Date: Mon, 06 Jul 2009 11:35:17 -0700 (PDT)
From: Ned Freed <ned.freed@mrochek.com>
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
In-reply-to: "Your message dated Mon, 06 Jul 2009 12:02:40 -0600" <1246903360.8602.78.camel@legolas.orthanc.ca>
To: Lyndon Nerenberg <lyndon@orthanc.ca>
Cc: Ned Freed <ned.freed@mrochek.com>, ietf-imapext@imc.org
Message-id: <01NAZXPHSOKI000078@mauve.mrochek.com>
MIME-version: 1.0
Content-type: TEXT/PLAIN
Content-transfer-encoding: 7BIT
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <28941.1246868682.560969@puncture> <4A51D1A1.5040507@isode.com> <01NAZVQAYWH4001DLH@mauve.mrochek.com> <1246903360.8602.78.camel@legolas.orthanc.ca>
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

> On Mon, 2009-07-06 at 10:39 -0700, Ned Freed wrote:
> > The problem with using keywords for this is that you are now
> > dependence on your
> > client supporting that particular set of keywords and their associated
> > semantics. Had we standardize such a keyword set 10 or so years ago
> > this
> > approach miight have had a chance, but the client base is now too
> > large and
> > too difficult to update.

> Is it really too late? We're making an request to process a particular
> message through the site's "learning" filters, i.e. this is explicit
> request to have the server take action on the message. I don't that
> behaviour is implied by any of the current schemes floating around.

Well, you have two choices: Try to extend an existing ad-hoc keyword
usage scheme that has some deployment but doesn't have quite these semaantics
currently, or start from scratch with a fresh keyword set that has zero
deployment and is unlikely to displace any of the ad-hoc schemes.

This choice is why I view using keywords as at a minimum suboptimal.

> I don't see how it could hurt to standardize a pair of new keywords (as
> I suggested in my previous message) with this behaviour documented. It
> lets the user request specific identified behaviour from the server; the
> existing keywords are all private switches for client behaviour, in my
> experience, so the new keyword behaviour would be complimentary to
> what's already out there.

I on't think it would hurt either, the question is whether it will actually
accomplish something.

				Ned



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66IXp3l076243 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 11:33:51 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n66IXpf4076242; Mon, 6 Jul 2009 11:33:51 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from mauve.mrochek.com (mauve.mrochek.com [66.59.230.40]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66IXoMR076235 for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 11:33:51 -0700 (MST) (envelope-from ned.freed@mrochek.com)
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NAZXI19NJK00SFNX@mauve.mrochek.com> for ietf-imapext@imc.org; Mon, 6 Jul 2009 11:33:46 -0700 (PDT)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NAZUBB7I4G000078@mauve.mrochek.com>; Mon, 06 Jul 2009 11:33:43 -0700 (PDT)
Date: Mon, 06 Jul 2009 11:07:18 -0700 (PDT)
From: Ned Freed <ned.freed@mrochek.com>
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
In-reply-to: "Your message dated Mon, 06 Jul 2009 10:26:10 -0700 (PDT)" <20090706100834.R49379@eng-bsd.muonics.com>
To: Michael Kirkham <mikek@muonics.com>
Cc: Ned Freed <ned.freed@mrochek.com>, Iljitsch van Beijnum <iljitsch@muada.com>, ietf-imapext@imc.org, George Michaelson <ggm@pobox.com>, discuss@ietf.org
Message-id: <01NAZXHZ47O8000078@mauve.mrochek.com>
MIME-version: 1.0
Content-type: TEXT/PLAIN; format=flowed
Content-transfer-encoding: 7BIT
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <01NAZUST62HQ001DLH@mauve.mrochek.com> <20090706100834.R49379@eng-bsd.muonics.com>
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

> On Sun, 5 Jul 2009, Ned Freed wrote:

> > Please explain how moving a message to a random, arbtirarily named
> > folder tells the server anything useful.

> Seems to me this is up to the server configuration and software running on
> the server.  I don't see why you need a specific contract between IMAP
> client and server for flagging spam,

The primary reason is so that the server can use the information to
automaticallly refine it's filtering policies on a per-user basis.

Other uses include, but are not limited to, setting expiration and arhival
policies.

> and it's of limited usefulness.

On the contrary, such a mechanism would be HUGELY useful. We have any number of
customers begging for it, as a matter of fact.

> For
> one thing, it was pretty painless for me to set up cron jobs to suck in
> whatever messages I save to certain folders (that could have been named
> anything) for training my spam filter.

Are you serious? Cron jobs? Do you have any idea how minute a fraction of the
users of email even know what a cron job is, let alone know how to set one up?

The minute you start talking about setting up your own cron jobs, it  pretty
much goes without saying that you're operating entirely outside the space where
99+% of the use of IMAP takes place.

> I also have other cron jobs set up
> to do other things with email completely unrelated to spam that also
> simply involve saving to a particular folder (e.g. a tickler file for
> email--I move messages to folders named for a month or day, and cron jobs
> automatiacally reinsert them back in my inbox later).  That kind of thing
> isn't addressed by having something spam-specific, but as far as my email
> client is concerned, they're flagged the same way.

And no doubt you also tweak SpamAssasin rules, set up milters, etc. etc. This
is far beyond the capability of most users.

				Ned



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66IUei8076008 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 11:30:40 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n66IUe9J076007; Mon, 6 Jul 2009 11:30:40 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from rufus.isode.com (rufus.isode.com [62.3.217.251]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66IUc3x076001 for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 11:30:39 -0700 (MST) (envelope-from alexey.melnikov@isode.com)
Received: from [92.40.185.222] (92.40.185.222.sub.mbb.three.co.uk [92.40.185.222])  by rufus.isode.com (submission channel) via TCP with ESMTPA  id <SlJCzABV9Cxd@rufus.isode.com>; Mon, 6 Jul 2009 19:30:37 +0100
Message-ID: <4A524280.9000106@isode.com>
Date: Mon, 06 Jul 2009 19:29:20 +0100
From: Alexey Melnikov <alexey.melnikov@isode.com>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/20050915
X-Accept-Language: en-us, en
To: Lyndon Nerenberg <lyndon@orthanc.ca>
CC: Ned Freed <ned.freed@mrochek.com>, ietf-imapext@imc.org, discuss@ietf.org
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <01NAZUST62HQ001DLH@mauve.mrochek.com> <1246902821.8602.70.camel@legolas.orthanc.ca> <01NAZWHC66ZU000078@mauve.mrochek.com> <1246904761.8602.86.camel@legolas.orthanc.ca>
In-Reply-To: <1246904761.8602.86.camel@legolas.orthanc.ca>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

Lyndon Nerenberg wrote:

>But if your concern is adoption rates, a keyword-only solution will get
>wider take-up than something that requires support for annotations. (And
>from a purely engineering standpoint, the simplest solution wins.)
>  
>
+1.



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66IQLrS075707 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 11:26:21 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n66IQL2x075706; Mon, 6 Jul 2009 11:26:21 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from orthanc.ca (orthanc.ca [208.86.224.138]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66IQK0p075700 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 11:26:21 -0700 (MST) (envelope-from lyndon@orthanc.ca)
Received: from [192.168.0.42] (S01060011242eafe2.cg.shawcable.net [68.145.131.177]) (authenticated bits=0) by orthanc.ca (8.14.3/8.14.3) with ESMTP id n66IQ7lm095621; Mon, 6 Jul 2009 12:26:08 -0600 (MDT) (envelope-from lyndon@orthanc.ca)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=orthanc.ca; s=2009-1; t=1246904768; bh=WwrMcL9AQ6/lx87Ocpx19pPgFGa77TTioXFJrUQ6PG0=; h=Subject:From:To:Cc:In-Reply-To:References:Content-Type:Date: Message-Id:Mime-Version:Content-Transfer-Encoding; b=UpIRbLSvwCIbBPMKoSj32racmtHCxgrbXDONA7nswDpkJvUQBkRfgEKOzqw9ehBSX YqeF4VAVWmpI7RWOIjmTAI2ZPbGUTdryjtWGZE602ZKCP9EZ9hrIJ9Qtr+GQg6GHMa lMllguVaer1Sv+6qwOzmPhrWvCjFsB3G14Ss4qZmLMAieFrdu7Gv5Ak7bUoT71w/XN zcSGgCEzIAq0IddWJXtOZcAKiIkrAwDGtz3tXN0q0wdB4wSn4uCM0RBfucYRP0e+nY Lef60eIg9cG+xn15i8e9WRCjqp2JLP0tub0DmBDo/XyWz7DdCXRxDgJYNDclblVKa8 5pEpPz6qEeLag==
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
From: Lyndon Nerenberg <lyndon@orthanc.ca>
To: Ned Freed <ned.freed@mrochek.com>
Cc: ietf-imapext@imc.org, discuss@ietf.org
In-Reply-To: <01NAZWHC66ZU000078@mauve.mrochek.com>
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <01NAZUST62HQ001DLH@mauve.mrochek.com> <1246902821.8602.70.camel@legolas.orthanc.ca> <01NAZWHC66ZU000078@mauve.mrochek.com>
Content-Type: text/plain
Date: Mon, 06 Jul 2009 12:26:01 -0600
Message-Id: <1246904761.8602.86.camel@legolas.orthanc.ca>
Mime-Version: 1.0
X-Mailer: Evolution 2.26.2 FreeBSD GNOME Team Port 
Content-Transfer-Encoding: 7bit
X-Spam-Status: No, score=2.1 required=5.0 tests=DKIM_SIGNED,DKIM_VERIFIED, RCVD_IN_PBL,RCVD_IN_SORBS_DUL autolearn=no version=3.2.5
X-Spam-Level: **
X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on orthanc.ca
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

On Mon, 2009-07-06 at 11:00 -0700, Ned Freed wrote:
> If there's a reason multiple folders can't be marked this way I sure
> don't
> know what it is.

You're right -- I didn't parse what you were saying correctly the first
time around.

But if your concern is adoption rates, a keyword-only solution will get
wider take-up than something that requires support for annotations. (And
from a purely engineering standpoint, the simplest solution wins.)

--lyndon



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66IGNeL075045 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 11:16:23 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n66IGN2k075044; Mon, 6 Jul 2009 11:16:23 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from rufus.isode.com (rufus.isode.com [62.3.217.251]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66IGBM6075027 for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 11:16:22 -0700 (MST) (envelope-from alexey.melnikov@isode.com)
Received: from [172.16.2.141] (shiny.isode.com [62.3.217.250])  by rufus.isode.com (submission channel) via TCP with ESMTPA  id <SlI=ZQBV9Lui@rufus.isode.com>; Mon, 6 Jul 2009 19:16:09 +0100
Message-ID: <4A523F0E.5090706@isode.com>
Date: Mon, 06 Jul 2009 19:14:38 +0100
From: Alexey Melnikov <alexey.melnikov@isode.com>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/20050915
X-Accept-Language: en-us, en
To: Arnt Gulbrandsen <arnt@gulbrandsen.priv.no>
CC: general discussion of application-layer protocols <discuss@ietf.org>, ietf-imapext@imc.org, Ned Freed <ned.freed@mrochek.com>, Iljitsch van Beijnum <iljitsch@muada.com>, George Michaelson <ggm@pobox.com>
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <01NAZUST62HQ001DLH@mauve.mrochek.com> <UP3Anyei29+Pc7cGSi2Sdg.md5@lochnagar.oryx.com>
In-Reply-To: <UP3Anyei29+Pc7cGSi2Sdg.md5@lochnagar.oryx.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

Arnt Gulbrandsen wrote:

> Ned Freed writes:
>
>> The obvious way to make this work in IMAP is with a standardized 
>> folder annotation saying "messages put in this folder were flagged as 
>> spam".
>
> I think what Thunderbird and Apple Mail do is to set specific message 
> flags on messages in whatever folders they happen to be in: $Junk if 
> the user said a particular message is spam, $autojunk if tb 
> autonomously decided that message is spam, $nonjunk and $autononjunk 
> for the opposite. (I may be wrong about the exact flag names.)

Something like that, yes.

> This was specified in some expired draft,

Which was mine.

> which IMO now has been frozen by deployment. Apple Mail and 
> Thunderbird add up to >10% for me. Time for an informational RFC, 
> perhaps. "Widely-used IMAP flag names"?

Yes. I think even more important to create the IANA registry for keywords.
Do you want to help finish my draft?



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66I4Fwi074004 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 11:04:15 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n66I4FMU074003; Mon, 6 Jul 2009 11:04:15 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from mauve.mrochek.com (mauve.mrochek.com [66.59.230.40]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66I4E7T073996 for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 11:04:14 -0700 (MST) (envelope-from ned.freed@mrochek.com)
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NAZWHDBNTS005LCB@mauve.mrochek.com> for ietf-imapext@imc.org; Mon, 6 Jul 2009 11:04:12 -0700 (PDT)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NAZUBB7I4G000078@mauve.mrochek.com>; Mon, 06 Jul 2009 11:04:10 -0700 (PDT)
Date: Mon, 06 Jul 2009 11:00:24 -0700 (PDT)
From: Ned Freed <ned.freed@mrochek.com>
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
In-reply-to: "Your message dated Mon, 06 Jul 2009 11:53:41 -0600" <1246902821.8602.70.camel@legolas.orthanc.ca>
To: Lyndon Nerenberg <lyndon@orthanc.ca>
Cc: Ned Freed <ned.freed@mrochek.com>, ietf-imapext@imc.org, discuss@ietf.org
Message-id: <01NAZWHC66ZU000078@mauve.mrochek.com>
MIME-version: 1.0
Content-type: TEXT/PLAIN
Content-transfer-encoding: 7BIT
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <01NAZUST62HQ001DLH@mauve.mrochek.com> <1246902821.8602.70.camel@legolas.orthanc.ca>
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

> On Sun, 2009-07-05 at 17:40 -0700, Ned Freed wrote:
> > The obvious way to make this work in IMAP is with a standardized
> > folder
> > annotation saying "messages put in this folder were flagged as spam".
> > Then
> > moves of messages to the folder with this annotation provide the
> > server with
> > the necessary information that this message is considered to be spam
> > by the
> > user. And by the same token, moving a message from this folder can be
> > construed
> > as the message not being considered spam after all.

> This is just trying to turn a folder into a keyword, but without full
> keyword semantics.

Actually, it's a hack t work around the lack of client support, which is why
I didn't propose a keyword-based mechanism.

> The most obvious flaw is that it constrains the
> client to storing all the spam in one bucket.

If there's a reason multiple folders can't be marked this way I sure don't
know what it is.

> It doesn't allow for (say)
> sorting junk by type into different folders (perhaps on different
> servers). A message's 'state of spamminess' is an attribute of the
> message, properly denoted by keywords that stick to (and with) the
> message.

I see no reason why this couldn't be done, if you really want to.

> If you want this functionality (and I think it's a useful service for
> servers to provide) you need to implement two keywords: $spam and
> $nospam (these are just example names). Setting $spam on a message would
> instruct the server to pass the message off to the filtering component
> as spam; setting $nospam would instruct the server to pass the message
> off to the filtering component as valid (non-spam). The two keywords are
> exclusive, and it would be up to the server to decide what action to
> take if the client tries to light up both at once (e.g. unset the other
> flag and take the corresponding action, reject the command, or silently
> ignore the request).

Again, the problem with using keywords is now you have to get a significant
fraction of clients on board before it's useful. I'd be willing to go along
with a keyword-based approach if I could convinced that's going to happen, but
everything recentlts has supported exactly the opposite conclusion.

				Ned



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66I2nHe073879 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 11:02:49 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n66I2nuu073878; Mon, 6 Jul 2009 11:02:49 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from orthanc.ca (orthanc.ca [208.86.224.138]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66I2lok073870 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 11:02:48 -0700 (MST) (envelope-from lyndon@orthanc.ca)
Received: from [192.168.0.42] (S01060011242eafe2.cg.shawcable.net [68.145.131.177]) (authenticated bits=0) by orthanc.ca (8.14.3/8.14.3) with ESMTP id n66I2jxa095178; Mon, 6 Jul 2009 12:02:46 -0600 (MDT) (envelope-from lyndon@orthanc.ca)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=orthanc.ca; s=2009-1; t=1246903366; bh=IaIoBTuR8KNs1Prrx1DWRJHGYAhCgKsDSqZgGFvSJU8=; h=Subject:From:To:Cc:In-Reply-To:References:Content-Type:Date: Message-Id:Mime-Version:Content-Transfer-Encoding; b=DWAl8lPHNm+tcsPjH8YhV1CW28atPInh4lE7MFgls71WXpZ5ZwPZqfPVQNAbeMEKO bKWOuk+eg9enx41+z1NavXTfNT76KPjPNsMcD+6mQQWk8DD42mEi9ArK1axgjP8Ksl pP+wvE/Pg9Xhl5u3EWfcJZmMzSf1J0UywQ5dtsvNoUTZEJ0hudctPD+TJozZa1VqYy 8B9Q42QE304Da8Dtoxj5c4E4NrXWD9Ad/1lt8RU8bHtWMQM5MYafwozoCB1/sIco7a jzwh2UNEPJpcI4O0fb8FVKzxL290Fp+xbOmshBD1zFtkd8JHVftc0tlpvGySPu3A9V 0WPA/tyv6SebA==
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
From: Lyndon Nerenberg <lyndon@orthanc.ca>
To: Ned Freed <ned.freed@mrochek.com>
Cc: ietf-imapext@imc.org
In-Reply-To: <01NAZVQAYWH4001DLH@mauve.mrochek.com>
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <28941.1246868682.560969@puncture> <4A51D1A1.5040507@isode.com> <01NAZVQAYWH4001DLH@mauve.mrochek.com>
Content-Type: text/plain
Date: Mon, 06 Jul 2009 12:02:40 -0600
Message-Id: <1246903360.8602.78.camel@legolas.orthanc.ca>
Mime-Version: 1.0
X-Mailer: Evolution 2.26.2 FreeBSD GNOME Team Port 
Content-Transfer-Encoding: 7bit
X-Spam-Status: No, score=2.1 required=5.0 tests=DKIM_SIGNED,DKIM_VERIFIED, RCVD_IN_PBL,RCVD_IN_SORBS_DUL autolearn=no version=3.2.5
X-Spam-Level: **
X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on orthanc.ca
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

On Mon, 2009-07-06 at 10:39 -0700, Ned Freed wrote:
> The problem with using keywords for this is that you are now
> dependence on your
> client supporting that particular set of keywords and their associated
> semantics. Had we standardize such a keyword set 10 or so years ago
> this
> approach miight have had a chance, but the client base is now too
> large and
> too difficult to update.

Is it really too late? We're making an request to process a particular
message through the site's "learning" filters, i.e. this is explicit
request to have the server take action on the message. I don't that
behaviour is implied by any of the current schemes floating around.

I don't see how it could hurt to standardize a pair of new keywords (as
I suggested in my previous message) with this behaviour documented. It
lets the user request specific identified behaviour from the server; the
existing keywords are all private switches for client behaviour, in my
experience, so the new keyword behaviour would be complimentary to
what's already out there.

--lyndon



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66I2Ss8073854 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 11:02:29 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n66I2SPw073853; Mon, 6 Jul 2009 11:02:28 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from sca-ea-mail-2.sun.com (sca-ea-mail-2.Sun.COM [192.18.43.25]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66I25Sn073823 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO) for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 11:02:16 -0700 (MST) (envelope-from Nicolas.Williams@sun.com)
Received: from dm-central-02.central.sun.com ([129.147.62.5]) by sca-ea-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n66I24VB028574 for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 18:02:05 GMT
Received: from binky.Central.Sun.COM (binky.Central.Sun.COM [129.153.128.104]) by dm-central-02.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n66I246j054013 for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 12:02:04 -0600 (MDT)
Received: from binky.Central.Sun.COM (localhost [127.0.0.1]) by binky.Central.Sun.COM (8.14.3+Sun/8.14.3) with ESMTP id n66HiKfZ019363; Mon, 6 Jul 2009 12:44:20 -0500 (CDT)
Received: (from nw141292@localhost) by binky.Central.Sun.COM (8.14.3+Sun/8.14.3/Submit) id n66HiEk1019362; Mon, 6 Jul 2009 12:44:14 -0500 (CDT)
X-Authentication-Warning: binky.Central.Sun.COM: nw141292 set sender to Nicolas.Williams@sun.com using -f
Date: Mon, 6 Jul 2009 12:44:14 -0500
From: Nicolas Williams <Nicolas.Williams@sun.com>
To: Ned Freed <ned.freed@mrochek.com>
Cc: Iljitsch van Beijnum <iljitsch@muada.com>, ietf-imapext@imc.org, George Michaelson <ggm@pobox.com>, discuss@ietf.org
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
Message-ID: <20090706174414.GM15302@Sun.COM>
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <01NAZUST62HQ001DLH@mauve.mrochek.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <01NAZUST62HQ001DLH@mauve.mrochek.com>
User-Agent: Mutt/1.5.7i
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

On Sun, Jul 05, 2009 at 05:40:31PM -0700, Ned Freed wrote:
> 
> >On 3 jul 2009, at 1:05, George Michaelson wrote:
> 
> >> As an example, at the moment if I wish to inform google that I have
> >> known spam in local folders, I have to go to the web interface and
> >> manually tag. If there was an IMAP extension, I could review my
> >> local baysian junk folder, remove all non-spam (and flag the senders
> >> as white-listed if need be), and request the rest to be flagged as
> >> spam back on the IMAP backed MS.
> 
> >Doesn't moving the spam messages to the spam folder accomplish this
> >already? That's what my client does.
> 
> Please explain how moving a message to a random, arbtirarily named
> folder tells the server anything useful. People use all sorts of
> different naming schemes for folders, and it's common for clients to
> provide lots of flexibility in this area.

In particular the word 'spam' is really an English-language word.  L10N
would demand that the name of this folder not be 'spam' in all locales,
or at least that clients know how to map that folder name to the user's
language(s).  But even if we go for the latter, having special folder
names means denying or complicating the use of those names in other
languages for other purposes; it's bad enough we have one well-known
folder name (INBOX), let's have no more.

> The obvious way to make this work in IMAP is with a standardized
> folder annotation saying "messages put in this folder were flagged as
> spam". Then moves of messages to the folder with this annotation
> provide the server with the necessary information that this message is
> considered to be spam by the user. And by the same token, moving a
> message from this folder can be construed as the message not being
> considered spam after all.

+1

Or even just a message flag/annotation.  I understand the "move junk
mail to the junk mail folder" metaphor for flagging junk mail as such,
but one might want to flag junk mail and still move it to an "archived
mail" folder.  I don't find my argument for a per-message flag terribly
convincing, so I won't push it.

> It isn't perfect, but as long as this is taken as an indicator, not as
> an absolute statement, it should get the job done.
> 
> >But if you want this, I'd say that it needs to be a fractional thing,
> >not a binary spam/no spam indication. For instance, the server could
> >give something a spam score of 2 and the client also 2 and together
> >that would be 4 so the message is presumed to be spam (assuming the
> >spam threshold is 3), but in a binary system no spam OR no spam = no
> >spam.
> 
> There's a place for fractional scoring in spam filtering, but this isn't
> it. A user doesn't think "this message is 90% likely to be spam", they
> either think it's spam or it isn't.

+1

Fractional scoring by the server, or by software running on the client
can be useful, but the users themselves will generally make binary spam/
not-spam decisions.

Nico
-- 



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66Hx0oW073500 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 10:59:00 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n66Hx066073499; Mon, 6 Jul 2009 10:59:00 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from orthanc.ca (orthanc.ca [208.86.224.138]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66HwnKx073488 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 10:59:00 -0700 (MST) (envelope-from lyndon@orthanc.ca)
Received: from [192.168.0.42] (S01060011242eafe2.cg.shawcable.net [68.145.131.177]) (authenticated bits=0) by orthanc.ca (8.14.3/8.14.3) with ESMTP id n66HrlfS094952; Mon, 6 Jul 2009 11:53:47 -0600 (MDT) (envelope-from lyndon@orthanc.ca)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=orthanc.ca; s=2009-1; t=1246902828; bh=43yaytwJNSzxZ5RcTMNBAjKIwGNjzZ4YtRQZoliwL+M=; h=Subject:From:To:Cc:In-Reply-To:References:Content-Type:Date: Message-Id:Mime-Version:Content-Transfer-Encoding; b=SiB2C1/Pib6nDPH+BvnP1LLBdtfZGOBmv1Ifkl7CFQsN8W6leE2GPuD2pWLLmEq7p m8JIt+jG6YoCjg9osnORaz1AaXz5EDGObCkkfc12rK1tPWFybSFIcbDeRs6JMXtEQI HLujERdLnxCngkOVdrcrfCNS/8fKe0eecVM2SkEFtRx6tYgrFarb6Zfh/JOH+hIvKj o8ftiLby6kxMszh0mviaX+WDdehBTSllN7GJ7HajGs6OlCgZgglBDGwZpc6hIfAqBc QzLGt5QhcI0cbqleDaZxkrEK5UawakmhEZKqnNbwLl/cvz48ph2SlAG6sXnjkpTIQC tsbWon+QvYcLQ==
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
From: Lyndon Nerenberg <lyndon@orthanc.ca>
To: Ned Freed <ned.freed@mrochek.com>
Cc: ietf-imapext@imc.org, discuss@ietf.org
In-Reply-To: <01NAZUST62HQ001DLH@mauve.mrochek.com>
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <01NAZUST62HQ001DLH@mauve.mrochek.com>
Content-Type: text/plain
Date: Mon, 06 Jul 2009 11:53:41 -0600
Message-Id: <1246902821.8602.70.camel@legolas.orthanc.ca>
Mime-Version: 1.0
X-Mailer: Evolution 2.26.2 FreeBSD GNOME Team Port 
Content-Transfer-Encoding: 7bit
X-Spam-Status: No, score=2.1 required=5.0 tests=DKIM_SIGNED,DKIM_VERIFIED, RCVD_IN_PBL,RCVD_IN_SORBS_DUL autolearn=no version=3.2.5
X-Spam-Level: **
X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on orthanc.ca
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

On Sun, 2009-07-05 at 17:40 -0700, Ned Freed wrote:
> The obvious way to make this work in IMAP is with a standardized
> folder
> annotation saying "messages put in this folder were flagged as spam".
> Then
> moves of messages to the folder with this annotation provide the
> server with
> the necessary information that this message is considered to be spam
> by the
> user. And by the same token, moving a message from this folder can be
> construed
> as the message not being considered spam after all.

This is just trying to turn a folder into a keyword, but without full
keyword semantics. The most obvious flaw is that it constrains the
client to storing all the spam in one bucket. It doesn't allow for (say)
sorting junk by type into different folders (perhaps on different
servers). A message's 'state of spamminess' is an attribute of the
message, properly denoted by keywords that stick to (and with) the
message.

If you want this functionality (and I think it's a useful service for
servers to provide) you need to implement two keywords: $spam and
$nospam (these are just example names). Setting $spam on a message would
instruct the server to pass the message off to the filtering component
as spam; setting $nospam would instruct the server to pass the message
off to the filtering component as valid (non-spam). The two keywords are
exclusive, and it would be up to the server to decide what action to
take if the client tries to light up both at once (e.g. unset the other
flag and take the corresponding action, reject the command, or silently
ignore the request).



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66HnmSm072964 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 10:49:48 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n66Hnm4M072963; Mon, 6 Jul 2009 10:49:48 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from kalyani.oryx.com (kalyani.oryx.com [195.30.37.30]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66HnaXR072932 for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 10:49:47 -0700 (MST) (envelope-from arnt@gulbrandsen.priv.no)
Received: from kalyani.oryx.com (localhost [127.0.0.1]) by kalyani.oryx.com (Postfix) with ESMTP id D25AD2E2F5; Mon,  6 Jul 2009 19:48:06 +0200 (CEST)
Received: from arnt@oryx.com (HELO lochnagar.oryx.com) by kalyani.oryx.com (Archiveopteryx 3.1.0) with esmtp id 1246902486-80172-80171/6/64 (6 recipients); Mon, 6 Jul 2009 19:48:06 +0200
Message-Id: <UP3Anyei29+Pc7cGSi2Sdg.md5@lochnagar.oryx.com>
Date: Mon, 6 Jul 2009 19:50:08 +0200
From: Arnt Gulbrandsen <arnt@gulbrandsen.priv.no>
To: general discussion of application-layer protocols <discuss@ietf.org>, ietf-imapext@imc.org
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
Cc: Ned Freed <ned.freed@mrochek.com>, Iljitsch van Beijnum <iljitsch@muada.com>, George Michaelson <ggm@pobox.com>
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <01NAZUST62HQ001DLH@mauve.mrochek.com>
In-Reply-To: <01NAZUST62HQ001DLH@mauve.mrochek.com>
Content-Type: text/plain; format=flowed
Mime-Version: 1.0
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

Ned Freed writes:
> The obvious way to make this work in IMAP is with a standardized 
> folder annotation saying "messages put in this folder were flagged as 
> spam".

I think what Thunderbird and Apple Mail do is to set specific message 
flags on messages in whatever folders they happen to be in: $Junk if 
the user said a particular message is spam, $autojunk if tb 
autonomously decided that message is spam, $nonjunk and $autononjunk 
for the opposite. (I may be wrong about the exact flag names.)

This was specified in some expired draft, which IMO now has been frozen 
by deployment. Apple Mail and Thunderbird add up to >10% for me. Time 
for an informational RFC, perhaps. "Widely-used IMAP flag names"?

Arnt



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66HhEDH072587 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 10:43:14 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n66HhDT0072586; Mon, 6 Jul 2009 10:43:13 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from mauve.mrochek.com (mauve.mrochek.com [66.59.230.40]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66HhDn1072580 for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 10:43:13 -0700 (MST) (envelope-from ned.freed@mrochek.com)
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NAZVQBTCS0002DWJ@mauve.mrochek.com> for ietf-imapext@imc.org; Mon, 6 Jul 2009 10:43:11 -0700 (PDT)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NAUEIO1ZRK001DLH@mauve.mrochek.com>; Mon, 06 Jul 2009 10:43:09 -0700 (PDT)
Date: Mon, 06 Jul 2009 10:39:40 -0700 (PDT)
From: Ned Freed <ned.freed@mrochek.com>
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
In-reply-to: "Your message dated Mon, 06 Jul 2009 11:27:45 +0100" <4A51D1A1.5040507@isode.com>
To: Alexey Melnikov <alexey.melnikov@isode.com>
Cc: Iljitsch van Beijnum <iljitsch@muada.com>, Dave Cridland <dave@cridland.net>, ietf-imapext@imc.org, general discussion of application-layer protocols <discuss@ietf.org>, George Michaelson <ggm@pobox.com>
Message-id: <01NAZVQAYWH4001DLH@mauve.mrochek.com>
MIME-version: 1.0
Content-type: TEXT/PLAIN; format=flowed
Content-transfer-encoding: 7BIT
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <28941.1246868682.560969@puncture> <4A51D1A1.5040507@isode.com>
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

> Dave Cridland wrote:

> > On Sun Jul  5 20:03:30 2009, Iljitsch van Beijnum wrote:
> >
> >> On 3 jul 2009, at 1:05, George Michaelson wrote:
> >>
> >>> As an example, at the moment if I wish to inform google that I
> >>> have  known spam in local folders, I have to go to the web
> >>> interface and  manually tag. If there was an IMAP extension, I
> >>> could review my  local baysian junk folder, remove all non-spam
> >>> (and flag the senders  as white-listed if need be), and request  the
> >>> rest to be flagged as  spam back on the IMAP backed MS.
> >>
> >> Doesn't moving the spam messages to the spam folder accomplish this
> >>  already? That's what my client does.
> >
> > There's also a proposal (expired?) to use keywords to signal  spamminess.

> To expand on this: I believe the proposal is implemented by several
> major email clients (Thunderbird is one of them).

> [For people not familiar with IMAP: IMAP keywords is a basic IMAP
> feature implemented by majority of IMAP servers I've seen.]

The problem with using keywords for this is that you are now dependence on your
client supporting that particular set of keywords and their associated
semantics. Had we standardize such a keyword set 10 or so years ago this
approach miight have had a chance, but the client base is now too large and
too difficult to update.

This is why I think folder annotations are a better bet. You mark the spam
folder and use that marking to attach meaning to the movement of messages into
and out of that folder. It's not perfect, but it has the key advantage that it
works with any client (OK, any client that can use folders) once the annotation
is in place.

				Ned



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66HGOHH070959 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 10:16:24 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n66HGOq9070958; Mon, 6 Jul 2009 10:16:24 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from mauve.mrochek.com (mauve.mrochek.com [66.59.230.40]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66HGC4s070942 for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 10:16:22 -0700 (MST) (envelope-from ned.freed@mrochek.com)
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NAZUSTSTW000S1PD@mauve.mrochek.com> for ietf-imapext@imc.org; Mon, 6 Jul 2009 10:16:10 -0700 (PDT)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01NAUEIO1ZRK001DLH@mauve.mrochek.com>; Mon, 06 Jul 2009 10:16:09 -0700 (PDT)
Date: Sun, 05 Jul 2009 17:40:31 -0700 (PDT)
From: Ned Freed <ned.freed@mrochek.com>
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
In-reply-to: "Your message dated Sun, 05 Jul 2009 21:03:30 +0200" <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com>
To: Iljitsch van Beijnum <iljitsch@muada.com>
Cc: George Michaelson <ggm@pobox.com>, ietf-imapext@imc.org, discuss@ietf.org
Message-id: <01NAZUST62HQ001DLH@mauve.mrochek.com>
MIME-version: 1.0
Content-type: TEXT/PLAIN; format=flowed; delsp=yes
Content-transfer-encoding: 7BIT
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com>
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

> On 3 jul 2009, at 1:05, George Michaelson wrote:

> > As an example, at the moment if I wish to inform google that I have
> > known spam in local folders, I have to go to the web interface and
> > manually tag. If there was an IMAP extension, I could review my
> > local baysian junk folder, remove all non-spam (and flag the senders
> > as white-listed if need be), and request the rest to be flagged as
> > spam back on the IMAP backed MS.

> Doesn't moving the spam messages to the spam folder accomplish this
> already? That's what my client does.

Please explain how moving a message to a random, arbtirarily named folder tells
the server anything useful. People use all sorts of different naming schemes
for folders, and it's common for clients to provide lots of flexibility in this
area.

The obvious way to make this work in IMAP is with a standardized folder
annotation saying "messages put in this folder were flagged as spam". Then
moves of messages to the folder with this annotation provide the server with
the necessary information that this message is considered to be spam by the
user. And by the same token, moving a message from this folder can be construed
as the message not being considered spam after all.

It isn't perfect, but as long as this is taken as an indicator, not as an
absolute statement, it should get the job done.

> But if you want this, I'd say that it needs to be a fractional thing,
> not a binary spam/no spam indication. For instance, the server could
> give something a spam score of 2 and the client also 2 and together
> that would be 4 so the message is presumed to be spam (assuming the
> spam threshold is 3), but in a binary system no spam OR no spam = no
> spam.

There's a place for fractional scoring in spam filtering, but this isn't
it. A user doesn't think "this message is 90% likely to be spam", they
either think it's spam or it isn't.

				Ned



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66ASkjD042325 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 03:28:46 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n66ASk49042324; Mon, 6 Jul 2009 03:28:46 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from rufus.isode.com (rufus.isode.com [62.3.217.251]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n66ASY2Z042311 for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 03:28:45 -0700 (MST) (envelope-from alexey.melnikov@isode.com)
Received: from [92.40.185.222] (92.40.185.222.sub.mbb.three.co.uk [92.40.185.222])  by rufus.isode.com (submission channel) via TCP with ESMTPA  id <SlHRzwBV9F-C@rufus.isode.com>; Mon, 6 Jul 2009 11:28:33 +0100
Message-ID: <4A51D1A1.5040507@isode.com>
Date: Mon, 06 Jul 2009 11:27:45 +0100
From: Alexey Melnikov <alexey.melnikov@isode.com>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/20050915
X-Accept-Language: en-us, en
To: Iljitsch van Beijnum <iljitsch@muada.com>
CC: Dave Cridland <dave@cridland.net>, ietf-imapext@imc.org, general discussion of application-layer protocols <discuss@ietf.org>, George Michaelson <ggm@pobox.com>
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <28941.1246868682.560969@puncture>
In-Reply-To: <28941.1246868682.560969@puncture>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

Dave Cridland wrote:

> On Sun Jul  5 20:03:30 2009, Iljitsch van Beijnum wrote:
>
>> On 3 jul 2009, at 1:05, George Michaelson wrote:
>>
>>> As an example, at the moment if I wish to inform google that I  
>>> have  known spam in local folders, I have to go to the web  
>>> interface and  manually tag. If there was an IMAP extension, I  
>>> could review my  local baysian junk folder, remove all non-spam  
>>> (and flag the senders  as white-listed if need be), and request  the 
>>> rest to be flagged as  spam back on the IMAP backed MS.
>>
>> Doesn't moving the spam messages to the spam folder accomplish this  
>>  already? That's what my client does.
>
> There's also a proposal (expired?) to use keywords to signal  spamminess.

To expand on this: I believe the proposal is implemented by several 
major email clients (Thunderbird is one of them).

[For people not familiar with IMAP: IMAP keywords is a basic IMAP 
feature implemented by majority of IMAP servers I've seen.]



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n6695u1m036140 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 02:05:56 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n6695uci036139; Mon, 6 Jul 2009 02:05:56 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from peirce.dave.cridland.net (peirce.dave.cridland.net [217.155.137.61]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n6695ssS036133 for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 02:05:55 -0700 (MST) (envelope-from dave@cridland.net)
Received: from puncture ((unknown) [217.155.137.60]) by peirce.dave.cridland.net (submission) via TCP with ESMTPA id <SlG-bQAd5CTE@peirce.dave.cridland.net>; Mon, 6 Jul 2009 10:05:49 +0100
X-SMTP-Protocol-Errors: NORDNS
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <28941.1246868682.560969@puncture> <CEE69A99-B772-414A-9BE6-61C841BEDC29@muada.com>
In-Reply-To: <CEE69A99-B772-414A-9BE6-61C841BEDC29@muada.com>
MIME-Version: 1.0
Message-Id: <28941.1246871141.850024@puncture>
Date: Mon, 06 Jul 2009 10:05:41 +0100
From: Dave Cridland <dave@cridland.net>
To: Iljitsch van Beijnum <iljitsch@muada.com>, <ietf-imapext@imc.org>, general discussion of application-layer protocols <discuss@ietf.org>, George Michaelson <ggm@pobox.com>
Content-Type: text/plain; delsp="yes"; charset="iso-8859-1"; format="flowed"
Content-Transfer-Encoding: quoted-printable
X-Encoded: Changed encoding from 8Bit for 7bit transmission
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

On Mon Jul  6 09:37:06 2009, Iljitsch van Beijnum wrote:
> =

> On 6 jul 2009, at 10:24, Dave Cridland wrote:
> =

>>> But if you want this, I'd say that it needs to be a fractional   =

>>> thing,  not a binary spam/no spam indication. For instance, the   =

>>> server could  give something a spam score of 2 and the client  =

>>> also  2 and together  that would be 4 so the message is presumed  =

>>> to be  spam (assuming the  spam threshold is 3), but in a binary  =

>>> system no  spam OR no spam =3D no  spam.
> =

>> The notion of using two spam filters in concert to attempt to make  =

>>  an overall decision is basically flawed, because it fits into one  =

>> or  other case above - if you do go to the effort of having a  =

>> range  based spamminess from one, you'd have to use it as mere  =

>> input into  the other's decision process, since combining them  =

>> na=EFvely would  produce poorly weighted results.
> =

> Right. The filtering would have to be complimentary. So perhaps  =

> there  should be an indication of the type of filtering done, too.  =

> The reason  to combine server and client side approaches is that  =

> servers can  easily do blacklists, and running any type of  =

> filtering on the server  gives it the opportunity to reject the  =

> message during the SMTP  session, creating a good error message for  =

> real senders but no useless  bounces to innocent third parties in  =

> the case of spoofed senders. On  the client is useful because the  =

> client can typically better afford to  run CPU intensive stuff like  =

> baysian filters and the client probably  has a better list of  =

> previous correspondents that it can use to  whitelist.

Whitelists should, in principle, be easily comminicable to the server  =

(and easily processed).

Bayesian filters on the client are only really useful if the same  =

client is always used - this is often not the case (and, in my  =

experience, is becoming rarer). Otherwise the quality of your spam  =

filtering radically changes depending on the client you happen to be  =

using at the time.

And basically, any filtering at all that can be done on the server  =

saves a vast amount of bandwidth, and makes use of email on smaller  =

devices (like mobile phones) instantly more useful.

The one massive advantage that a client has is direct contact with  =

the user, and the eyeball remains the best spam filtering technology  =

we have.

I'm not suggesting that clients must not do spam filtering -  =

obviously they do, and it's often useful - but I think we should aim  =

to make server-side filtering the best we can. A very simple uniform  =

feedback mechanism of whether the user thinks a message is spam or  =

not makes this very much easier.

Dave.
-- =

Dave Cridland - mailto:dave@cridland.net - xmpp:dwd@dave.cridland.net
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n668cJVi034324 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 01:38:19 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n668cJNr034323; Mon, 6 Jul 2009 01:38:19 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n668c6R0034298 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 01:38:18 -0700 (MST) (envelope-from iljitsch@muada.com)
Received: from claw.it.uc3m.es (claw.it.uc3m.es [163.117.139.55] (may be forged)) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n668bAXQ048131 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Mon, 6 Jul 2009 10:37:11 +0200 (CEST) (envelope-from iljitsch@muada.com)
Cc: <ietf-imapext@imc.org>, general discussion of application-layer protocols <discuss@ietf.org>, George Michaelson <ggm@pobox.com>
Message-Id: <CEE69A99-B772-414A-9BE6-61C841BEDC29@muada.com>
From: Iljitsch van Beijnum <iljitsch@muada.com>
To: Dave Cridland <dave@cridland.net>
In-Reply-To: <28941.1246868682.560969@puncture>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed; delsp=yes
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (Apple Message framework v935.3)
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
Date: Mon, 6 Jul 2009 10:37:06 +0200
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com> <28941.1246868682.560969@puncture>
X-Mailer: Apple Mail (2.935.3)
X-Spam-Status: No, score=-2.6 required=3.4 tests=BAYES_00 autolearn=ham  version=3.0.2
X-Spam-Checker-Version: SpamAssassin 3.0.2 (2004-11-16) on sequoia.muada.com
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

On 6 jul 2009, at 10:24, Dave Cridland wrote:

>> But if you want this, I'd say that it needs to be a fractional =20
>> thing,  not a binary spam/no spam indication. For instance, the =20
>> server could  give something a spam score of 2 and the client also =20=

>> 2 and together  that would be 4 so the message is presumed to be =20
>> spam (assuming the  spam threshold is 3), but in a binary system no =20=

>> spam OR no spam =3D no  spam.

> The notion of using two spam filters in concert to attempt to make =20
> an overall decision is basically flawed, because it fits into one or =20=

> other case above - if you do go to the effort of having a range =20
> based spamminess from one, you'd have to use it as mere input into =20
> the other's decision process, since combining them na=EFvely would =20
> produce poorly weighted results.

Right. The filtering would have to be complimentary. So perhaps there =20=

should be an indication of the type of filtering done, too. The reason =20=

to combine server and client side approaches is that servers can =20
easily do blacklists, and running any type of filtering on the server =20=

gives it the opportunity to reject the message during the SMTP =20
session, creating a good error message for real senders but no useless =20=

bounces to innocent third parties in the case of spoofed senders. On =20
the client is useful because the client can typically better afford to =20=

run CPU intensive stuff like baysian filters and the client probably =20
has a better list of previous correspondents that it can use to =20
whitelist.=



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n668PBfu033253 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Jul 2009 01:25:11 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n668PB3C033252; Mon, 6 Jul 2009 01:25:11 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from peirce.dave.cridland.net (peirce.dave.cridland.net [217.155.137.61]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n668Ow8R033229 for <ietf-imapext@imc.org>; Mon, 6 Jul 2009 01:25:09 -0700 (MST) (envelope-from dave@cridland.net)
Received: from puncture ((unknown) [217.155.137.60]) by peirce.dave.cridland.net (submission) via TCP with ESMTPA id <SlG00gAd5Jez@peirce.dave.cridland.net>; Mon, 6 Jul 2009 09:24:51 +0100
X-SMTP-Protocol-Errors: NORDNS
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com> <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com>
In-Reply-To: <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com>
MIME-Version: 1.0
Message-Id: <28941.1246868682.560969@puncture>
Date: Mon, 06 Jul 2009 09:24:42 +0100
From: Dave Cridland <dave@cridland.net>
To: Iljitsch van Beijnum <iljitsch@muada.com>, <ietf-imapext@imc.org>, general discussion of application-layer protocols <discuss@ietf.org>, George Michaelson <ggm@pobox.com>
Content-Type: text/plain; delsp="yes"; charset="iso-8859-1"; format="flowed"
Content-Transfer-Encoding: quoted-printable
X-Encoded: Changed encoding from 8Bit for 7bit transmission
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

On Sun Jul  5 20:03:30 2009, Iljitsch van Beijnum wrote:
> On 3 jul 2009, at 1:05, George Michaelson wrote:
> =

>> As an example, at the moment if I wish to inform google that I  =

>> have  known spam in local folders, I have to go to the web  =

>> interface and  manually tag. If there was an IMAP extension, I  =

>> could review my  local baysian junk folder, remove all non-spam  =

>> (and flag the senders  as white-listed if need be), and request  =

>> the rest to be flagged as  spam back on the IMAP backed MS.
> =

> Doesn't moving the spam messages to the spam folder accomplish this  =

>  already? That's what my client does.
> =

> =

There's also a proposal (expired?) to use keywords to signal  =

spamminess.

> But if you want this, I'd say that it needs to be a fractional  =

> thing,  not a binary spam/no spam indication. For instance, the  =

> server could  give something a spam score of 2 and the client also  =

> 2 and together  that would be 4 so the message is presumed to be  =

> spam (assuming the  spam threshold is 3), but in a binary system no  =

> spam OR no spam =3D no  spam.

Of course, if both client and server use precisely the same criteria,  =

you've simply halved your threshold.

There's two cases:

1) The server has some feedback-based spam detection mechanism, like  =

a bayesian filter. You want to teach the server's filter about your  =

explicit decisions.

2) Your client has a spam filter (or some sort), you want the server  =

to tell your client about it's preliminary findings.

The notion of using two spam filters in concert to attempt to make an  =

overall decision is basically flawed, because it fits into one or  =

other case above - if you do go to the effort of having a range based  =

spamminess from one, you'd have to use it as mere input into the  =

other's decision process, since combining them na=EFvely would produce  =

poorly weighted results.

A simple keyword approach isn't quite as good as ranges, but it does  =

have the useful property that nearly every service already offers  =

arbitrary keywords, and so is quite likely to already offer it.

Ranges, on the other hand, require the use of annotations, which is a  =

rather more complex area.

Dave.
-- =

Dave Cridland - mailto:dave@cridland.net - xmpp:dwd@dave.cridland.net
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n65J4A4u086920 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 5 Jul 2009 12:04:10 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n65J4AKm086919; Sun, 5 Jul 2009 12:04:10 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n65J3w6a086896 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for <ietf-imapext@imc.org>; Sun, 5 Jul 2009 12:04:09 -0700 (MST) (envelope-from iljitsch@muada.com)
Received: from [192.168.2.4] (static-167-138-7-89.ipcom.comunitel.net [89.7.138.167] (may be forged)) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n65J3Z8C043962 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Sun, 5 Jul 2009 21:03:38 +0200 (CEST) (envelope-from iljitsch@muada.com)
Cc: ietf-imapext@imc.org, discuss@ietf.org
Message-Id: <F9D917EB-A09B-48BB-AA30-F31EADEDD7DD@muada.com>
From: Iljitsch van Beijnum <iljitsch@muada.com>
To: George Michaelson <ggm@pobox.com>
In-Reply-To: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
Content-Transfer-Encoding: 7bit
Mime-Version: 1.0 (Apple Message framework v935.3)
Subject: Re: IMAP extensions needed for SPAM/HAM and WHITE/BLACK listing
Date: Sun, 5 Jul 2009 21:03:30 +0200
References: <371C2EEC-8CBD-4AC1-B07A-928352E19DCB@pobox.com>
X-Mailer: Apple Mail (2.935.3)
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

On 3 jul 2009, at 1:05, George Michaelson wrote:

> As an example, at the moment if I wish to inform google that I have  
> known spam in local folders, I have to go to the web interface and  
> manually tag. If there was an IMAP extension, I could review my  
> local baysian junk folder, remove all non-spam (and flag the senders  
> as white-listed if need be), and request the rest to be flagged as  
> spam back on the IMAP backed MS.

Doesn't moving the spam messages to the spam folder accomplish this  
already? That's what my client does.

But if you want this, I'd say that it needs to be a fractional thing,  
not a binary spam/no spam indication. For instance, the server could  
give something a spam score of 2 and the client also 2 and together  
that would be 4 so the message is presumed to be spam (assuming the  
spam threshold is 3), but in a binary system no spam OR no spam = no  
spam.


