
Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n7RMqGks010821 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 27 Aug 2009 15:52:16 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n7RMqG63010820; Thu, 27 Aug 2009 15:52:16 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from orthanc.ca (orthanc.ca [208.86.224.138]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n7RMqEms010814 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for <ietf-imapext@imc.org>; Thu, 27 Aug 2009 15:52:15 -0700 (MST) (envelope-from lyndon@yyc.orthanc.ca)
Received: from orthanc.ca (localhost [127.0.0.1]) by orthanc.ca (8.14.3/8.14.3) with ESMTP id n7RMqEqW031064 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 27 Aug 2009 16:52:14 -0600 (MDT) (envelope-from lyndon@yyc.orthanc.ca)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=orthanc.ca; s=2009-1; t=1251413534; bh=I54fnTN4YqgGu1TxQkseSwEFvPvPgbCO+ZSwnJ7PFvs=; h=Message-ID:To:Subject:From:Date:In-Reply-To:MIME-Version: Content-Type:Content-Transfer-Encoding; b=XhNZI9ZKU8pglbfJmKyDGKsnt5SOriVwpSPWEB1ZgS/jI5PdXPuDmrNwQrzksb6ZO ANVr5mLNWUsNVyEkS2B/QUKGdyyOI9iYyuZ7CddYwCKtdaaloTc6aai61VJGyt+svr bWJNFI4LiNlsqCZDvkN8g1i7kQI2Q1Kx0QOzWst9OZh5WR4f0G2yDU9VEhrftsj2mW YbzDWFSPDYg1+BFQ8WkrBBpGjZQhm44OFnYZJOKI7hSBOr6fi/15XdQjB/9FKL3LAU RHjlISAsfZiRnrYBSJsW8/ZOP2unQIelq1mo7B1viu/TUu1vePjtpFMx1wQrgUZvr6 on2cK3zIxowxQ==
Received: (from uucp@localhost) by orthanc.ca (8.14.3/8.14.3/Submit) with UUCP id n7RMqEMj031063; Thu, 27 Aug 2009 16:52:14 -0600 (MDT) (envelope-from lyndon@yyc.orthanc.ca)
Received: from gandalf.yyc.orthanc.ca (gandalf.yyc.orthanc.ca [192.168.0.3]) by legolas.yyc.orthanc.ca (8.14.3/8.14.3) with ESMTP id n7RMqAim049524; Thu, 27 Aug 2009 16:52:10 -0600 (MDT) (envelope-from lyndon@yyc.orthanc.ca)
Message-ID: <926163938893b2f2c230a2d5b1acb908@yyc.orthanc.ca>
To: imap-protocol@u.washington.edu, ietf-imapext@imc.org
Subject: Resending David Harris' Keyword Comments
From: Lyndon Nerenberg - VE6BBM/VE7TFX  <lyndon@orthanc.ca>
Organization: The Frobozz Magic Homing Pigeon Company
Date: Thu, 27 Aug 2009 16:52:10 -0600
In-Reply-To: <4A666F2E.25104.17C99307@David.Harris.pmail.gen.nz>
MIME-Version: 1.0
Content-Type: text/plain; charset="US-ASCII"
Content-Transfer-Encoding: 7bit
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

[ I think the first part was truncated in the original message. ]

> First off, I assume we're talking about "keywords" as defined in 
> RFC3501 section 2.3.2 - or at least, I use the word "defined" loosely, 
> because the word "keyword" just sort of appears out of nowhere in that 
> section without any definition at all, and the BNF doesn't amplify the 
> situation much.
> 
> My client is Pegasus Mail - it has had IMAP support for about 14 years 
> now, but that support never been very advanced or ambitious, mostly 
> because in the very early days I ran into so many inconsistencies with 
> server implementations that I never felt it was safe to use anything 
> other than the most basic functions of the protocol (and even then I've 
> had my share of problems). Keywords are one of the things I've always 
> avoided, simply because in the early stages, I found that some servers 
> either did not support client-defined keywords, or defined them in 
> variable ways that made them difficult to use. As a general rule, you 
> can't hang an entire piece of functionality on a feature that might or 
> might not be implemented in the server, so it was best simply to avoid 
> keywords altogether. Instead, I check for specific message headers 
> and allow the user to define others. My client code is very heavily 
> cached, so the overhead of examining the headers is only a one-time 
> hit in most cases, and a hit that has to be done for other reasons in 
> any event.
> 
> [ As an aside, I've just re-read the section of RFC3501 on keywords 
> and find that it's still server-optional whether they can be client-defined, 
> which effectively renders them all but useless, in my view. ]
> 
> Unfortunately, one of the problems with a very long legacy is that 
> decisions made early in the life cycle can be hard to reverse at a later 
> stage. I'm currently engaged in a total rewrite of my message store 
> code: some of the code there is nearly 20 years old, and it's *really* 
> showing its age, so however painful, a total reappraisal and 
> redevelopment using more modern techniques and tools has become 
> unavoidable. As part of the rewrite, my IMAP code is getting rewritten 
> and much more compartmentalized allowing it to be much freer in how 
> it goes about doing a wide range of tasks. I expect this to allow me to 
> use facilities like keywords if it seems appropriate, and to make it 
> much easier to add support for such facilities if it becomes appropriate 
> in future.
> 
>> > I'd also really like to know if the client has "this is spam" or "this 
>> > is not spam" buttons in the UI, and if it does how those buttons 
>> > interact with the use of keywords.
> 
> Pegasus Mail has various spam-fighting facilities, including a regex-
> based content processing engine and a full Bayesian spam filter. 
> Identifying a message as spam is simply a matter of moving it into the 
> spam folder. Similarly, indicating that a message that is in the spam 
> folder is not spam is simply a matter of moving it out of the folder to 
> any other location. There is also a clickable status button that can be 
> used for in-place evaluation and reclassification, although our 
> experience is that most users are more at home with the "move-
> in/move-out" paradigm. None of the built-in anti-spam features has any 
> interaction with any IMAP keyword at any level, although it would be 
> very convenient if there were one I could use reliably.
> 
>> > I ask this because users of any client that has such buttons are clearly 
>> > going to use those buttons preferentially over any other labelling 
>> > mechanism. And if a significant number of clients have such buttons that 
>> > do a particular thing - regardless of what that thing is - that's pretty 
>> > much going to be determinative in finding the most easily deployed 
>> > approach.
> 
> Speaking purely as a client developer (since I develop both client and 
> server IMAP code), if there were a formally-defined keyword that I 
> could reliably expect to find on all server implementations, then I'd 
> definitely use it, but otherwise I won't go near it. It's really a pretty 
> simple equation for me - IMAP is only a part of my message store 
> strategy, and I can't shape the entire way I work on the basis of what it 
> does and does not support (I suspect this may be a fairly common 
> conundrum for client developers, but have no hard evidence of that).



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n7RMia4e010371 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 27 Aug 2009 15:44:36 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n7RMiawJ010370; Thu, 27 Aug 2009 15:44:36 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from orthanc.ca (orthanc.ca [208.86.224.138]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n7RMiTnx010359 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for <ietf-imapext@imc.org>; Thu, 27 Aug 2009 15:44:35 -0700 (MST) (envelope-from lyndon@yyc.orthanc.ca)
Received: from orthanc.ca (localhost [127.0.0.1]) by orthanc.ca (8.14.3/8.14.3) with ESMTP id n7RMiSuF030910 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 27 Aug 2009 16:44:28 -0600 (MDT) (envelope-from lyndon@yyc.orthanc.ca)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=orthanc.ca; s=2009-1; t=1251413069; bh=iNJ8qkUUHKo1mS+Y+RPS9KqLo7nSac+27LUn116Kw8M=; h=Message-ID:To:subject:From:Date:MIME-Version:Content-Type: Content-Transfer-Encoding; b=RBQbjd8ioKZkT1DqfuU+q22PRF/Nd3KaeGbrcYOpZWpeOwkTryJ7jj/YRIMhcYccH mUatLocrW2of9UOipwP/hUvs4jLN2X9Ja+3Ggy4e9WioRwHzUWorV+WlCkDg3G3eLr 8d31oRglheQdVTCU7k3kZ54fJYjWvqXe1EOI7z0lzmPRkXf9/KkDTALCFVIk9WvLMp L0BJ+IC+AK9z5rArVgV0lM7EyPg+fuLKOdXcrsSFIpBC2hhMVwuRyqfYhCvqmi0Uav 3Pw3MNlWKfi2xIv30b/V9MULjQEMjnylY3OA6/SdaxKD8/p7g8KdEu+x5wWGddy9RQ 0SQJtqhiWkkpw==
Received: (from uucp@localhost) by orthanc.ca (8.14.3/8.14.3/Submit) with UUCP id n7RMiSkt030908; Thu, 27 Aug 2009 16:44:28 -0600 (MDT) (envelope-from lyndon@yyc.orthanc.ca)
Received: from gandalf.yyc.orthanc.ca (gandalf.yyc.orthanc.ca [192.168.0.3]) by legolas.yyc.orthanc.ca (8.14.3/8.14.3) with ESMTP id n7RMiO7M049363; Thu, 27 Aug 2009 16:44:24 -0600 (MDT) (envelope-from lyndon@yyc.orthanc.ca)
Message-ID: <e0e7d4217140dffcc2265b5880ba0fe5@yyc.orthanc.ca>
To: imap-protocol@u.washington.edu, ietf-imapext@imc.org
subject: [Redux] Spam-related keyword usage survey
From: Lyndon Nerenberg - VE6BBM/VE7TFX  <lyndon@orthanc.ca>
Organization: The Frobozz Magic Homing Pigeon Company
Date: Thu, 27 Aug 2009 16:44:24 -0600
MIME-Version: 1.0
Content-Type: text/plain; charset="US-ASCII"
Content-Transfer-Encoding: 7bit
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

Oops -- I spoke too soon.  There were valid replies, but my summary
missed them due to a corrupted mail folder screwing up my results
search.  My apologies to the two Davids!  Their responses are included
below (along with Mark's server commentary). 


* Client: Pegasus
* From: David.Harris@pmail.gen.nz

   Unfortunately, one of the problems with a very long legacy is that
   decisions made early in the life cycle can be hard to reverse at a later
   stage.  I'm currently engaged in a total rewrite of my message store code:
   some of the code there is nearly 20 years old, and it's *really* showing
   its age, so however painful, a total reappraisal and redevelopment using
   more modern techniques and tools has become unavoidable.  As part of the
   rewrite, my IMAP code is getting rewritten and much more compartmentalized
   allowing it to be much freer in how it goes about doing a wide range of
   tasks.  I expect this to allow me to use facilities like keywords if it
   seems appropriate, and to make it much easier to add support for such
   facilities if it becomes appropriate in future.

   > > I'd also really like to know if the client has "this is spam" or "this
   > > is not spam" buttons in the UI, and if it does how those buttons > >
   interact with the use of keywords.

   Pegasus Mail has various spam-fighting facilities, including a regex- based
   content processing engine and a full Bayesian spam filter.  Identifying a
   message as spam is simply a matter of moving it into the spam folder.
   Similarly, indicating that a message that is in the spam folder is not spam
   is simply a matter of moving it out of the folder to any other location.
   There is also a clickable status button that can be used for in-place
   evaluation and reclassification, although our experience is that most users
   are more at home with the "move- in/move-out" paradigm.  None of the
   built-in anti-spam features has any interaction with any IMAP keyword at
   any level, although it would be very convenient if there were one I could
   use reliably.

   > > I ask this because users of any client that has such buttons are
   clearly > > going to use those buttons preferentially over any other
   labelling > > mechanism.  And if a significant number of clients have such
   buttons that > > do a particular thing - regardless of what that thing is -
   that's pretty > > much going to be determinative in finding the most easily
   deployed > > approach.

   Speaking purely as a client developer (since I develop both client and
   server IMAP code), if there were a formally-defined keyword that I could
   reliably expect to find on all server implementations, then I'd definitely
   use it, but otherwise I won't go near it.  It's really a pretty simple
   equation for me - IMAP is only a part of my message store strategy, and I
   can't shape the entire way I work on the basis of what it does and does not
   support (I suspect this may be a fairly common conundrum for client
   developers, but have no hard evidence of that).

* Client: Thunderbird
From: bienvenu@davidbienvenu.org

   We store Junk/NonJunk keywords when either the user or the bayesian junk
   filter have determined a message to be spam/not spam.  We treat any message
   with a keyword containing the strings "NonJunk" or "NotJunk" (for mail.app)
   as a spam message, and if we don't find those strings in the keywords, if
   we find "Junk" in any keyword, we treat the message as spam.  So we'll
   treat $Junk as junk, for example.  We do a case-insensitive find on the
   sub-strings.

* Server: M*Guardian
* From: mrcrispin@Panda.COM

   This isn't done by an IMAP client, but it is exposed to IMAP clients by our
   product.

   We use the following keywords in M+Guardian.  We quarantine both incoming
   and outgoing.  The content and attachment quarantines are based upon site
   configurable policy.

   $MPlusSpam	quarantined due to spam

   $MPlusVirus	quarantined due to virus

   $MPlusContent quarantined due to text content

   $MPlusAttach	quarantined due to attachment

   $MPlusOutbound	outbound vs.  inbound message (e.g., you sent
			    something that you aren't allowed to send)



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n7P3CYkY058995 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 24 Aug 2009 20:12:34 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n7P3CYTh058994; Mon, 24 Aug 2009 20:12:34 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from orthanc.ca (orthanc.ca [208.86.224.138]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n7P3CUTu058986 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for <ietf-imapext@imc.org>; Mon, 24 Aug 2009 20:12:31 -0700 (MST) (envelope-from lyndon@orthanc.ca)
Received: from S0106003048db30f7.cg.shawcable.net (S0106003048db30f7.cg.shawcable.net [68.145.186.10]) (authenticated bits=0) by orthanc.ca (8.14.3/8.14.3) with ESMTP id n7P3CQnf098699 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 24 Aug 2009 21:12:28 -0600 (MDT) (envelope-from lyndon@orthanc.ca)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=orthanc.ca; s=2009-1; t=1251169949; bh=UOjxYdXX61yYOKIYEXLR3rMdqzvZwFjnuyxCFibaa4E=; h=Date:From:To:Subject:In-Reply-To:Message-ID:References: MIME-Version:Content-Type; b=JJYpL8qSZwRdMAuwXfQUAY0hq3fxUXRyvKFLSdXxVo2ybHPP4mYan14MMCSWKqfwO mh/iioydLEF2ib9N0mysUdcYPyAnAMaVtXtWBM6blrHe6RQZ98XCUNatL2lTjKRJJZ AFrzVcPtQjfiYEaIYC5EvCyMFpXEyXJrmSXlrU8LsxFjHz8DRJeVL9d7QVAlQghAMA pAzb8ztV9Kv2TzLCPFjp3T9E/y40d7npNyr7eSCi9YbvkN4PycPJ+gxE57syLwKmLA sK8PWstfCn3JLEf6o3BhuTg19j1VPN99ZoB2RFZeH59HCyHuyu3oT0uUyYgkic7pzI w5U0YWHovNSSA==
Date: Mon, 24 Aug 2009 21:12:25 -0600 (MDT)
From: Lyndon Nerenberg <lyndon@orthanc.ca>
X-X-Sender: lyndon@legolas.yyc.orthanc.ca
To: imap-protocol@u.washington.edu, ietf-imapext@imc.org
Subject: Survey Results for Spam-related keyword usage by IMAP clients.
In-Reply-To: <1247001507.8602.109.camel@legolas.orthanc.ca>
Message-ID: <alpine.BSF.2.00.0908242105240.18476@legolas.yyc.orthanc.ca>
References: <1247001507.8602.109.camel@legolas.orthanc.ca>
User-Agent: Alpine 2.00 (BSF 1167 2008-08-23)
Organization: The Frobozz Magic Homing Pigeon Company
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII
X-Spam-Status: No, score=1.0 required=5.0 tests=AWL,BAYES_20,DKIM_SIGNED, DKIM_VERIFIED,RCVD_IN_PBL,RCVD_IN_SORBS_DUL autolearn=no version=3.2.5
X-Spam-Level: *
X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on orthanc.ca
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

I realize this is a bit later than promised, but I was holding out hope 
for (more) responses.

The results were underwhelming. Two unofficial reports of client 
behaviour, and a note from Mark describing how their server products look 
at spam-related keywords.

Apparently SPAM isn't an issue after all.

I'll let Mark elaborate about his stuff if he thinks it's relevant. The 
rest of the info was anecdotal and shouldn't go into the record.

--lyndon



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n7RMqGks010821 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 27 Aug 2009 15:52:16 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n7RMqG63010820; Thu, 27 Aug 2009 15:52:16 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from orthanc.ca (orthanc.ca [208.86.224.138]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n7RMqEms010814 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for <ietf-imapext@imc.org>; Thu, 27 Aug 2009 15:52:15 -0700 (MST) (envelope-from lyndon@yyc.orthanc.ca)
Received: from orthanc.ca (localhost [127.0.0.1]) by orthanc.ca (8.14.3/8.14.3) with ESMTP id n7RMqEqW031064 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 27 Aug 2009 16:52:14 -0600 (MDT) (envelope-from lyndon@yyc.orthanc.ca)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=orthanc.ca; s=2009-1; t=1251413534; bh=I54fnTN4YqgGu1TxQkseSwEFvPvPgbCO+ZSwnJ7PFvs=; h=Message-ID:To:Subject:From:Date:In-Reply-To:MIME-Version: Content-Type:Content-Transfer-Encoding; b=XhNZI9ZKU8pglbfJmKyDGKsnt5SOriVwpSPWEB1ZgS/jI5PdXPuDmrNwQrzksb6ZO ANVr5mLNWUsNVyEkS2B/QUKGdyyOI9iYyuZ7CddYwCKtdaaloTc6aai61VJGyt+svr bWJNFI4LiNlsqCZDvkN8g1i7kQI2Q1Kx0QOzWst9OZh5WR4f0G2yDU9VEhrftsj2mW YbzDWFSPDYg1+BFQ8WkrBBpGjZQhm44OFnYZJOKI7hSBOr6fi/15XdQjB/9FKL3LAU RHjlISAsfZiRnrYBSJsW8/ZOP2unQIelq1mo7B1viu/TUu1vePjtpFMx1wQrgUZvr6 on2cK3zIxowxQ==
Received: (from uucp@localhost) by orthanc.ca (8.14.3/8.14.3/Submit) with UUCP id n7RMqEMj031063; Thu, 27 Aug 2009 16:52:14 -0600 (MDT) (envelope-from lyndon@yyc.orthanc.ca)
Received: from gandalf.yyc.orthanc.ca (gandalf.yyc.orthanc.ca [192.168.0.3]) by legolas.yyc.orthanc.ca (8.14.3/8.14.3) with ESMTP id n7RMqAim049524; Thu, 27 Aug 2009 16:52:10 -0600 (MDT) (envelope-from lyndon@yyc.orthanc.ca)
Message-ID: <926163938893b2f2c230a2d5b1acb908@yyc.orthanc.ca>
To: imap-protocol@u.washington.edu, ietf-imapext@imc.org
Subject: Resending David Harris' Keyword Comments
From: Lyndon Nerenberg - VE6BBM/VE7TFX  <lyndon@orthanc.ca>
Organization: The Frobozz Magic Homing Pigeon Company
Date: Thu, 27 Aug 2009 16:52:10 -0600
In-Reply-To: <4A666F2E.25104.17C99307@David.Harris.pmail.gen.nz>
MIME-Version: 1.0
Content-Type: text/plain; charset="US-ASCII"
Content-Transfer-Encoding: 7bit
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

[ I think the first part was truncated in the original message. ]

> First off, I assume we're talking about "keywords" as defined in 
> RFC3501 section 2.3.2 - or at least, I use the word "defined" loosely, 
> because the word "keyword" just sort of appears out of nowhere in that 
> section without any definition at all, and the BNF doesn't amplify the 
> situation much.
> 
> My client is Pegasus Mail - it has had IMAP support for about 14 years 
> now, but that support never been very advanced or ambitious, mostly 
> because in the very early days I ran into so many inconsistencies with 
> server implementations that I never felt it was safe to use anything 
> other than the most basic functions of the protocol (and even then I've 
> had my share of problems). Keywords are one of the things I've always 
> avoided, simply because in the early stages, I found that some servers 
> either did not support client-defined keywords, or defined them in 
> variable ways that made them difficult to use. As a general rule, you 
> can't hang an entire piece of functionality on a feature that might or 
> might not be implemented in the server, so it was best simply to avoid 
> keywords altogether. Instead, I check for specific message headers 
> and allow the user to define others. My client code is very heavily 
> cached, so the overhead of examining the headers is only a one-time 
> hit in most cases, and a hit that has to be done for other reasons in 
> any event.
> 
> [ As an aside, I've just re-read the section of RFC3501 on keywords 
> and find that it's still server-optional whether they can be client-defined, 
> which effectively renders them all but useless, in my view. ]
> 
> Unfortunately, one of the problems with a very long legacy is that 
> decisions made early in the life cycle can be hard to reverse at a later 
> stage. I'm currently engaged in a total rewrite of my message store 
> code: some of the code there is nearly 20 years old, and it's *really* 
> showing its age, so however painful, a total reappraisal and 
> redevelopment using more modern techniques and tools has become 
> unavoidable. As part of the rewrite, my IMAP code is getting rewritten 
> and much more compartmentalized allowing it to be much freer in how 
> it goes about doing a wide range of tasks. I expect this to allow me to 
> use facilities like keywords if it seems appropriate, and to make it 
> much easier to add support for such facilities if it becomes appropriate 
> in future.
> 
>> > I'd also really like to know if the client has "this is spam" or "this 
>> > is not spam" buttons in the UI, and if it does how those buttons 
>> > interact with the use of keywords.
> 
> Pegasus Mail has various spam-fighting facilities, including a regex-
> based content processing engine and a full Bayesian spam filter. 
> Identifying a message as spam is simply a matter of moving it into the 
> spam folder. Similarly, indicating that a message that is in the spam 
> folder is not spam is simply a matter of moving it out of the folder to 
> any other location. There is also a clickable status button that can be 
> used for in-place evaluation and reclassification, although our 
> experience is that most users are more at home with the "move-
> in/move-out" paradigm. None of the built-in anti-spam features has any 
> interaction with any IMAP keyword at any level, although it would be 
> very convenient if there were one I could use reliably.
> 
>> > I ask this because users of any client that has such buttons are clearly 
>> > going to use those buttons preferentially over any other labelling 
>> > mechanism. And if a significant number of clients have such buttons that 
>> > do a particular thing - regardless of what that thing is - that's pretty 
>> > much going to be determinative in finding the most easily deployed 
>> > approach.
> 
> Speaking purely as a client developer (since I develop both client and 
> server IMAP code), if there were a formally-defined keyword that I 
> could reliably expect to find on all server implementations, then I'd 
> definitely use it, but otherwise I won't go near it. It's really a pretty 
> simple equation for me - IMAP is only a part of my message store 
> strategy, and I can't shape the entire way I work on the basis of what it 
> does and does not support (I suspect this may be a fairly common 
> conundrum for client developers, but have no hard evidence of that).



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n7RMia4e010371 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 27 Aug 2009 15:44:36 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n7RMiawJ010370; Thu, 27 Aug 2009 15:44:36 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from orthanc.ca (orthanc.ca [208.86.224.138]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n7RMiTnx010359 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for <ietf-imapext@imc.org>; Thu, 27 Aug 2009 15:44:35 -0700 (MST) (envelope-from lyndon@yyc.orthanc.ca)
Received: from orthanc.ca (localhost [127.0.0.1]) by orthanc.ca (8.14.3/8.14.3) with ESMTP id n7RMiSuF030910 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 27 Aug 2009 16:44:28 -0600 (MDT) (envelope-from lyndon@yyc.orthanc.ca)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=orthanc.ca; s=2009-1; t=1251413069; bh=iNJ8qkUUHKo1mS+Y+RPS9KqLo7nSac+27LUn116Kw8M=; h=Message-ID:To:subject:From:Date:MIME-Version:Content-Type: Content-Transfer-Encoding; b=RBQbjd8ioKZkT1DqfuU+q22PRF/Nd3KaeGbrcYOpZWpeOwkTryJ7jj/YRIMhcYccH mUatLocrW2of9UOipwP/hUvs4jLN2X9Ja+3Ggy4e9WioRwHzUWorV+WlCkDg3G3eLr 8d31oRglheQdVTCU7k3kZ54fJYjWvqXe1EOI7z0lzmPRkXf9/KkDTALCFVIk9WvLMp L0BJ+IC+AK9z5rArVgV0lM7EyPg+fuLKOdXcrsSFIpBC2hhMVwuRyqfYhCvqmi0Uav 3Pw3MNlWKfi2xIv30b/V9MULjQEMjnylY3OA6/SdaxKD8/p7g8KdEu+x5wWGddy9RQ 0SQJtqhiWkkpw==
Received: (from uucp@localhost) by orthanc.ca (8.14.3/8.14.3/Submit) with UUCP id n7RMiSkt030908; Thu, 27 Aug 2009 16:44:28 -0600 (MDT) (envelope-from lyndon@yyc.orthanc.ca)
Received: from gandalf.yyc.orthanc.ca (gandalf.yyc.orthanc.ca [192.168.0.3]) by legolas.yyc.orthanc.ca (8.14.3/8.14.3) with ESMTP id n7RMiO7M049363; Thu, 27 Aug 2009 16:44:24 -0600 (MDT) (envelope-from lyndon@yyc.orthanc.ca)
Message-ID: <e0e7d4217140dffcc2265b5880ba0fe5@yyc.orthanc.ca>
To: imap-protocol@u.washington.edu, ietf-imapext@imc.org
subject: [Redux] Spam-related keyword usage survey
From: Lyndon Nerenberg - VE6BBM/VE7TFX  <lyndon@orthanc.ca>
Organization: The Frobozz Magic Homing Pigeon Company
Date: Thu, 27 Aug 2009 16:44:24 -0600
MIME-Version: 1.0
Content-Type: text/plain; charset="US-ASCII"
Content-Transfer-Encoding: 7bit
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

Oops -- I spoke too soon.  There were valid replies, but my summary
missed them due to a corrupted mail folder screwing up my results
search.  My apologies to the two Davids!  Their responses are included
below (along with Mark's server commentary). 


* Client: Pegasus
* From: David.Harris@pmail.gen.nz

   Unfortunately, one of the problems with a very long legacy is that
   decisions made early in the life cycle can be hard to reverse at a later
   stage.  I'm currently engaged in a total rewrite of my message store code:
   some of the code there is nearly 20 years old, and it's *really* showing
   its age, so however painful, a total reappraisal and redevelopment using
   more modern techniques and tools has become unavoidable.  As part of the
   rewrite, my IMAP code is getting rewritten and much more compartmentalized
   allowing it to be much freer in how it goes about doing a wide range of
   tasks.  I expect this to allow me to use facilities like keywords if it
   seems appropriate, and to make it much easier to add support for such
   facilities if it becomes appropriate in future.

   > > I'd also really like to know if the client has "this is spam" or "this
   > > is not spam" buttons in the UI, and if it does how those buttons > >
   interact with the use of keywords.

   Pegasus Mail has various spam-fighting facilities, including a regex- based
   content processing engine and a full Bayesian spam filter.  Identifying a
   message as spam is simply a matter of moving it into the spam folder.
   Similarly, indicating that a message that is in the spam folder is not spam
   is simply a matter of moving it out of the folder to any other location.
   There is also a clickable status button that can be used for in-place
   evaluation and reclassification, although our experience is that most users
   are more at home with the "move- in/move-out" paradigm.  None of the
   built-in anti-spam features has any interaction with any IMAP keyword at
   any level, although it would be very convenient if there were one I could
   use reliably.

   > > I ask this because users of any client that has such buttons are
   clearly > > going to use those buttons preferentially over any other
   labelling > > mechanism.  And if a significant number of clients have such
   buttons that > > do a particular thing - regardless of what that thing is -
   that's pretty > > much going to be determinative in finding the most easily
   deployed > > approach.

   Speaking purely as a client developer (since I develop both client and
   server IMAP code), if there were a formally-defined keyword that I could
   reliably expect to find on all server implementations, then I'd definitely
   use it, but otherwise I won't go near it.  It's really a pretty simple
   equation for me - IMAP is only a part of my message store strategy, and I
   can't shape the entire way I work on the basis of what it does and does not
   support (I suspect this may be a fairly common conundrum for client
   developers, but have no hard evidence of that).

* Client: Thunderbird
From: bienvenu@davidbienvenu.org

   We store Junk/NonJunk keywords when either the user or the bayesian junk
   filter have determined a message to be spam/not spam.  We treat any message
   with a keyword containing the strings "NonJunk" or "NotJunk" (for mail.app)
   as a spam message, and if we don't find those strings in the keywords, if
   we find "Junk" in any keyword, we treat the message as spam.  So we'll
   treat $Junk as junk, for example.  We do a case-insensitive find on the
   sub-strings.

* Server: M*Guardian
* From: mrcrispin@Panda.COM

   This isn't done by an IMAP client, but it is exposed to IMAP clients by our
   product.

   We use the following keywords in M+Guardian.  We quarantine both incoming
   and outgoing.  The content and attachment quarantines are based upon site
   configurable policy.

   $MPlusSpam	quarantined due to spam

   $MPlusVirus	quarantined due to virus

   $MPlusContent quarantined due to text content

   $MPlusAttach	quarantined due to attachment

   $MPlusOutbound	outbound vs.  inbound message (e.g., you sent
			    something that you aren't allowed to send)



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n7P3CYkY058995 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 24 Aug 2009 20:12:34 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
Received: (from majordom@localhost) by balder-227.proper.com (8.14.2/8.13.5/Submit) id n7P3CYTh058994; Mon, 24 Aug 2009 20:12:34 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org)
X-Authentication-Warning: balder-227.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f
Received: from orthanc.ca (orthanc.ca [208.86.224.138]) by balder-227.proper.com (8.14.2/8.14.2) with ESMTP id n7P3CUTu058986 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for <ietf-imapext@imc.org>; Mon, 24 Aug 2009 20:12:31 -0700 (MST) (envelope-from lyndon@orthanc.ca)
Received: from S0106003048db30f7.cg.shawcable.net (S0106003048db30f7.cg.shawcable.net [68.145.186.10]) (authenticated bits=0) by orthanc.ca (8.14.3/8.14.3) with ESMTP id n7P3CQnf098699 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 24 Aug 2009 21:12:28 -0600 (MDT) (envelope-from lyndon@orthanc.ca)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=orthanc.ca; s=2009-1; t=1251169949; bh=UOjxYdXX61yYOKIYEXLR3rMdqzvZwFjnuyxCFibaa4E=; h=Date:From:To:Subject:In-Reply-To:Message-ID:References: MIME-Version:Content-Type; b=JJYpL8qSZwRdMAuwXfQUAY0hq3fxUXRyvKFLSdXxVo2ybHPP4mYan14MMCSWKqfwO mh/iioydLEF2ib9N0mysUdcYPyAnAMaVtXtWBM6blrHe6RQZ98XCUNatL2lTjKRJJZ AFrzVcPtQjfiYEaIYC5EvCyMFpXEyXJrmSXlrU8LsxFjHz8DRJeVL9d7QVAlQghAMA pAzb8ztV9Kv2TzLCPFjp3T9E/y40d7npNyr7eSCi9YbvkN4PycPJ+gxE57syLwKmLA sK8PWstfCn3JLEf6o3BhuTg19j1VPN99ZoB2RFZeH59HCyHuyu3oT0uUyYgkic7pzI w5U0YWHovNSSA==
Date: Mon, 24 Aug 2009 21:12:25 -0600 (MDT)
From: Lyndon Nerenberg <lyndon@orthanc.ca>
X-X-Sender: lyndon@legolas.yyc.orthanc.ca
To: imap-protocol@u.washington.edu, ietf-imapext@imc.org
Subject: Survey Results for Spam-related keyword usage by IMAP clients.
In-Reply-To: <1247001507.8602.109.camel@legolas.orthanc.ca>
Message-ID: <alpine.BSF.2.00.0908242105240.18476@legolas.yyc.orthanc.ca>
References: <1247001507.8602.109.camel@legolas.orthanc.ca>
User-Agent: Alpine 2.00 (BSF 1167 2008-08-23)
Organization: The Frobozz Magic Homing Pigeon Company
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII
X-Spam-Status: No, score=1.0 required=5.0 tests=AWL,BAYES_20,DKIM_SIGNED, DKIM_VERIFIED,RCVD_IN_PBL,RCVD_IN_SORBS_DUL autolearn=no version=3.2.5
X-Spam-Level: *
X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on orthanc.ca
Sender: owner-ietf-imapext@mail.imc.org
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-imapext/mail-archive/>
List-ID: <ietf-imapext.imc.org>
List-Unsubscribe: <mailto:ietf-imapext-request@imc.org?body=unsubscribe>

I realize this is a bit later than promised, but I was holding out hope 
for (more) responses.

The results were underwhelming. Two unofficial reports of client 
behaviour, and a note from Mark describing how their server products look 
at spam-related keywords.

Apparently SPAM isn't an issue after all.

I'll let Mark elaborate about his stuff if he thinks it's relevant. The 
rest of the info was anecdotal and shouldn't go into the record.

--lyndon


