
From stephan@rename-it.nl  Mon Jan 14 04:24:14 2013
Return-Path: <stephan@rename-it.nl>
X-Original-To: sieve@ietfa.amsl.com
Delivered-To: sieve@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EDE5021F861A for <sieve@ietfa.amsl.com>; Mon, 14 Jan 2013 04:24:14 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.504
X-Spam-Level: 
X-Spam-Status: No, score=-0.504 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_NL=0.55, HOST_EQ_NL=1.545]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Zh3VFdtbYG1G for <sieve@ietfa.amsl.com>; Mon, 14 Jan 2013 04:24:14 -0800 (PST)
Received: from drpepper.rename-it.nl (drpepper.rename-it.nl [217.119.238.16]) by ietfa.amsl.com (Postfix) with ESMTP id 2984B21F8609 for <sieve@ietf.org>; Mon, 14 Jan 2013 04:24:12 -0800 (PST)
Received: from ewi1299.ewi.utwente.nl ([130.89.145.113]:52286) by drpepper.rename-it.nl with esmtpsa (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.72) (envelope-from <stephan@rename-it.nl>) id 1Tuj5B-0007tU-NC for sieve@ietf.org; Mon, 14 Jan 2013 13:24:10 +0100
Message-ID: <50F3F8C7.9030804@rename-it.nl>
Date: Mon, 14 Jan 2013 13:23:35 +0100
From: Stephan Bosch <stephan@rename-it.nl>
User-Agent: Mozilla/5.0 (Windows NT 6.0; rv:17.0) Gecko/20130107 Thunderbird/17.0.2
MIME-Version: 1.0
To: Sieve mailing list <sieve@ietf.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-RenameIT-MailScanner-SpamScore: -2.3 (--)
X-RenameIT-MailScanner-SpamCheck: No, score=-2.3 required=5.0 tests=ALL_TRUSTED, BAYES_00 autolearn=ham version=3.3.1
Subject: [sieve] Duplicate test
X-BeenThere: sieve@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: SIEVE Working Group <sieve.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/sieve>, <mailto:sieve-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/sieve>
List-Post: <mailto:sieve@ietf.org>
List-Help: <mailto:sieve-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/sieve>, <mailto:sieve-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 14 Jan 2013 12:24:15 -0000

Hi,

A while back I built a non-standard Sieve extension for detecting 
duplicate message deliveries, e.g. as seen when people cross-post over 
multiple mailing lists. This functionality is very similar to what could 
be achieved with Procmail using formail -D. Someone requested this at 
some point and in a moment of boredom I built the initial 
implementation. Last year I extended it a bit and I made it more 
generic. I never intended for this to be standardized, but - now that I 
look at it again - I think this may yet be of some interest to other 
Sieve implementers as well.

I make a habit of documenting my custom Sieve extensions in a familiar 
RFC-like manner:

http://hg.rename-it.nl/dovecot-2.1-pigeonhole/raw-file/tip/doc/rfc/spec-bosch-sieve-duplicate.txt

What do you think? Would you be interested in having functionality like 
this standardized? Should I submit this as a draft?

Regards,

Stephan.





From NED+mta-filters@mauve.mrochek.com  Tue Jan 15 19:22:24 2013
Return-Path: <NED+mta-filters@mauve.mrochek.com>
X-Original-To: sieve@ietfa.amsl.com
Delivered-To: sieve@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A27AB11E80AD for <sieve@ietfa.amsl.com>; Tue, 15 Jan 2013 19:22:24 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.599
X-Spam-Level: 
X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tbPxRTnEzw+f for <sieve@ietfa.amsl.com>; Tue, 15 Jan 2013 19:22:23 -0800 (PST)
Received: from mauve.mrochek.com (mauve.mrochek.com [66.59.230.40]) by ietfa.amsl.com (Postfix) with ESMTP id BE8B421F8635 for <sieve@ietf.org>; Tue, 15 Jan 2013 19:22:23 -0800 (PST)
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01OP13BZTYG000469A@mauve.mrochek.com> for sieve@ietf.org; Tue, 15 Jan 2013 19:17:21 -0800 (PST)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01OP10N6JLKW00008T@mauve.mrochek.com> (original mail from NED@mauve.mrochek.com) for sieve@ietf.org; Tue, 15 Jan 2013 19:17:19 -0800 (PST)
From: NED+mta-filters@mauve.mrochek.com
Message-id: <01OP13BY38UQ00008T@mauve.mrochek.com>
Date: Tue, 15 Jan 2013 19:10:13 -0800 (PST)
In-reply-to: "Your message dated Mon, 14 Jan 2013 13:23:35 +0100" <50F3F8C7.9030804@rename-it.nl>
MIME-version: 1.0
Content-type: TEXT/PLAIN; Format=flowed
References: <50F3F8C7.9030804@rename-it.nl>
To: Stephan Bosch <stephan@rename-it.nl>
Cc: Sieve mailing list <sieve@ietf.org>
Subject: Re: [sieve] Duplicate test
X-BeenThere: sieve@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: SIEVE Working Group <sieve.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/sieve>, <mailto:sieve-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/sieve>
List-Post: <mailto:sieve@ietf.org>
List-Help: <mailto:sieve-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/sieve>, <mailto:sieve-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 16 Jan 2013 03:22:24 -0000

> Hi,

> A while back I built a non-standard Sieve extension for detecting
> duplicate message deliveries, e.g. as seen when people cross-post over
> multiple mailing lists. This functionality is very similar to what could
> be achieved with Procmail using formail -D. Someone requested this at
> some point and in a moment of boredom I built the initial
> implementation. Last year I extended it a bit and I made it more
> generic. I never intended for this to be standardized, but - now that I
> look at it again - I think this may yet be of some interest to other
> Sieve implementers as well.

> I make a habit of documenting my custom Sieve extensions in a familiar
> RFC-like manner:

> http://hg.rename-it.nl/dovecot-2.1-pigeonhole/raw-file/tip/doc/rfc/spec-bosch-sieve-duplicate.txt

> What do you think? Would you be interested in having functionality like
> this standardized? Should I submit this as a draft?

I think the underlying idea is excellent and deserves to be standardized in one
form or another. I also like the way the optional parameters make the test easy
to use in the default way, but provide the flexibility to test arbitrary string
parameters.

About the only thing that's missing is the ability to do things like count
occurances, and I'm pretty strongly inclined not to add that.

So yes, please submit this and let's work on getting it standardized.

				Ned

From stephan@rename-it.nl  Thu Jan 17 03:53:20 2013
Return-Path: <stephan@rename-it.nl>
X-Original-To: sieve@ietfa.amsl.com
Delivered-To: sieve@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DB08021F892D for <sieve@ietfa.amsl.com>; Thu, 17 Jan 2013 03:53:20 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.504
X-Spam-Level: 
X-Spam-Status: No, score=-0.504 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_NL=0.55, HOST_EQ_NL=1.545]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KXth2bwlsVo6 for <sieve@ietfa.amsl.com>; Thu, 17 Jan 2013 03:53:20 -0800 (PST)
Received: from drpepper.rename-it.nl (drpepper.rename-it.nl [217.119.238.16]) by ietfa.amsl.com (Postfix) with ESMTP id 5A0B021F8910 for <sieve@ietf.org>; Thu, 17 Jan 2013 03:53:18 -0800 (PST)
Received: from ewi1299.ewi.utwente.nl ([130.89.145.113]:56560) by drpepper.rename-it.nl with esmtpsa (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.72) (envelope-from <stephan@rename-it.nl>) id 1Tvo1z-0003QT-CA; Thu, 17 Jan 2013 12:53:17 +0100
Message-ID: <50F7E625.1080509@rename-it.nl>
Date: Thu, 17 Jan 2013 12:53:09 +0100
From: Stephan Bosch <stephan@rename-it.nl>
User-Agent: Mozilla/5.0 (Windows NT 6.0; rv:17.0) Gecko/20130107 Thunderbird/17.0.2
MIME-Version: 1.0
To: Ned Freed <ned.freed@mrochek.com>
References: <50F3F8C7.9030804@rename-it.nl> <01OP13BY38UQ00008T@mauve.mrochek.com>
In-Reply-To: <01OP13BY38UQ00008T@mauve.mrochek.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-RenameIT-MailScanner-SpamScore: -2.3 (--)
X-RenameIT-MailScanner-SpamCheck: No, score=-2.3 required=5.0 tests=ALL_TRUSTED, BAYES_00 autolearn=ham version=3.3.1
Cc: Sieve mailing list <sieve@ietf.org>
Subject: Re: [sieve] Duplicate test
X-BeenThere: sieve@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: SIEVE Working Group <sieve.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/sieve>, <mailto:sieve-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/sieve>
List-Post: <mailto:sieve@ietf.org>
List-Help: <mailto:sieve-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/sieve>, <mailto:sieve-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 17 Jan 2013 11:53:21 -0000

Hi Ned,

On 1/16/2013 4:10 AM, Ned Freed wrote:
>
>> What do you think? Would you be interested in having functionality like
>> this standardized? Should I submit this as a draft?
>
> I think the underlying idea is excellent and deserves to be 
> standardized in one
> form or another. I also like the way the optional parameters make the 
> test easy
> to use in the default way, but provide the flexibility to test 
> arbitrary string
> parameters.
>
> About the only thing that's missing is the ability to do things like 
> count
> occurances, and I'm pretty strongly inclined not to add that.
>
> So yes, please submit this and let's work on getting it standardized.

Ok, I turned it into an Internet draft.

Here is the submission:

https://datatracker.ietf.org/doc/draft-bosch-sieve-duplicate/
http://tools.ietf.org/html/draft-bosch-sieve-duplicate-00

Regards,

Stephan.


From alexey.melnikov@isode.com  Thu Jan 17 07:17:53 2013
Return-Path: <alexey.melnikov@isode.com>
X-Original-To: sieve@ietfa.amsl.com
Delivered-To: sieve@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3005821F8584 for <sieve@ietfa.amsl.com>; Thu, 17 Jan 2013 07:17:53 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -101.925
X-Spam-Level: 
X-Spam-Status: No, score=-101.925 tagged_above=-999 required=5 tests=[AWL=-0.318, BAYES_00=-2.599, DATE_IN_PAST_12_24=0.992, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vNC5YQlOsjQ8 for <sieve@ietfa.amsl.com>; Thu, 17 Jan 2013 07:17:52 -0800 (PST)
Received: from statler.isode.com (statler.isode.com [62.3.217.254]) by ietfa.amsl.com (Postfix) with ESMTP id DCEE121F8586 for <sieve@ietf.org>; Thu, 17 Jan 2013 07:17:51 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1358435868; d=isode.com; s=selector; i=@isode.com; bh=SryxzKkQBlLtY2dmDO/MFiNVngVg8J2ZV3F/ae/A6og=; h=From:Sender:Reply-To:Subject:Date:Message-ID:To:Cc:MIME-Version: In-Reply-To:References:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description; b=xToUKWUajNKi86a7TbAJ9F0pcFnpLIQreuQyuTPeYM775u25PzyNMt9xP8aQloNk1+8v5d YDvYLR8hpnqPZIxcE0dfV9DwYSSSMRhDMkNYFk1OOL983PjiMDevuYaSCixhb2xGCdHOOb Rloepv3yLzT6uFCgTw4uXnDynpTqQ+s=;
Received: from [172.16.1.29] (shiny.isode.com [62.3.217.250])  by statler.isode.com (submission channel) via TCP with ESMTPA  id <UPgWHAAYIVdV@statler.isode.com>; Thu, 17 Jan 2013 15:17:48 +0000
Message-ID: <50F72B63.3080004@isode.com>
Date: Wed, 16 Jan 2013 22:36:19 +0000
From: Alexey Melnikov <alexey.melnikov@isode.com>
User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:13.0) Gecko/20120614 Thunderbird/13.0.1
To: Stephan Bosch <stephan@rename-it.nl>
References: <50F3F8C7.9030804@rename-it.nl>
In-Reply-To: <50F3F8C7.9030804@rename-it.nl>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Cc: Sieve mailing list <sieve@ietf.org>
Subject: Re: [sieve] Duplicate test
X-BeenThere: sieve@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: SIEVE Working Group <sieve.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/sieve>, <mailto:sieve-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/sieve>
List-Post: <mailto:sieve@ietf.org>
List-Help: <mailto:sieve-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/sieve>, <mailto:sieve-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 17 Jan 2013 15:17:53 -0000

On 14/01/2013 12:23, Stephan Bosch wrote:
> Hi,
>
> A while back I built a non-standard Sieve extension for detecting 
> duplicate message deliveries, e.g. as seen when people cross-post over 
> multiple mailing lists. This functionality is very similar to what 
> could be achieved with Procmail using formail -D. Someone requested 
> this at some point and in a moment of boredom I built the initial 
> implementation. Last year I extended it a bit and I made it more 
> generic. I never intended for this to be standardized, but - now that 
> I look at it again - I think this may yet be of some interest to other 
> Sieve implementers as well.
>
> I make a habit of documenting my custom Sieve extensions in a familiar 
> RFC-like manner:
>
> http://hg.rename-it.nl/dovecot-2.1-pigeonhole/raw-file/tip/doc/rfc/spec-bosch-sieve-duplicate.txt 
>
>
> What do you think? Would you be interested in having functionality 
> like this standardized? Should I submit this as a draft?
I think this would be worth standardizing. Some comments on the document:

1). I think the text motivating existence of this extension needs a bit 
of restructuring: it first introduces a very generic concept of matching 
and only later it talks about duplicate suppression. I would reverse 
that: describe some specific use cases first, then generalize.

2). I found that "the side effect of the test takes force at the end of 
a successful script execution" to be odd/awkward. Maybe we can have an 
explicit action? Or maybe leaving this as is is Ok... Need to think a 
bit more about that.

Otherwise this looks quite good to me.

Best Regards,
Alexey


From NED+mta-filters@mauve.mrochek.com  Fri Jan 18 11:41:49 2013
Return-Path: <NED+mta-filters@mauve.mrochek.com>
X-Original-To: sieve@ietfa.amsl.com
Delivered-To: sieve@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D06C021F8790 for <sieve@ietfa.amsl.com>; Fri, 18 Jan 2013 11:41:49 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.599
X-Spam-Level: 
X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1FzhpbSLFEPX for <sieve@ietfa.amsl.com>; Fri, 18 Jan 2013 11:41:49 -0800 (PST)
Received: from mauve.mrochek.com (mauve.mrochek.com [66.59.230.40]) by ietfa.amsl.com (Postfix) with ESMTP id F1F9821F8795 for <sieve@ietf.org>; Fri, 18 Jan 2013 11:41:48 -0800 (PST)
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01OP4U4Z4K34005L9C@mauve.mrochek.com> for sieve@ietf.org; Fri, 18 Jan 2013 11:36:47 -0800 (PST)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01OOJ8U2Q2U800008S@mauve.mrochek.com> (original mail from NED@mauve.mrochek.com) for sieve@ietf.org; Fri, 18 Jan 2013 11:36:45 -0800 (PST)
From: NED+mta-filters@mauve.mrochek.com
Message-id: <01OP4U4XTT0U00008S@mauve.mrochek.com>
Date: Fri, 18 Jan 2013 11:22:30 -0800 (PST)
In-reply-to: "Your message dated Wed, 16 Jan 2013 22:36:19 +0000" <50F72B63.3080004@isode.com>
MIME-version: 1.0
Content-type: TEXT/PLAIN; Format=flowed
References: <50F3F8C7.9030804@rename-it.nl> <50F72B63.3080004@isode.com>
To: Alexey Melnikov <alexey.melnikov@isode.com>
Cc: Sieve mailing list <sieve@ietf.org>
Subject: Re: [sieve] Duplicate test
X-BeenThere: sieve@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: SIEVE Working Group <sieve.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/sieve>, <mailto:sieve-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/sieve>
List-Post: <mailto:sieve@ietf.org>
List-Help: <mailto:sieve-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/sieve>, <mailto:sieve-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 18 Jan 2013 19:41:49 -0000

> 2). I found that "the side effect of the test takes force at the end of
> a successful script execution" to be odd/awkward. Maybe we can have an
> explicit action? Or maybe leaving this as is is Ok... Need to think a
> bit more about that.

I'm not wild about this either, mostly because when you do this as two steps
there's a window where overlapping script executions can cause a duplicate to
be missed. Of course locking can be used to prevent this, but given the scale
we operate at any use of locking has to be carefully considered.

That said, the problem where a script goes wonky and ends up marking something
as a duplicate is much more severe, so IMO the draft is handling this the right
way.

tl;dr: Missing a duplicate is preferable to marking a message as a duplicate
incorrectly.

				Ned

From stephan@rename-it.nl  Sun Jan 20 04:57:44 2013
Return-Path: <stephan@rename-it.nl>
X-Original-To: sieve@ietfa.amsl.com
Delivered-To: sieve@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1231421F84E7 for <sieve@ietfa.amsl.com>; Sun, 20 Jan 2013 04:57:44 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.504
X-Spam-Level: 
X-Spam-Status: No, score=-0.504 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_NL=0.55, HOST_EQ_NL=1.545]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VkuoMHK73kiM for <sieve@ietfa.amsl.com>; Sun, 20 Jan 2013 04:57:43 -0800 (PST)
Received: from drpepper.rename-it.nl (drpepper.rename-it.nl [217.119.238.16]) by ietfa.amsl.com (Postfix) with ESMTP id E648821F84DB for <sieve@ietf.org>; Sun, 20 Jan 2013 04:57:41 -0800 (PST)
Received: from klara.student.utwente.nl ([130.89.162.218]:57543 helo=[10.168.3.2]) by drpepper.rename-it.nl with esmtpsa (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.72) (envelope-from <stephan@rename-it.nl>) id 1TwuSu-0004nh-JC; Sun, 20 Jan 2013 13:57:38 +0100
Message-ID: <50FBE9B0.40209@rename-it.nl>
Date: Sun, 20 Jan 2013 13:57:20 +0100
From: Stephan Bosch <stephan@rename-it.nl>
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130107 Thunderbird/17.0.2
MIME-Version: 1.0
To: Alexey Melnikov <alexey.melnikov@isode.com>
References: <50F3F8C7.9030804@rename-it.nl> <50F72B63.3080004@isode.com>
In-Reply-To: <50F72B63.3080004@isode.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
X-RenameIT-MailScanner-SpamScore: -2.3 (--)
X-RenameIT-MailScanner-SpamCheck: No, score=-2.3 required=5.0 tests=ALL_TRUSTED, BAYES_00 autolearn=ham version=3.3.1
Cc: Sieve mailing list <sieve@ietf.org>
Subject: Re: [sieve] Duplicate test
X-BeenThere: sieve@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: SIEVE Working Group <sieve.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/sieve>, <mailto:sieve-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/sieve>
List-Post: <mailto:sieve@ietf.org>
List-Help: <mailto:sieve-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/sieve>, <mailto:sieve-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sun, 20 Jan 2013 12:57:44 -0000

On 1/16/2013 11:36 PM, Alexey Melnikov wrote:
> On 14/01/2013 12:23, Stephan Bosch wrote:
>> Hi,
>>
>> A while back I built a non-standard Sieve extension for detecting 
>> duplicate message deliveries, e.g. as seen when people cross-post 
>> over multiple mailing lists. This functionality is very similar to 
>> what could be achieved with Procmail using formail -D. Someone 
>> requested this at some point and in a moment of boredom I built the 
>> initial implementation. Last year I extended it a bit and I made it 
>> more generic. I never intended for this to be standardized, but - now 
>> that I look at it again - I think this may yet be of some interest to 
>> other Sieve implementers as well.
>>
>> I make a habit of documenting my custom Sieve extensions in a 
>> familiar RFC-like manner:
>>
>> http://hg.rename-it.nl/dovecot-2.1-pigeonhole/raw-file/tip/doc/rfc/spec-bosch-sieve-duplicate.txt 
>>
>>
>> What do you think? Would you be interested in having functionality 
>> like this standardized? Should I submit this as a draft?
> I think this would be worth standardizing. Some comments on the document:
>
> 1). I think the text motivating existence of this extension needs a 
> bit of restructuring: it first introduces a very generic concept of 
> matching and only later it talks about duplicate suppression. I would 
> reverse that: describe some specific use cases first, then generalize.

Agreed. I'll revise that in the next version.

Regards,

Stephan.

From stephan@rename-it.nl  Sun Jan 20 05:00:57 2013
Return-Path: <stephan@rename-it.nl>
X-Original-To: sieve@ietfa.amsl.com
Delivered-To: sieve@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2FC1221F84E9 for <sieve@ietfa.amsl.com>; Sun, 20 Jan 2013 05:00:57 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.504
X-Spam-Level: 
X-Spam-Status: No, score=-0.504 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_NL=0.55, HOST_EQ_NL=1.545]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id n+-Czm0bj350 for <sieve@ietfa.amsl.com>; Sun, 20 Jan 2013 05:00:56 -0800 (PST)
Received: from drpepper.rename-it.nl (drpepper.rename-it.nl [217.119.238.16]) by ietfa.amsl.com (Postfix) with ESMTP id A936221F84E7 for <sieve@ietf.org>; Sun, 20 Jan 2013 05:00:56 -0800 (PST)
Received: from klara.student.utwente.nl ([130.89.162.218]:57548 helo=[10.168.3.2]) by drpepper.rename-it.nl with esmtpsa (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.72) (envelope-from <stephan@rename-it.nl>) id 1TwuW5-0004rd-8m; Sun, 20 Jan 2013 14:00:54 +0100
Message-ID: <50FBEA74.90603@rename-it.nl>
Date: Sun, 20 Jan 2013 14:00:36 +0100
From: Stephan Bosch <stephan@rename-it.nl>
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130107 Thunderbird/17.0.2
MIME-Version: 1.0
To: Ned Freed <ned.freed@mrochek.com>
References: <50F3F8C7.9030804@rename-it.nl> <50F72B63.3080004@isode.com> <01OP4U4XTT0U00008S@mauve.mrochek.com>
In-Reply-To: <01OP4U4XTT0U00008S@mauve.mrochek.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
X-RenameIT-MailScanner-SpamScore: -2.3 (--)
X-RenameIT-MailScanner-SpamCheck: No, score=-2.3 required=5.0 tests=ALL_TRUSTED, BAYES_00 autolearn=ham version=3.3.1
Cc: Sieve mailing list <sieve@ietf.org>
Subject: Re: [sieve] Duplicate test
X-BeenThere: sieve@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: SIEVE Working Group <sieve.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/sieve>, <mailto:sieve-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/sieve>
List-Post: <mailto:sieve@ietf.org>
List-Help: <mailto:sieve-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/sieve>, <mailto:sieve-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sun, 20 Jan 2013 13:00:57 -0000

On 1/18/2013 8:22 PM, Ned Freed wrote:
>> 2). I found that "the side effect of the test takes force at the end of
>> a successful script execution" to be odd/awkward. Maybe we can have an
>> explicit action? Or maybe leaving this as is is Ok... Need to think a
>> bit more about that.
>
> I'm not wild about this either, mostly because when you do this as two 
> steps
> there's a window where overlapping script executions can cause a 
> duplicate to
> be missed. Of course locking can be used to prevent this, but given 
> the scale
> we operate at any use of locking has to be carefully considered.
>
> That said, the problem where a script goes wonky and ends up marking 
> something
> as a duplicate is much more severe, so IMO the draft is handling this 
> the right
> way.
>
> tl;dr: Missing a duplicate is preferable to marking a message as a 
> duplicate
> incorrectly.

Exactly.

Perhaps I should mention these considerations more explicitly in the draft?

Regards,

Stephan.


From NED+mta-filters@mauve.mrochek.com  Sun Jan 20 14:33:36 2013
Return-Path: <NED+mta-filters@mauve.mrochek.com>
X-Original-To: sieve@ietfa.amsl.com
Delivered-To: sieve@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EDF1D21F8801 for <sieve@ietfa.amsl.com>; Sun, 20 Jan 2013 14:33:36 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: 0
X-Spam-Level: 
X-Spam-Status: No, score=0 tagged_above=-999 required=5 tests=[none]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id niOE4lIN-yQJ for <sieve@ietfa.amsl.com>; Sun, 20 Jan 2013 14:33:36 -0800 (PST)
Received: from mauve.mrochek.com (mauve.mrochek.com [66.59.230.40]) by ietfa.amsl.com (Postfix) with ESMTP id 7B9B621F87FF for <sieve@ietf.org>; Sun, 20 Jan 2013 14:33:33 -0800 (PST)
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01OP7SPKWJTS005T7L@mauve.mrochek.com> for sieve@ietf.org; Sun, 20 Jan 2013 14:28:31 -0800 (PST)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01OOJ8U2Q2U800008S@mauve.mrochek.com> (original mail from NED@mauve.mrochek.com) for sieve@ietf.org; Sun, 20 Jan 2013 14:28:26 -0800 (PST)
From: NED+mta-filters@mauve.mrochek.com
Message-id: <01OP7SPIEZWS00008S@mauve.mrochek.com>
Date: Sun, 20 Jan 2013 14:27:53 -0800 (PST)
In-reply-to: "Your message dated Sun, 20 Jan 2013 14:00:36 +0100" <50FBEA74.90603@rename-it.nl>
MIME-version: 1.0
Content-type: TEXT/PLAIN; format=flowed
References: <50F3F8C7.9030804@rename-it.nl> <50F72B63.3080004@isode.com> <01OP4U4XTT0U00008S@mauve.mrochek.com> <50FBEA74.90603@rename-it.nl>
To: Stephan Bosch <stephan@rename-it.nl>
Cc: Ned Freed <ned.freed@mrochek.com>, Sieve mailing list <sieve@ietf.org>
Subject: Re: [sieve] Duplicate test
X-BeenThere: sieve@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: SIEVE Working Group <sieve.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/sieve>, <mailto:sieve-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/sieve>
List-Post: <mailto:sieve@ietf.org>
List-Help: <mailto:sieve-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/sieve>, <mailto:sieve-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sun, 20 Jan 2013 22:33:37 -0000

> On 1/18/2013 8:22 PM, Ned Freed wrote:
> >> 2). I found that "the side effect of the test takes force at the end of
> >> a successful script execution" to be odd/awkward. Maybe we can have an
> >> explicit action? Or maybe leaving this as is is Ok... Need to think a
> >> bit more about that.
> >
> > I'm not wild about this either, mostly because when you do this as two
> > steps
> > there's a window where overlapping script executions can cause a
> > duplicate to
> > be missed. Of course locking can be used to prevent this, but given
> > the scale
> > we operate at any use of locking has to be carefully considered.
> >
> > That said, the problem where a script goes wonky and ends up marking
> > something
> > as a duplicate is much more severe, so IMO the draft is handling this
> > the right
> > way.
> >
> > tl;dr: Missing a duplicate is preferable to marking a message as a
> > duplicate
> > incorrectly.

> Exactly.

> Perhaps I should mention these considerations more explicitly in the draft?

Yes, I think that's a good idea.

				Ned
