Received: from hoffman.proper.com (localhost [127.0.0.1]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q1DDUG6k094020 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 13 Feb 2012 06:30:16 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) Received: (from majordom@localhost) by hoffman.proper.com (8.14.5/8.13.5/Submit) id q1DDUGNh094019; Mon, 13 Feb 2012 06:30:16 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) X-Authentication-Warning: hoffman.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q1DDUFIL094014 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Mon, 13 Feb 2012 06:30:15 -0700 (MST) (envelope-from brong@fastmail.fm) Received: from compute3.internal (compute3.nyi.mail.srv.osa [10.202.2.43]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 2A4D22092A for ; Mon, 13 Feb 2012 08:30:15 -0500 (EST) Received: from web1.nyi.mail.srv.osa ([10.202.2.211]) by compute3.internal (MEProxy); Mon, 13 Feb 2012 08:30:15 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=fastmail.fm; h= message-id:from:to:cc:mime-version:content-transfer-encoding :content-type:in-reply-to:references:subject:date; s=mesmtp; bh= wmyd/hTl2PoiwcFPqUUo4b8FMGg=; b=MmZ2rn5E04MVvST7UTRW0piSdZk6s3IP yuy3Ya+7csQ0r+jwbtP+hFQ9I1Kbch4IomyDjB+mHJG4IjGjcPDJVRzFII79xokM MhjoMgo2Oqf9LUMvSpZ10tlJ7BFcw674AwWnZ2zUxE7HP11RwoVjfjaE2qedyh2X zF3rux3sT7w= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=message-id:from:to:cc:mime-version :content-transfer-encoding:content-type:in-reply-to:references :subject:date; s=smtpout; bh=wmyd/hTl2PoiwcFPqUUo4b8FMGg=; b=IAJ umcTSq2sSCMxylAGa5Y0TijYa2h9E7+u3BEg1IbcSjuI4umVaCZn1TIJoJ0dYQmD jWZWQETKTSibyeNTVQseKeBvQO0+02zW2qdqkk4QHRRCN5+vMu5aDb9WfGmQVy2c z0u1NxNn8TwGaKf+VyPGhxq/SNnl159TaK7w4Ax8= Received: by web1.nyi.mail.srv.osa (Postfix, from userid 99) id 00E1DA000C2; Mon, 13 Feb 2012 08:30:14 -0500 (EST) Message-Id: <1329139814.1318.140661035865773@webmail.messagingengine.com> X-Sasl-Enc: bU3o/mY85q0GCRRsrAK5UTE/nMDvjO5xREezkEDkhx+/ 1329139814 From: Bron Gondwana To: Arnt Gulbrandsen Cc: Timo Sirainen , ietf-imapext@imc.org MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii" X-Mailer: MessagingEngine.com Webmail Interface In-Reply-To: <4F390E77.2060808@gulbrandsen.priv.no> References: <8CAAFEFF-A302-4359-9B59-685795F49B21@iki.fi><4F38D595.4030904@gulbrandsen.priv.no><1329125560.14070.140661035785429@webmail.messagingengine.com><4F38DCB8.4040802@gulbrandsen.priv.no><1329138251.27672.140661035856225@webmail.messagingengine.com> <4F390E77.2060808@gulbrandsen.priv.no> Subject: Re: ANNOTATE-EXPERIMENT-1 Date: Mon, 13 Feb 2012 14:30:14 +0100 Sender: owner-ietf-imapext@mail.imc.org Precedence: bulk List-Archive: List-ID: List-Unsubscribe: On Mon, Feb 13, 2012, at 02:21 PM, Arnt Gulbrandsen wrote: > On 02/13/2012 02:04 PM, Bron Gondwana wrote: > > And of course there's normalisation as well. > > Normalisation doesn't affect meaning at all. It's just changes such as > "o umlaut-above-previous-letter" into "o-with-umlaut" or the other way. > Useful for text processing. Also useful for checking if a particular keyword is already set, and merging the various forms together into a single keyword. Bron. -- Bron Gondwana brong@fastmail.fm Received: from hoffman.proper.com (localhost [127.0.0.1]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q1DDLVPi093474 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 13 Feb 2012 06:21:32 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) Received: (from majordom@localhost) by hoffman.proper.com (8.14.5/8.13.5/Submit) id q1DDLV3d093473; Mon, 13 Feb 2012 06:21:31 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) X-Authentication-Warning: hoffman.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f Received: from strange.aox.org (strange.aox.org [80.244.248.170]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q1DDLURa093464 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Mon, 13 Feb 2012 06:21:31 -0700 (MST) (envelope-from arnt@gulbrandsen.priv.no) Received: from fri.gulbrandsen.priv.no (unknown [127.0.0.1]) by strange.aox.org (Postfix) with ESMTP id 0AD77F8C62D; Mon, 13 Feb 2012 13:21:29 +0000 (UTC) Received: from arnt@gulbrandsen.priv.no by fri.gulbrandsen.priv.no (Archiveopteryx 3.1.4) with esmtpsa id 1329139288-16404-16403/10/12; Mon, 13 Feb 2012 13:21:28 +0000 Message-Id: <4F390E77.2060808@gulbrandsen.priv.no> Date: Mon, 13 Feb 2012 14:21:59 +0100 From: Arnt Gulbrandsen User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:9.0) Gecko/20111229 Thunderbird/9.0 Mime-Version: 1.0 To: Bron Gondwana Cc: Timo Sirainen , ietf-imapext@imc.org Subject: Re: ANNOTATE-EXPERIMENT-1 References: <8CAAFEFF-A302-4359-9B59-685795F49B21@iki.fi> <4F38D595.4030904@gulbrandsen.priv.no> <1329125560.14070.140661035785429@webmail.messagingengine.com> <4F38DCB8.4040802@gulbrandsen.priv.no> <1329138251.27672.140661035856225@webmail.messagingengine.com> In-Reply-To: <1329138251.27672.140661035856225@webmail.messagingengine.com> Content-Type: text/plain; charset=utf-8 Sender: owner-ietf-imapext@mail.imc.org Precedence: bulk List-Archive: List-ID: List-Unsubscribe: On 02/13/2012 02:04 PM, Bron Gondwana wrote: > And of course there's normalisation as well. Normalisation doesn't affect meaning at all. It's just changes such as "o umlaut-above-previous-letter" into "o-with-umlaut" or the other way. Useful for text processing. Arnt Received: from hoffman.proper.com (localhost [127.0.0.1]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q1DD4EXa092736 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 13 Feb 2012 06:04:14 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) Received: (from majordom@localhost) by hoffman.proper.com (8.14.5/8.13.5/Submit) id q1DD4EfU092733; Mon, 13 Feb 2012 06:04:14 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) X-Authentication-Warning: hoffman.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q1DD4B32092727 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Mon, 13 Feb 2012 06:04:12 -0700 (MST) (envelope-from brong@fastmail.fm) Received: from compute5.internal (compute5.nyi.mail.srv.osa [10.202.2.45]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 7A9A9208CA for ; Mon, 13 Feb 2012 08:04:11 -0500 (EST) Received: from web1.nyi.mail.srv.osa ([10.202.2.211]) by compute5.internal (MEProxy); Mon, 13 Feb 2012 08:04:11 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=fastmail.fm; h= message-id:from:to:cc:mime-version:content-transfer-encoding :content-type:in-reply-to:references:subject:date; s=mesmtp; bh= FZaKGGjRyL24Wvdw+vDGnxWkREc=; b=eEkCUnQdJtHGAnkedza8mRTUN/wBiNdK LJeLRnmKyppxYMeF+NAr/8hO0rDLuB2lmwua2WcY4h2n2+uRoBhQcnEYwl5EYP/o BBg0XBVkhjuVYClrZOE86GvVGQ9+00K2sYYEMfRpcEWSC1X0uV3sWW+dnYYUBZfd UqAnbqeLgeg= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=message-id:from:to:cc:mime-version :content-transfer-encoding:content-type:in-reply-to:references :subject:date; s=smtpout; bh=FZaKGGjRyL24Wvdw+vDGnxWkREc=; b=Woy PNR0dd7DkJk4rM2+WD4Dhskdu8p+83UL02JWjwO6YA1gZVxniMtXLRJLMVHag1Jj ZLCO7S8ZdmVhwWRJ5dxSCPLafD46bqcsiKr8nw+gQp3nUt2QHt4NIWS+AqOKfGun 5D4FPyZVJlkOGEweMvKQvKChNvrV1Aq0ax2tFGAg= Received: by web1.nyi.mail.srv.osa (Postfix, from userid 99) id 5363AA000C2; Mon, 13 Feb 2012 08:04:11 -0500 (EST) Message-Id: <1329138251.27672.140661035856225@webmail.messagingengine.com> X-Sasl-Enc: NcsGXtugB+HxBZOJ3A5TlrPcSfTnJ2FRKUdxGynk8c78 1329138251 From: Bron Gondwana To: Arnt Gulbrandsen Cc: Timo Sirainen , ietf-imapext@imc.org MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Mailer: MessagingEngine.com Webmail Interface In-Reply-To: <4F38DCB8.4040802@gulbrandsen.priv.no> References: <8CAAFEFF-A302-4359-9B59-685795F49B21@iki.fi><4F38D595.4030904@gulbrandsen.priv.no><1329125560.14070.140661035785429@webmail.messagingengine.com> <4F38DCB8.4040802@gulbrandsen.priv.no> Subject: Re: ANNOTATE-EXPERIMENT-1 Date: Mon, 13 Feb 2012 14:04:11 +0100 Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by hoffman.proper.com id q1DD4C31092728 Sender: owner-ietf-imapext@mail.imc.org Precedence: bulk List-Archive: List-ID: List-Unsubscribe: On Mon, Feb 13, 2012, at 10:49 AM, Arnt Gulbrandsen wrote: > On 02/13/2012 10:32 AM, Bron Gondwana wrote: > > Keywords are already case insensitive. Which is going to be a pig in poke if we try to make UTF8 keywords. > > Unicode case insensitivity isn't so bad. > > One rule for Turkish, one rule of the rest of the world, and anyone who > can define i to be equal to ı can sidestep the difference. In IMAP's > case, these two commands could be defined to be equivalent: > > a store +flags 1 istanbul > b store +flags 1 ıstanbul > > Not pretty but also not the end of the world. > > (Neat: This spell checker, which I must find out how to turn off btw, > thinks istanbul is wrong and ıstanbul right.) And of course there's normalisation as well. I really don't know as much about this area as I should. I guess the obvious is "require best practice as defined ". > >> But I don't mind dropping that in the least. AFAIK noone has actually > >> wanted to sort by an annotation. > > > > If nobody wants it, it doesn't matter if it's inefficient, right? > > Mostly, but not quite. > > A fetch item that noone ever calls can still require copy and append to > do extra work. True - so long as it's stored rather than calculated on request. > >> Why would store be able to change only some entries? > > > > ACLs is the obvious case. Another thing which the Gmail guys mentioned > > recently is "partial storage failure" - IMAP presents a view in which > > everything is either fully functional or fully broken, but the reality > > under the hood is not always so. > > Good point. > > Still, I think that's an IMAP problem. It's not made worse by covering > all of IMAP, as opposed to all except ANNOTATE. Agree. Bron. -- Bron Gondwana brong@fastmail.fm Received: from hoffman.proper.com (localhost [127.0.0.1]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q1D9nHuh084565 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 13 Feb 2012 02:49:17 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) Received: (from majordom@localhost) by hoffman.proper.com (8.14.5/8.13.5/Submit) id q1D9nHMt084564; Mon, 13 Feb 2012 02:49:17 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) X-Authentication-Warning: hoffman.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f Received: from strange.aox.org (strange.aox.org [80.244.248.170]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q1D9nFuL084559 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Mon, 13 Feb 2012 02:49:17 -0700 (MST) (envelope-from arnt@gulbrandsen.priv.no) Received: from fri.gulbrandsen.priv.no (unknown [127.0.0.1]) by strange.aox.org (Postfix) with ESMTP id 55E6CF8C60A; Mon, 13 Feb 2012 09:49:15 +0000 (UTC) Received: from arnt@gulbrandsen.priv.no by fri.gulbrandsen.priv.no (Archiveopteryx 3.1.4) with esmtpsa id 1329126554-16404-16403/10/5; Mon, 13 Feb 2012 09:49:14 +0000 Message-Id: <4F38DCB8.4040802@gulbrandsen.priv.no> Date: Mon, 13 Feb 2012 10:49:44 +0100 From: Arnt Gulbrandsen User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:9.0) Gecko/20111229 Thunderbird/9.0 Mime-Version: 1.0 To: Bron Gondwana Cc: Timo Sirainen , ietf-imapext@imc.org Subject: Re: ANNOTATE-EXPERIMENT-1 References: <8CAAFEFF-A302-4359-9B59-685795F49B21@iki.fi> <4F38D595.4030904@gulbrandsen.priv.no> <1329125560.14070.140661035785429@webmail.messagingengine.com> In-Reply-To: <1329125560.14070.140661035785429@webmail.messagingengine.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by hoffman.proper.com id q1D9nHuK084560 Sender: owner-ietf-imapext@mail.imc.org Precedence: bulk List-Archive: List-ID: List-Unsubscribe: On 02/13/2012 10:32 AM, Bron Gondwana wrote: > Keywords are already case insensitive. Which is going to be a pig in poke if we try to make UTF8 keywords. Unicode case insensitivity isn't so bad. One rule for Turkish, one rule of the rest of the world, and anyone who can define i to be equal to ı can sidestep the difference. In IMAP's case, these two commands could be defined to be equivalent: a store +flags 1 istanbul b store +flags 1 ıstanbul Not pretty but also not the end of the world. (Neat: This spell checker, which I must find out how to turn off btw, thinks istanbul is wrong and ıstanbul right.) >> But I don't mind dropping that in the least. AFAIK noone has actually >> wanted to sort by an annotation. > > If nobody wants it, it doesn't matter if it's inefficient, right? Mostly, but not quite. A fetch item that noone ever calls can still require copy and append to do extra work. >> Why would store be able to change only some entries? > > ACLs is the obvious case. Another thing which the Gmail guys mentioned > recently is "partial storage failure" - IMAP presents a view in which > everything is either fully functional or fully broken, but the reality > under the hood is not always so. Good point. Still, I think that's an IMAP problem. It's not made worse by covering all of IMAP, as opposed to all except ANNOTATE. Arnt Received: from hoffman.proper.com (localhost [127.0.0.1]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q1D9WgmR084060 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 13 Feb 2012 02:32:42 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) Received: (from majordom@localhost) by hoffman.proper.com (8.14.5/8.13.5/Submit) id q1D9WgWM084059; Mon, 13 Feb 2012 02:32:42 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) X-Authentication-Warning: hoffman.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q1D9WfOL084052 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Mon, 13 Feb 2012 02:32:41 -0700 (MST) (envelope-from brong@fastmail.fm) Received: from compute3.internal (compute3.nyi.mail.srv.osa [10.202.2.43]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 8687820ABF for ; Mon, 13 Feb 2012 04:32:40 -0500 (EST) Received: from web1.nyi.mail.srv.osa ([10.202.2.211]) by compute3.internal (MEProxy); Mon, 13 Feb 2012 04:32:40 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=fastmail.fm; h= message-id:from:to:cc:mime-version:content-transfer-encoding :content-type:references:subject:in-reply-to:date; s=mesmtp; bh= YJC2pT2SjR/gZ7Dwa1Znn5UucfI=; b=bYW5SQ64lgWdb2JW96txIN+ADUHTADKf K/h7uZRpgQZCx4IUWLoU6hdN/d/DBYycKF8go3zjIlUv0qtUO5TlxY7adOjRyyUu qMDKX3wYTWe/lD0h29jzlEN77tf94Ih/nmf6gX1saJmkfPidPZhMm1mlU+UMadqC POv58MhaSLM= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=message-id:from:to:cc:mime-version :content-transfer-encoding:content-type:references:subject :in-reply-to:date; s=smtpout; bh=YJC2pT2SjR/gZ7Dwa1Znn5UucfI=; b= cct2WzOSyBykG9hAca+/JuElXycSRblWExp9sOGeX64SCdXmDTsBpgd1uX3yqHFE GTA691gK95sJ6G+s71EKbYGR8a0SJCaqe07NvewFAOrQJG8DtI+vD8SYxQpAnfwK SUjUwUz2RFCVO+H0HxzIe/InO0FpMxjb8iXYV6/pd1k= Received: by web1.nyi.mail.srv.osa (Postfix, from userid 99) id 62BEEA000E4; Mon, 13 Feb 2012 04:32:40 -0500 (EST) Message-Id: <1329125560.14070.140661035785429@webmail.messagingengine.com> X-Sasl-Enc: HlnWH8bF3d8ZEPT3NiOCxH41g2SzLBSk/HfCoA8nN0kZ 1329125560 From: Bron Gondwana To: Arnt Gulbrandsen , Timo Sirainen Cc: ietf-imapext@imc.org MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii" X-Mailer: MessagingEngine.com Webmail Interface References: <8CAAFEFF-A302-4359-9B59-685795F49B21@iki.fi> <4F38D595.4030904@gulbrandsen.priv.no> Subject: Re: ANNOTATE-EXPERIMENT-1 In-Reply-To: <4F38D595.4030904@gulbrandsen.priv.no> Date: Mon, 13 Feb 2012 10:32:40 +0100 Sender: owner-ietf-imapext@mail.imc.org Precedence: bulk List-Archive: List-ID: List-Unsubscribe: On Mon, Feb 13, 2012, at 10:19 AM, Arnt Gulbrandsen wrote: > > On 02/13/2012 01:45 AM, Timo Sirainen wrote: > > > > I was thinking about finally implementing METADATA + per-mail annotations to Dovecot so I read through the RFCs. There are mainly two things I'd do differently in ANNOTATE-EXPERIMENT-2: > > > > 1. METADATA has case-insensitive entry names, ANNOTATE has case-sensitive. Probably better to keep them case-insensitive in both. > > I wouldn't mind. Keywords are already case insensitive. Which is going to be a pig in poke if we try to make UTF8 keywords. > > 2. SORT (ANNOTATION) - is this really worth the trouble? There's really no good way for servers to implement this in an efficient way. It would pretty much require building and maintaining an index for every single annotation entry name. > > Sorting at read time isn't bad, considering the very low number of > annotations used. > > But I don't mind dropping that in the least. AFAIK noone has actually > wanted to sort by an annotation. If nobody wants it, it doesn't matter if it's inefficient, right? If people do want it, then you can track the annotations they are actually using to sort by and index those... at least that's the theory behind having adaptive indexing. (just being contrary now!) > > And other things I wondered about: > > > > - If APPEND has annotations, and annotation limits are reached, the whole APPEND fails with TOOBIG/TOOMANY error? I guess it has to, although that's a bit annoying. > > Append is transactional. I agree it can be annoying sometimes, such as > when ten-thousand-message multiappends break. But IMAP already has > enough funny little exceptions, please don't try to add a rule such as > "append is transactional except in regard to annotations". Agree - if it's transactional, it's transactional. > > - ANNOTATE doesn't specify what happens if STORE can change only some of the entries. Should it rollback the changes to all the previous entries as well? Including changes to previous messages if multiple messages were specified? Would have been simpler if both METADATA and ANNOTATE supported changing only one annotation at a time. > > My code is transactional, so I don't have the problem. > > Why would store be able to change only some entries? ACLs is the obvious case. Another thing which the Gmail guys mentioned recently is "partial storage failure" - IMAP presents a view in which everything is either fully functional or fully broken, but the reality under the hood is not always so. More simply, could just be permission problems on a file somewhere, or partial database corruption. Shit happens. Bron. -- Bron Gondwana brong@fastmail.fm Received: from hoffman.proper.com (localhost [127.0.0.1]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q1D9IqoY083722 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 13 Feb 2012 02:18:52 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) Received: (from majordom@localhost) by hoffman.proper.com (8.14.5/8.13.5/Submit) id q1D9Iqst083721; Mon, 13 Feb 2012 02:18:52 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) X-Authentication-Warning: hoffman.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f Received: from strange.aox.org (strange.aox.org [80.244.248.170]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q1D9Indp083712 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Mon, 13 Feb 2012 02:18:51 -0700 (MST) (envelope-from arnt@gulbrandsen.priv.no) Received: from fri.gulbrandsen.priv.no (unknown [127.0.0.1]) by strange.aox.org (Postfix) with ESMTP id 53753F8C571; Mon, 13 Feb 2012 09:18:48 +0000 (UTC) Received: from arnt@gulbrandsen.priv.no by fri.gulbrandsen.priv.no (Archiveopteryx 3.1.4) with esmtpsa id 1329124726-16404-16403/10/3; Mon, 13 Feb 2012 09:18:46 +0000 Message-Id: <4F38D595.4030904@gulbrandsen.priv.no> Date: Mon, 13 Feb 2012 10:19:17 +0100 From: Arnt Gulbrandsen User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:9.0) Gecko/20111229 Thunderbird/9.0 Mime-Version: 1.0 To: Timo Sirainen Cc: ietf-imapext@imc.org Subject: Re: ANNOTATE-EXPERIMENT-1 References: <8CAAFEFF-A302-4359-9B59-685795F49B21@iki.fi> In-Reply-To: <8CAAFEFF-A302-4359-9B59-685795F49B21@iki.fi> Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by hoffman.proper.com id q1D9Ipdo083717 Sender: owner-ietf-imapext@mail.imc.org Precedence: bulk List-Archive: List-ID: List-Unsubscribe: On 02/13/2012 01:45 AM, Timo Sirainen wrote: > > I was thinking about finally implementing METADATA + per-mail annotations to Dovecot so I read through the RFCs. There are mainly two things I'd do differently in ANNOTATE-EXPERIMENT-2: > > 1. METADATA has case-insensitive entry names, ANNOTATE has case-sensitive. Probably better to keep them case-insensitive in both. I wouldn't mind. > 2. SORT (ANNOTATION) - is this really worth the trouble? There's really no good way for servers to implement this in an efficient way. It would pretty much require building and maintaining an index for every single annotation entry name. Sorting at read time isn't bad, considering the very low number of annotations used. But I don't mind dropping that in the least. AFAIK noone has actually wanted to sort by an annotation. > And other things I wondered about: > > - If APPEND has annotations, and annotation limits are reached, the whole APPEND fails with TOOBIG/TOOMANY error? I guess it has to, although that's a bit annoying. Append is transactional. I agree it can be annoying sometimes, such as when ten-thousand-message multiappends break. But IMAP already has enough funny little exceptions, please don't try to add a rule such as "append is transactional except in regard to annotations". > - ANNOTATE doesn't specify what happens if STORE can change only some of the entries. Should it rollback the changes to all the previous entries as well? Including changes to previous messages if multiple messages were specified? Would have been simpler if both METADATA and ANNOTATE supported changing only one annotation at a time. My code is transactional, so I don't have the problem. Why would store be able to change only some entries? > - TOOBIG/TOOMANY doesn't map very nicely if I want to have global "max. annotation size" and "max. annotation count". Perhaps a global max. count shouldn't exist, since the number of mails/mailboxes could be limited instead, but a global size similar to mail quota should exist. I guess when that global size is reached it just fails with TOOBIG until some annotations are deleted. Yes. Arnt Received: from hoffman.proper.com (localhost [127.0.0.1]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q1D0jeAk070021 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 12 Feb 2012 17:45:40 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) Received: (from majordom@localhost) by hoffman.proper.com (8.14.5/8.13.5/Submit) id q1D0je2x070020; Sun, 12 Feb 2012 17:45:40 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) X-Authentication-Warning: hoffman.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f Received: from dovecot.org (dovecot.org [193.210.130.67]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q1D0jdji070015 for ; Sun, 12 Feb 2012 17:45:40 -0700 (MST) (envelope-from tss@iki.fi) Received: from [192.168.10.101] (a88-112-255-76.elisa-laajakaista.fi [88.112.255.76]) by dovecot.org (Postfix) with ESMTP id 0ACFA1AE876C for ; Mon, 13 Feb 2012 02:45:33 +0200 (EET) From: Timo Sirainen Content-Type: text/plain; charset=us-ascii Subject: ANNOTATE-EXPERIMENT-1 Date: Mon, 13 Feb 2012 02:45:32 +0200 Message-Id: <8CAAFEFF-A302-4359-9B59-685795F49B21@iki.fi> To: ietf-imapext@imc.org Mime-Version: 1.0 (Apple Message framework v1084) X-Mailer: Apple Mail (2.1084) Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by hoffman.proper.com id q1D0jeji070016 Sender: owner-ietf-imapext@mail.imc.org Precedence: bulk List-Archive: List-ID: List-Unsubscribe: I was thinking about finally implementing METADATA + per-mail annotations to Dovecot so I read through the RFCs. There are mainly two things I'd do differently in ANNOTATE-EXPERIMENT-2: 1. METADATA has case-insensitive entry names, ANNOTATE has case-sensitive. Probably better to keep them case-insensitive in both. 2. SORT (ANNOTATION) - is this really worth the trouble? There's really no good way for servers to implement this in an efficient way. It would pretty much require building and maintaining an index for every single annotation entry name. And other things I wondered about: - If APPEND has annotations, and annotation limits are reached, the whole APPEND fails with TOOBIG/TOOMANY error? I guess it has to, although that's a bit annoying. - ANNOTATE doesn't specify what happens if STORE can change only some of the entries. Should it rollback the changes to all the previous entries as well? Including changes to previous messages if multiple messages were specified? Would have been simpler if both METADATA and ANNOTATE supported changing only one annotation at a time. - TOOBIG/TOOMANY doesn't map very nicely if I want to have global "max. annotation size" and "max. annotation count". Perhaps a global max. count shouldn't exist, since the number of mails/mailboxes could be limited instead, but a global size similar to mail quota should exist. I guess when that global size is reached it just fails with TOOBIG until some annotations are deleted. Received: from hoffman.proper.com (localhost [127.0.0.1]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q1AEj7jI043094 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 10 Feb 2012 07:45:07 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) Received: (from majordom@localhost) by hoffman.proper.com (8.14.5/8.13.5/Submit) id q1AEj7S3043093; Fri, 10 Feb 2012 07:45:07 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) X-Authentication-Warning: hoffman.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f Received: from koch.ro (koch.ro [88.198.2.104]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q1AEj4aU043088 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NO) for ; Fri, 10 Feb 2012 07:45:05 -0700 (MST) (envelope-from thomas@koch.ro) Received: from aktaia.intevation.org ([212.95.126.10] helo=t61.localnet) by koch.ro with esmtpsa (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.72) (envelope-from ) id 1RvriZ-0007SW-3v; Fri, 10 Feb 2012 15:44:55 +0100 From: Thomas Koch Reply-To: thomas@koch.ro To: Adrien de Croy Subject: Re: [imap5] Designing a new replacement protocol for IMAP Date: Fri, 10 Feb 2012 15:44:50 +0100 User-Agent: KMail/1.13.7 (Linux/3.1.0-1-amd64; KDE/4.6.5; x86_64; ; ) Cc: imap5@ietf.org, Bron Gondwana , IMAP Protocol Interest List , ietf-imapext@imc.org References: <1328732126.32086.140661033971485@webmail.messagingengine.com> <201202090820.28260.thomas@koch.ro> <4F337E61.5040702@qbik.com> In-Reply-To: <4F337E61.5040702@qbik.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Message-Id: <201202101544.51364.thomas@koch.ro> Sender: owner-ietf-imapext@mail.imc.org Precedence: bulk List-Archive: List-ID: List-Unsubscribe: Hi Adrien, thank you for your insights. I've some more questions inline. Adrien de Croy: > * updates. Would need to poll or use long polling to get real-time > updates (e.g. notification of incoming mail). Could Websockets or http://en.wikipedia.org/wiki/Comet_(programming) be of help here? > * complex. In order to reduce round-trips, you'd have to do many things > in a single transaction, which would create complexity issues in the > server and client I was thinking restful HTTP which by definition is not complex. The question is, whether a useful mail fetch protocol could be designed in the constraints of rest. > I think it would also be very problematic in the wild, since any http > intermediary would feel entitled to mess with traffic. E.g. heuristic > caching, intercepting proxies etc. There is an HTTP header to instruct any cache to pipe the request through and do no caching. But anybody on any line in any protocol could mess with your data. Have to go now, sorry. :-) Best regards, Thomas Koch, http://www.koch.ro Received: from hoffman.proper.com (localhost [127.0.0.1]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q19BWB5f081688 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 9 Feb 2012 04:32:11 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) Received: (from majordom@localhost) by hoffman.proper.com (8.14.5/8.13.5/Submit) id q19BWBKv081687; Thu, 9 Feb 2012 04:32:11 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) X-Authentication-Warning: hoffman.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q19BWABW081682 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Thu, 9 Feb 2012 04:32:10 -0700 (MST) (envelope-from brong@fastmail.fm) Received: from compute5.internal (compute5.nyi.mail.srv.osa [10.202.2.45]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 075C0209D5 for ; Thu, 9 Feb 2012 06:32:10 -0500 (EST) Received: from frontend1.nyi.mail.srv.osa ([10.202.2.160]) by compute5.internal (MEProxy); Thu, 09 Feb 2012 06:32:10 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=fastmail.fm; h= date:from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=mesmtp; bh=wPCn9RtD+I2RGzRY5ggh/VUe mtU=; b=Gbr3CvUxKMNaThkEvyic5Oq5+9OcqBQcSlwN37QLthMcsBeG3p+iPHTk QJDgK6n8ioJ5n80LOntvgVBKDbqkvkXFWmFSPsKW2fk0GOqU78Jnl1XgH9MW3biV 2zPn2VFN9PZ1SbmGXPFjOOJLMHAZNJ7gFPR5OZdhaSJnPwO73ec= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=date:from:to:cc:subject:message-id :references:mime-version:content-type:in-reply-to; s=smtpout; bh=wPCn9RtD+I2RGzRY5ggh/VUemtU=; b=AmuL2A2HKAkyqSjv4v9gqaDqkqI/ knnqmOnIzrzbmHyd7JVeks6wdkL73KodYdunG43A+Gmin2r0BO84+ozqfmu6Y1D8 KiC+VOrF/29YminiWVyu6P/o6vW8w10NxsE1LhFaZmq6YuO7l5ISsL6fZp2xjxkz y2fasYKE3tk+Ms0= X-Sasl-enc: t+LLM2k4dsiT7QwkYEiqOLrfMqwb1O0wfKAnksCsqsBx 1328787129 Received: from localhost (99.249.9.46.customer.cdi.no [46.9.249.99]) by mail.messagingengine.com (Postfix) with ESMTPSA id AF14D8E0085; Thu, 9 Feb 2012 06:32:09 -0500 (EST) Received: by localhost (Postfix, from userid 1000) id 380351C6FBC; Thu, 9 Feb 2012 12:32:08 +0100 (CET) Date: Thu, 9 Feb 2012 12:32:08 +0100 From: Bron Gondwana To: Giovanni Panozzo Cc: imap5@ietf.org, ietf-imapext@imc.org, imap-protocol@u.washington.edu Subject: Re: [imap5] Designing a new replacement protocol for IMAP Message-ID: <20120209113208.GA31355@launde.brong.net> References: <1328732126.32086.140661033971485@webmail.messagingengine.com> <201202090820.28260.thomas@koch.ro> <4F33AD2D.7050904@panozzo.it> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4F33AD2D.7050904@panozzo.it> Organization: brong.net User-Agent: Mutt/1.5.21 (2010-09-15) Sender: owner-ietf-imapext@mail.imc.org Precedence: bulk List-Archive: List-ID: List-Unsubscribe: On Thu, Feb 09, 2012 at 12:25:33PM +0100, Giovanni Panozzo wrote: > PS: I think that crossposting in 3 ML is too much. Which one will be > the official ML for this discussion ? (posting to all three for this...) Let's take the rest of the traffic to just imap5@ietf.org Bron. Received: from hoffman.proper.com (localhost [127.0.0.1]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q19BOGZk081318 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 9 Feb 2012 04:24:16 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) Received: (from majordom@localhost) by hoffman.proper.com (8.14.5/8.13.5/Submit) id q19BOGDa081317; Thu, 9 Feb 2012 04:24:16 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) X-Authentication-Warning: hoffman.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q19BODSG081311 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Thu, 9 Feb 2012 04:24:14 -0700 (MST) (envelope-from brong@fastmail.fm) Received: from compute6.internal (compute6.nyi.mail.srv.osa [10.202.2.46]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 5AA852108E for ; Thu, 9 Feb 2012 06:24:13 -0500 (EST) Received: from frontend2.nyi.mail.srv.osa ([10.202.2.161]) by compute6.internal (MEProxy); Thu, 09 Feb 2012 06:24:13 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=fastmail.fm; h= date:from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=mesmtp; bh=eyPXuJRWfA3AXuiX2tZFA2H0 IS0=; b=bUbkW2yF0fyEFxTkMD/plKdAWJ2qG6A4e84VHJaStPeYC9hMWNQOhSs7 0DM6BDPU3BqoSlX7Dt2R/nnveC3/OuvHB7IFjxDLI9XcA5cCdgIvb/z3ilRX437t CqqaQirhtWqoR27sY1nq35CLs2d9zy7sFVnJPMmepQlY4iSRLgo= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=date:from:to:cc:subject:message-id :references:mime-version:content-type:in-reply-to; s=smtpout; bh=eyPXuJRWfA3AXuiX2tZFA2H0IS0=; b=W+Y72hA1zXC5iq/s3oZjJfy4gyaJ Sl0F3ZfeHDTb3uXhX3Yz14grU5MBuGlN6TsoB5N0slVzpdmk3WH6KAY2ve+BEJ9G 55sBIszGJji853jHfyzwQAmZv5elgOVW17rMfDLLKSA0szaez5aSFFKfFUBsvn4J K5jed5OPxBfmyPc= X-Sasl-enc: DRLqcQnkDZ/s9vPvcFYXo6cyO3ntFszOGgVzj6jfL1Pj 1328786653 Received: from localhost (99.249.9.46.customer.cdi.no [46.9.249.99]) by mail.messagingengine.com (Postfix) with ESMTPSA id 07DA9483525; Thu, 9 Feb 2012 06:24:13 -0500 (EST) Received: by localhost (Postfix, from userid 1000) id 4D35819BF04; Thu, 9 Feb 2012 12:24:11 +0100 (CET) Date: Thu, 9 Feb 2012 12:24:11 +0100 From: Bron Gondwana To: Adrien de Croy Cc: thomas@koch.ro, imap5@ietf.org, Bron Gondwana , IMAP Protocol Interest List , ietf-imapext@imc.org Subject: Re: [imap5] Designing a new replacement protocol for IMAP Message-ID: <20120209112411.GB29734@launde.brong.net> References: <1328732126.32086.140661033971485@webmail.messagingengine.com> <201202090820.28260.thomas@koch.ro> <4F337E61.5040702@qbik.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4F337E61.5040702@qbik.com> Organization: brong.net User-Agent: Mutt/1.5.21 (2010-09-15) Sender: owner-ietf-imapext@mail.imc.org Precedence: bulk List-Archive: List-ID: List-Unsubscribe: On Thu, Feb 09, 2012 at 09:05:53PM +1300, Adrien de Croy wrote: > > Whilst HTTP could certainly be used to put together an email system, > I don't think it would perform well enough to replace existing > protocols on desktops. > > The main issues I see are: > > * updates. Would need to poll or use long polling to get real-time > updates (e.g. notification of incoming mail). Yes - long polling or "out of band" notification systems. We have an HTTP protocol defined kind of "on top of" IMAP plus our own extensions here at FastMail/Opera (mail.opera.com if you want to play with it) We are currently using EventSource to notify the client that "there is something new", after which the client queries to find what's updated. This is possible to be efficient because we use a single counter for uidvalidity and a single counter for highestmodseq across the entire user's folders - so any change to uidvalidity means the folder list needs refreshing, and any change to highestmodseq means the message view needs checking. Combined with efficient "what's changed in this view since my previous modseq value" queries, it's very bandwidth efficient. > * slow. Gut feel based on 17 years writing http proxies is that it > would be too slow Not really actually - so long as you can query multiple things in a single "transaction" > * complex. In order to reduce round-trips, you'd have to do many > things in a single transaction, which would create complexity issues > in the server and client But here's the rub. Yes, complex. > * protocol bloat. if you think IMAP is bloated, try HTTP. > * fundamentally an off-line style protocol for mail, which has > strong incentives to be online (esp for in-office use). Mostly facebook's "live feed" is plenty fast enough. People are doing all sorts of close-enough-to-real-time stuff on top of HTTP. It doesn't mean it's the right choice, but I wouldn't dismiss it out of hand either. > I think it would also be very problematic in the wild, since any > http intermediary would feel entitled to mess with traffic. E.g. > heuristic caching, intercepting proxies etc. That's a problem for sure. Mind you, proxyability is high on my list of goals. Immutable stuff should be cachable. > It's almost too ubiquitous. Everyone and their dog thinks they know > how http works. It's not strictly-enough defined, and has a LOT of > optional features which aren't relevant for mail (e.g. content > negotiation). Interop in http is already problematic, but start > getting people writing mail servers in php etc, and see what starts > to happen. I think it would be a nightmare. Like IMAP then, except only their dogs actually understand it ;) Good enough tests should, in theory, allow people to write their servers in PHP if they want to. Why you would choose to write in PHP, I'm not sure... but still. It takes all types. Bron. Received: from hoffman.proper.com (localhost [127.0.0.1]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q1985v1a073478 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 9 Feb 2012 01:05:57 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) Received: (from majordom@localhost) by hoffman.proper.com (8.14.5/8.13.5/Submit) id q1985v6B073477; Thu, 9 Feb 2012 01:05:57 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) X-Authentication-Warning: hoffman.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f Received: from smtp.qbik.com (smtp.qbik.com [210.55.214.35]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q1985tMY073472 for ; Thu, 9 Feb 2012 01:05:56 -0700 (MST) (envelope-from adrien@qbik.com) Received: From [192.168.1.10] (unverified [219.89.217.118]) by SMTP Server [210.55.214.35] (WinGate SMTP Receiver v7.1.0 (Build 3379)) with SMTP id <0018854642@smtp.qbik.com>; Thu, 09 Feb 2012 21:05:53 +1300 Message-ID: <4F337E61.5040702@qbik.com> Date: Thu, 09 Feb 2012 21:05:53 +1300 From: Adrien de Croy User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:11.0) Gecko/20120202 Thunderbird/11.0 MIME-Version: 1.0 To: thomas@koch.ro CC: imap5@ietf.org, Bron Gondwana , IMAP Protocol Interest List , ietf-imapext@imc.org Subject: Re: [imap5] Designing a new replacement protocol for IMAP References: <1328732126.32086.140661033971485@webmail.messagingengine.com> <201202090820.28260.thomas@koch.ro> In-Reply-To: <201202090820.28260.thomas@koch.ro> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-ietf-imapext@mail.imc.org Precedence: bulk List-Archive: List-ID: List-Unsubscribe: Whilst HTTP could certainly be used to put together an email system, I don't think it would perform well enough to replace existing protocols on desktops. The main issues I see are: * updates. Would need to poll or use long polling to get real-time updates (e.g. notification of incoming mail). * slow. Gut feel based on 17 years writing http proxies is that it would be too slow * complex. In order to reduce round-trips, you'd have to do many things in a single transaction, which would create complexity issues in the server and client * protocol bloat. if you think IMAP is bloated, try HTTP. * fundamentally an off-line style protocol for mail, which has strong incentives to be online (esp for in-office use). I think it would also be very problematic in the wild, since any http intermediary would feel entitled to mess with traffic. E.g. heuristic caching, intercepting proxies etc. It's almost too ubiquitous. Everyone and their dog thinks they know how http works. It's not strictly-enough defined, and has a LOT of optional features which aren't relevant for mail (e.g. content negotiation). Interop in http is already problematic, but start getting people writing mail servers in php etc, and see what starts to happen. I think it would be a nightmare. Adrien On 9/02/2012 8:20 p.m., Thomas Koch wrote: > Bron Gondwana: >> Snide comments from those who don't agree with the goals are welcome too... >> I just won't listen to you. These goals are very important to me. > Hi Bron, > > I'm not an expert in IMAP, just got here out of curiosity about Kolab[1]'s use > of IMAP folders as data stores. > I found some older initiatives to replace IMAP with an HTTP based approach.[2] > What do you think of HTTP for mail access? For my bachelor thesis[3] I > currently argue that CardDAV/CalDAV could be perfectly replaced by AtomPub > (RFC 5023). > The main advantage would be that most software developers on this planet have > some vague understanding of HTTP and that imense infrastructure and software > already exists. > > [1] http://wiki.kolab.org/Why_IMAP_Is_Used_For_Storage > [2] http://www.prescod.net/rest/restmail/ > https://www.ietf.org/mailman/listinfo/httpmail > [3] https://github.com/thkoch2001/bachelor-thesis > > Best regards, > > Thomas Koch, http://www.koch.ro > -- Adrien de Croy - WinGate Proxy Server - http://www.wingate.com Received: from hoffman.proper.com (localhost [127.0.0.1]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q197KllW072460 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 9 Feb 2012 00:20:47 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) Received: (from majordom@localhost) by hoffman.proper.com (8.14.5/8.13.5/Submit) id q197KlQ4072459; Thu, 9 Feb 2012 00:20:47 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) X-Authentication-Warning: hoffman.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f Received: from koch.ro (koch.ro [88.198.2.104]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q197KhB6072448 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NO) for ; Thu, 9 Feb 2012 00:20:44 -0700 (MST) (envelope-from thomas@koch.ro) Received: from 43-172.77-83.cust.bluewin.ch ([83.77.172.43] helo=t61.localnet) by koch.ro with esmtpsa (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.72) (envelope-from ) id 1RvOJ3-0002cX-0q; Thu, 09 Feb 2012 08:20:37 +0100 From: Thomas Koch Reply-To: thomas@koch.ro To: imap5@ietf.org Subject: Re: [imap5] Designing a new replacement protocol for IMAP Date: Thu, 9 Feb 2012 08:20:27 +0100 User-Agent: KMail/1.13.7 (Linux/3.1.0-1-amd64; KDE/4.6.5; x86_64; ; ) Cc: Bron Gondwana , IMAP Protocol Interest List , ietf-imapext@imc.org References: <1328732126.32086.140661033971485@webmail.messagingengine.com> In-Reply-To: <1328732126.32086.140661033971485@webmail.messagingengine.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Message-Id: <201202090820.28260.thomas@koch.ro> Sender: owner-ietf-imapext@mail.imc.org Precedence: bulk List-Archive: List-ID: List-Unsubscribe: Bron Gondwana: > Snide comments from those who don't agree with the goals are welcome too... > I just won't listen to you. These goals are very important to me. Hi Bron, I'm not an expert in IMAP, just got here out of curiosity about Kolab[1]'s use of IMAP folders as data stores. I found some older initiatives to replace IMAP with an HTTP based approach.[2] What do you think of HTTP for mail access? For my bachelor thesis[3] I currently argue that CardDAV/CalDAV could be perfectly replaced by AtomPub (RFC 5023). The main advantage would be that most software developers on this planet have some vague understanding of HTTP and that imense infrastructure and software already exists. [1] http://wiki.kolab.org/Why_IMAP_Is_Used_For_Storage [2] http://www.prescod.net/rest/restmail/ https://www.ietf.org/mailman/listinfo/httpmail [3] https://github.com/thkoch2001/bachelor-thesis Best regards, Thomas Koch, http://www.koch.ro Received: from hoffman.proper.com (localhost [127.0.0.1]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q18KIls5049373 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 8 Feb 2012 13:18:47 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) Received: (from majordom@localhost) by hoffman.proper.com (8.14.5/8.13.5/Submit) id q18KIl1O049372; Wed, 8 Feb 2012 13:18:47 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) X-Authentication-Warning: hoffman.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q18KIkIm049367 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Wed, 8 Feb 2012 13:18:46 -0700 (MST) (envelope-from gnb@fastmail.fm) Received: from compute5.internal (compute5.nyi.mail.srv.osa [10.202.2.45]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 3B22E20996 for ; Wed, 8 Feb 2012 15:18:46 -0500 (EST) Received: from frontend1.nyi.mail.srv.osa ([10.202.2.160]) by compute5.internal (MEProxy); Wed, 08 Feb 2012 15:18:46 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=fastmail.fm; h= references:message-id:from:to:in-reply-to:content-type :content-transfer-encoding:mime-version:subject:date:cc; s= mesmtp; bh=O3wS1xgz0pMNbYlOa3YJDDnfQ7M=; b=XvDjPyzvYisJLcb76Yx2t 0hBNTH7t4iS7C/gP8ncTogH38Teq1ILlQhxcrWNM4zkd2iaqBZj1+cUdijazedKM DBr+jB2fkpN5BIT78M9Jvrk+qc9If1F6nXeqHCGQsTXx3fC8a2FeDxHNGo9RbcjL Ueqy6ftdcGpVCNvCgHHpYI= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=references:message-id:from:to:in-reply-to :content-type:content-transfer-encoding:mime-version:subject :date:cc; s=smtpout; bh=O3wS1xgz0pMNbYlOa3YJDDnfQ7M=; b=WwbixYs0 JCEuBGRmY8QiGvWHyIK8GzfN8V5sKkNmP7m4pJ3adcvNfJKhpBVmngj22RDD4Xgb dt6qzmVGtRCG9D6IVd09z4nix50Hv8PBPTf+9dgshg2W+//v+repZM5S6kVLGx9R jOHM8LT+ujDA632OejlkRReBPGnGpwHfs5s= X-Sasl-enc: lnvuHRYW/RfVVKz26beJ4NXlg80aPouFDaq0tDhfviT0 1328732325 Received: from [10.0.0.1] (CPE-124-180-8-240.lns1.lon.bigpond.net.au [124.180.8.240]) by mail.messagingengine.com (Postfix) with ESMTPSA id 2919F8E0163; Wed, 8 Feb 2012 15:18:45 -0500 (EST) References: <52DA24FE-4065-4CF6-9322-96A5DDA065A9@orthanc.ca> Message-Id: From: Greg Banks To: Lyndon Nerenberg In-Reply-To: <52DA24FE-4065-4CF6-9322-96A5DDA065A9@orthanc.ca> Content-Type: text/plain; charset=us-ascii; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit X-Mailer: iPhone Mail (7C144) Mime-Version: 1.0 (iPhone Mail 7C144) Subject: Re: Metadata (5654) Implementation Limits Date: Thu, 9 Feb 2012 07:18:39 +1100 Cc: "ietf-imapext@imc.org" Sender: owner-ietf-imapext@mail.imc.org Precedence: bulk List-Archive: List-ID: List-Unsubscribe: On 09/02/2012, at 5:33, Lyndon Nerenberg wrote: > > [I didn't get much response from the imap-protocol list, so I'm > trying here, too.] > > I'm curious to know what the implementation limits are on servers > that support the Metadata extension. I would appreciate it if the > server authors out there could take a minute to fill out this survey. > > I will send a summary to the list once the responses trickle off. > (If you want your results to be anonymous, say so in the comments.) > > Thanks, > > --lyndon > > Implementation name and version: Cyrus (METADATA support rewritten and ANNOTATE support added to future 2.5 release). > Do you support /private?: Yes > Do you support /private/vendor?: Yes, with a subset of that space reserved for the server itself. Users cannot create new entries in that space, but they can create /private/ vendor/theirname/*. > Do you support unsolicited Metadata responses without values?: No > Do you support unsolicited Metadata responses with values?: No - no unsolicited responses. > Maximum number of annotations per mailbox: Not explicitly limited. > Maximum size of an : Not explicitly limited, but database keys include entry names and are limited to 4KiB. > Maximum size of a : Not explicitly limited. The old code had an undocumented limit of around 2KiB (iirc) at which it would silently truncate data in a stack buffer. The new code deals with values dynamically. There is presumably a limit to the size of the database values. For ANNOTATE support we report a 64KiB limit at SELECT time but this is a lie. > Maximum depth supported in : Not limited except by the length. > Other comments: > By default, Cyrus only accepts annotations whose entries are hardcoded or explicitly listed in a Cyrus configuration file (which is empty by default). A config option allows users to set arbitrary entries. We thought carefully about limits when implementing ANNOTATE as the per-message annotations pose a more extreme challenge. The solution we came up with was to track and limit the total size of all values of annotations on mailboxes and on messages, using a new QUOTA resource which we called X-ANNOTATION-STORAGE, with semantics similar to STORAGE. Per-server annotations are not tracked but can only be set by admin users. Greg. Received: from hoffman.proper.com (localhost [127.0.0.1]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q18KFSe5049193 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 8 Feb 2012 13:15:28 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) Received: (from majordom@localhost) by hoffman.proper.com (8.14.5/8.13.5/Submit) id q18KFSfv049192; Wed, 8 Feb 2012 13:15:28 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) X-Authentication-Warning: hoffman.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q18KFQah049186 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Wed, 8 Feb 2012 13:15:27 -0700 (MST) (envelope-from brong@fastmail.fm) Received: from compute3.internal (compute3.nyi.mail.srv.osa [10.202.2.43]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 6320E20A5C for ; Wed, 8 Feb 2012 15:15:26 -0500 (EST) Received: from web3.nyi.mail.srv.osa ([10.202.2.213]) by compute3.internal (MEProxy); Wed, 08 Feb 2012 15:15:26 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=fastmail.fm; h= message-id:from:to:mime-version:content-transfer-encoding :content-type:subject:date; s=mesmtp; bh=GeNe1ttnOq1n/NloKRSQGsm anwc=; b=ost2Ay/p0TnBqefBy2OARnPGZgJHIoX2NqHOer1xhDpmw9UkDM1kcMa YPckh5iwS/0gxaXPfISkkAiZoscPbGkEIlQvn7IIEnbTUNwk69N+xxQWSU+DyFF1 Cyi/t/PuQi+BlDGyCARq6NupzdTjMFhhMfeWL2gwOf4ODhDxGfdg= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=message-id:from:to:mime-version :content-transfer-encoding:content-type:subject:date; s=smtpout; bh=GeNe1ttnOq1n/NloKRSQGsmanwc=; b=NmbsgWtFf4UkQfk4eEij5HzSuZ1E nd/CkVAAY0z89BoHOKgeDfa/HNSG+Yy5zswK081cRGPjwb5ve7uE0KQfXocOXf7f q3Hisk6nASGegsHhfoFv3AD06szR46U/7lKnEig/9wHgoj+J7GzksWbiayNX8iSI o6rsswnmF3al8nE= Received: by web3.nyi.mail.srv.osa (Postfix, from userid 99) id 3E653400E0; Wed, 8 Feb 2012 15:15:26 -0500 (EST) Message-Id: <1328732126.32086.140661033971485@webmail.messagingengine.com> X-Sasl-Enc: CzgA3IIG77nbd4ZYPwvK6TfQVTbMqJS2ZNj6rq9hgZVQ 1328732126 From: Bron Gondwana To: IMAP Protocol Interest List , "Discussion on drastically slimming-down IMAP." , ietf-imapext@imc.org MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii" X-Mailer: MessagingEngine.com Webmail Interface Subject: Designing a new replacement protocol for IMAP Date: Wed, 08 Feb 2012 21:15:26 +0100 Sender: owner-ietf-imapext@mail.imc.org Precedence: bulk List-Archive: List-ID: List-Unsubscribe: I've given up on the IMAP5 mailing list going endlessly around in circles, though I'm definitely going to read through the backlogs and remind myself again what everyone spoke about, because there was some excellent discussion. So - I will build something and prove that it's good by showing improved user experiences. And I'm happy to keep talking here on these mailing lists too. I've started a wiki page: http://imapwiki.org/ReplacementProtocol and plan to blog about what I'm doing on our own (Opera's) blogging platform, to practice using it: http://my.opera.com/brongondwana/blog/ I plan to make on a protocol which can replace IMAP in most of its current uses. I want to get (at least) two working server implementations, and two working clients. I have at least in-principle support from enough project maintainers to do that. My strong philosophical ideals are: * Simple to build clients - morons welcome. If it's not clear the right way to do something to someone just reading a protocol dump, we haven't done a good enough job. We can't expect you to read 30 RFCs and resolve their conflicts yourself before starting coding, and we won't insult you if you didn't get it right. Where there is necessary complexity, the server authors bear the brunt. * Server side data model compatible with IMAP4 + extensions; we need to be able to co-exist. * Handle disconnection more gracefully - able to cheaply resync state (I plan to reuse the QRESYNC/CONDSTORE logic here - the model is sane). Getting reliable updates on mailbox state changes should not require a continuous connection. * Easy to proxy - regular syntax (UTF-8 for all communications where possible, but won't screw about trying to "fix" MIME - message file format will remain unchanged) * NO MAGIC - everything is explicitly requested. No UNSELECT/CLOSE/ SELECT another mailbox special behaviours, no implicit FETCH + SET \Seen, no pre-calculated "UNSEEN X" that wasn't asked for. (This probably means some way to compose actions and fetches, but it's better than being complex and magical. Remember our friends the morons.) * COMPLETE TEST SUITE that exercises every constraint in the spec, both positive and negative constraints. An implementation which passes the tests is compliant. An implementation which doesn't is not. This is much more likely to produce inter-operable implementations than a wordy spec ... ... if the authors want inter-operability that is. Nothing can protect against a deliberate embrace and extend - though negative constraint tests (MUST NOT) should help. I would love input. I spent most of the time at FOSDEM this year talking to other people involved in email about these ideas. I quite like the mailbox model behind IMAP, but I want to address the pain points I have seen so many times over the years. Snide comments from those who don't agree with the goals are welcome too... I just won't listen to you. These goals are very important to me. Regards, Bron. -- Bron Gondwana brong@fastmail.fm Received: from hoffman.proper.com (localhost [127.0.0.1]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q18IXIso043463 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 8 Feb 2012 11:33:18 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) Received: (from majordom@localhost) by hoffman.proper.com (8.14.5/8.13.5/Submit) id q18IXIhk043462; Wed, 8 Feb 2012 11:33:18 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) X-Authentication-Warning: hoffman.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f Received: from orthanc.ca (orthanc.ca [199.48.133.202]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q18IXI2Z043451 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for ; Wed, 8 Feb 2012 11:33:18 -0700 (MST) (envelope-from lyndon@orthanc.ca) Received: from [IPv6:2604:8800:137::21b:63ff:fe91:4123] ([IPv6:2604:8800:137:0:21b:63ff:fe91:4123]) (authenticated bits=0) by orthanc.ca (8.14.5/8.14.5) with ESMTP id q18IXEun017438 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO) for ; Wed, 8 Feb 2012 10:33:16 -0800 (PST) (envelope-from lyndon@orthanc.ca) From: Lyndon Nerenberg Content-Type: text/plain; charset=us-ascii Subject: Metadata (5654) Implementation Limits Date: Wed, 8 Feb 2012 10:33:13 -0800 Message-Id: <52DA24FE-4065-4CF6-9322-96A5DDA065A9@orthanc.ca> To: ietf-imapext@imc.org Mime-Version: 1.0 (Apple Message framework v1084) X-Mailer: Apple Mail (2.1084) Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by hoffman.proper.com id q18IXI2Y043457 Sender: owner-ietf-imapext@mail.imc.org Precedence: bulk List-Archive: List-ID: List-Unsubscribe: [I didn't get much response from the imap-protocol list, so I'm trying here, too.] I'm curious to know what the implementation limits are on servers that support the Metadata extension. I would appreciate it if the server authors out there could take a minute to fill out this survey. I will send a summary to the list once the responses trickle off. (If you want your results to be anonymous, say so in the comments.) Thanks, --lyndon Implementation name and version: Do you support /private?: Do you support /private/vendor?: Do you support unsolicited Metadata responses without values?: Do you support unsolicited Metadata responses with values?: Maximum number of annotations per mailbox: Maximum size of an : Maximum size of a : Maximum depth supported in : Other comments: Received: from hoffman.proper.com (localhost [127.0.0.1]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q1DDUG6k094020 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 13 Feb 2012 06:30:16 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) Received: (from majordom@localhost) by hoffman.proper.com (8.14.5/8.13.5/Submit) id q1DDUGNh094019; Mon, 13 Feb 2012 06:30:16 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) X-Authentication-Warning: hoffman.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q1DDUFIL094014 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Mon, 13 Feb 2012 06:30:15 -0700 (MST) (envelope-from brong@fastmail.fm) Received: from compute3.internal (compute3.nyi.mail.srv.osa [10.202.2.43]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 2A4D22092A for ; Mon, 13 Feb 2012 08:30:15 -0500 (EST) Received: from web1.nyi.mail.srv.osa ([10.202.2.211]) by compute3.internal (MEProxy); Mon, 13 Feb 2012 08:30:15 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=fastmail.fm; h= message-id:from:to:cc:mime-version:content-transfer-encoding :content-type:in-reply-to:references:subject:date; s=mesmtp; bh= wmyd/hTl2PoiwcFPqUUo4b8FMGg=; b=MmZ2rn5E04MVvST7UTRW0piSdZk6s3IP yuy3Ya+7csQ0r+jwbtP+hFQ9I1Kbch4IomyDjB+mHJG4IjGjcPDJVRzFII79xokM MhjoMgo2Oqf9LUMvSpZ10tlJ7BFcw674AwWnZ2zUxE7HP11RwoVjfjaE2qedyh2X zF3rux3sT7w= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=message-id:from:to:cc:mime-version :content-transfer-encoding:content-type:in-reply-to:references :subject:date; s=smtpout; bh=wmyd/hTl2PoiwcFPqUUo4b8FMGg=; b=IAJ umcTSq2sSCMxylAGa5Y0TijYa2h9E7+u3BEg1IbcSjuI4umVaCZn1TIJoJ0dYQmD jWZWQETKTSibyeNTVQseKeBvQO0+02zW2qdqkk4QHRRCN5+vMu5aDb9WfGmQVy2c z0u1NxNn8TwGaKf+VyPGhxq/SNnl159TaK7w4Ax8= Received: by web1.nyi.mail.srv.osa (Postfix, from userid 99) id 00E1DA000C2; Mon, 13 Feb 2012 08:30:14 -0500 (EST) Message-Id: <1329139814.1318.140661035865773@webmail.messagingengine.com> X-Sasl-Enc: bU3o/mY85q0GCRRsrAK5UTE/nMDvjO5xREezkEDkhx+/ 1329139814 From: Bron Gondwana To: Arnt Gulbrandsen Cc: Timo Sirainen , ietf-imapext@imc.org MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii" X-Mailer: MessagingEngine.com Webmail Interface In-Reply-To: <4F390E77.2060808@gulbrandsen.priv.no> References: <8CAAFEFF-A302-4359-9B59-685795F49B21@iki.fi><4F38D595.4030904@gulbrandsen.priv.no><1329125560.14070.140661035785429@webmail.messagingengine.com><4F38DCB8.4040802@gulbrandsen.priv.no><1329138251.27672.140661035856225@webmail.messagingengine.com> <4F390E77.2060808@gulbrandsen.priv.no> Subject: Re: ANNOTATE-EXPERIMENT-1 Date: Mon, 13 Feb 2012 14:30:14 +0100 Sender: owner-ietf-imapext@mail.imc.org Precedence: bulk List-Archive: List-ID: List-Unsubscribe: On Mon, Feb 13, 2012, at 02:21 PM, Arnt Gulbrandsen wrote: > On 02/13/2012 02:04 PM, Bron Gondwana wrote: > > And of course there's normalisation as well. > > Normalisation doesn't affect meaning at all. It's just changes such as > "o umlaut-above-previous-letter" into "o-with-umlaut" or the other way. > Useful for text processing. Also useful for checking if a particular keyword is already set, and merging the various forms together into a single keyword. Bron. -- Bron Gondwana brong@fastmail.fm Received: from hoffman.proper.com (localhost [127.0.0.1]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q1DDLVPi093474 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 13 Feb 2012 06:21:32 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) Received: (from majordom@localhost) by hoffman.proper.com (8.14.5/8.13.5/Submit) id q1DDLV3d093473; Mon, 13 Feb 2012 06:21:31 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) X-Authentication-Warning: hoffman.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f Received: from strange.aox.org (strange.aox.org [80.244.248.170]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q1DDLURa093464 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Mon, 13 Feb 2012 06:21:31 -0700 (MST) (envelope-from arnt@gulbrandsen.priv.no) Received: from fri.gulbrandsen.priv.no (unknown [127.0.0.1]) by strange.aox.org (Postfix) with ESMTP id 0AD77F8C62D; Mon, 13 Feb 2012 13:21:29 +0000 (UTC) Received: from arnt@gulbrandsen.priv.no by fri.gulbrandsen.priv.no (Archiveopteryx 3.1.4) with esmtpsa id 1329139288-16404-16403/10/12; Mon, 13 Feb 2012 13:21:28 +0000 Message-Id: <4F390E77.2060808@gulbrandsen.priv.no> Date: Mon, 13 Feb 2012 14:21:59 +0100 From: Arnt Gulbrandsen User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:9.0) Gecko/20111229 Thunderbird/9.0 Mime-Version: 1.0 To: Bron Gondwana Cc: Timo Sirainen , ietf-imapext@imc.org Subject: Re: ANNOTATE-EXPERIMENT-1 References: <8CAAFEFF-A302-4359-9B59-685795F49B21@iki.fi> <4F38D595.4030904@gulbrandsen.priv.no> <1329125560.14070.140661035785429@webmail.messagingengine.com> <4F38DCB8.4040802@gulbrandsen.priv.no> <1329138251.27672.140661035856225@webmail.messagingengine.com> In-Reply-To: <1329138251.27672.140661035856225@webmail.messagingengine.com> Content-Type: text/plain; charset=utf-8 Sender: owner-ietf-imapext@mail.imc.org Precedence: bulk List-Archive: List-ID: List-Unsubscribe: On 02/13/2012 02:04 PM, Bron Gondwana wrote: > And of course there's normalisation as well. Normalisation doesn't affect meaning at all. It's just changes such as "o umlaut-above-previous-letter" into "o-with-umlaut" or the other way. Useful for text processing. Arnt Received: from hoffman.proper.com (localhost [127.0.0.1]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q1DD4EXa092736 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 13 Feb 2012 06:04:14 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) Received: (from majordom@localhost) by hoffman.proper.com (8.14.5/8.13.5/Submit) id q1DD4EfU092733; Mon, 13 Feb 2012 06:04:14 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) X-Authentication-Warning: hoffman.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q1DD4B32092727 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Mon, 13 Feb 2012 06:04:12 -0700 (MST) (envelope-from brong@fastmail.fm) Received: from compute5.internal (compute5.nyi.mail.srv.osa [10.202.2.45]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 7A9A9208CA for ; Mon, 13 Feb 2012 08:04:11 -0500 (EST) Received: from web1.nyi.mail.srv.osa ([10.202.2.211]) by compute5.internal (MEProxy); Mon, 13 Feb 2012 08:04:11 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=fastmail.fm; h= message-id:from:to:cc:mime-version:content-transfer-encoding :content-type:in-reply-to:references:subject:date; s=mesmtp; bh= FZaKGGjRyL24Wvdw+vDGnxWkREc=; b=eEkCUnQdJtHGAnkedza8mRTUN/wBiNdK LJeLRnmKyppxYMeF+NAr/8hO0rDLuB2lmwua2WcY4h2n2+uRoBhQcnEYwl5EYP/o BBg0XBVkhjuVYClrZOE86GvVGQ9+00K2sYYEMfRpcEWSC1X0uV3sWW+dnYYUBZfd UqAnbqeLgeg= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=message-id:from:to:cc:mime-version :content-transfer-encoding:content-type:in-reply-to:references :subject:date; s=smtpout; bh=FZaKGGjRyL24Wvdw+vDGnxWkREc=; b=Woy PNR0dd7DkJk4rM2+WD4Dhskdu8p+83UL02JWjwO6YA1gZVxniMtXLRJLMVHag1Jj ZLCO7S8ZdmVhwWRJ5dxSCPLafD46bqcsiKr8nw+gQp3nUt2QHt4NIWS+AqOKfGun 5D4FPyZVJlkOGEweMvKQvKChNvrV1Aq0ax2tFGAg= Received: by web1.nyi.mail.srv.osa (Postfix, from userid 99) id 5363AA000C2; Mon, 13 Feb 2012 08:04:11 -0500 (EST) Message-Id: <1329138251.27672.140661035856225@webmail.messagingengine.com> X-Sasl-Enc: NcsGXtugB+HxBZOJ3A5TlrPcSfTnJ2FRKUdxGynk8c78 1329138251 From: Bron Gondwana To: Arnt Gulbrandsen Cc: Timo Sirainen , ietf-imapext@imc.org MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Mailer: MessagingEngine.com Webmail Interface In-Reply-To: <4F38DCB8.4040802@gulbrandsen.priv.no> References: <8CAAFEFF-A302-4359-9B59-685795F49B21@iki.fi><4F38D595.4030904@gulbrandsen.priv.no><1329125560.14070.140661035785429@webmail.messagingengine.com> <4F38DCB8.4040802@gulbrandsen.priv.no> Subject: Re: ANNOTATE-EXPERIMENT-1 Date: Mon, 13 Feb 2012 14:04:11 +0100 Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by hoffman.proper.com id q1DD4C31092728 Sender: owner-ietf-imapext@mail.imc.org Precedence: bulk List-Archive: List-ID: List-Unsubscribe: On Mon, Feb 13, 2012, at 10:49 AM, Arnt Gulbrandsen wrote: > On 02/13/2012 10:32 AM, Bron Gondwana wrote: > > Keywords are already case insensitive. Which is going to be a pig in poke if we try to make UTF8 keywords. > > Unicode case insensitivity isn't so bad. > > One rule for Turkish, one rule of the rest of the world, and anyone who > can define i to be equal to ı can sidestep the difference. In IMAP's > case, these two commands could be defined to be equivalent: > > a store +flags 1 istanbul > b store +flags 1 ıstanbul > > Not pretty but also not the end of the world. > > (Neat: This spell checker, which I must find out how to turn off btw, > thinks istanbul is wrong and ıstanbul right.) And of course there's normalisation as well. I really don't know as much about this area as I should. I guess the obvious is "require best practice as defined ". > >> But I don't mind dropping that in the least. AFAIK noone has actually > >> wanted to sort by an annotation. > > > > If nobody wants it, it doesn't matter if it's inefficient, right? > > Mostly, but not quite. > > A fetch item that noone ever calls can still require copy and append to > do extra work. True - so long as it's stored rather than calculated on request. > >> Why would store be able to change only some entries? > > > > ACLs is the obvious case. Another thing which the Gmail guys mentioned > > recently is "partial storage failure" - IMAP presents a view in which > > everything is either fully functional or fully broken, but the reality > > under the hood is not always so. > > Good point. > > Still, I think that's an IMAP problem. It's not made worse by covering > all of IMAP, as opposed to all except ANNOTATE. Agree. Bron. -- Bron Gondwana brong@fastmail.fm Received: from hoffman.proper.com (localhost [127.0.0.1]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q1D9nHuh084565 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 13 Feb 2012 02:49:17 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) Received: (from majordom@localhost) by hoffman.proper.com (8.14.5/8.13.5/Submit) id q1D9nHMt084564; Mon, 13 Feb 2012 02:49:17 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) X-Authentication-Warning: hoffman.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f Received: from strange.aox.org (strange.aox.org [80.244.248.170]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q1D9nFuL084559 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Mon, 13 Feb 2012 02:49:17 -0700 (MST) (envelope-from arnt@gulbrandsen.priv.no) Received: from fri.gulbrandsen.priv.no (unknown [127.0.0.1]) by strange.aox.org (Postfix) with ESMTP id 55E6CF8C60A; Mon, 13 Feb 2012 09:49:15 +0000 (UTC) Received: from arnt@gulbrandsen.priv.no by fri.gulbrandsen.priv.no (Archiveopteryx 3.1.4) with esmtpsa id 1329126554-16404-16403/10/5; Mon, 13 Feb 2012 09:49:14 +0000 Message-Id: <4F38DCB8.4040802@gulbrandsen.priv.no> Date: Mon, 13 Feb 2012 10:49:44 +0100 From: Arnt Gulbrandsen User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:9.0) Gecko/20111229 Thunderbird/9.0 Mime-Version: 1.0 To: Bron Gondwana Cc: Timo Sirainen , ietf-imapext@imc.org Subject: Re: ANNOTATE-EXPERIMENT-1 References: <8CAAFEFF-A302-4359-9B59-685795F49B21@iki.fi> <4F38D595.4030904@gulbrandsen.priv.no> <1329125560.14070.140661035785429@webmail.messagingengine.com> In-Reply-To: <1329125560.14070.140661035785429@webmail.messagingengine.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by hoffman.proper.com id q1D9nHuK084560 Sender: owner-ietf-imapext@mail.imc.org Precedence: bulk List-Archive: List-ID: List-Unsubscribe: On 02/13/2012 10:32 AM, Bron Gondwana wrote: > Keywords are already case insensitive. Which is going to be a pig in poke if we try to make UTF8 keywords. Unicode case insensitivity isn't so bad. One rule for Turkish, one rule of the rest of the world, and anyone who can define i to be equal to ı can sidestep the difference. In IMAP's case, these two commands could be defined to be equivalent: a store +flags 1 istanbul b store +flags 1 ıstanbul Not pretty but also not the end of the world. (Neat: This spell checker, which I must find out how to turn off btw, thinks istanbul is wrong and ıstanbul right.) >> But I don't mind dropping that in the least. AFAIK noone has actually >> wanted to sort by an annotation. > > If nobody wants it, it doesn't matter if it's inefficient, right? Mostly, but not quite. A fetch item that noone ever calls can still require copy and append to do extra work. >> Why would store be able to change only some entries? > > ACLs is the obvious case. Another thing which the Gmail guys mentioned > recently is "partial storage failure" - IMAP presents a view in which > everything is either fully functional or fully broken, but the reality > under the hood is not always so. Good point. Still, I think that's an IMAP problem. It's not made worse by covering all of IMAP, as opposed to all except ANNOTATE. Arnt Received: from hoffman.proper.com (localhost [127.0.0.1]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q1D9WgmR084060 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 13 Feb 2012 02:32:42 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) Received: (from majordom@localhost) by hoffman.proper.com (8.14.5/8.13.5/Submit) id q1D9WgWM084059; Mon, 13 Feb 2012 02:32:42 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) X-Authentication-Warning: hoffman.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q1D9WfOL084052 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Mon, 13 Feb 2012 02:32:41 -0700 (MST) (envelope-from brong@fastmail.fm) Received: from compute3.internal (compute3.nyi.mail.srv.osa [10.202.2.43]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 8687820ABF for ; Mon, 13 Feb 2012 04:32:40 -0500 (EST) Received: from web1.nyi.mail.srv.osa ([10.202.2.211]) by compute3.internal (MEProxy); Mon, 13 Feb 2012 04:32:40 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=fastmail.fm; h= message-id:from:to:cc:mime-version:content-transfer-encoding :content-type:references:subject:in-reply-to:date; s=mesmtp; bh= YJC2pT2SjR/gZ7Dwa1Znn5UucfI=; b=bYW5SQ64lgWdb2JW96txIN+ADUHTADKf K/h7uZRpgQZCx4IUWLoU6hdN/d/DBYycKF8go3zjIlUv0qtUO5TlxY7adOjRyyUu qMDKX3wYTWe/lD0h29jzlEN77tf94Ih/nmf6gX1saJmkfPidPZhMm1mlU+UMadqC POv58MhaSLM= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=message-id:from:to:cc:mime-version :content-transfer-encoding:content-type:references:subject :in-reply-to:date; s=smtpout; bh=YJC2pT2SjR/gZ7Dwa1Znn5UucfI=; b= cct2WzOSyBykG9hAca+/JuElXycSRblWExp9sOGeX64SCdXmDTsBpgd1uX3yqHFE GTA691gK95sJ6G+s71EKbYGR8a0SJCaqe07NvewFAOrQJG8DtI+vD8SYxQpAnfwK SUjUwUz2RFCVO+H0HxzIe/InO0FpMxjb8iXYV6/pd1k= Received: by web1.nyi.mail.srv.osa (Postfix, from userid 99) id 62BEEA000E4; Mon, 13 Feb 2012 04:32:40 -0500 (EST) Message-Id: <1329125560.14070.140661035785429@webmail.messagingengine.com> X-Sasl-Enc: HlnWH8bF3d8ZEPT3NiOCxH41g2SzLBSk/HfCoA8nN0kZ 1329125560 From: Bron Gondwana To: Arnt Gulbrandsen , Timo Sirainen Cc: ietf-imapext@imc.org MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii" X-Mailer: MessagingEngine.com Webmail Interface References: <8CAAFEFF-A302-4359-9B59-685795F49B21@iki.fi> <4F38D595.4030904@gulbrandsen.priv.no> Subject: Re: ANNOTATE-EXPERIMENT-1 In-Reply-To: <4F38D595.4030904@gulbrandsen.priv.no> Date: Mon, 13 Feb 2012 10:32:40 +0100 Sender: owner-ietf-imapext@mail.imc.org Precedence: bulk List-Archive: List-ID: List-Unsubscribe: On Mon, Feb 13, 2012, at 10:19 AM, Arnt Gulbrandsen wrote: > > On 02/13/2012 01:45 AM, Timo Sirainen wrote: > > > > I was thinking about finally implementing METADATA + per-mail annotations to Dovecot so I read through the RFCs. There are mainly two things I'd do differently in ANNOTATE-EXPERIMENT-2: > > > > 1. METADATA has case-insensitive entry names, ANNOTATE has case-sensitive. Probably better to keep them case-insensitive in both. > > I wouldn't mind. Keywords are already case insensitive. Which is going to be a pig in poke if we try to make UTF8 keywords. > > 2. SORT (ANNOTATION) - is this really worth the trouble? There's really no good way for servers to implement this in an efficient way. It would pretty much require building and maintaining an index for every single annotation entry name. > > Sorting at read time isn't bad, considering the very low number of > annotations used. > > But I don't mind dropping that in the least. AFAIK noone has actually > wanted to sort by an annotation. If nobody wants it, it doesn't matter if it's inefficient, right? If people do want it, then you can track the annotations they are actually using to sort by and index those... at least that's the theory behind having adaptive indexing. (just being contrary now!) > > And other things I wondered about: > > > > - If APPEND has annotations, and annotation limits are reached, the whole APPEND fails with TOOBIG/TOOMANY error? I guess it has to, although that's a bit annoying. > > Append is transactional. I agree it can be annoying sometimes, such as > when ten-thousand-message multiappends break. But IMAP already has > enough funny little exceptions, please don't try to add a rule such as > "append is transactional except in regard to annotations". Agree - if it's transactional, it's transactional. > > - ANNOTATE doesn't specify what happens if STORE can change only some of the entries. Should it rollback the changes to all the previous entries as well? Including changes to previous messages if multiple messages were specified? Would have been simpler if both METADATA and ANNOTATE supported changing only one annotation at a time. > > My code is transactional, so I don't have the problem. > > Why would store be able to change only some entries? ACLs is the obvious case. Another thing which the Gmail guys mentioned recently is "partial storage failure" - IMAP presents a view in which everything is either fully functional or fully broken, but the reality under the hood is not always so. More simply, could just be permission problems on a file somewhere, or partial database corruption. Shit happens. Bron. -- Bron Gondwana brong@fastmail.fm Received: from hoffman.proper.com (localhost [127.0.0.1]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q1D9IqoY083722 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 13 Feb 2012 02:18:52 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) Received: (from majordom@localhost) by hoffman.proper.com (8.14.5/8.13.5/Submit) id q1D9Iqst083721; Mon, 13 Feb 2012 02:18:52 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) X-Authentication-Warning: hoffman.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f Received: from strange.aox.org (strange.aox.org [80.244.248.170]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q1D9Indp083712 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Mon, 13 Feb 2012 02:18:51 -0700 (MST) (envelope-from arnt@gulbrandsen.priv.no) Received: from fri.gulbrandsen.priv.no (unknown [127.0.0.1]) by strange.aox.org (Postfix) with ESMTP id 53753F8C571; Mon, 13 Feb 2012 09:18:48 +0000 (UTC) Received: from arnt@gulbrandsen.priv.no by fri.gulbrandsen.priv.no (Archiveopteryx 3.1.4) with esmtpsa id 1329124726-16404-16403/10/3; Mon, 13 Feb 2012 09:18:46 +0000 Message-Id: <4F38D595.4030904@gulbrandsen.priv.no> Date: Mon, 13 Feb 2012 10:19:17 +0100 From: Arnt Gulbrandsen User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:9.0) Gecko/20111229 Thunderbird/9.0 Mime-Version: 1.0 To: Timo Sirainen Cc: ietf-imapext@imc.org Subject: Re: ANNOTATE-EXPERIMENT-1 References: <8CAAFEFF-A302-4359-9B59-685795F49B21@iki.fi> In-Reply-To: <8CAAFEFF-A302-4359-9B59-685795F49B21@iki.fi> Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by hoffman.proper.com id q1D9Ipdo083717 Sender: owner-ietf-imapext@mail.imc.org Precedence: bulk List-Archive: List-ID: List-Unsubscribe: On 02/13/2012 01:45 AM, Timo Sirainen wrote: > > I was thinking about finally implementing METADATA + per-mail annotations to Dovecot so I read through the RFCs. There are mainly two things I'd do differently in ANNOTATE-EXPERIMENT-2: > > 1. METADATA has case-insensitive entry names, ANNOTATE has case-sensitive. Probably better to keep them case-insensitive in both. I wouldn't mind. > 2. SORT (ANNOTATION) - is this really worth the trouble? There's really no good way for servers to implement this in an efficient way. It would pretty much require building and maintaining an index for every single annotation entry name. Sorting at read time isn't bad, considering the very low number of annotations used. But I don't mind dropping that in the least. AFAIK noone has actually wanted to sort by an annotation. > And other things I wondered about: > > - If APPEND has annotations, and annotation limits are reached, the whole APPEND fails with TOOBIG/TOOMANY error? I guess it has to, although that's a bit annoying. Append is transactional. I agree it can be annoying sometimes, such as when ten-thousand-message multiappends break. But IMAP already has enough funny little exceptions, please don't try to add a rule such as "append is transactional except in regard to annotations". > - ANNOTATE doesn't specify what happens if STORE can change only some of the entries. Should it rollback the changes to all the previous entries as well? Including changes to previous messages if multiple messages were specified? Would have been simpler if both METADATA and ANNOTATE supported changing only one annotation at a time. My code is transactional, so I don't have the problem. Why would store be able to change only some entries? > - TOOBIG/TOOMANY doesn't map very nicely if I want to have global "max. annotation size" and "max. annotation count". Perhaps a global max. count shouldn't exist, since the number of mails/mailboxes could be limited instead, but a global size similar to mail quota should exist. I guess when that global size is reached it just fails with TOOBIG until some annotations are deleted. Yes. Arnt Received: from hoffman.proper.com (localhost [127.0.0.1]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q1D0jeAk070021 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 12 Feb 2012 17:45:40 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) Received: (from majordom@localhost) by hoffman.proper.com (8.14.5/8.13.5/Submit) id q1D0je2x070020; Sun, 12 Feb 2012 17:45:40 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) X-Authentication-Warning: hoffman.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f Received: from dovecot.org (dovecot.org [193.210.130.67]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q1D0jdji070015 for ; Sun, 12 Feb 2012 17:45:40 -0700 (MST) (envelope-from tss@iki.fi) Received: from [192.168.10.101] (a88-112-255-76.elisa-laajakaista.fi [88.112.255.76]) by dovecot.org (Postfix) with ESMTP id 0ACFA1AE876C for ; Mon, 13 Feb 2012 02:45:33 +0200 (EET) From: Timo Sirainen Content-Type: text/plain; charset=us-ascii Subject: ANNOTATE-EXPERIMENT-1 Date: Mon, 13 Feb 2012 02:45:32 +0200 Message-Id: <8CAAFEFF-A302-4359-9B59-685795F49B21@iki.fi> To: ietf-imapext@imc.org Mime-Version: 1.0 (Apple Message framework v1084) X-Mailer: Apple Mail (2.1084) Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by hoffman.proper.com id q1D0jeji070016 Sender: owner-ietf-imapext@mail.imc.org Precedence: bulk List-Archive: List-ID: List-Unsubscribe: I was thinking about finally implementing METADATA + per-mail annotations to Dovecot so I read through the RFCs. There are mainly two things I'd do differently in ANNOTATE-EXPERIMENT-2: 1. METADATA has case-insensitive entry names, ANNOTATE has case-sensitive. Probably better to keep them case-insensitive in both. 2. SORT (ANNOTATION) - is this really worth the trouble? There's really no good way for servers to implement this in an efficient way. It would pretty much require building and maintaining an index for every single annotation entry name. And other things I wondered about: - If APPEND has annotations, and annotation limits are reached, the whole APPEND fails with TOOBIG/TOOMANY error? I guess it has to, although that's a bit annoying. - ANNOTATE doesn't specify what happens if STORE can change only some of the entries. Should it rollback the changes to all the previous entries as well? Including changes to previous messages if multiple messages were specified? Would have been simpler if both METADATA and ANNOTATE supported changing only one annotation at a time. - TOOBIG/TOOMANY doesn't map very nicely if I want to have global "max. annotation size" and "max. annotation count". Perhaps a global max. count shouldn't exist, since the number of mails/mailboxes could be limited instead, but a global size similar to mail quota should exist. I guess when that global size is reached it just fails with TOOBIG until some annotations are deleted. Received: from hoffman.proper.com (localhost [127.0.0.1]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q1AEj7jI043094 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 10 Feb 2012 07:45:07 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) Received: (from majordom@localhost) by hoffman.proper.com (8.14.5/8.13.5/Submit) id q1AEj7S3043093; Fri, 10 Feb 2012 07:45:07 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) X-Authentication-Warning: hoffman.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f Received: from koch.ro (koch.ro [88.198.2.104]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q1AEj4aU043088 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NO) for ; Fri, 10 Feb 2012 07:45:05 -0700 (MST) (envelope-from thomas@koch.ro) Received: from aktaia.intevation.org ([212.95.126.10] helo=t61.localnet) by koch.ro with esmtpsa (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.72) (envelope-from ) id 1RvriZ-0007SW-3v; Fri, 10 Feb 2012 15:44:55 +0100 From: Thomas Koch Reply-To: thomas@koch.ro To: Adrien de Croy Subject: Re: [imap5] Designing a new replacement protocol for IMAP Date: Fri, 10 Feb 2012 15:44:50 +0100 User-Agent: KMail/1.13.7 (Linux/3.1.0-1-amd64; KDE/4.6.5; x86_64; ; ) Cc: imap5@ietf.org, Bron Gondwana , IMAP Protocol Interest List , ietf-imapext@imc.org References: <1328732126.32086.140661033971485@webmail.messagingengine.com> <201202090820.28260.thomas@koch.ro> <4F337E61.5040702@qbik.com> In-Reply-To: <4F337E61.5040702@qbik.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Message-Id: <201202101544.51364.thomas@koch.ro> Sender: owner-ietf-imapext@mail.imc.org Precedence: bulk List-Archive: List-ID: List-Unsubscribe: Hi Adrien, thank you for your insights. I've some more questions inline. Adrien de Croy: > * updates. Would need to poll or use long polling to get real-time > updates (e.g. notification of incoming mail). Could Websockets or http://en.wikipedia.org/wiki/Comet_(programming) be of help here? > * complex. In order to reduce round-trips, you'd have to do many things > in a single transaction, which would create complexity issues in the > server and client I was thinking restful HTTP which by definition is not complex. The question is, whether a useful mail fetch protocol could be designed in the constraints of rest. > I think it would also be very problematic in the wild, since any http > intermediary would feel entitled to mess with traffic. E.g. heuristic > caching, intercepting proxies etc. There is an HTTP header to instruct any cache to pipe the request through and do no caching. But anybody on any line in any protocol could mess with your data. Have to go now, sorry. :-) Best regards, Thomas Koch, http://www.koch.ro Received: from hoffman.proper.com (localhost [127.0.0.1]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q19BWB5f081688 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 9 Feb 2012 04:32:11 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) Received: (from majordom@localhost) by hoffman.proper.com (8.14.5/8.13.5/Submit) id q19BWBKv081687; Thu, 9 Feb 2012 04:32:11 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) X-Authentication-Warning: hoffman.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q19BWABW081682 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Thu, 9 Feb 2012 04:32:10 -0700 (MST) (envelope-from brong@fastmail.fm) Received: from compute5.internal (compute5.nyi.mail.srv.osa [10.202.2.45]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 075C0209D5 for ; Thu, 9 Feb 2012 06:32:10 -0500 (EST) Received: from frontend1.nyi.mail.srv.osa ([10.202.2.160]) by compute5.internal (MEProxy); Thu, 09 Feb 2012 06:32:10 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=fastmail.fm; h= date:from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=mesmtp; bh=wPCn9RtD+I2RGzRY5ggh/VUe mtU=; b=Gbr3CvUxKMNaThkEvyic5Oq5+9OcqBQcSlwN37QLthMcsBeG3p+iPHTk QJDgK6n8ioJ5n80LOntvgVBKDbqkvkXFWmFSPsKW2fk0GOqU78Jnl1XgH9MW3biV 2zPn2VFN9PZ1SbmGXPFjOOJLMHAZNJ7gFPR5OZdhaSJnPwO73ec= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=date:from:to:cc:subject:message-id :references:mime-version:content-type:in-reply-to; s=smtpout; bh=wPCn9RtD+I2RGzRY5ggh/VUemtU=; b=AmuL2A2HKAkyqSjv4v9gqaDqkqI/ knnqmOnIzrzbmHyd7JVeks6wdkL73KodYdunG43A+Gmin2r0BO84+ozqfmu6Y1D8 KiC+VOrF/29YminiWVyu6P/o6vW8w10NxsE1LhFaZmq6YuO7l5ISsL6fZp2xjxkz y2fasYKE3tk+Ms0= X-Sasl-enc: t+LLM2k4dsiT7QwkYEiqOLrfMqwb1O0wfKAnksCsqsBx 1328787129 Received: from localhost (99.249.9.46.customer.cdi.no [46.9.249.99]) by mail.messagingengine.com (Postfix) with ESMTPSA id AF14D8E0085; Thu, 9 Feb 2012 06:32:09 -0500 (EST) Received: by localhost (Postfix, from userid 1000) id 380351C6FBC; Thu, 9 Feb 2012 12:32:08 +0100 (CET) Date: Thu, 9 Feb 2012 12:32:08 +0100 From: Bron Gondwana To: Giovanni Panozzo Cc: imap5@ietf.org, ietf-imapext@imc.org, imap-protocol@u.washington.edu Subject: Re: [imap5] Designing a new replacement protocol for IMAP Message-ID: <20120209113208.GA31355@launde.brong.net> References: <1328732126.32086.140661033971485@webmail.messagingengine.com> <201202090820.28260.thomas@koch.ro> <4F33AD2D.7050904@panozzo.it> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4F33AD2D.7050904@panozzo.it> Organization: brong.net User-Agent: Mutt/1.5.21 (2010-09-15) Sender: owner-ietf-imapext@mail.imc.org Precedence: bulk List-Archive: List-ID: List-Unsubscribe: On Thu, Feb 09, 2012 at 12:25:33PM +0100, Giovanni Panozzo wrote: > PS: I think that crossposting in 3 ML is too much. Which one will be > the official ML for this discussion ? (posting to all three for this...) Let's take the rest of the traffic to just imap5@ietf.org Bron. Received: from hoffman.proper.com (localhost [127.0.0.1]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q19BOGZk081318 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 9 Feb 2012 04:24:16 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) Received: (from majordom@localhost) by hoffman.proper.com (8.14.5/8.13.5/Submit) id q19BOGDa081317; Thu, 9 Feb 2012 04:24:16 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) X-Authentication-Warning: hoffman.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q19BODSG081311 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Thu, 9 Feb 2012 04:24:14 -0700 (MST) (envelope-from brong@fastmail.fm) Received: from compute6.internal (compute6.nyi.mail.srv.osa [10.202.2.46]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 5AA852108E for ; Thu, 9 Feb 2012 06:24:13 -0500 (EST) Received: from frontend2.nyi.mail.srv.osa ([10.202.2.161]) by compute6.internal (MEProxy); Thu, 09 Feb 2012 06:24:13 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=fastmail.fm; h= date:from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=mesmtp; bh=eyPXuJRWfA3AXuiX2tZFA2H0 IS0=; b=bUbkW2yF0fyEFxTkMD/plKdAWJ2qG6A4e84VHJaStPeYC9hMWNQOhSs7 0DM6BDPU3BqoSlX7Dt2R/nnveC3/OuvHB7IFjxDLI9XcA5cCdgIvb/z3ilRX437t CqqaQirhtWqoR27sY1nq35CLs2d9zy7sFVnJPMmepQlY4iSRLgo= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=date:from:to:cc:subject:message-id :references:mime-version:content-type:in-reply-to; s=smtpout; bh=eyPXuJRWfA3AXuiX2tZFA2H0IS0=; b=W+Y72hA1zXC5iq/s3oZjJfy4gyaJ Sl0F3ZfeHDTb3uXhX3Yz14grU5MBuGlN6TsoB5N0slVzpdmk3WH6KAY2ve+BEJ9G 55sBIszGJji853jHfyzwQAmZv5elgOVW17rMfDLLKSA0szaez5aSFFKfFUBsvn4J K5jed5OPxBfmyPc= X-Sasl-enc: DRLqcQnkDZ/s9vPvcFYXo6cyO3ntFszOGgVzj6jfL1Pj 1328786653 Received: from localhost (99.249.9.46.customer.cdi.no [46.9.249.99]) by mail.messagingengine.com (Postfix) with ESMTPSA id 07DA9483525; Thu, 9 Feb 2012 06:24:13 -0500 (EST) Received: by localhost (Postfix, from userid 1000) id 4D35819BF04; Thu, 9 Feb 2012 12:24:11 +0100 (CET) Date: Thu, 9 Feb 2012 12:24:11 +0100 From: Bron Gondwana To: Adrien de Croy Cc: thomas@koch.ro, imap5@ietf.org, Bron Gondwana , IMAP Protocol Interest List , ietf-imapext@imc.org Subject: Re: [imap5] Designing a new replacement protocol for IMAP Message-ID: <20120209112411.GB29734@launde.brong.net> References: <1328732126.32086.140661033971485@webmail.messagingengine.com> <201202090820.28260.thomas@koch.ro> <4F337E61.5040702@qbik.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4F337E61.5040702@qbik.com> Organization: brong.net User-Agent: Mutt/1.5.21 (2010-09-15) Sender: owner-ietf-imapext@mail.imc.org Precedence: bulk List-Archive: List-ID: List-Unsubscribe: On Thu, Feb 09, 2012 at 09:05:53PM +1300, Adrien de Croy wrote: > > Whilst HTTP could certainly be used to put together an email system, > I don't think it would perform well enough to replace existing > protocols on desktops. > > The main issues I see are: > > * updates. Would need to poll or use long polling to get real-time > updates (e.g. notification of incoming mail). Yes - long polling or "out of band" notification systems. We have an HTTP protocol defined kind of "on top of" IMAP plus our own extensions here at FastMail/Opera (mail.opera.com if you want to play with it) We are currently using EventSource to notify the client that "there is something new", after which the client queries to find what's updated. This is possible to be efficient because we use a single counter for uidvalidity and a single counter for highestmodseq across the entire user's folders - so any change to uidvalidity means the folder list needs refreshing, and any change to highestmodseq means the message view needs checking. Combined with efficient "what's changed in this view since my previous modseq value" queries, it's very bandwidth efficient. > * slow. Gut feel based on 17 years writing http proxies is that it > would be too slow Not really actually - so long as you can query multiple things in a single "transaction" > * complex. In order to reduce round-trips, you'd have to do many > things in a single transaction, which would create complexity issues > in the server and client But here's the rub. Yes, complex. > * protocol bloat. if you think IMAP is bloated, try HTTP. > * fundamentally an off-line style protocol for mail, which has > strong incentives to be online (esp for in-office use). Mostly facebook's "live feed" is plenty fast enough. People are doing all sorts of close-enough-to-real-time stuff on top of HTTP. It doesn't mean it's the right choice, but I wouldn't dismiss it out of hand either. > I think it would also be very problematic in the wild, since any > http intermediary would feel entitled to mess with traffic. E.g. > heuristic caching, intercepting proxies etc. That's a problem for sure. Mind you, proxyability is high on my list of goals. Immutable stuff should be cachable. > It's almost too ubiquitous. Everyone and their dog thinks they know > how http works. It's not strictly-enough defined, and has a LOT of > optional features which aren't relevant for mail (e.g. content > negotiation). Interop in http is already problematic, but start > getting people writing mail servers in php etc, and see what starts > to happen. I think it would be a nightmare. Like IMAP then, except only their dogs actually understand it ;) Good enough tests should, in theory, allow people to write their servers in PHP if they want to. Why you would choose to write in PHP, I'm not sure... but still. It takes all types. Bron. Received: from hoffman.proper.com (localhost [127.0.0.1]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q1985v1a073478 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 9 Feb 2012 01:05:57 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) Received: (from majordom@localhost) by hoffman.proper.com (8.14.5/8.13.5/Submit) id q1985v6B073477; Thu, 9 Feb 2012 01:05:57 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) X-Authentication-Warning: hoffman.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f Received: from smtp.qbik.com (smtp.qbik.com [210.55.214.35]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q1985tMY073472 for ; Thu, 9 Feb 2012 01:05:56 -0700 (MST) (envelope-from adrien@qbik.com) Received: From [192.168.1.10] (unverified [219.89.217.118]) by SMTP Server [210.55.214.35] (WinGate SMTP Receiver v7.1.0 (Build 3379)) with SMTP id <0018854642@smtp.qbik.com>; Thu, 09 Feb 2012 21:05:53 +1300 Message-ID: <4F337E61.5040702@qbik.com> Date: Thu, 09 Feb 2012 21:05:53 +1300 From: Adrien de Croy User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:11.0) Gecko/20120202 Thunderbird/11.0 MIME-Version: 1.0 To: thomas@koch.ro CC: imap5@ietf.org, Bron Gondwana , IMAP Protocol Interest List , ietf-imapext@imc.org Subject: Re: [imap5] Designing a new replacement protocol for IMAP References: <1328732126.32086.140661033971485@webmail.messagingengine.com> <201202090820.28260.thomas@koch.ro> In-Reply-To: <201202090820.28260.thomas@koch.ro> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-ietf-imapext@mail.imc.org Precedence: bulk List-Archive: List-ID: List-Unsubscribe: Whilst HTTP could certainly be used to put together an email system, I don't think it would perform well enough to replace existing protocols on desktops. The main issues I see are: * updates. Would need to poll or use long polling to get real-time updates (e.g. notification of incoming mail). * slow. Gut feel based on 17 years writing http proxies is that it would be too slow * complex. In order to reduce round-trips, you'd have to do many things in a single transaction, which would create complexity issues in the server and client * protocol bloat. if you think IMAP is bloated, try HTTP. * fundamentally an off-line style protocol for mail, which has strong incentives to be online (esp for in-office use). I think it would also be very problematic in the wild, since any http intermediary would feel entitled to mess with traffic. E.g. heuristic caching, intercepting proxies etc. It's almost too ubiquitous. Everyone and their dog thinks they know how http works. It's not strictly-enough defined, and has a LOT of optional features which aren't relevant for mail (e.g. content negotiation). Interop in http is already problematic, but start getting people writing mail servers in php etc, and see what starts to happen. I think it would be a nightmare. Adrien On 9/02/2012 8:20 p.m., Thomas Koch wrote: > Bron Gondwana: >> Snide comments from those who don't agree with the goals are welcome too... >> I just won't listen to you. These goals are very important to me. > Hi Bron, > > I'm not an expert in IMAP, just got here out of curiosity about Kolab[1]'s use > of IMAP folders as data stores. > I found some older initiatives to replace IMAP with an HTTP based approach.[2] > What do you think of HTTP for mail access? For my bachelor thesis[3] I > currently argue that CardDAV/CalDAV could be perfectly replaced by AtomPub > (RFC 5023). > The main advantage would be that most software developers on this planet have > some vague understanding of HTTP and that imense infrastructure and software > already exists. > > [1] http://wiki.kolab.org/Why_IMAP_Is_Used_For_Storage > [2] http://www.prescod.net/rest/restmail/ > https://www.ietf.org/mailman/listinfo/httpmail > [3] https://github.com/thkoch2001/bachelor-thesis > > Best regards, > > Thomas Koch, http://www.koch.ro > -- Adrien de Croy - WinGate Proxy Server - http://www.wingate.com Received: from hoffman.proper.com (localhost [127.0.0.1]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q197KllW072460 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 9 Feb 2012 00:20:47 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) Received: (from majordom@localhost) by hoffman.proper.com (8.14.5/8.13.5/Submit) id q197KlQ4072459; Thu, 9 Feb 2012 00:20:47 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) X-Authentication-Warning: hoffman.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f Received: from koch.ro (koch.ro [88.198.2.104]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q197KhB6072448 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NO) for ; Thu, 9 Feb 2012 00:20:44 -0700 (MST) (envelope-from thomas@koch.ro) Received: from 43-172.77-83.cust.bluewin.ch ([83.77.172.43] helo=t61.localnet) by koch.ro with esmtpsa (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.72) (envelope-from ) id 1RvOJ3-0002cX-0q; Thu, 09 Feb 2012 08:20:37 +0100 From: Thomas Koch Reply-To: thomas@koch.ro To: imap5@ietf.org Subject: Re: [imap5] Designing a new replacement protocol for IMAP Date: Thu, 9 Feb 2012 08:20:27 +0100 User-Agent: KMail/1.13.7 (Linux/3.1.0-1-amd64; KDE/4.6.5; x86_64; ; ) Cc: Bron Gondwana , IMAP Protocol Interest List , ietf-imapext@imc.org References: <1328732126.32086.140661033971485@webmail.messagingengine.com> In-Reply-To: <1328732126.32086.140661033971485@webmail.messagingengine.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Message-Id: <201202090820.28260.thomas@koch.ro> Sender: owner-ietf-imapext@mail.imc.org Precedence: bulk List-Archive: List-ID: List-Unsubscribe: Bron Gondwana: > Snide comments from those who don't agree with the goals are welcome too... > I just won't listen to you. These goals are very important to me. Hi Bron, I'm not an expert in IMAP, just got here out of curiosity about Kolab[1]'s use of IMAP folders as data stores. I found some older initiatives to replace IMAP with an HTTP based approach.[2] What do you think of HTTP for mail access? For my bachelor thesis[3] I currently argue that CardDAV/CalDAV could be perfectly replaced by AtomPub (RFC 5023). The main advantage would be that most software developers on this planet have some vague understanding of HTTP and that imense infrastructure and software already exists. [1] http://wiki.kolab.org/Why_IMAP_Is_Used_For_Storage [2] http://www.prescod.net/rest/restmail/ https://www.ietf.org/mailman/listinfo/httpmail [3] https://github.com/thkoch2001/bachelor-thesis Best regards, Thomas Koch, http://www.koch.ro Received: from hoffman.proper.com (localhost [127.0.0.1]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q18KIls5049373 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 8 Feb 2012 13:18:47 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) Received: (from majordom@localhost) by hoffman.proper.com (8.14.5/8.13.5/Submit) id q18KIl1O049372; Wed, 8 Feb 2012 13:18:47 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) X-Authentication-Warning: hoffman.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q18KIkIm049367 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Wed, 8 Feb 2012 13:18:46 -0700 (MST) (envelope-from gnb@fastmail.fm) Received: from compute5.internal (compute5.nyi.mail.srv.osa [10.202.2.45]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 3B22E20996 for ; Wed, 8 Feb 2012 15:18:46 -0500 (EST) Received: from frontend1.nyi.mail.srv.osa ([10.202.2.160]) by compute5.internal (MEProxy); Wed, 08 Feb 2012 15:18:46 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=fastmail.fm; h= references:message-id:from:to:in-reply-to:content-type :content-transfer-encoding:mime-version:subject:date:cc; s= mesmtp; bh=O3wS1xgz0pMNbYlOa3YJDDnfQ7M=; b=XvDjPyzvYisJLcb76Yx2t 0hBNTH7t4iS7C/gP8ncTogH38Teq1ILlQhxcrWNM4zkd2iaqBZj1+cUdijazedKM DBr+jB2fkpN5BIT78M9Jvrk+qc9If1F6nXeqHCGQsTXx3fC8a2FeDxHNGo9RbcjL Ueqy6ftdcGpVCNvCgHHpYI= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=references:message-id:from:to:in-reply-to :content-type:content-transfer-encoding:mime-version:subject :date:cc; s=smtpout; bh=O3wS1xgz0pMNbYlOa3YJDDnfQ7M=; b=WwbixYs0 JCEuBGRmY8QiGvWHyIK8GzfN8V5sKkNmP7m4pJ3adcvNfJKhpBVmngj22RDD4Xgb dt6qzmVGtRCG9D6IVd09z4nix50Hv8PBPTf+9dgshg2W+//v+repZM5S6kVLGx9R jOHM8LT+ujDA632OejlkRReBPGnGpwHfs5s= X-Sasl-enc: lnvuHRYW/RfVVKz26beJ4NXlg80aPouFDaq0tDhfviT0 1328732325 Received: from [10.0.0.1] (CPE-124-180-8-240.lns1.lon.bigpond.net.au [124.180.8.240]) by mail.messagingengine.com (Postfix) with ESMTPSA id 2919F8E0163; Wed, 8 Feb 2012 15:18:45 -0500 (EST) References: <52DA24FE-4065-4CF6-9322-96A5DDA065A9@orthanc.ca> Message-Id: From: Greg Banks To: Lyndon Nerenberg In-Reply-To: <52DA24FE-4065-4CF6-9322-96A5DDA065A9@orthanc.ca> Content-Type: text/plain; charset=us-ascii; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit X-Mailer: iPhone Mail (7C144) Mime-Version: 1.0 (iPhone Mail 7C144) Subject: Re: Metadata (5654) Implementation Limits Date: Thu, 9 Feb 2012 07:18:39 +1100 Cc: "ietf-imapext@imc.org" Sender: owner-ietf-imapext@mail.imc.org Precedence: bulk List-Archive: List-ID: List-Unsubscribe: On 09/02/2012, at 5:33, Lyndon Nerenberg wrote: > > [I didn't get much response from the imap-protocol list, so I'm > trying here, too.] > > I'm curious to know what the implementation limits are on servers > that support the Metadata extension. I would appreciate it if the > server authors out there could take a minute to fill out this survey. > > I will send a summary to the list once the responses trickle off. > (If you want your results to be anonymous, say so in the comments.) > > Thanks, > > --lyndon > > Implementation name and version: Cyrus (METADATA support rewritten and ANNOTATE support added to future 2.5 release). > Do you support /private?: Yes > Do you support /private/vendor?: Yes, with a subset of that space reserved for the server itself. Users cannot create new entries in that space, but they can create /private/ vendor/theirname/*. > Do you support unsolicited Metadata responses without values?: No > Do you support unsolicited Metadata responses with values?: No - no unsolicited responses. > Maximum number of annotations per mailbox: Not explicitly limited. > Maximum size of an : Not explicitly limited, but database keys include entry names and are limited to 4KiB. > Maximum size of a : Not explicitly limited. The old code had an undocumented limit of around 2KiB (iirc) at which it would silently truncate data in a stack buffer. The new code deals with values dynamically. There is presumably a limit to the size of the database values. For ANNOTATE support we report a 64KiB limit at SELECT time but this is a lie. > Maximum depth supported in : Not limited except by the length. > Other comments: > By default, Cyrus only accepts annotations whose entries are hardcoded or explicitly listed in a Cyrus configuration file (which is empty by default). A config option allows users to set arbitrary entries. We thought carefully about limits when implementing ANNOTATE as the per-message annotations pose a more extreme challenge. The solution we came up with was to track and limit the total size of all values of annotations on mailboxes and on messages, using a new QUOTA resource which we called X-ANNOTATION-STORAGE, with semantics similar to STORAGE. Per-server annotations are not tracked but can only be set by admin users. Greg. Received: from hoffman.proper.com (localhost [127.0.0.1]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q18KFSe5049193 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 8 Feb 2012 13:15:28 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) Received: (from majordom@localhost) by hoffman.proper.com (8.14.5/8.13.5/Submit) id q18KFSfv049192; Wed, 8 Feb 2012 13:15:28 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) X-Authentication-Warning: hoffman.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q18KFQah049186 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Wed, 8 Feb 2012 13:15:27 -0700 (MST) (envelope-from brong@fastmail.fm) Received: from compute3.internal (compute3.nyi.mail.srv.osa [10.202.2.43]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 6320E20A5C for ; Wed, 8 Feb 2012 15:15:26 -0500 (EST) Received: from web3.nyi.mail.srv.osa ([10.202.2.213]) by compute3.internal (MEProxy); Wed, 08 Feb 2012 15:15:26 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=fastmail.fm; h= message-id:from:to:mime-version:content-transfer-encoding :content-type:subject:date; s=mesmtp; bh=GeNe1ttnOq1n/NloKRSQGsm anwc=; b=ost2Ay/p0TnBqefBy2OARnPGZgJHIoX2NqHOer1xhDpmw9UkDM1kcMa YPckh5iwS/0gxaXPfISkkAiZoscPbGkEIlQvn7IIEnbTUNwk69N+xxQWSU+DyFF1 Cyi/t/PuQi+BlDGyCARq6NupzdTjMFhhMfeWL2gwOf4ODhDxGfdg= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=message-id:from:to:mime-version :content-transfer-encoding:content-type:subject:date; s=smtpout; bh=GeNe1ttnOq1n/NloKRSQGsmanwc=; b=NmbsgWtFf4UkQfk4eEij5HzSuZ1E nd/CkVAAY0z89BoHOKgeDfa/HNSG+Yy5zswK081cRGPjwb5ve7uE0KQfXocOXf7f q3Hisk6nASGegsHhfoFv3AD06szR46U/7lKnEig/9wHgoj+J7GzksWbiayNX8iSI o6rsswnmF3al8nE= Received: by web3.nyi.mail.srv.osa (Postfix, from userid 99) id 3E653400E0; Wed, 8 Feb 2012 15:15:26 -0500 (EST) Message-Id: <1328732126.32086.140661033971485@webmail.messagingengine.com> X-Sasl-Enc: CzgA3IIG77nbd4ZYPwvK6TfQVTbMqJS2ZNj6rq9hgZVQ 1328732126 From: Bron Gondwana To: IMAP Protocol Interest List , "Discussion on drastically slimming-down IMAP." , ietf-imapext@imc.org MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii" X-Mailer: MessagingEngine.com Webmail Interface Subject: Designing a new replacement protocol for IMAP Date: Wed, 08 Feb 2012 21:15:26 +0100 Sender: owner-ietf-imapext@mail.imc.org Precedence: bulk List-Archive: List-ID: List-Unsubscribe: I've given up on the IMAP5 mailing list going endlessly around in circles, though I'm definitely going to read through the backlogs and remind myself again what everyone spoke about, because there was some excellent discussion. So - I will build something and prove that it's good by showing improved user experiences. And I'm happy to keep talking here on these mailing lists too. I've started a wiki page: http://imapwiki.org/ReplacementProtocol and plan to blog about what I'm doing on our own (Opera's) blogging platform, to practice using it: http://my.opera.com/brongondwana/blog/ I plan to make on a protocol which can replace IMAP in most of its current uses. I want to get (at least) two working server implementations, and two working clients. I have at least in-principle support from enough project maintainers to do that. My strong philosophical ideals are: * Simple to build clients - morons welcome. If it's not clear the right way to do something to someone just reading a protocol dump, we haven't done a good enough job. We can't expect you to read 30 RFCs and resolve their conflicts yourself before starting coding, and we won't insult you if you didn't get it right. Where there is necessary complexity, the server authors bear the brunt. * Server side data model compatible with IMAP4 + extensions; we need to be able to co-exist. * Handle disconnection more gracefully - able to cheaply resync state (I plan to reuse the QRESYNC/CONDSTORE logic here - the model is sane). Getting reliable updates on mailbox state changes should not require a continuous connection. * Easy to proxy - regular syntax (UTF-8 for all communications where possible, but won't screw about trying to "fix" MIME - message file format will remain unchanged) * NO MAGIC - everything is explicitly requested. No UNSELECT/CLOSE/ SELECT another mailbox special behaviours, no implicit FETCH + SET \Seen, no pre-calculated "UNSEEN X" that wasn't asked for. (This probably means some way to compose actions and fetches, but it's better than being complex and magical. Remember our friends the morons.) * COMPLETE TEST SUITE that exercises every constraint in the spec, both positive and negative constraints. An implementation which passes the tests is compliant. An implementation which doesn't is not. This is much more likely to produce inter-operable implementations than a wordy spec ... ... if the authors want inter-operability that is. Nothing can protect against a deliberate embrace and extend - though negative constraint tests (MUST NOT) should help. I would love input. I spent most of the time at FOSDEM this year talking to other people involved in email about these ideas. I quite like the mailbox model behind IMAP, but I want to address the pain points I have seen so many times over the years. Snide comments from those who don't agree with the goals are welcome too... I just won't listen to you. These goals are very important to me. Regards, Bron. -- Bron Gondwana brong@fastmail.fm Received: from hoffman.proper.com (localhost [127.0.0.1]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q18IXIso043463 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 8 Feb 2012 11:33:18 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) Received: (from majordom@localhost) by hoffman.proper.com (8.14.5/8.13.5/Submit) id q18IXIhk043462; Wed, 8 Feb 2012 11:33:18 -0700 (MST) (envelope-from owner-ietf-imapext@mail.imc.org) X-Authentication-Warning: hoffman.proper.com: majordom set sender to owner-ietf-imapext@mail.imc.org using -f Received: from orthanc.ca (orthanc.ca [199.48.133.202]) by hoffman.proper.com (8.14.5/8.14.3) with ESMTP id q18IXI2Z043451 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for ; Wed, 8 Feb 2012 11:33:18 -0700 (MST) (envelope-from lyndon@orthanc.ca) Received: from [IPv6:2604:8800:137::21b:63ff:fe91:4123] ([IPv6:2604:8800:137:0:21b:63ff:fe91:4123]) (authenticated bits=0) by orthanc.ca (8.14.5/8.14.5) with ESMTP id q18IXEun017438 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO) for ; Wed, 8 Feb 2012 10:33:16 -0800 (PST) (envelope-from lyndon@orthanc.ca) From: Lyndon Nerenberg Content-Type: text/plain; charset=us-ascii Subject: Metadata (5654) Implementation Limits Date: Wed, 8 Feb 2012 10:33:13 -0800 Message-Id: <52DA24FE-4065-4CF6-9322-96A5DDA065A9@orthanc.ca> To: ietf-imapext@imc.org Mime-Version: 1.0 (Apple Message framework v1084) X-Mailer: Apple Mail (2.1084) Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by hoffman.proper.com id q18IXI2Y043457 Sender: owner-ietf-imapext@mail.imc.org Precedence: bulk List-Archive: List-ID: List-Unsubscribe: [I didn't get much response from the imap-protocol list, so I'm trying here, too.] I'm curious to know what the implementation limits are on servers that support the Metadata extension. I would appreciate it if the server authors out there could take a minute to fill out this survey. I will send a summary to the list once the responses trickle off. (If you want your results to be anonymous, say so in the comments.) Thanks, --lyndon Implementation name and version: Do you support /private?: Do you support /private/vendor?: Do you support unsolicited Metadata responses without values?: Do you support unsolicited Metadata responses with values?: Maximum number of annotations per mailbox: Maximum size of an : Maximum size of a : Maximum depth supported in : Other comments: