From rg+ietf@qualcomm.com  Tue Apr  7 16:03:55 2009
Return-Path: <rg+ietf@qualcomm.com>
X-Original-To: morg@core3.amsl.com
Delivered-To: morg@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id B3C3128C10E for <morg@core3.amsl.com>; Tue,  7 Apr 2009 16:03:55 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -104.964
X-Spam-Level: 
X-Spam-Status: No, score=-104.964 tagged_above=-999 required=5 tests=[AWL=-0.423, BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_64=0.6, MIME_HTML_ONLY=1.457, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hR0wltG3lHqH for <morg@core3.amsl.com>; Tue,  7 Apr 2009 16:03:54 -0700 (PDT)
Received: from wolverine01.qualcomm.com (wolverine01.qualcomm.com [199.106.114.254]) by core3.amsl.com (Postfix) with ESMTP id 58FCA3A6A6D for <morg@ietf.org>; Tue,  7 Apr 2009 16:03:54 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=qualcomm.com; i=rg+ietf@qualcomm.com; q=dns/txt; s=qcdkim; t=1239145501; x=1270681501; h=message-id:x-mailer:x-message-note:date:to:from:subject: content-type:x-random-sig-tag:x-ironport-av; z=Message-Id:=20<p0624060ac6018b9b2818@[192.168.99.12]> |X-Mailer:=20Eudora=20for=20Mac=20OS=20X|X-message-Note: =20Warning:=20Outlook=20in=20use.=20=20Upgrade=20to=20Eud ora:=20<http://www.eudora.com>|Date:=20Tue,=207=20Apr=202 009=2016:01:18=20-0700|To:=20morg@ietf.org|From:=20Randal l=20Gellens=20<rg+ietf@qualcomm.com>|Subject:=20MORG=20no tes=20from=20IETF=2074|Content-Type:=20text/html=3B=20cha rset=3D"us-ascii"|X-Random-Sig-Tag:=201.0b28 |X-IronPort-AV:=20E=3DMcAfee=3Bi=3D"5300,2777,5577"=3B=20 a=3D"16965913"; bh=iP15tVJ0XEqoOuOdg2FoFsQxJ59GKvt77G0AnCrLR44=; b=FqCZrfodngFG4twvMmxDdb988I9HyKZ+xkid7NstohUyBhh5rTIxB9A3 f69ocyMiYovxHtKzqThP+Z7WDDSC7ftfvdB9QDk2TugpSBexsCPcdy1Li yTJ4QYtF1WeAwFBUPxrySbDu6TZeIHTh6RyLVBGzKqh8KcjKAmkHsJ5Y+ Y=;
X-IronPort-AV: E=McAfee;i="5300,2777,5577"; a="16965913"
Received: from pdmz-ns-mip.qualcomm.com (HELO ithilien.qualcomm.com) ([199.106.114.10]) by wolverine01.qualcomm.com with ESMTP/TLS/DHE-RSA-AES256-SHA; 07 Apr 2009 16:05:01 -0700
Received: from msgtransport04.qualcomm.com (msgtransport04.qualcomm.com [129.46.61.156]) by ithilien.qualcomm.com (8.14.2/8.14.2/1.0) with ESMTP id n37N50QY023451 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for <morg@ietf.org>; Tue, 7 Apr 2009 16:05:00 -0700
Received: from [192.168.99.12] ([10.64.0.118]) by msgtransport04.qualcomm.com (8.14.2/8.14.2/1.0) with ESMTP id n37N4wg8007544 for <morg@ietf.org>; Tue, 7 Apr 2009 16:04:59 -0700
Mime-Version: 1.0
Message-Id: <p0624060ac6018b9b2818@[192.168.99.12]>
X-Mailer: Eudora for Mac OS X
X-message-Note: Warning: Outlook in use. Upgrade to Eudora: <http://www.eudora.com>
Date: Tue, 7 Apr 2009 16:01:18 -0700
To: morg@ietf.org
From: Randall Gellens <rg+ietf@qualcomm.com>
Mime-Version: 1.0
Content-Type: text/html; charset="us-ascii"
X-Random-Sig-Tag: 1.0b28
Subject: [MORG] MORG notes from IETF 74
X-BeenThere: morg@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: Messaging Organization <morg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/morg>, <mailto:morg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/morg>
List-Post: <mailto:morg@ietf.org>
List-Help: <mailto:morg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/morg>, <mailto:morg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 07 Apr 2009 23:03:55 -0000

<!doctype html public "-//W3C//DTD W3 HTML//EN">
<html><head><style type="text/css"><!--
blockquote, dl, ul, ol, li { padding-top: 0 ; padding-bottom: 0 }
 --></style><title>MORG notes from IETF 74</title></head><body>
<div>Here are my own notes from the MORG session.</div>
<div><br></div>
<ul>
<li><u>STATUS in LIST</u></ul>
<blockquote>
<ul>
<li>Barry asks if there is real demand for this, specifically, who
will implement it?&nbsp; Four server people in the room say they will,
some client people (one via Jabber) say they will.
<li>Discussion regarding failure case (server can't return status for
some requested mailboxes).
<li>Barry, Chris, and Alexey prefer choice #2.</ul>
</blockquote>
<div><u><br></u></div>
<ul>
<li><u>Sort display</u></ul>
<blockquote>
<ul>
<li>Open issue: Should there be a DISPLAYTO sort-key and how should it
work?
<li>Alexey wants it, e.g., in a &quot;sent items&quot; folder where
everything is from the user so sorting by &quot;From&quot; is
meaningless, but sorting by &quot;To&quot; is useful.</ul>
</blockquote>
<div><u><br></u></div>
<ul>
<li><u>INTHREAD</u></ul>
<blockquote>
<ul>
<li>Open issue: How to encode Message-ID?
<li>Discussion if this needs to be a WG item; usability of extension.
<li>Barry says this is clearly usable, but questions as to who will
implement.&nbsp; Only two of the server vendors present say they will
or have implemented, no client people say so.
<li>Discussion on THREAD=REFS compared to rest.&nbsp; Barry questions
need for in-thread SEARCH.
<li>Cyrus notes that this is most useful for webmail clients, most of
which use c-client as base, so if c-client won't implement, then this
won't be used.
<li>Chris notes that he has a webmail implementation that doesn't use
c-client.
<li>Status: open question as to real need for this; Randy suggests
leaving draft as low-priority until there's demonstrated need for it.
<li>Alexey notes that first part of this extension adds new basic IMAP
syntax, which breaks many generic IMAP parsers (which tokenize into
strings, atoms, etc.); Alexey suggests using STRING.
<li>Barry agrees and says that Message-ID should be treated as a blob
and not parsed.
<li>Consensus in room to go with last option on slide</ul>
</blockquote>
<div><u><br></u></div>
<ul>
<li><u>Address Search</u></ul>
<blockquote>
<ul>
<li><u>ANY field</u> (list of many headers to search) -- should server
be allowed to search additional headers?
<li>Alexey suggests MUST search listed fields, MAY search others.
<li>Discussion as to possible user confusion if server says there is a
match but matching header isn't clear
<li>Barry and Arnt prefer option 1A (just RFC5322).
<li>I suggest that there isn't an interoperability harm to server
searching extra headers.
<li>Discussion of headers such as List-Unsubscribe: nonsense if this
is searched.
<li>Barry suggests 5322 only or 5322+server choice.
<li>Cyrus suggests we be very explicit here, and do the fuzzy search
in fuzzy search extension.
<li><u>Consensus</u> for this.
<li>Discussion on searching arbitrary headers.&nbsp; Some support for
this.
<li>Question on searching address headers in<u> nested body parts</u>.
<li>Some agreement that this is useful (e.g., forwarded messages,
multipart/digest) .
<li>Discussion on server MAY or MUST, and on client option.
<li>Tony likes having a client option.
<li>I suggest that a client option is the most complex choice, since
the server MUST then implement everything, and with conditional code
paths, and further the client needs to choose, but without any clear
basis for doing so.
<li>Cyrus wonders what the user interface would look like to control
this.
<li>Cyrus suggests making this extension search top-level only, and
doing a new extension for nested body parts.
<li><u>Consensus</u> call to go with this choice.
<li>Open issue: fallback or not.
<li>Suggestion to leave it up to editor.&nbsp; No one cares much.</ul>
</blockquote>
<div><u><br></u></div>
<ul>
<li><u>Fuzzy Search</u></ul>
<blockquote>
<ul>
<li>A few people had read the draft (posted a few days ago).
<li>Question on handling of<u> non-string matching data</u>.
<li>Barry says fuzzy search means it is up to the server, so this
choice is left up to the server.
<li>Alexey asks about<u> fuzzy date searching</u>. (Looking for
&quot;recent&quot; message, time zone boundaries, etc.)
<li>Consensus that fuzzy means server chooses.
<li>Alexey suggests draft include examples of this to make it more
clear.
<li>Randy suggests that if we do fuzzy, it means searcher choice, and
if we want to specify everything then it isn't fuzzy.
<li><u>Consensus</u> to move forward&nbsp; with fuzzy search.
<li>Chris reiterates need for lots of examples and informative text
regarding what's helpful and what's not.
<li>Question on<u> returning relevancy scores</u>.&nbsp; Is this
useful?
<li>Barry suggests standardizing the range (0-100).
<li>Chris says we have some experience that scores in the 0-100 range
are useful; displayed as percentage; users seem to handle it.&nbsp;
Google, e.g., shows scores.
<li><u>Consensus</u> that servers return relevancy scores (probably
0-100) and the document needs text to say that scores from different
servers can't be correlated.
<li>Barry and Cryus note that server can choose how to normalize
internal values.
<li>Question on returning relevancy without fuzzy search.&nbsp; No one
does.</ul>
</blockquote>
<div><br></div>
<ul>
<li>Collations/Comparators</ul>
<blockquote>
<ul>
<li>Ned proposed space-ignore
<li>proposal for non-case-mapping unicode comparitor
<li>Alexey added two numeric comparators: signed numeric and ignore
leading white space.
<li>Barry asks about one that ignores separators, and one that
recognizes decimals (including national differences).
<li>I note that even if the user knows what he uses, he doesn't know
what the author of an email may have used, so will need to search all
possibilities (US, European, etc.)
<li>Chris asks which server implementers want to do this.&nbsp; Three
raise their hands.&nbsp; Chris asks for client people.&nbsp; No one
raises their hands.
<li>Zoltan says most useful search is phone numbers, and this doesn't
do that.
<li>Discussion that this is more useful in Sieve (which doesn't
currently permit signed numbers).
<li>No enthusiasm for<u> i-ascii-signed-numeric</u> but Alexey
suggests leaving it for now and we can discuss on list and abandon
later.
<li><u>i-ascii-punc-ignore-numeric</u>
<li>Cyrus asks if digits on either side of comma followed by
whitespace should be treated as one number or two.
<li>Chris says we should look at use cases first, and only if we have
a clear use case that needs a comparator do we create one; this is
creating comparators first.
<li>Ned's proposed space ignoring comparator (e.g., Japanese texts).
<li>Case-sensitive version of i;unicode-casemap -- Barry and Pete
suggest we don't know what this means.
<li><u>Consensus</u> that only space-ignore is clearly needed.&nbsp;
Chris or Ned will take on editorship.</ul>
</blockquote>
<div><br></div>
<ul>
<li>Multi-Mailbox SEARCH</ul>
<blockquote>
<ul>
<li>Can't pipeline SEARCH since untagged search responses are
indistinguishable.
<li>Proposed extension tags search results, runs in authenticated,
non-selected state, always returns UIDs.
<li>Options for wildcards and depth limit; allows for future extension
for result option; one ESEARCH response per mailbox with hits.
<li>Discussion on new CAPABILITY for server to report that it executes
pipelined commands in parallel.
<li>Discussion on<u> wildcards</u>.&nbsp; Use LIST syntax or NOTIFY
syntax?&nbsp; NOTIFY uses subtree.&nbsp; Several people voice support
for this, no one argues against it.
<li>Discussion on<u> depth</u> limit.&nbsp; What should be used by
default?&nbsp; Should default be specified or left to server?&nbsp;
Support for saying client MUST specify depth.
<li>Barry notes that some IMAP implementations can have loops in
hierarchy, and that some implementations loop infinitely on SEARCH.&nbsp;
Chris suggests making depth 0, 1, or infinitely, and adding text that
infinity means that the server searches each mailbox once.
<li>Discussion on security considerations.&nbsp; Barry notes ACL.&nbsp;
Chris suggests shared mailboxes could be used for force spam into
search results by owner of shared mailbox.&nbsp; Chris wonders about a
parameter that says search only user's own mailboxes (no shared).&nbsp;
Cyrus says you can use metadata to tag mailboxes you don't want to
search, and use search criteria to exclude them.
<li>Suggestions to add EXCLUDE clause (with a nested search criteria).
<li>Suggestions to use metadata tags to mark those mailboxes that you
want to search (to not</ul>
</blockquote>
<div><br></div>
<ul>
<li>VIEW vs Multi-Mailbox Search</ul>
<blockquote>
<ul>
<li>Timo asks if anyone besides him is interested in VIEW.
<li>Barry says VIEW was attractive but a big rat-hole.
<li>Much debate regarding how to return or limit large results.</ul>
</blockquote>
<div><br></div>
<x-sigsep><pre>-- 
</pre></x-sigsep>
<div><font color="#000000">Randall Gellens<br>
Opinions are personal;&nbsp;&nbsp;&nbsp; facts are
suspect;&nbsp;&nbsp;&nbsp; I speak for myself only<br>
-------------- Randomly selected tag: ---------------<br>
The whole problem with the world is that fools and fanatics are always<br>
so certain of themselves, but wiser people so full of doubts.<br>
 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;--Bertrand Russell<br>
</font></div></body>
</html>

From arnt@oryx.com  Wed Apr  8 05:37:31 2009
Return-Path: <arnt@oryx.com>
X-Original-To: morg@core3.amsl.com
Delivered-To: morg@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id CAB6228C169 for <morg@core3.amsl.com>; Wed,  8 Apr 2009 05:37:31 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.686
X-Spam-Level: 
X-Spam-Status: No, score=-1.686 tagged_above=-999 required=5 tests=[AWL=-0.947, BAYES_20=-0.74, UNPARSEABLE_RELAY=0.001]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id F1wL8FEGsF2H for <morg@core3.amsl.com>; Wed,  8 Apr 2009 05:37:31 -0700 (PDT)
Received: from kalyani.oryx.com (kalyani.oryx.com [195.30.37.30]) by core3.amsl.com (Postfix) with ESMTP id 3E8FD3A6AB3 for <morg@ietf.org>; Wed,  8 Apr 2009 05:37:30 -0700 (PDT)
Received: from kalyani.oryx.com (localhost [127.0.0.1]) by kalyani.oryx.com (Postfix) with ESMTP id 727332E0EC; Wed,  8 Apr 2009 14:38:33 +0200 (CEST)
Received: from arnt@oryx.com (HELO lochnagar) by kalyani.oryx.com (Archiveopteryx 3.1.0) with esmtp id 1239194313-53296-53284/6/23 for morg@ietf.org; Wed, 8 Apr 2009 14:38:33 +0200
Message-Id: <UDJylvAUqX5dqLKmB+C1Qg.md5@lochnagar>
Date: Wed, 8 Apr 2009 14:38:54 +0200
From: Arnt Gulbrandsen <arnt@oryx.com>
To: morg@ietf.org
References: <1234565453.6132.886.camel@timo-desktop> <E8EBD6F423B52FD02CD04B8D@socrates.local> <5lt3qVJvy+jaTjngz/M+Fg.md5@lochnagar.oryx.com> <alpine.OSX.2.00.0903260758350.28287@hsinghsing.panda.com> <nhZ/ucOkEVUShLAg2vFAeQ.md5@lochnagar.oryx.com> <alpine.OSX.2.00.0903260837300.28287@hsinghsing.panda.com> <6c9fcc2a0903260917o8d5c3c1p86230e23c67ed854@mail.gmail.com> <xUbz6YGmIraZ0Kii/x30qQ.md5@lochnagar.oryx.com> <alpine.OSX.2.00.0903260937000.28287@hsinghsing.panda.com> <i7Ov0Sh7CnXn1spKeiryJQ.md5@lochnagar.oryx.com> <alpine.OSX.2.00.0903260945070.28287@hsinghsing.panda.com> <74E86DC1-440F-42D2-BB42-FD2291F51672@mumbo.ca> <P+hWnMz5o6F5yG+pRevlCQ.md5@lochnagar.oryx.com>
In-Reply-To: <P+hWnMz5o6F5yG+pRevlCQ.md5@lochnagar.oryx.com>
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset=iso-8859-1; format=flowed
Mime-Version: 1.0
Subject: [MORG] Why IMAP extensions are (not) used
X-BeenThere: morg@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: Messaging Organization <morg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/morg>, <mailto:morg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/morg>
List-Post: <mailto:morg@ietf.org>
List-Help: <mailto:morg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/morg>, <mailto:morg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 08 Apr 2009 12:37:31 -0000

Arnt Gulbrandsen writes:
> Curtis King writes:
>> I'm sure there are some more clients which use a few more extensions,=20
>> but that still means ~16 extensions we (Isode) have implemented are=20
>> not being used. Not a very good ratio. Yet, here we are planning on=20
>> adding more.
>
> A bit of discussion of the reasons seems like a good idea. I have=20
> opinions but it's late and I'm too tired now. I'll try to post=20
> something during the coming week.

Here goes. I'm sure someone will tell me I'm all wrong. Go on, but=20
please do say something about what's right too.

I believe that IMAP extensions aren't used as much as they should be for=20
a combination of five reasons. I believe that all five are factors in=20
every case, but the weight varies widely.

Further, I believe that all five can be mitigated, in some cases quite=20
easily. Mitigating is not the same as eliminating, and of course=20
mitigating the known problems is enough to ensure takeup of a=20
particular new extension.

1. RFCs written with the wrong audience in mind.

Many of the RFCs don't make much effort to serve client authors or=20
answer their questions, such as =ABhow should I use this?=BB, =ABhow =
should I=20
fall back when it isn't supported?=BB. Some RFCs have few examples, some=20
have many, but =ABmany=BB is not the same as =ABuseful for a particular=20
audience=BB.

(Without making a detailed survey, I'd guess that the ones with many are=20
fairly good for server implementers, less so for clients. Server=20
implementers need more text and examples about corner cases, clients=20
more that's related to their use cases.)

2. RFCs can be hard to find.

We don't have a list mapping from problem/benefit to the relevant=20
extension(s), and googling doesn't solve that problem well.

3. What if/ifnot.

Clients generally need to implement stuff both for servers that support=20
a given extensions, and for servers that don't. We haven't given much=20
thought to minimising the duplicated code paths that result, or do it=20
cleanly.

(I don't mean textually this time, I mean functionally.)

4. Uncertainty about compliant servers.

It's needlessly difficult for a client to find out which servers support=20
something, which makes it harder to test with/without a particular=20
extension than it should be, and also harder to know whether=20
implementing an extension will have a benefit in the wild.

Another aspect of this is interoperation.

5. Combinatorial behaviour.

Several non-IETF people I've spoken to complain about the number of=20
combinations.

The number of combinations is effectively higher than the number of=20
extensions, since some extensions interact in tricky ways. This strikes=20
me as wrong, and most of the interactions are IMO just neat-to-have.

It could be lower in practice, since a client can always do
   if ( server supports a, b and c)
      use all three
   else
      use none

We could try harder to avoid the interactions, and perhaps occasionally=20
call attention to the a && b && c trick. (That trick also needs=20
information about how many servers implement a and b but not c, etc.)

Arnt

(PS: I've tried to avoid digression in this little missive, as I think=20
we would otherwise get bogged down the third sentence of RFC x section=20
y.x, instead of why IMAP discussions aren't used as much as we'd like=20
and how MORG can have a good fate.)

From markrcrispin@panda.com  Wed Apr  8 12:24:31 2009
Return-Path: <markrcrispin@panda.com>
X-Original-To: morg@core3.amsl.com
Delivered-To: morg@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 87A4C3A69C2 for <morg@core3.amsl.com>; Wed,  8 Apr 2009 12:24:31 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.669
X-Spam-Level: 
X-Spam-Status: No, score=-1.669 tagged_above=-999 required=5 tests=[AWL=-0.929, BAYES_20=-0.74]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SHn65k7NxGu8 for <morg@core3.amsl.com>; Wed,  8 Apr 2009 12:24:30 -0700 (PDT)
Received: from Panda.COM (panda.com [206.124.149.114]) by core3.amsl.com (Postfix) with ESMTP id 4DD2F3A6A5C for <morg@ietf.org>; Wed,  8 Apr 2009 12:24:30 -0700 (PDT)
Received: from hsinghsing.panda.com markrcrispin@Panda.COM [206.124.149.116] by Panda.COM with M+ Extreme Email Engine 2008.3.internal via secured & encrypted transport (TLS); Wed, 08 Apr 2009 12:25:36 -0700
X-MailFrom: markrcrispin@panda.com
Date: Wed, 8 Apr 2009 12:25:35 -0700 (PDT)
From: Mark Crispin <markrcrispin@panda.com>
Sender: root@hsinghsing.panda.com
To: morg@ietf.org
In-Reply-To: <UDJylvAUqX5dqLKmB+C1Qg.md5@lochnagar>
Message-ID: <alpine.OSX.2.00.0904081042310.1327@hsinghsing.panda.com>
References: <1234565453.6132.886.camel@timo-desktop> <E8EBD6F423B52FD02CD04B8D@socrates.local> <5lt3qVJvy+jaTjngz/M+Fg.md5@lochnagar.oryx.com> <alpine.OSX.2.00.0903260758350.28287@hsinghsing.panda.com> <nhZ/ucOkEVUShLAg2vFAeQ.md5@lochnagar.oryx.com> <alpine.OSX.2.00.0903260837300.28287@hsinghsing.panda.com> <6c9fcc2a0903260917o8d5c3c1p86230e23c67ed854@mail.gmail.com> <xUbz6YGmIraZ0Kii/x30qQ.md5@lochnagar.oryx.com> <alpine.OSX.2.00.0903260937000.28287@hsinghsing.panda.com> <i7Ov0Sh7CnXn1spKeiryJQ.md5@lochnagar.oryx.com> <alpine.OSX.2.00.0903260945070.28287@hsinghsing.panda.com> <74E86DC1-440F-42D2-BB42-FD2291F51672@mumbo.ca> <P+hWnMz5o6F5yG+pRevlCQ.md5@lochnagar.oryx.com> <UDJylvAUqX5dqLKmB+C1Qg.md5@lochnagar>
User-Agent: Alpine 2.00 (OSX 1167 2008-08-23)
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII
Subject: Re: [MORG] Why IMAP extensions are (not) used
X-BeenThere: morg@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: Messaging Organization <morg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/morg>, <mailto:morg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/morg>
List-Post: <mailto:morg@ietf.org>
List-Help: <mailto:morg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/morg>, <mailto:morg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 08 Apr 2009 19:24:31 -0000

IMAP extensions are not used much for the following reasons:

[1] Almost all IMAP extensions are worthless garbage.

Oh, they may be valuable to some limited constituency, but not generally.

Some servers may do UNSELECT or NAMESPACE or CHILDREN because they are 
low-hanging fruit, or IDLE for mobile devices.  It is nearly impossible to 
make a business case for the rest, especially the recent batch.


[2] Non-compliant servers destroy what limited value there is to an 
extension.

Any client developer who depends upon an extension quickly discovers that 
the client fails when talking to some server.  No amount of "it's the 
server's fault" will help; the customer will invariably say "it works with 
Outlook and Thunderbird, so it MUST be a bug in your client."

To some extent, the customer is right; the client author depended upon an 
extension.  That is a bug.

Courier is particularly ruthless in destroying by mis-implementation.


[3] Combinatorial behavior.

This is more a matter of perception than a real technical problem.

Nonetheless the people who do QA and regression testing are alarmed by the 
2^n combinations that need testing.

You should be too.


Last but not least...

[4] The refusal to allow IMAP Extensions to die.

IMAPEXT was dragged out long beyond the point where it should have been 
killed.

When IMAPEXT was finally killed, IMAP Extensions moved to LEMONADE. 
LEMONADE completely missed its window of opportunity, due in part to 
everybody sneaking in their favorite IMAP extension as a LEMONADE 
requirement.  Deep down inside, they knew that this extension didn't stand 
a snowball's chance in hell of any market penetration.

Now that LEMONADE is dying, along comes MORG which is simply a new name 
for IMAPEXT

Slay this dragon.   By whatever means necessary.  Do it now.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

From barryleiba.mailing.lists@gmail.com  Wed Apr  8 12:28:12 2009
Return-Path: <barryleiba.mailing.lists@gmail.com>
X-Original-To: morg@core3.amsl.com
Delivered-To: morg@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 5AF673A6900 for <morg@core3.amsl.com>; Wed,  8 Apr 2009 12:28:12 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.172
X-Spam-Level: 
X-Spam-Status: No, score=-2.172 tagged_above=-999 required=5 tests=[AWL=-0.195, BAYES_00=-2.599, FM_FORGED_GMAIL=0.622]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xJ8q2mZB7iBg for <morg@core3.amsl.com>; Wed,  8 Apr 2009 12:28:11 -0700 (PDT)
Received: from mail-bw0-f169.google.com (mail-bw0-f169.google.com [209.85.218.169]) by core3.amsl.com (Postfix) with ESMTP id 189903A697F for <morg@ietf.org>; Wed,  8 Apr 2009 12:28:10 -0700 (PDT)
Received: by bwz17 with SMTP id 17so280691bwz.37 for <morg@ietf.org>; Wed, 08 Apr 2009 12:29:17 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type:content-transfer-encoding; bh=1M1KJ/V/zpNQtUCtiCDp4XWPs8rqpgphfbLihdGdLyc=; b=mKOQWo0heHLarnb/qO2QkRARuQ27Dlgiei0tSV2dK0EaSvZ+3PtFdKwXL2j/ID+JIp Q5JCodi19CvZ0LOjRssuGrnqj1kJW1EysYG9HACaNZKXM9q3WUUbipn+F1eqf7T1k5Jq xmlLZ6Smnp2vLK1xBWmb5xOUdUOzEsu2eFBDE=
DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=dcP/u9Wzw3zPYacdd7jb/hSnMFY0Em0mGUdE8OgOOCEXZ0hE2uCrLI/olHWolB42Yq QzzBZj/yLxM4Iyfd5SATsUqHyfB2Hrr76YqbhTn15alRNNaA9RggeJWCrDBFh54LeRqP cTfuIfGDK9kJufQRDfgqy2M9MUdy+0MVmjorg=
MIME-Version: 1.0
Sender: barryleiba.mailing.lists@gmail.com
Received: by 10.223.119.84 with SMTP id y20mr521929faq.14.1239218957382; Wed,  08 Apr 2009 12:29:17 -0700 (PDT)
In-Reply-To: <UDJylvAUqX5dqLKmB+C1Qg.md5@lochnagar>
References: <1234565453.6132.886.camel@timo-desktop> <alpine.OSX.2.00.0903260837300.28287@hsinghsing.panda.com> <6c9fcc2a0903260917o8d5c3c1p86230e23c67ed854@mail.gmail.com> <xUbz6YGmIraZ0Kii/x30qQ.md5@lochnagar.oryx.com> <alpine.OSX.2.00.0903260937000.28287@hsinghsing.panda.com> <i7Ov0Sh7CnXn1spKeiryJQ.md5@lochnagar.oryx.com> <alpine.OSX.2.00.0903260945070.28287@hsinghsing.panda.com> <74E86DC1-440F-42D2-BB42-FD2291F51672@mumbo.ca> <P+hWnMz5o6F5yG+pRevlCQ.md5@lochnagar.oryx.com> <UDJylvAUqX5dqLKmB+C1Qg.md5@lochnagar>
Date: Wed, 8 Apr 2009 15:29:17 -0400
X-Google-Sender-Auth: 400222296091c59f
Message-ID: <6c9fcc2a0904081229k11028061x67ef997fd33ba4ba@mail.gmail.com>
From: Barry Leiba <barryleiba@computer.org>
To: Arnt Gulbrandsen <arnt@oryx.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Cc: morg@ietf.org
Subject: Re: [MORG] Why IMAP extensions are (not) used
X-BeenThere: morg@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: Messaging Organization <morg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/morg>, <mailto:morg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/morg>
List-Post: <mailto:morg@ietf.org>
List-Help: <mailto:morg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/morg>, <mailto:morg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 08 Apr 2009 19:28:12 -0000

> 1. RFCs written with the wrong audience in mind.
>
> Many of the RFCs don't make much effort to serve client authors or answer
> their questions, such as =ABhow should I use this?=BB, =ABhow should I fa=
ll back
> when it isn't supported?=BB. Some RFCs have few examples, some have many,=
 but
> =ABmany=BB is not the same as =ABuseful for a particular audience=BB.

I suspect the number of examples is not a factor at all in decisions
whether to implement an extension or not.

In fact, I suspect that how the RFC is written is, in general, a minor poin=
t.

> 2. RFCs can be hard to find.
>
> We don't have a list mapping from problem/benefit to the relevant
> extension(s), and googling doesn't solve that problem well.

This amounts to "publicity", and, surely, the most implemented ones
may correlate with the most publicized ones.  Alexey once has a web
page with a list of which implementations support which extensions.
That also addresses your point 4.

> 3. What if/ifnot.
>
> Clients generally need to implement stuff both for servers that support a
> given extensions, and for servers that don't. We haven't given much thoug=
ht
> to minimising the duplicated code paths that result, or do it cleanly.

This is really the nut of it, I think.
Servers will implement an extension if (1) it's so easy that they
might as well, or (2) they perceive a demand for it (from clients or
from customers -- the latter is sometimes just a box they have to
tick... yes, we support that).

Clients will implement an extension if (1) it really saves them a lot
of programming work, (2) it really saves a lot of performance, (3) it
provides a feature that they can't get without it, or (4) they
perceive a demand for it (see above).

Your point 3 shows that (1) is basically not possible the way
extensions are implemented in IMAP.  For POP, the TOP and UIDP
extensions, for example, are essentially required.  Any server that
doesn't support them will fail, and clients regularly just assume
they're available and refuse to work with servers that don't have
them.  So it's easy.

Lacking that, though, client developers have to do *extra* programming
to support extensions, because things have to work both ways, as you
say.

(2) mostly coves things like searching, sorting, and threading on the
server, and a few things that save a lot of bandwidth or round trips
for situations with slow connections or ones with high latency.

(3) mostly covers IDLE.  Pretty much anything else can be done another way.


What it comes down to is that (1), above, is the critical point.  As
long as the extensions are "extensions", sparsely (and perhaps
buggily) supported, things will be difficult.

There was a small effort started to trim things from IMAP that might
be better done in other ways (like folders) and to fold in the
"important" extensions to a new base.  The effort was called "IMAP5",
and it seems to have died down.

Barry

From arnt@oryx.com  Wed Apr  8 13:41:17 2009
Return-Path: <arnt@oryx.com>
X-Original-To: morg@core3.amsl.com
Delivered-To: morg@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 8B4DE3A6A2D for <morg@core3.amsl.com>; Wed,  8 Apr 2009 13:41:17 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.593
X-Spam-Level: 
X-Spam-Status: No, score=-2.593 tagged_above=-999 required=5 tests=[AWL=0.005,  BAYES_00=-2.599, UNPARSEABLE_RELAY=0.001]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7A-84L7wsLPs for <morg@core3.amsl.com>; Wed,  8 Apr 2009 13:41:16 -0700 (PDT)
Received: from kalyani.oryx.com (kalyani.oryx.com [195.30.37.30]) by core3.amsl.com (Postfix) with ESMTP id BBC383A689E for <morg@ietf.org>; Wed,  8 Apr 2009 13:41:14 -0700 (PDT)
Received: from kalyani.oryx.com (localhost [127.0.0.1]) by kalyani.oryx.com (Postfix) with ESMTP id 4EE432E0F7; Wed,  8 Apr 2009 22:42:18 +0200 (CEST)
Received: from arnt@oryx.com (HELO lochnagar) by kalyani.oryx.com (Archiveopteryx 3.1.0) with esmtp id 1239223338-53296-53284/6/30 (3 recipients); Wed, 8 Apr 2009 22:42:18 +0200
Message-Id: <W7El7F9shiifFw/D6B5PKw.md5@lochnagar>
Date: Wed, 8 Apr 2009 22:42:39 +0200
From: Arnt Gulbrandsen <arnt@oryx.com>
To: Barry Leiba <barryleiba@computer.org>
References: <1234565453.6132.886.camel@timo-desktop> <alpine.OSX.2.00.0903260837300.28287@hsinghsing.panda.com> <6c9fcc2a0903260917o8d5c3c1p86230e23c67ed854@mail.gmail.com> <xUbz6YGmIraZ0Kii/x30qQ.md5@lochnagar.oryx.com> <alpine.OSX.2.00.0903260937000.28287@hsinghsing.panda.com> <i7Ov0Sh7CnXn1spKeiryJQ.md5@lochnagar.oryx.com> <alpine.OSX.2.00.0903260945070.28287@hsinghsing.panda.com> <74E86DC1-440F-42D2-BB42-FD2291F51672@mumbo.ca> <P+hWnMz5o6F5yG+pRevlCQ.md5@lochnagar.oryx.com> <UDJylvAUqX5dqLKmB+C1Qg.md5@lochnagar> <6c9fcc2a0904081229k11028061x67ef997fd33ba4ba@mail.gmail.com>
In-Reply-To: <6c9fcc2a0904081229k11028061x67ef997fd33ba4ba@mail.gmail.com>
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset=iso-8859-1; format=flowed
Mime-Version: 1.0
Cc: morg@ietf.org
Subject: Re: [MORG] Why IMAP extensions are (not) used
X-BeenThere: morg@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: Messaging Organization <morg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/morg>, <mailto:morg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/morg>
List-Post: <mailto:morg@ietf.org>
List-Help: <mailto:morg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/morg>, <mailto:morg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 08 Apr 2009 20:41:17 -0000

Thank you for this very good response.

Barry Leiba writes:
>>  1. RFCs written with the wrong audience in mind.
>>
>>  Many of the RFCs don't make much effort to serve client authors or=20
>>  answer their questions, such as =ABhow should I use this?=BB, =ABhow=20
>>  should I fall back when it isn't supported?=BB. Some RFCs have few=20
>>  examples, some have many, but =ABmany=BB is not the same as =ABuseful=
 for=20
>>  a particular audience=BB.
>
> I suspect the number of examples is not a factor at all in decisions=20
> whether to implement an extension or not.
>
> In fact, I suspect that how the RFC is written is, in general, a minor =
point.

Maybe you're right. But I know two cases where a perceived lack of=20
readability/blah delayed implementation for literally years, and my=20
part of the world isn't very big.

(And yes, I'm as guilty as anyone else.)

>>  2. RFCs can be hard to find.
>>
>>  We don't have a list mapping from problem/benefit to the relevant=20
>>  extension(s), and googling doesn't solve that problem well.
>
> This amounts to "publicity", and, surely, the most implemented ones=20
> may correlate with the most publicized ones. Alexey once has a web=20
> page with a list of which implementations support which extensions.=20
> That also addresses your point 4.

I know the page and think not, but it's close-ish. This would be more=20
helpful, e.g.:

   [ extension name ]
      known supporting servers: [ list ]
      known unsupporting: [ list ]
      test accounts/downloads with: [ list ]

>> 3. What if/ifnot.
>>
>>  Clients generally need to implement stuff both for servers that=20
>>  support a given extensions, and for servers that don't. We haven't=20
>>  given much thought to minimising the duplicated code paths that=20
>>  result, or do it cleanly.
>
> This is really the nut of it, I think.
> Servers will implement an extension if (1) it's so easy that they
> might as well, or (2) they perceive a demand for it (from clients or
> from customers -- the latter is sometimes just a box they have to
> tick... yes, we support that).

or, pedantically, (3) they need the functionality and providing an IMAP=20
interface is essentially cost-free.

> Clients will implement an extension if (1) it really saves them a lot=20
> of programming work, (2) it really saves a lot of performance, (3) it=20
> provides a feature that they can't get without it, or (4) they=20
> perceive a demand for it (see above).
>
> Your point 3 shows that (1) is basically not possible the way=20
> extensions are implemented in IMAP.

I think you're right, at least for all the relevant extensions I can=20
think of now.

Do me a favour. Repost that list at each WGLC.

> ...
>
> (2) mostly coves things like searching, sorting, and threading on the=20
> server, and a few things that save a lot of bandwidth or round trips=20
> for situations with slow connections or ones with high latency.
>
> (3) mostly covers IDLE. Pretty much anything else can be done another =
way.

A couple of hours ago someone sent me mail saying essentially =ABblah=20
issues 350 SELECTs and 350 SEARCHes and it's unpleasantly slow=BB. I=20
think the sender might just possibly consider multimailbox search to=20
fit group 3 ;)

INTHREAD searches are a bit like that too. With that you can add a=20
checkbox in the search dialog:

    [ ] return entires conversations

That's a pure added feature. It's also possible to use INTHREAD=20
differently in clients (e.g. send 350 THREAD commands in addition to=20
the 350 SELECTs above), of course.

> There was a small effort started to trim things from IMAP that might=20
> be better done in other ways (like folders) and to fold in the=20
> "important" extensions to a new base. The effort was called "IMAP5",=20
> and it seems to have died down.

Yes. I though that was a little too... shall we say, the likelihood of=20
highly unpleasant discussions seemed high and the advantage for me=20
small.

Arnt

From markrcrispin@panda.com  Wed Apr  8 14:17:54 2009
Return-Path: <markrcrispin@panda.com>
X-Original-To: morg@core3.amsl.com
Delivered-To: morg@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id B26793A687D for <morg@core3.amsl.com>; Wed,  8 Apr 2009 14:17:54 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.444
X-Spam-Level: 
X-Spam-Status: No, score=-2.444 tagged_above=-999 required=5 tests=[AWL=0.155,  BAYES_00=-2.599]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EJzCAslXtB4j for <morg@core3.amsl.com>; Wed,  8 Apr 2009 14:17:53 -0700 (PDT)
Received: from Panda.COM (panda.com [206.124.149.114]) by core3.amsl.com (Postfix) with ESMTP id B55443A6EB4 for <morg@ietf.org>; Wed,  8 Apr 2009 14:17:42 -0700 (PDT)
Received: from hsinghsing.panda.com markrcrispin@Panda.COM [206.124.149.116] by Panda.COM with M+ Extreme Email Engine 2008.3.internal via secured & encrypted transport (TLS); Wed, 08 Apr 2009 14:18:41 -0700
X-MailFrom: markrcrispin@panda.com
Date: Wed, 8 Apr 2009 14:18:40 -0700 (PDT)
From: Mark Crispin <markrcrispin@panda.com>
Sender: mrc@hsinghsing.panda.com
To: Arnt Gulbrandsen <arnt@oryx.com>
In-Reply-To: <W7El7F9shiifFw/D6B5PKw.md5@lochnagar>
Message-ID: <alpine.OSX.2.00.0904081346250.2675@hsinghsing.panda.com>
References: <1234565453.6132.886.camel@timo-desktop> <alpine.OSX.2.00.0903260837300.28287@hsinghsing.panda.com> <6c9fcc2a0903260917o8d5c3c1p86230e23c67ed854@mail.gmail.com> <xUbz6YGmIraZ0Kii/x30qQ.md5@lochnagar.oryx.com> <alpine.OSX.2.00.0903260937000.28287@hsinghsing.panda.com> <i7Ov0Sh7CnXn1spKeiryJQ.md5@lochnagar.oryx.com> <alpine.OSX.2.00.0903260945070.28287@hsinghsing.panda.com> <74E86DC1-440F-42D2-BB42-FD2291F51672@mumbo.ca> <P+hWnMz5o6F5yG+pRevlCQ.md5@lochnagar.oryx.com> <UDJylvAUqX5dqLKmB+C1Qg.md5@lochnagar> <6c9fcc2a0904081229k11028061x67ef997fd33ba4ba@mail.gmail.com> <W7El7F9shiifFw/D6B5PKw.md5@lochnagar>
User-Agent: Alpine 2.00 (OSX 1167 2008-08-23)
MIME-Version: 1.0
Content-Type: MULTIPART/MIXED; BOUNDARY="0-730147924-1239225521=:2675"
Cc: morg@ietf.org, Barry Leiba <barryleiba@computer.org>
Subject: Re: [MORG] Why IMAP extensions are (not) used
X-BeenThere: morg@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: Messaging Organization <morg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/morg>, <mailto:morg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/morg>
List-Post: <mailto:morg@ietf.org>
List-Help: <mailto:morg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/morg>, <mailto:morg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 08 Apr 2009 21:17:54 -0000

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--0-730147924-1239225521=:2675
Content-Type: TEXT/PLAIN; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8BIT

On Wed, 8 Apr 2009, Arnt Gulbrandsen wrote:
> or, pedantically, (3) they need the functionality and providing an IMAP 
> interface is essentially cost-free.

There is no such thing as "cost-free" for features, at least not if you 
have a real company with QA and support.  Freeware server implementors may 
have the luxury of implementing random stuff for fun.

> A couple of hours ago someone sent me mail saying essentially «blah issues 
> 350 SELECTs and 350 SEARCHes and it's unpleasantly slow». I think the sender 
> might just possibly consider multimailbox search to fit group 3 ;)

If 350 SELECTs and 350 SEARCHes are unpleasantly slow, what makes you 
think that multimailbox search will be different?

It will save 700 RTTs, which can certainly be unpleasant.  But, unless the 
server has instantaneous SELECT and SEARCH, there are more substantial 
costs elsewhere.

The advantage of the 700 RTTs it is clear to anyone looking at the 
resulting network traffic that what the client is doing is idiotic. 
Compare this to a server command that takes 20 minutes to complete.

> It's also possible to use INTHREAD differently 
> in clients (e.g. send 350 THREAD commands in addition to the 350 SELECTs 
> above), of course.

It's also possible to have a more intelligent client that uses the 
facilities that exist already.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.
--0-730147924-1239225521=:2675--

From arnt@oryx.com  Wed Apr  8 14:35:04 2009
Return-Path: <arnt@oryx.com>
X-Original-To: morg@core3.amsl.com
Delivered-To: morg@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 3E7493A68FD for <morg@core3.amsl.com>; Wed,  8 Apr 2009 14:35:04 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.593
X-Spam-Level: 
X-Spam-Status: No, score=-2.593 tagged_above=-999 required=5 tests=[AWL=0.005,  BAYES_00=-2.599, UNPARSEABLE_RELAY=0.001]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id x8lQQpi5wg+4 for <morg@core3.amsl.com>; Wed,  8 Apr 2009 14:35:03 -0700 (PDT)
Received: from kalyani.oryx.com (kalyani.oryx.com [195.30.37.30]) by core3.amsl.com (Postfix) with ESMTP id 173B13A6B8C for <morg@ietf.org>; Wed,  8 Apr 2009 14:35:03 -0700 (PDT)
Received: from kalyani.oryx.com (localhost [127.0.0.1]) by kalyani.oryx.com (Postfix) with ESMTP id 1B7C42E1FF; Wed,  8 Apr 2009 23:36:05 +0200 (CEST)
Received: from arnt@oryx.com (HELO lochnagar) by kalyani.oryx.com (Archiveopteryx 3.1.0) with esmtp id 1239226565-53296-53284/6/31 (2 recipients); Wed, 8 Apr 2009 23:36:05 +0200
Message-Id: <JpVMu9f7Ego+2plOHIR4DA.md5@lochnagar>
Date: Wed, 8 Apr 2009 23:36:26 +0200
From: Arnt Gulbrandsen <arnt@oryx.com>
To: morg@ietf.org
References: <1234565453.6132.886.camel@timo-desktop> <alpine.OSX.2.00.0903260837300.28287@hsinghsing.panda.com> <6c9fcc2a0903260917o8d5c3c1p86230e23c67ed854@mail.gmail.com> <xUbz6YGmIraZ0Kii/x30qQ.md5@lochnagar.oryx.com> <alpine.OSX.2.00.0903260937000.28287@hsinghsing.panda.com> <i7Ov0Sh7CnXn1spKeiryJQ.md5@lochnagar.oryx.com> <alpine.OSX.2.00.0903260945070.28287@hsinghsing.panda.com> <74E86DC1-440F-42D2-BB42-FD2291F51672@mumbo.ca> <P+hWnMz5o6F5yG+pRevlCQ.md5@lochnagar.oryx.com> <UDJylvAUqX5dqLKmB+C1Qg.md5@lochnagar> <6c9fcc2a0904081229k11028061x67ef997fd33ba4ba@mail.gmail.com> <W7El7F9shiifFw/D6B5PKw.md5@lochnagar> <alpine.OSX.2.00.0904081346250.2675@hsinghsing.panda.com>
In-Reply-To: <alpine.OSX.2.00.0904081346250.2675@hsinghsing.panda.com>
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset=iso-8859-1; format=flowed
Mime-Version: 1.0
Subject: Re: [MORG] Why IMAP extensions are (not) used
X-BeenThere: morg@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: Messaging Organization <morg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/morg>, <mailto:morg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/morg>
List-Post: <mailto:morg@ietf.org>
List-Help: <mailto:morg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/morg>, <mailto:morg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 08 Apr 2009 21:35:04 -0000

Mark Crispin writes:
>>  A couple of hours ago someone sent me mail saying essentially =
=ABblah=20
>>  issues 350 SELECTs and 350 SEARCHes and it's unpleasantly slow=BB. I=20
>>  think the sender might just possibly consider multimailbox search=20
>>  to fit group 3 ;)
>
> If 350 SELECTs and 350 SEARCHes are unpleasantly slow, what makes you=20
> think that multimailbox search will be different?

Not =ABwill be=BB, =ABis=BB.

The speedup (up to orders of magnitude depending on the search=20
expression) comes from two factors: The search itself is much better=20
optimised, and there's no need to compute those 350 UNSEENs, EXISTSs=20
and RECENTs.

Arnt

From markrcrispin@panda.com  Wed Apr  8 14:45:30 2009
Return-Path: <markrcrispin@panda.com>
X-Original-To: morg@core3.amsl.com
Delivered-To: morg@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 34ADD3A6E75 for <morg@core3.amsl.com>; Wed,  8 Apr 2009 14:45:30 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.466
X-Spam-Level: 
X-Spam-Status: No, score=-2.466 tagged_above=-999 required=5 tests=[AWL=0.133,  BAYES_00=-2.599]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cwvLVVvXqFy8 for <morg@core3.amsl.com>; Wed,  8 Apr 2009 14:45:29 -0700 (PDT)
Received: from Panda.COM (panda.com [206.124.149.114]) by core3.amsl.com (Postfix) with ESMTP id 298273A6B2A for <morg@ietf.org>; Wed,  8 Apr 2009 14:45:29 -0700 (PDT)
Received: from hsinghsing.panda.com markrcrispin@Panda.COM [206.124.149.116] by Panda.COM with M+ Extreme Email Engine 2008.3.internal via secured & encrypted transport (TLS); Wed, 08 Apr 2009 14:46:35 -0700
X-MailFrom: markrcrispin@panda.com
Date: Wed, 8 Apr 2009 14:46:35 -0700 (PDT)
From: Mark Crispin <markrcrispin@panda.com>
Sender: mrc@hsinghsing.panda.com
To: Arnt Gulbrandsen <arnt@oryx.com>
In-Reply-To: <JpVMu9f7Ego+2plOHIR4DA.md5@lochnagar>
Message-ID: <alpine.OSX.2.00.0904081437400.40001@hsinghsing.panda.com>
References: <1234565453.6132.886.camel@timo-desktop> <alpine.OSX.2.00.0903260837300.28287@hsinghsing.panda.com> <6c9fcc2a0903260917o8d5c3c1p86230e23c67ed854@mail.gmail.com> <xUbz6YGmIraZ0Kii/x30qQ.md5@lochnagar.oryx.com> <alpine.OSX.2.00.0903260937000.28287@hsinghsing.panda.com> <i7Ov0Sh7CnXn1spKeiryJQ.md5@lochnagar.oryx.com> <alpine.OSX.2.00.0903260945070.28287@hsinghsing.panda.com> <74E86DC1-440F-42D2-BB42-FD2291F51672@mumbo.ca> <P+hWnMz5o6F5yG+pRevlCQ.md5@lochnagar.oryx.com> <UDJylvAUqX5dqLKmB+C1Qg.md5@lochnagar> <6c9fcc2a0904081229k11028061x67ef997fd33ba4ba@mail.gmail.com> <W7El7F9shiifFw/D6B5PKw.md5@lochnagar> <alpine.OSX.2.00.0904081346250.2675@hsinghsing.panda.com> <JpVMu9f7Ego+2plOHIR4DA.md5@lochnagar>
User-Agent: Alpine 2.00 (OSX 1167 2008-08-23)
MIME-Version: 1.0
Content-Type: MULTIPART/MIXED; BOUNDARY="0-1510078944-1239227195=:40001"
Cc: morg@ietf.org
Subject: Re: [MORG] Why IMAP extensions are (not) used
X-BeenThere: morg@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: Messaging Organization <morg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/morg>, <mailto:morg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/morg>
List-Post: <mailto:morg@ietf.org>
List-Help: <mailto:morg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/morg>, <mailto:morg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 08 Apr 2009 21:45:30 -0000

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--0-1510078944-1239227195=:40001
Content-Type: TEXT/PLAIN; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8BIT

On Wed, 8 Apr 2009, Arnt Gulbrandsen wrote:
>> If 350 SELECTs and 350 SEARCHes are unpleasantly slow, what makes you think 
>> that multimailbox search will be different?
> Not «will be», «is».
> The speedup (up to orders of magnitude depending on the search expression) 
> comes from two factors: The search itself is much better optimised, and 
> there's no need to compute those 350 UNSEENs, EXISTSs and RECENTs.

You are engaged in the fallacy of believing that an implementation aspect 
of your server is common to all other servers.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.
--0-1510078944-1239227195=:40001--

From arnt@oryx.com  Wed Apr  8 14:57:55 2009
Return-Path: <arnt@oryx.com>
X-Original-To: morg@core3.amsl.com
Delivered-To: morg@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 3D8373A6EA3 for <morg@core3.amsl.com>; Wed,  8 Apr 2009 14:57:55 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.593
X-Spam-Level: 
X-Spam-Status: No, score=-2.593 tagged_above=-999 required=5 tests=[AWL=0.005,  BAYES_00=-2.599, UNPARSEABLE_RELAY=0.001]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id L2vQtuuNNoxZ for <morg@core3.amsl.com>; Wed,  8 Apr 2009 14:57:54 -0700 (PDT)
Received: from kalyani.oryx.com (kalyani.oryx.com [195.30.37.30]) by core3.amsl.com (Postfix) with ESMTP id 3F9A23A6E87 for <morg@ietf.org>; Wed,  8 Apr 2009 14:57:54 -0700 (PDT)
Received: from kalyani.oryx.com (localhost [127.0.0.1]) by kalyani.oryx.com (Postfix) with ESMTP id E7E1D2E0EF; Wed,  8 Apr 2009 23:58:57 +0200 (CEST)
Received: from arnt@oryx.com (HELO lochnagar) by kalyani.oryx.com (Archiveopteryx 3.1.0) with esmtp id 1239227937-53296-53284/6/32 for morg@ietf.org; Wed, 8 Apr 2009 23:58:57 +0200
Message-Id: <zJDgSl598yxuAQmgE7JgNA.md5@lochnagar>
Date: Wed, 8 Apr 2009 23:59:19 +0200
From: Arnt Gulbrandsen <arnt@oryx.com>
To: morg@ietf.org
References: <1234565453.6132.886.camel@timo-desktop> <alpine.OSX.2.00.0903260837300.28287@hsinghsing.panda.com> <6c9fcc2a0903260917o8d5c3c1p86230e23c67ed854@mail.gmail.com> <xUbz6YGmIraZ0Kii/x30qQ.md5@lochnagar.oryx.com> <alpine.OSX.2.00.0903260937000.28287@hsinghsing.panda.com> <i7Ov0Sh7CnXn1spKeiryJQ.md5@lochnagar.oryx.com> <alpine.OSX.2.00.0903260945070.28287@hsinghsing.panda.com> <74E86DC1-440F-42D2-BB42-FD2291F51672@mumbo.ca> <P+hWnMz5o6F5yG+pRevlCQ.md5@lochnagar.oryx.com> <UDJylvAUqX5dqLKmB+C1Qg.md5@lochnagar> <6c9fcc2a0904081229k11028061x67ef997fd33ba4ba@mail.gmail.com> <W7El7F9shiifFw/D6B5PKw.md5@lochnagar> <alpine.OSX.2.00.0904081346250.2675@hsinghsing.panda.com> <JpVMu9f7Ego+2plOHIR4DA.md5@lochnagar> <alpine.OSX.2.00.0904081437400.40001@hsinghsing.panda.com>
In-Reply-To: <alpine.OSX.2.00.0904081437400.40001@hsinghsing.panda.com>
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset=iso-8859-1; format=flowed
Mime-Version: 1.0
Subject: Re: [MORG] Why IMAP extensions are (not) used
X-BeenThere: morg@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: Messaging Organization <morg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/morg>, <mailto:morg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/morg>
List-Post: <mailto:morg@ietf.org>
List-Help: <mailto:morg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/morg>, <mailto:morg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 08 Apr 2009 21:57:55 -0000

Mark Crispin writes:
> On Wed, 8 Apr 2009, Arnt Gulbrandsen wrote:
>>>  If 350 SELECTs and 350 SEARCHes are unpleasantly slow, what makes=20
>>>  you think that multimailbox search will be different?
>>  Not =ABwill be=BB, =ABis=BB.
>>  The speedup (up to orders of magnitude depending on the search=20
>>  expression) comes from two factors: The search itself is much=20
>>  better optimised, and there's no need to compute those 350 UNSEENs,=20
>>  EXISTSs and RECENTs.
>
> You are engaged in the fallacy of believing that an implementation=20
> aspect of your server is common to all other servers.

Locking the mailbox in order to compute RECENT isn't free for anyone.=20
Scanning for UNSEEN and EXISTS isn't free for anyone. Many databases=20
and libraries that will use indexes to search quickly are available to=20
everyone. Maybe the query planner I use is better than most. So what?

Arnt

From barryleiba.mailing.lists@gmail.com  Wed Apr  8 15:36:11 2009
Return-Path: <barryleiba.mailing.lists@gmail.com>
X-Original-To: morg@core3.amsl.com
Delivered-To: morg@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 67CF33A6A8C for <morg@core3.amsl.com>; Wed,  8 Apr 2009 15:36:11 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.169
X-Spam-Level: 
X-Spam-Status: No, score=-2.169 tagged_above=-999 required=5 tests=[AWL=-0.192, BAYES_00=-2.599, FM_FORGED_GMAIL=0.622]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pbSQvaiTMd6U for <morg@core3.amsl.com>; Wed,  8 Apr 2009 15:36:10 -0700 (PDT)
Received: from mail-bw0-f169.google.com (mail-bw0-f169.google.com [209.85.218.169]) by core3.amsl.com (Postfix) with ESMTP id E973A28C1B5 for <morg@ietf.org>; Wed,  8 Apr 2009 15:36:03 -0700 (PDT)
Received: by bwz17 with SMTP id 17so344234bwz.37 for <morg@ietf.org>; Wed, 08 Apr 2009 15:37:10 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type:content-transfer-encoding; bh=TlpGsW84otExk7AKpif3B4ttYshUngXmQ1QLPGsR4Ss=; b=V+YYuBa8wVDXteepeNCmmV8KIru8ViWt6uCd6wOuLzNhEjvN+EOec+jlc8gDpOOPra nqiZ9dSvx9rVzIVjEcGip6LizEufHVUGuHpkbpz2HEW15a7H1LxE/bBnPzRjRaLIDOcZ bl2abfdxTjbvpqImV5gaKba1A/V6uacUab3zQ=
DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=L93iuyrPTQO6rvQcAVq0HgUAT8Fqps0aNx5ExKW9fZ4bAjP3tB54HQMuvN2xGMHApj O0NL36H4lb+ZYReromXlI2Iq7nfHE+7B/LAdttDxC70T00QJ/jWpOcBhuKrqHKNtRBTs jRpEpwARhyDZlTHXPUCDg8PJEnjanFUHDqCX0=
MIME-Version: 1.0
Sender: barryleiba.mailing.lists@gmail.com
Received: by 10.223.108.15 with SMTP id d15mr527473fap.105.1239230230546; Wed,  08 Apr 2009 15:37:10 -0700 (PDT)
In-Reply-To: <zJDgSl598yxuAQmgE7JgNA.md5@lochnagar>
References: <1234565453.6132.886.camel@timo-desktop> <74E86DC1-440F-42D2-BB42-FD2291F51672@mumbo.ca> <P+hWnMz5o6F5yG+pRevlCQ.md5@lochnagar.oryx.com> <UDJylvAUqX5dqLKmB+C1Qg.md5@lochnagar> <6c9fcc2a0904081229k11028061x67ef997fd33ba4ba@mail.gmail.com> <W7El7F9shiifFw/D6B5PKw.md5@lochnagar> <alpine.OSX.2.00.0904081346250.2675@hsinghsing.panda.com> <JpVMu9f7Ego+2plOHIR4DA.md5@lochnagar> <alpine.OSX.2.00.0904081437400.40001@hsinghsing.panda.com> <zJDgSl598yxuAQmgE7JgNA.md5@lochnagar>
Date: Wed, 8 Apr 2009 18:37:10 -0400
X-Google-Sender-Auth: bde947b3fcc8dcac
Message-ID: <6c9fcc2a0904081537w340954b6ld0608d6a38bc57a3@mail.gmail.com>
From: Barry Leiba <barryleiba@computer.org>
To: Arnt Gulbrandsen <arnt@oryx.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Cc: morg@ietf.org
Subject: Re: [MORG] Why IMAP extensions are (not) used
X-BeenThere: morg@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: Messaging Organization <morg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/morg>, <mailto:morg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/morg>
List-Post: <mailto:morg@ietf.org>
List-Help: <mailto:morg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/morg>, <mailto:morg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 08 Apr 2009 22:36:11 -0000

> Locking the mailbox in order to compute RECENT isn't free for anyone.
> Scanning for UNSEEN and EXISTS isn't free for anyone. Many databases and
> libraries that will use indexes to search quickly are available to everyone.
> Maybe the query planner I use is better than most. So what?

If you have (say) a DB2 back-end to your mail store, a search of 350
mailboxes might be very efficient.

If you have (say) a set of Unix mailboxes as the back-end to your mail
store, a search of 350 mailboxes might be pretty much the same,
whether you do it in one command or 700, apart from the protocol
overhead.

Mark's point is that for a lot of these sorts of things, you kind of
need to know something about the server to know whether doing [X] is a
good idea or not.

Perhaps one might say that a server SHOULD only implement multimailbox
search if it has the sort of back-end that makes it significantly more
efficient.  Or perhaps there should also be some sort of clue to the
client as to the efficiency.  Or perhaps clients and servers should do
what they like, and the market will sort out which ones match users'
expectations best.

Whatever the decision, Mark's point, which is correct, is that you
can't make assumptions.  A client that wants to let a user search 350
mailboxes will do it, one way or another.  But it might or might not
perform badly with or without the extension, and you can't know in
advance which it'll be.

Barry

From markrcrispin@panda.com  Wed Apr  8 15:40:01 2009
Return-Path: <markrcrispin@panda.com>
X-Original-To: morg@core3.amsl.com
Delivered-To: morg@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 2BE6F3A6B6A for <morg@core3.amsl.com>; Wed,  8 Apr 2009 15:40:01 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.483
X-Spam-Level: 
X-Spam-Status: No, score=-2.483 tagged_above=-999 required=5 tests=[AWL=0.116,  BAYES_00=-2.599]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id H-OuYRGeBSrE for <morg@core3.amsl.com>; Wed,  8 Apr 2009 15:40:00 -0700 (PDT)
Received: from Panda.COM (panda.com [206.124.149.114]) by core3.amsl.com (Postfix) with ESMTP id 23AD53A6A20 for <morg@ietf.org>; Wed,  8 Apr 2009 15:40:00 -0700 (PDT)
Received: from hsinghsing.panda.com markrcrispin@Panda.COM [206.124.149.116] by Panda.COM with M+ Extreme Email Engine 2008.3.internal via secured & encrypted transport (TLS); Wed, 08 Apr 2009 15:41:06 -0700
X-MailFrom: markrcrispin@panda.com
Date: Wed, 8 Apr 2009 15:41:06 -0700 (PDT)
From: Mark Crispin <markrcrispin@panda.com>
Sender: mrc@hsinghsing.panda.com
To: Arnt Gulbrandsen <arnt@oryx.com>
In-Reply-To: <zJDgSl598yxuAQmgE7JgNA.md5@lochnagar>
Message-ID: <alpine.OSX.2.00.0904081506080.40001@hsinghsing.panda.com>
References: <1234565453.6132.886.camel@timo-desktop> <alpine.OSX.2.00.0903260837300.28287@hsinghsing.panda.com> <6c9fcc2a0903260917o8d5c3c1p86230e23c67ed854@mail.gmail.com> <xUbz6YGmIraZ0Kii/x30qQ.md5@lochnagar.oryx.com> <alpine.OSX.2.00.0903260937000.28287@hsinghsing.panda.com> <i7Ov0Sh7CnXn1spKeiryJQ.md5@lochnagar.oryx.com> <alpine.OSX.2.00.0903260945070.28287@hsinghsing.panda.com> <74E86DC1-440F-42D2-BB42-FD2291F51672@mumbo.ca> <P+hWnMz5o6F5yG+pRevlCQ.md5@lochnagar.oryx.com> <UDJylvAUqX5dqLKmB+C1Qg.md5@lochnagar> <6c9fcc2a0904081229k11028061x67ef997fd33ba4ba@mail.gmail.com> <W7El7F9shiifFw/D6B5PKw.md5@lochnagar> <alpine.OSX.2.00.0904081346250.2675@hsinghsing.panda.com> <JpVMu9f7Ego+2plOHIR4DA.md5@lochnagar> <alpine.OSX.2.00.0904081437400.40001@hsinghsing.panda.com> <zJDgSl598yxuAQmgE7JgNA.md5@lochnagar>
User-Agent: Alpine 2.00 (OSX 1167 2008-08-23)
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
Cc: morg@ietf.org
Subject: Re: [MORG] Why IMAP extensions are (not) used
X-BeenThere: morg@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: Messaging Organization <morg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/morg>, <mailto:morg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/morg>
List-Post: <mailto:morg@ietf.org>
List-Help: <mailto:morg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/morg>, <mailto:morg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 08 Apr 2009 22:40:01 -0000

On Wed, 8 Apr 2009, Arnt Gulbrandsen wrote:
>> You are engaged in the fallacy of believing that an implementation aspect 
>> of your server is common to all other servers.
> Locking the mailbox in order to compute RECENT isn't free for anyone.

What is "locking the mailbox in order to compute RECENT"?

> Scanning for UNSEEN and EXISTS isn't free for anyone.

What is "scanning for UNSEEN and EXISTS"?

You are assuming, falsely, that concepts that exist in your implementation 
are common to all other implementations.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

From tss@iki.fi  Wed Apr  8 16:13:01 2009
Return-Path: <tss@iki.fi>
X-Original-To: morg@core3.amsl.com
Delivered-To: morg@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id D73133A6B2E for <morg@core3.amsl.com>; Wed,  8 Apr 2009 16:13:01 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -6.399
X-Spam-Level: 
X-Spam-Status: No, score=-6.399 tagged_above=-999 required=5 tests=[AWL=0.200,  BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1GGgvdpi9FSw for <morg@core3.amsl.com>; Wed,  8 Apr 2009 16:13:00 -0700 (PDT)
Received: from dovecot.org (dovecot.org [82.118.211.50]) by core3.amsl.com (Postfix) with ESMTP id AA0623A6ABA for <morg@ietf.org>; Wed,  8 Apr 2009 16:13:00 -0700 (PDT)
Received: from [10.4.192.51] (unknown [74.205.24.229]) by dovecot.org (Postfix) with ESMTP id ABE5A16471F2; Thu,  9 Apr 2009 02:14:05 +0300 (EEST)
From: Timo Sirainen <tss@iki.fi>
To: Arnt Gulbrandsen <arnt@oryx.com>
In-Reply-To: <W7El7F9shiifFw/D6B5PKw.md5@lochnagar>
References: <1234565453.6132.886.camel@timo-desktop> <alpine.OSX.2.00.0903260837300.28287@hsinghsing.panda.com> <6c9fcc2a0903260917o8d5c3c1p86230e23c67ed854@mail.gmail.com> <xUbz6YGmIraZ0Kii/x30qQ.md5@lochnagar.oryx.com> <alpine.OSX.2.00.0903260937000.28287@hsinghsing.panda.com> <i7Ov0Sh7CnXn1spKeiryJQ.md5@lochnagar.oryx.com> <alpine.OSX.2.00.0903260945070.28287@hsinghsing.panda.com> <74E86DC1-440F-42D2-BB42-FD2291F51672@mumbo.ca> <P+hWnMz5o6F5yG+pRevlCQ.md5@lochnagar.oryx.com> <UDJylvAUqX5dqLKmB+C1Qg.md5@lochnagar> <6c9fcc2a0904081229k11028061x67ef997fd33ba4ba@mail.gmail.com> <W7El7F9shiifFw/D6B5PKw.md5@lochnagar>
Content-Type: multipart/signed; micalg="pgp-sha1"; protocol="application/pgp-signature"; boundary="=-o9OjqRH9Eek5jo2MTKKl"
Date: Wed, 08 Apr 2009 19:14:03 -0400
Message-Id: <1239232443.5523.218.camel@timo-desktop>
Mime-Version: 1.0
X-Mailer: Evolution 2.24.3 
Cc: morg <morg@ietf.org>
Subject: Re: [MORG] Why IMAP extensions are (not) used
X-BeenThere: morg@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: Messaging Organization <morg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/morg>, <mailto:morg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/morg>
List-Post: <mailto:morg@ietf.org>
List-Help: <mailto:morg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/morg>, <mailto:morg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 08 Apr 2009 23:13:01 -0000

--=-o9OjqRH9Eek5jo2MTKKl
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

On Wed, 2009-04-08 at 22:42 +0200, Arnt Gulbrandsen wrote:
> I know the page and think not, but it's close-ish. This would be more=20
> helpful, e.g.:
>=20
>    [ extension name ]
>       known supporting servers: [ list ]
>       known unsupporting: [ list ]
>       test accounts/downloads with: [ list ]

I started such a list, add your own servers (and their "home pages"):
http://imapwiki.org/Extensions


--=-o9OjqRH9Eek5jo2MTKKl
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEABECAAYFAkndL7sACgkQyUhSUUBViskeUACfTQRe2oqZbHrVAXWsevzcFn7N
cgMAn2unCzMdMDFgPw59rjdkCQIMM7gP
=MKWV
-----END PGP SIGNATURE-----

--=-o9OjqRH9Eek5jo2MTKKl--


From tss@iki.fi  Wed Apr  8 16:42:54 2009
Return-Path: <tss@iki.fi>
X-Original-To: morg@core3.amsl.com
Delivered-To: morg@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 2CAB03A6BB0 for <morg@core3.amsl.com>; Wed,  8 Apr 2009 16:42:54 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -6.449
X-Spam-Level: 
X-Spam-Status: No, score=-6.449 tagged_above=-999 required=5 tests=[AWL=0.150,  BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mnxYkcK0kST1 for <morg@core3.amsl.com>; Wed,  8 Apr 2009 16:42:53 -0700 (PDT)
Received: from dovecot.org (dovecot.org [82.118.211.50]) by core3.amsl.com (Postfix) with ESMTP id 25ED73A6BAE for <morg@ietf.org>; Wed,  8 Apr 2009 16:42:53 -0700 (PDT)
Received: from [10.4.192.51] (unknown [74.205.24.229]) by dovecot.org (Postfix) with ESMTP id BE56C16471F2; Thu,  9 Apr 2009 02:43:59 +0300 (EEST)
From: Timo Sirainen <tss@iki.fi>
To: Mark Crispin <markrcrispin@panda.com>
In-Reply-To: <alpine.OSX.2.00.0904081042310.1327@hsinghsing.panda.com>
References: <1234565453.6132.886.camel@timo-desktop> <E8EBD6F423B52FD02CD04B8D@socrates.local> <5lt3qVJvy+jaTjngz/M+Fg.md5@lochnagar.oryx.com> <alpine.OSX.2.00.0903260758350.28287@hsinghsing.panda.com> <nhZ/ucOkEVUShLAg2vFAeQ.md5@lochnagar.oryx.com> <alpine.OSX.2.00.0903260837300.28287@hsinghsing.panda.com> <6c9fcc2a0903260917o8d5c3c1p86230e23c67ed854@mail.gmail.com> <xUbz6YGmIraZ0Kii/x30qQ.md5@lochnagar.oryx.com> <alpine.OSX.2.00.0903260937000.28287@hsinghsing.panda.com> <i7Ov0Sh7CnXn1spKeiryJQ.md5@lochnagar.oryx.com> <alpine.OSX.2.00.0903260945070.28287@hsinghsing.panda.com> <74E86DC1-440F-42D2-BB42-FD2291F51672@mumbo.ca> <P+hWnMz5o6F5yG+pRevlCQ.md5@lochnagar.oryx.com> <UDJylvAUqX5dqLKmB+C1Qg.md5@lochnagar> <alpine.OSX.2.00.0904081042310.1327@hsinghsing.panda.com>
Content-Type: multipart/signed; micalg="pgp-sha1"; protocol="application/pgp-signature"; boundary="=-uQ+Y8XuMy1xrGfiOsXmb"
Date: Wed, 08 Apr 2009 19:43:58 -0400
Message-Id: <1239234238.5523.262.camel@timo-desktop>
Mime-Version: 1.0
X-Mailer: Evolution 2.24.3 
Cc: morg@ietf.org
Subject: Re: [MORG] Why IMAP extensions are (not) used
X-BeenThere: morg@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: Messaging Organization <morg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/morg>, <mailto:morg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/morg>
List-Post: <mailto:morg@ietf.org>
List-Help: <mailto:morg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/morg>, <mailto:morg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 08 Apr 2009 23:42:54 -0000

--=-uQ+Y8XuMy1xrGfiOsXmb
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

On Wed, 2009-04-08 at 12:25 -0700, Mark Crispin wrote:
> IMAP extensions are not used much for the following reasons:
>=20
> [1] Almost all IMAP extensions are worthless garbage.
>=20
> Oh, they may be valuable to some limited constituency, but not generally.

I guess that depends on your target audience. For traditional desktop
IMAP clients most extensions are worthless. For mobile clients I guess
many would be valuable in theory, but since the server support for
extensions is (still) so low I have some doubts as to if/when they start
supporting e.g. Lemonade extensions.

Anyway, most of the extensions I implement nowadays are going to be used
by webmails. Webmail users want to do everything and they want
everything to work immediately, so anything that helps there is useful.
I guess those features could also be implemented in a non-standard way,
but I do see it useful to standardize them so that webmails aren't tied
to specific server implementations.

--=-uQ+Y8XuMy1xrGfiOsXmb
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEABECAAYFAkndNr4ACgkQyUhSUUBVislXWACeLx1HFP3ucN3rNB0yh/ogqSKz
OlcAn1UOEKZjM8Vcb+0JtievPnuubufe
=4LbQ
-----END PGP SIGNATURE-----

--=-uQ+Y8XuMy1xrGfiOsXmb--


From tss@iki.fi  Wed Apr  8 17:24:17 2009
Return-Path: <tss@iki.fi>
X-Original-To: morg@core3.amsl.com
Delivered-To: morg@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id A9C1F3A6BBE for <morg@core3.amsl.com>; Wed,  8 Apr 2009 17:24:17 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -6.479
X-Spam-Level: 
X-Spam-Status: No, score=-6.479 tagged_above=-999 required=5 tests=[AWL=0.120,  BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PDEJYJ6SHk6X for <morg@core3.amsl.com>; Wed,  8 Apr 2009 17:24:17 -0700 (PDT)
Received: from dovecot.org (dovecot.org [82.118.211.50]) by core3.amsl.com (Postfix) with ESMTP id D8C3C3A6BB1 for <morg@ietf.org>; Wed,  8 Apr 2009 17:24:16 -0700 (PDT)
Received: from [10.4.192.51] (unknown [74.205.24.229]) by dovecot.org (Postfix) with ESMTP id 2ABE816472C8; Thu,  9 Apr 2009 03:25:23 +0300 (EEST)
From: Timo Sirainen <tss@iki.fi>
To: Randall Gellens <rg+ietf@qualcomm.com>
In-Reply-To: <p0624060ac6018b9b2818@[192.168.99.12]>
References: <p0624060ac6018b9b2818@[192.168.99.12]>
Content-Type: multipart/signed; micalg="pgp-sha1"; protocol="application/pgp-signature"; boundary="=-kpDl8JhfmNlqG6sWUNb1"
Date: Wed, 08 Apr 2009 20:25:20 -0400
Message-Id: <1239236720.5523.309.camel@timo-desktop>
Mime-Version: 1.0
X-Mailer: Evolution 2.24.3 
Cc: morg@ietf.org
Subject: Re: [MORG] MORG notes from IETF 74
X-BeenThere: morg@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: Messaging Organization <morg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/morg>, <mailto:morg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/morg>
List-Post: <mailto:morg@ietf.org>
List-Help: <mailto:morg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/morg>, <mailto:morg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 09 Apr 2009 00:24:17 -0000

--=-kpDl8JhfmNlqG6sWUNb1
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

On Tue, 2009-04-07 at 16:01 -0700, Randall Gellens wrote:
> Here are my own notes from the MORG session.

Merged Randall's and Chris's notes to=20
http://www.ietf.org/proceedings/09mar/minutes/morg.txt


--=-kpDl8JhfmNlqG6sWUNb1
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEABECAAYFAkndQHAACgkQyUhSUUBVisn9gQCdH6GK7ryHNktv9Kl3vnTWT6ca
nxEAoIWfSirnV2k6ZTrSjt6ameKxs2hH
=N4fi
-----END PGP SIGNATURE-----

--=-kpDl8JhfmNlqG6sWUNb1--


From markrcrispin@panda.com  Wed Apr  8 17:29:15 2009
Return-Path: <markrcrispin@panda.com>
X-Original-To: morg@core3.amsl.com
Delivered-To: morg@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 309683A6BB7 for <morg@core3.amsl.com>; Wed,  8 Apr 2009 17:29:15 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.496
X-Spam-Level: 
X-Spam-Status: No, score=-2.496 tagged_above=-999 required=5 tests=[AWL=0.103,  BAYES_00=-2.599]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id v61RNiYG1-8E for <morg@core3.amsl.com>; Wed,  8 Apr 2009 17:29:14 -0700 (PDT)
Received: from Panda.COM (panda.com [206.124.149.114]) by core3.amsl.com (Postfix) with ESMTP id 6B9123A67F6 for <morg@ietf.org>; Wed,  8 Apr 2009 17:29:10 -0700 (PDT)
Received: from hsinghsing.panda.com markrcrispin@Panda.COM [206.124.149.116] by Panda.COM with M+ Extreme Email Engine 2008.3.internal via secured & encrypted transport (TLS); Wed, 08 Apr 2009 17:30:10 -0700
X-MailFrom: markrcrispin@panda.com
Date: Wed, 8 Apr 2009 17:30:10 -0700 (PDT)
From: Mark Crispin <markrcrispin@panda.com>
Sender: mrc@hsinghsing.panda.com
To: Barry Leiba <barryleiba@computer.org>
In-Reply-To: <6c9fcc2a0904081537w340954b6ld0608d6a38bc57a3@mail.gmail.com>
Message-ID: <alpine.OSX.2.00.0904081700060.40001@hsinghsing.panda.com>
References: <1234565453.6132.886.camel@timo-desktop> <74E86DC1-440F-42D2-BB42-FD2291F51672@mumbo.ca> <P+hWnMz5o6F5yG+pRevlCQ.md5@lochnagar.oryx.com> <UDJylvAUqX5dqLKmB+C1Qg.md5@lochnagar> <6c9fcc2a0904081229k11028061x67ef997fd33ba4ba@mail.gmail.com> <W7El7F9shiifFw/D6B5PKw.md5@lochnagar> <alpine.OSX.2.00.0904081346250.2675@hsinghsing.panda.com> <JpVMu9f7Ego+2plOHIR4DA.md5@lochnagar> <alpine.OSX.2.00.0904081437400.40001@hsinghsing.panda.com> <zJDgSl598yxuAQmgE7JgNA.md5@lochnagar> <6c9fcc2a0904081537w340954b6ld0608d6a38bc57a3@mail.gmail.com>
User-Agent: Alpine 2.00 (OSX 1167 2008-08-23)
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII
Cc: morg@ietf.org
Subject: Re: [MORG] Why IMAP extensions are (not) used
X-BeenThere: morg@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: Messaging Organization <morg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/morg>, <mailto:morg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/morg>
List-Post: <mailto:morg@ietf.org>
List-Help: <mailto:morg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/morg>, <mailto:morg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 09 Apr 2009 00:29:15 -0000

On Wed, 8 Apr 2009, Barry Leiba wrote:
> If you have (say) a DB2 back-end to your mail store, a search of 350
> mailboxes might be very efficient.

At some other costs... ;-)

> If you have (say) a set of Unix mailboxes as the back-end to your mail
> store, a search of 350 mailboxes might be pretty much the same,
> whether you do it in one command or 700, apart from the protocol
> overhead.

Note that to do IMAP SEARCH right, you have to be cognizant of the MIME 
structure, select only the textual parts, and search the textual data 
using the i;unicode-casemap algorithm.  "Just grep through the mailboxes" 
doesn't cut it any more.  grep doesn't know about a string that may be in 
a non-UTF8 charset within BASE64 encoded content, or about text vs. image 
MIME parts.

Either you do the necessary computations on the fly at demand, or do the 
calculations in advance and store the results separate from the message 
data, or do something in between.

No database can somehow make this choice go away.  The form may change, 
but not the function.

Nor is it as easy as saying "we have lots of disk space, just precompute 
and store everything".  The more you store, the more complexity that you 
have to deal with.  Also, delivery time is a really bad time to be doing 
anything time-consuming.

> Whatever the decision, Mark's point, which is correct, is that you
> can't make assumptions.  A client that wants to let a user search 350
> mailboxes will do it, one way or another.  But it might or might not
> perform badly with or without the extension, and you can't know in
> advance which it'll be.

There's more than that.  A server may decide that the client that searches 
350 mailboxes is abuse, and will "implement" the multimailbox search to 
block the abuse.  Only it won't really search.

Don't think for one minute that this isn't going on now.

For better or worse, most servers are designed with one goal, and only one 
goal, in mind: to implement synchronization with Outlook.  Everything 
else, and I do mean EVERYTHING else, is secondary.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

From markrcrispin@panda.com  Wed Apr  8 18:00:09 2009
Return-Path: <markrcrispin@panda.com>
X-Original-To: morg@core3.amsl.com
Delivered-To: morg@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 292933A6EB7 for <morg@core3.amsl.com>; Wed,  8 Apr 2009 18:00:09 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.506
X-Spam-Level: 
X-Spam-Status: No, score=-2.506 tagged_above=-999 required=5 tests=[AWL=0.093,  BAYES_00=-2.599]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CvNLZ+nDTZgf for <morg@core3.amsl.com>; Wed,  8 Apr 2009 18:00:08 -0700 (PDT)
Received: from Panda.COM (panda.com [206.124.149.114]) by core3.amsl.com (Postfix) with ESMTP id B36ED3A6BDE for <morg@ietf.org>; Wed,  8 Apr 2009 18:00:07 -0700 (PDT)
Received: from hsinghsing.panda.com markrcrispin@Panda.COM [206.124.149.116] by Panda.COM with M+ Extreme Email Engine 2008.3.internal via secured & encrypted transport (TLS); Wed, 08 Apr 2009 18:01:12 -0700
X-MailFrom: markrcrispin@panda.com
Date: Wed, 8 Apr 2009 18:01:11 -0700 (PDT)
From: Mark Crispin <markrcrispin@panda.com>
Sender: mrc@hsinghsing.panda.com
To: Timo Sirainen <tss@iki.fi>
In-Reply-To: <1239234238.5523.262.camel@timo-desktop>
Message-ID: <alpine.OSX.2.00.0904081730210.40001@hsinghsing.panda.com>
References: <1234565453.6132.886.camel@timo-desktop> <E8EBD6F423B52FD02CD04B8D@socrates.local> <5lt3qVJvy+jaTjngz/M+Fg.md5@lochnagar.oryx.com> <alpine.OSX.2.00.0903260758350.28287@hsinghsing.panda.com> <nhZ/ucOkEVUShLAg2vFAeQ.md5@lochnagar.oryx.com>  <alpine.OSX.2.00.0903260837300.28287@hsinghsing.panda.com> <6c9fcc2a0903260917o8d5c3c1p86230e23c67ed854@mail.gmail.com> <xUbz6YGmIraZ0Kii/x30qQ.md5@lochnagar.oryx.com> <alpine.OSX.2.00.0903260937000.28287@hsinghsing.panda.com> <i7Ov0Sh7CnXn1spKeiryJQ.md5@lochnagar.oryx.com> <alpine.OSX.2.00.0903260945070.28287@hsinghsing.panda.com> <74E86DC1-440F-42D2-BB42-FD2291F51672@mumbo.ca> <P+hWnMz5o6F5yG+pRevlCQ.md5@lochnagar.oryx.com> <UDJylvAUqX5dqLKmB+C1Qg.md5@lochnagar> <alpine.OSX.2.00.0904081042310.1327@hsinghsing.panda.com> <1239234238.5523.262.camel@timo-desktop>
User-Agent: Alpine 2.00 (OSX 1167 2008-08-23)
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII
Cc: morg@ietf.org
Subject: Re: [MORG] Why IMAP extensions are (not) used
X-BeenThere: morg@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: Messaging Organization <morg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/morg>, <mailto:morg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/morg>
List-Post: <mailto:morg@ietf.org>
List-Help: <mailto:morg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/morg>, <mailto:morg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 09 Apr 2009 01:00:09 -0000

On Wed, 8 Apr 2009, Timo Sirainen wrote:
> For traditional desktop
> IMAP clients most extensions are worthless.

Completely agree.  The only extensions that make a significant difference 
for Pine/Alpine are SORT/THREAD and to a lesser extent MULTIAPPEND.

The thing that strikes me with the recent batch of extensions is that they 
are all stupid little tweaks that do nothing to enhance functionality.

> For mobile clients I guess
> many would be valuable in theory, but since the server support for
> extensions is (still) so low I have some doubts as to if/when they start
> supporting e.g. Lemonade extensions.

I have $100 that says that this is not going to change.

LEMONADE missed its window of opportunity.  The LEMONADE extensions are a 
joke in the vendor community.

The BIS model makes much more sense for mobile clients than the humungous 
kludge tower of LEMONADE.  BES makes even more sense for the fully 
integrated environment (will IETF get calendaring done before the sun 
novas??).  You can damn well bet that bean counters like it too.

Windows Mobile and iPhone allow you to consume IMAP/POP directly, and 
iPhone's MobileMe is like BES.  But I won't be at all surprised if Apple 
soon offers an "optional" MobileMeLite BIS service that works much better 
(from the end user viewpoint) than directly consuming IMAP/POP.

> Anyway, most of the extensions I implement nowadays are going to be used
> by webmails.

This assumes that the webmails will consume IMAP.  I see plenty of free 
webmails that do that, some (e.g., Web Alpine) smarter than others.  But 
none of the industrial webmails consume IMAP now, or have any intention of 
consuming IMAP except as part of a migration tool.

Our webmail doesn't consume IMAP.  Webmail and IMAP consume the same 
distributed store; there are operations in that store specifically for 
webmail, and others specifically for IMAP.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

From arnt@oryx.com  Thu Apr  9 01:05:52 2009
Return-Path: <arnt@oryx.com>
X-Original-To: morg@core3.amsl.com
Delivered-To: morg@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 5A1853A6A6E for <morg@core3.amsl.com>; Thu,  9 Apr 2009 01:05:52 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.593
X-Spam-Level: 
X-Spam-Status: No, score=-2.593 tagged_above=-999 required=5 tests=[AWL=0.005,  BAYES_00=-2.599, UNPARSEABLE_RELAY=0.001]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8SFrH6ReEch8 for <morg@core3.amsl.com>; Thu,  9 Apr 2009 01:05:51 -0700 (PDT)
Received: from kalyani.oryx.com (kalyani.oryx.com [195.30.37.30]) by core3.amsl.com (Postfix) with ESMTP id D5B933A6B69 for <morg@ietf.org>; Thu,  9 Apr 2009 01:05:50 -0700 (PDT)
Received: from kalyani.oryx.com (localhost [127.0.0.1]) by kalyani.oryx.com (Postfix) with ESMTP id 738FF2E0E8; Thu,  9 Apr 2009 10:06:53 +0200 (CEST)
Received: from arnt@oryx.com (HELO lochnagar) by kalyani.oryx.com (Archiveopteryx 3.1.0) with esmtp id 1239264413-53296-53284/6/33 (3 recipients); Thu, 9 Apr 2009 10:06:53 +0200
Message-Id: <4b1ZuJ9lDjeH7l278sVQaA.md5@lochnagar>
Date: Thu, 9 Apr 2009 10:07:16 +0200
From: Arnt Gulbrandsen <arnt@oryx.com>
To: Barry Leiba <barryleiba@computer.org>
References: <1234565453.6132.886.camel@timo-desktop> <74E86DC1-440F-42D2-BB42-FD2291F51672@mumbo.ca> <P+hWnMz5o6F5yG+pRevlCQ.md5@lochnagar.oryx.com> <UDJylvAUqX5dqLKmB+C1Qg.md5@lochnagar> <6c9fcc2a0904081229k11028061x67ef997fd33ba4ba@mail.gmail.com> <W7El7F9shiifFw/D6B5PKw.md5@lochnagar> <alpine.OSX.2.00.0904081346250.2675@hsinghsing.panda.com> <JpVMu9f7Ego+2plOHIR4DA.md5@lochnagar> <alpine.OSX.2.00.0904081437400.40001@hsinghsing.panda.com> <zJDgSl598yxuAQmgE7JgNA.md5@lochnagar> <6c9fcc2a0904081537w340954b6ld0608d6a38bc57a3@mail.gmail.com>
In-Reply-To: <6c9fcc2a0904081537w340954b6ld0608d6a38bc57a3@mail.gmail.com>
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset=iso-8859-1; format=flowed
Mime-Version: 1.0
Cc: morg@ietf.org
Subject: Re: [MORG] Why IMAP extensions are (not) used
X-BeenThere: morg@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: Messaging Organization <morg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/morg>, <mailto:morg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/morg>
List-Post: <mailto:morg@ietf.org>
List-Help: <mailto:morg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/morg>, <mailto:morg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 09 Apr 2009 08:05:52 -0000

Barry Leiba writes:
>>  Locking the mailbox in order to compute RECENT isn't free for=20
>>  anyone. Scanning for UNSEEN and EXISTS isn't free for anyone. Many=20
>>  databases and libraries that will use indexes to search quickly are=20
>>  available to everyone. Maybe the query planner I use is better than=20
>>  most. So what?
>
> If you have (say) a DB2 back-end to your mail store, a search of 350=20
> mailboxes might be very efficient.

Yes. That's my experience. I was surprised by how fast it was, to tell=20
the truth.

> If you have (say) a set of Unix mailboxes as the back-end to your mail=20
> store, a search of 350 mailboxes might be pretty much the same,=20
> whether you do it in one command or 700, apart from the protocol=20
> overhead.

And because I was surprised by the speed, I did some analysis to find=20
out what made the difference. It's not just =ABdb good, rest of world=20
bad=BB.

If that mbox server uses no indexing, then you're right.

If it uses lucene (IIRC that's what Timo mentioned using it for his mbox=20
store) for each mailbox, then the story will be different. In that case=20
you may see that SELECT is as much effort as the actual searching for=20
some important searches (and not much improvement for some other=20
searches) and that RTTs dominate the user experience. It depends on the=20
search expression, of course.

If it uses lucene, but has a per-user index instead of a per-mailbox=20
one, then the best-case improvement is even bigger, particularly when=20
the results are in few mailboxes.

Using a DB2-like backend will boost multmailbox searches for more and=20
more varied queries than just a lucene-like index. But the lucene-like=20
index will be really fast for an important subset of queries (such as a=20
simple SEARCH BODY x for all the user's mailboxes).

> Mark's point is that for a lot of these sorts of things, you kind of=20
> need to know something about the server to know whether doing [X] is=20
> a good idea or not.

Yes... and I have a feeling that time spent implementing and analysing=20
aids knowledge.

> Perhaps one might say that a server SHOULD only implement multimailbox=20
> search if it has the sort of back-end that makes it significantly=20
> more efficient. Or perhaps there should also be some sort of clue to=20
> the client as to the efficiency.

I think not: The efficiency is difficult to describe concisely, and=20
people will leanr from experience which buttons are best not pressed=20
when the db is big.

> Or perhaps clients and servers should do what they like, and the=20
> market will sort out which ones match users' expectations best.
>
> Whatever the decision, Mark's point, which is correct, is that you=20
> can't make assumptions. A client that wants to let a user search 350=20
> mailboxes will do it, one way or another. But it might or might not=20
> perform badly with or without the extension, and you can't know in=20
> advance which it'll be.

But you can write the code, run some searches, analyse the logfiles and=20
then you'll know quite a bit.

This is a digression. I'll stop now.

Arnt

From tss@iki.fi  Thu Apr  9 12:11:16 2009
Return-Path: <tss@iki.fi>
X-Original-To: morg@core3.amsl.com
Delivered-To: morg@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 3CF593A6B47 for <morg@core3.amsl.com>; Thu,  9 Apr 2009 12:11:16 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -6.499
X-Spam-Level: 
X-Spam-Status: No, score=-6.499 tagged_above=-999 required=5 tests=[AWL=0.100,  BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id I1+dupznb+QO for <morg@core3.amsl.com>; Thu,  9 Apr 2009 12:11:15 -0700 (PDT)
Received: from dovecot.org (dovecot.org [82.118.211.50]) by core3.amsl.com (Postfix) with ESMTP id 353C23A6A17 for <morg@ietf.org>; Thu,  9 Apr 2009 12:11:15 -0700 (PDT)
Received: from [10.4.192.51] (unknown [74.205.24.229]) by dovecot.org (Postfix) with ESMTP id 0649816472B7; Thu,  9 Apr 2009 22:12:20 +0300 (EEST)
From: Timo Sirainen <tss@iki.fi>
To: Arnt Gulbrandsen <arnt@oryx.com>
In-Reply-To: <4b1ZuJ9lDjeH7l278sVQaA.md5@lochnagar>
References: <1234565453.6132.886.camel@timo-desktop> <74E86DC1-440F-42D2-BB42-FD2291F51672@mumbo.ca> <P+hWnMz5o6F5yG+pRevlCQ.md5@lochnagar.oryx.com> <UDJylvAUqX5dqLKmB+C1Qg.md5@lochnagar> <6c9fcc2a0904081229k11028061x67ef997fd33ba4ba@mail.gmail.com> <W7El7F9shiifFw/D6B5PKw.md5@lochnagar> <alpine.OSX.2.00.0904081346250.2675@hsinghsing.panda.com> <JpVMu9f7Ego+2plOHIR4DA.md5@lochnagar> <alpine.OSX.2.00.0904081437400.40001@hsinghsing.panda.com> <zJDgSl598yxuAQmgE7JgNA.md5@lochnagar> <6c9fcc2a0904081537w340954b6ld0608d6a38bc57a3@mail.gmail.com> <4b1ZuJ9lDjeH7l278sVQaA.md5@lochnagar>
Content-Type: multipart/signed; micalg="pgp-sha1"; protocol="application/pgp-signature"; boundary="=-ffVFAVxkQx+fxXRSCjNp"
Date: Thu, 09 Apr 2009 15:12:18 -0400
Message-Id: <1239304338.5523.1455.camel@timo-desktop>
Mime-Version: 1.0
X-Mailer: Evolution 2.24.3 
Cc: morg@ietf.org
Subject: Re: [MORG] Why IMAP extensions are (not) used
X-BeenThere: morg@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: Messaging Organization <morg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/morg>, <mailto:morg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/morg>
List-Post: <mailto:morg@ietf.org>
List-Help: <mailto:morg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/morg>, <mailto:morg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 09 Apr 2009 19:11:16 -0000

--=-ffVFAVxkQx+fxXRSCjNp
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

On Thu, 2009-04-09 at 10:07 +0200, Arnt Gulbrandsen wrote:
> Using a DB2-like backend will boost multmailbox searches for more and=20
> more varied queries than just a lucene-like index. But the lucene-like=20
> index will be really fast for an important subset of queries (such as a=20
> simple SEARCH BODY x for all the user's mailboxes).

Just one point: Using Lucene or pretty much anything else for SEARCH
TEXT or SEARCH BODY violates RFC 3501, since they can't do substring
searches. SEARCH FUZZY helps here.


--=-ffVFAVxkQx+fxXRSCjNp
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEABECAAYFAkneSJIACgkQyUhSUUBVisk2CQCfScFXJaqsMLl39jGyunnCE/hv
ZoYAnRuFGrzTkB5p4f+AmW0u2+vpJ9j7
=yyBQ
-----END PGP SIGNATURE-----

--=-ffVFAVxkQx+fxXRSCjNp--