From nobody Tue Apr 15 12:35:29 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4B4C91A0488 for ; Tue, 15 Apr 2014 12:35:28 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.8 X-Spam-Level: X-Spam-Status: No, score=0.8 tagged_above=-999 required=5 tests=[BAYES_50=0.8] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XNlA7MEUU8eW for ; Tue, 15 Apr 2014 12:35:27 -0700 (PDT) Received: from strange.aox.org (strange.aox.org [IPv6:2001:4d88:100c::1]) by ietfa.amsl.com (Postfix) with ESMTP id DD6751A0186 for ; Tue, 15 Apr 2014 12:35:26 -0700 (PDT) Received: from fri.gulbrandsen.priv.no (localhost [127.0.0.1]) by strange.aox.org (Postfix) with ESMTP id 4FA1FF8C001; Tue, 15 Apr 2014 19:35:22 +0000 (UTC) Received: from arnt@gulbrandsen.priv.no by fri.gulbrandsen.priv.no (Archiveopteryx 3.2.0) with esmtpsa id 1397590521-26712-26712/11/3; Tue, 15 Apr 2014 19:35:21 +0000 From: Arnt Gulbrandsen To: imapext@ietf.org Date: Tue, 15 Apr 2014 21:35:20 +0200 User-Agent: Trojita/v0.3.96-176-g713fcbb; Qt/4.8.4; X11; Linux; Ubuntu 13.10 Mime-Version: 1.0 Message-Id: <82b53701-77da-4bcb-8ca7-14fae2e5f9d3@gulbrandsen.priv.no> Content-Type: text/plain; charset=utf-8; format=flowed Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/CYn4p136A34qhOVp0VNFrYj2f3Q Subject: [imapext] what we need is obviously .imap, so we can have dinner.friendsof.imap X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Apr 2014 19:35:28 -0000 Wrote one of my registrars: "You may be aware that a batch of new Top Level Domains (TLDs) from ICANN have started rolling out" and the first mentioned domain is: .EMAIL God have mercy. From nobody Tue Apr 15 14:24:17 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0DA341A0499 for ; Tue, 15 Apr 2014 14:24:16 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.528 X-Spam-Level: X-Spam-Status: No, score=0.528 tagged_above=-999 required=5 tests=[BAYES_50=0.8, RCVD_IN_DNSWL_NONE=-0.0001, RP_MATCHES_RCVD=-0.272] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AJHF-T4V9bvV for ; Tue, 15 Apr 2014 14:24:15 -0700 (PDT) Received: from egssmtp02.att.com (egssmtp02.att.com [144.160.128.166]) by ietfa.amsl.com (Postfix) with ESMTP id EB1811A0488 for ; Tue, 15 Apr 2014 14:24:14 -0700 (PDT) Received: from mailgw1.maillennium.att.com (maillennium.att.com [135.25.114.99]) by egssmtp02.att.com ( EGS R6 8.14.5/8.14.5) with ESMTP id s3FLOBWo028056 for ; Tue, 15 Apr 2014 14:24:11 -0700 Received: from vpn-135-70-99-191.vpn.swst.att.com ([135.70.99.191]) by maillennium.att.com (mailgw1) with ESMTP id <20140415212406gw100j0cike>; Tue, 15 Apr 2014 21:24:10 +0000 X-Originating-IP: [135.70.99.191] Message-ID: <534DA375.6060601@att.com> Date: Tue, 15 Apr 2014 17:24:05 -0400 From: Tony Hansen User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: imapext@ietf.org References: <82b53701-77da-4bcb-8ca7-14fae2e5f9d3@gulbrandsen.priv.no> In-Reply-To: <82b53701-77da-4bcb-8ca7-14fae2e5f9d3@gulbrandsen.priv.no> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/VdozyKY8itVYEjRqVWJqv1n47og Subject: Re: [imapext] what we need is obviously .imap, so we can have dinner.friendsof.imap X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Apr 2014 21:24:16 -0000 On 4/15/14, 3:35 PM, Arnt Gulbrandsen wrote: > Wrote one of my registrars: "You may be aware that a batch of new Top > Level Domains (TLDs) from ICANN have started rolling out" and the > first mentioned domain is: > > .EMAIL > > God have mercy. +1000 for dinner.friendsof.imap :-) Tony Hansen From nobody Fri Apr 25 05:30:32 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F34EA1A0494 for ; Fri, 25 Apr 2014 05:30:30 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.8 X-Spam-Level: X-Spam-Status: No, score=0.8 tagged_above=-999 required=5 tests=[BAYES_50=0.8] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pZCnNNd1u70g for ; Fri, 25 Apr 2014 05:30:28 -0700 (PDT) Received: from strange.aox.org (strange.aox.org [IPv6:2001:4d88:100c::1]) by ietfa.amsl.com (Postfix) with ESMTP id 843351A048C for ; Fri, 25 Apr 2014 05:30:28 -0700 (PDT) Received: from fri.gulbrandsen.priv.no (localhost [127.0.0.1]) by strange.aox.org (Postfix) with ESMTP id 60E32FA006B; Fri, 25 Apr 2014 12:30:21 +0000 (UTC) Received: from arnt@gulbrandsen.priv.no by fri.gulbrandsen.priv.no (Archiveopteryx 3.2.0) with esmtpsa id 1398429020-29782-29782/12/5; Fri, 25 Apr 2014 12:30:20 +0000 From: Arnt Gulbrandsen To: imapext@ietf.org Date: Fri, 25 Apr 2014 14:30:20 +0200 User-Agent: Trojita/v0.3.96-176-g713fcbb; Qt/4.8.4; X11; Linux; Ubuntu 13.10 Mime-Version: 1.0 Message-Id: <0627dd7a-2203-40b6-b1c0-5b58613cbad8@gulbrandsen.priv.no> Content-Type: multipart/mixed; boundary="trojita=_2e6fea95-2536-4231-a17b-b4190756e876" Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/uhwzhQNWqTHoS7gUkqENqzQuknU Subject: [imapext] a very simple little imap extension, really quite magnificently simple X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 12:30:31 -0000 --trojita=_2e6fea95-2536-4231-a17b-b4190756e876 Content-Type: text/plain; charset=utf-8; format=flowed Hi, as some of you will have discovered, opening large mailboxes takes real time. So I discussed that with Brandon and wrote a draft, really short, perhaps it's actually shorter than the boilerplate. The gist: "If a client runs ENABLE UIDONLY, then that client cannot later use any MSNs in any commands, and if syntax forces the server needs to send an MSN, then it just sends a phony number." There are a couple of details. Attached. Comments most welcome. Arnt --trojita=_2e6fea95-2536-4231-a17b-b4190756e876 Content-Type: text/plain Content-Disposition: attachment; filename=draft-gulbrandsen-imap-uidonly-00.txt Content-Transfer-Encoding: quoted-printable Network Working Group Arnt Gulbrandsen Internet-Draft April 2014 Intended Status: Proposed Standard The IMAP UIDONLY Extension draft-gulbrandsen-imap-uidonly-00.txt Status of this Memo This Internet-Draft is submitted to IETF in full conformance with the provisions of BCP 78 and BCP 79. Copyright (c) 2014 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet- Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft expires in October 2014. Gulbrandsen Expires September 2014 [Page 1] =0C Internet-draft April 2014 Abstract Opening a large mailbox in IMAP can take mailbox; 30 seconds is realistic if the mailbox contains ten million messages. Most of that time is needed to number the messages consecutively. This extension provides a way to avoid having to number the messages consecutively. 1. Conventions Used in This Document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. Formal syntax is defined by [RFC5234]. Example lines prefaced by "C:" are sent by the client and ones prefaced by "S:" by the server. The five characters [...] means that something has been elided. 2. Overview Some/many clients do not use IMAP MSNs at all, and rely entirely on UIDs. Some servers (notably gmail) take a long time to open very large mailboxes, since they have to compute an MSN for each message. This extension allows a client to declare that it only uses UIDs, and that the server can skip all processing related to MSNs. A client declares that it wishes to use this extension with the command ENABLE UIDONLY. 3.1. Client requirements A client hat has issued ENABLE UIDONLY cannot use MSNs in any commands. This means that it MUST NOT use the FETCH, STORE and SEARCH commmands, that it MUST NOT use MSNs as search-key in the UID SEARCH command, and that it MUST NOT expect to receive an EXISTS response. While the client will still receive MSNs, it MUST NOT expect them to mean anything. (RFC editor: Remove this parenthesis. It seems to be easier to add support for UIDONLY by disregarding parsed MSNs than to change both the parser and the layer above, so I chose to leave dummy Gulbrandsen Expires September 2014 [Page 2] =0C Internet-draft April 2014 MSNs in.) 3.1. Server requirements These rules apply from the time the server has received an ENABLE command that enables UIDONLY until the TCP connection is closed. The server MUST send the number 999999999999999 when it needs to send an MSN. [RFC Editor: Change 999999999999999 to the number of this RFC.] The server MUST send the UID data item in all FETCH responses, including untagged FETCH responses. The server MUST respond with BAD to any command that uses an MSN, including a UID SEARCH command that uses MSNs. This search command would retrieve the UID of the last message: C: a uid search * S: a BAD [CLIENTBUG] You promised not to use MSNs The following alternative is legal, since in this case the * is a UID rather than an MSN: C: a uid search uid * C: * SEARCH 10240901 S: a OK done The server MUST NOT send EXISTS responses at any time (not even during SELECT/EXAMINE processing), and MUST send UIDNEXT when it would have sent EXISTS. For example: C: a idle S: + S: * OK [UIDNEXT 10240902] next UID The server MUST send VANISHED responses (see [RFC5162], section 3.6) where it would ordinarily send EXPUNGED responses. ([RFC5162] defines a large extension called Quick Resync, of which VANISHED is a small part. Note that neither the server nor the client need support or use Quick Resync in general.) 5. Formal Syntax The following syntax specification uses the Augmented Backus-Naur Gulbrandsen Expires September 2014 [Page 3] =0C Internet-draft April 2014 Form (ABNF) notation as specified in [RFC5234]. [RFC3501] defines the non-terminal "capability". Except as noted otherwise, all alphabetic characters are case- insensitive. The use of upper or lower case characters to define token strings is for editorial clarity only. Implementations MUST accept these strings in a case-insensitive fashion. capability =3D/ "UIDONLY" 6. Security Considerations This document is believed not to have any security implications. 7. IANA Considerations The IANA is requested to add UIDONLY to the list of IMAP extensions, http://www.iana.org/assignments/imap4-capabilities. 8. Acknowledgements Thanks to Brandon Long for helpful comments. 9. Normative References [RFC2119] Bradner, "Key words for use in RFCs to Indicate Requirement Levels", RFC 2119, Harvard University, March 1997. [RFC3501] Crispin, "Internet Message Access Protocol - Version 4rev1", RFC 3501, University of Washington, June 2003. [RFC5162] Melnikov, A. and D. Cridland, "IMAP4 Extensions for Quick Mailbox Resynchronization", RFC 5162, March 2008. [RFC5234] Crocker, D. and P. Overell, "Augmented BNF for Syntax Specifications: ABNF", RFC 5234, January 2008. 10. Author's Address Arnt Gulbrandsen Schweppermannstr. 8 D-81671 Muenchen Gulbrandsen Expires September 2014 [Page 4] =0C Internet-draft April 2014 Germany Email: arnt@gulbrandsen.priv.no Gulbrandsen Expires September 2014 [Page 5] =0C Internet-draft April 2014 (RFC Editor: Please delete everything after this point) Open Issues None. Changes since -00 Gulbrandsen Expires September 2014 [Page 6] =0C --trojita=_2e6fea95-2536-4231-a17b-b4190756e876-- From nobody Fri Apr 25 05:43:00 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 614071A04A8 for ; Fri, 25 Apr 2014 05:42:58 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.7 X-Spam-Level: X-Spam-Status: No, score=-2.7 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NpSPpu4Ggf9y for ; Fri, 25 Apr 2014 05:42:56 -0700 (PDT) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) by ietfa.amsl.com (Postfix) with ESMTP id 39ABC1A048D for ; Fri, 25 Apr 2014 05:42:56 -0700 (PDT) Received: from compute6.internal (compute6.nyi.mail.srv.osa [10.202.2.46]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id B2BFD212CE; Fri, 25 Apr 2014 08:42:48 -0400 (EDT) Received: from web2 ([10.202.2.212]) by compute6.internal (MEProxy); Fri, 25 Apr 2014 08:42:49 -0400 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=fastmail.fm; h= message-id:from:to:mime-version:content-transfer-encoding :content-type:in-reply-to:references:subject:date; s=mesmtp; bh= H4SWtsoJ4TepR3aPVn6ZGOYg0ek=; b=PxCP/i0Wpacg/l3xPSnb9QqcHYip0XLG VKP8tRMqGjHDUH0Y9tZYpMVnUjdGtBqt9jhNirTsw1YcbiN+nvnFiPzCF0bIXYaC QsXadFRRgqrYWfcAp112QyQuik7r5gO8th61KnqV8SwYmNkHhnCH56CJT9AeqrB2 aO93Dm4Yqxs= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=message-id:from:to:mime-version :content-transfer-encoding:content-type:in-reply-to:references :subject:date; s=smtpout; bh=H4SWtsoJ4TepR3aPVn6ZGOYg0ek=; b=NIj 8+OZsUzdy2HhsPrf1vjq3nF5LbqJFQG2YjjB4uCOWv3CUFiEnBcCkRDIeYyyiPhY SqnP0D5pu2CVhTpeDU3uQaPIy35j5T6ohpB4laSd77y6PyLG09FHMlgCXLvcdpJc 891IdQGchinKusMEvjseuB5ZFkoa/+TOU7lKuDUA= Received: by web2.nyi.mail.srv.osa (Postfix, from userid 99) id A0CCD540139; Fri, 25 Apr 2014 08:42:48 -0400 (EDT) Message-Id: <1398429768.12216.110316901.769BD1B9@webmail.messagingengine.com> X-Sasl-Enc: wKFJsgezbBjlhp+Gs+KE4Bckh0HaCwUuEFmQjk0dnU70 1398429768 From: Bron Gondwana To: Arnt Gulbrandsen , imapext@ietf.org MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain X-Mailer: MessagingEngine.com Webmail Interface - ajax-aafac2c3 In-Reply-To: <0627dd7a-2203-40b6-b1c0-5b58613cbad8@gulbrandsen.priv.no> References: <0627dd7a-2203-40b6-b1c0-5b58613cbad8@gulbrandsen.priv.no> Date: Fri, 25 Apr 2014 22:42:48 +1000 Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/ipci-OvPYn86wKxgZsZRJPj0_54 Subject: Re: [imapext] a very simple little imap extension, really quite magnificently simple X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 12:42:58 -0000 On Fri, Apr 25, 2014, at 10:30 PM, Arnt Gulbrandsen wrote: > Hi, > > as some of you will have discovered, opening large mailboxes takes real > time. So I discussed that with Brandon and wrote a draft, really short, > perhaps it's actually shorter than the boilerplate. The gist: "If a client > runs ENABLE UIDONLY, then that client cannot later use any MSNs in any > commands, and if syntax forces the server needs to send an MSN, then it > just sends a phony number." There are a couple of details. I'm sold on the idea. MSNs are a wart. They require the server to calculate and maintain state that the client may not actually need. I would also have been happy with a different type of SELECT command, but entire session ENABLE command makes sense too. It's a pity you can't use 0 as the MSN, but I don't see a problem with just using 1, rather than some random 4-5 digit number. Bron. -- Bron Gondwana brong@fastmail.fm From nobody Fri Apr 25 06:47:25 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2CBB51A01DF for ; Fri, 25 Apr 2014 06:47:18 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.871 X-Spam-Level: X-Spam-Status: No, score=-1.871 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, MIME_8BIT_HEADER=0.3, MIME_QP_LONG_LINE=0.001, RP_MATCHES_RCVD=-0.272] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jXeK_GVcLAGR for ; Fri, 25 Apr 2014 06:47:08 -0700 (PDT) Received: from latimerie.flaska.net (latimerie.flaska.net [IPv6:2a02:2b88:2:1::4a7:333]) by ietfa.amsl.com (Postfix) with ESMTP id D9F0E1A01E9 for ; Fri, 25 Apr 2014 06:47:06 -0700 (PDT) Received: by latimerie.flaska.net (Postfix, from userid 1000) id E9445615CE; Fri, 25 Apr 2014 15:46:54 +0200 (CEST) From: =?iso-8859-1?Q?Jan_Kundr=E1t?= To: Date: Fri, 25 Apr 2014 15:46:57 +0200 User-Agent: Trojita/v0.4.1-145-g3e4c215; Qt/4.8.5; X11; Linux; MIME-Version: 1.0 Message-ID: <8e45ff5c-1173-4119-bd53-47194d0f056b@flaska.net> In-Reply-To: <0627dd7a-2203-40b6-b1c0-5b58613cbad8@gulbrandsen.priv.no> References: <0627dd7a-2203-40b6-b1c0-5b58613cbad8@gulbrandsen.priv.no> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/N6dLZxcuey-E-kn1WlgoQUxSa9Y Subject: Re: [imapext] =?iso-8859-1?q?a_very_simple_little_imap_extension=2C_r?= =?iso-8859-1?q?eally_quite_magnificently_simple?= X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 13:47:18 -0000 On Friday, 25 April 2014 14:30:20 CEST, Arnt Gulbrandsen wrote: > Opening a large mailbox in IMAP can take mailbox; 30 seconds is "...can take a long time"? > This extension provides a way to avoid having to number the messages > consecutively. Are there any other mail servers out there which actually have a problem=20 with sequence numbers, besides GMail? Just wondering what is more probable=20= -- GMail adding $code which makes this faster because it's a pain for their=20= clients, or $developers updating N clients to make use of this extension. > command, and that it MUST NOT expect to receive an EXISTS response. How is the client supposed to notice that a new mail has arrived, then? I=20 see it addressed later in the doc, but it startled me when I read this. > While the client will still receive MSNs, it MUST NOT expect them to > mean anything. (RFC editor: Remove this parenthesis. It seems to be > easier to add support for UIDONLY by disregarding parsed MSNs than to > change both the parser and the layer above, so I chose to leave dummy > MSNs in.) I dislike this a lot. If the MSNs do not mean anything, just send a BAD and=20= possibly even terminate the connection. Otherwise the garbage which is sent=20= will only lead to further confusion of a non-compliant client. > These rules apply from the time the server has received an ENABLE > command that enables UIDONLY until the TCP connection is closed. "until the IMAP session terminates". There's no requirement for IMAP to run=20= over TCP. In fact, I used to use IMAP-over-SSH-with-PREAUTH on a daily=20 basis for many years, and only stopped when they cut down the SSH access. I=20= expect to use that in future as well. > The server MUST send the UID data item in all FETCH responses, > including untagged FETCH responses. Which means that you effectively require QRESYNC and its VANISHED. Without=20= that option it won't work. I see that you talk about that later in the=20 text, but it should probably be a hard dep on 5162-bis. Given that this extension disables EXISTS, you need soemthing like ARRIVED=20= [1]. This change on its own will complicate this dratf a lot. Also, what about STATUS and querying the offset of first message having a=20 certain property through that? Or even STATUS EXISTS? > The server MUST respond with BAD to any command that uses an MSN, Add a requirement for [CLIENTBUG], please. We have these response codes for=20= a reason. > including a UID SEARCH command that uses MSNs. This search command > would retrieve the UID of the last message: "...but the reference to the "last message" is specified as a MSN." > The server MUST NOT send EXISTS responses at any time (not even > during SELECT/EXAMINE processing), and MUST send UIDNEXT when it > would have sent EXISTS. And the client doesn't know how many messages are in the mailbox anymore.=20 That's a rather radical change. I don't like it at all, it's a nice=20 safeguard to detect broken sync and this extension takes it away. > For example: > > C: a idle > S: + > S: * OK [UIDNEXT 10240902] next UID This *will* lead to race conditions. Basically, this extension makes an=20 unspoken assumption on UIDs being assigned in a consecutive manner, without=20= any gaps. I bet there is at least one server implementaiton which can sleep=20= and will notice that two messages have arrived and the first of them got=20 deleted. How do you convey that? > The server MUST send VANISHED responses (see [RFC5162], section 3.6) > where it would ordinarily send EXPUNGED responses. ([RFC5162] defines > a large extension called Quick Resync, of which VANISHED is a small > part. Note that neither the server nor the client need support or use > Quick Resync in general.) I don't like mandating VANISHED without also requiring the rest of the=20 QRESYNC functionality. I for one cannot imagine how to sync a mailbox at=20 all in at least semi efficient manner when I cannot rely on UIDNEXT/EXISTS=20= combination and without QRESYNC's availability at the same time. > This document is believed not to have any security implications. This extension, as-is, opens up a few opportunities for client and server=20 to go out of sync, and potentially could lead to loss of data. When these=20 problems are mitigated, the doc IMHO still needs a brief section saying how=20= that was done and what bugs and gotchas are expected when an implementor=20 botches up something. > [RFC5162] Melnikov, A. and D. Cridland, "IMAP4 Extensions for Quick > Mailbox Resynchronization", RFC 5162, March 2008. Would be cool to have a reference to 5162-bis when it gets approved. Cheers, Jan [1] http://tools.ietf.org/html/draft-kundrat-qresync-arrived-00 --=20 Trojit=C3=A1, a fast Qt IMAP e-mail client -- http://trojita.flaska.net/ From nobody Fri Apr 25 09:57:53 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1AC941A055D for ; Fri, 25 Apr 2014 09:57:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.6 X-Spam-Level: X-Spam-Status: No, score=-1.6 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, MIME_8BIT_HEADER=0.3] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QtdlR3dzOtSy for ; Fri, 25 Apr 2014 09:57:48 -0700 (PDT) Received: from strange.aox.org (strange.aox.org [80.244.248.170]) by ietfa.amsl.com (Postfix) with ESMTP id 4D7841A03B3 for ; Fri, 25 Apr 2014 09:57:47 -0700 (PDT) Received: from fri.gulbrandsen.priv.no (localhost [127.0.0.1]) by strange.aox.org (Postfix) with ESMTP id 67091FA006B; Fri, 25 Apr 2014 16:57:40 +0000 (UTC) Received: from arnt@gulbrandsen.priv.no by fri.gulbrandsen.priv.no (Archiveopteryx 3.2.0) with esmtpsa id 1398445059-29782-29782/12/7; Fri, 25 Apr 2014 16:57:39 +0000 From: Arnt Gulbrandsen To: Jan =?ISO-8859-1?q?Kundr=E1t?= Date: Fri, 25 Apr 2014 18:57:38 +0200 User-Agent: Trojita/v0.3.96-176-g713fcbb; Qt/4.8.4; X11; Linux; Ubuntu 13.10 Mime-Version: 1.0 Message-Id: <7cb76610-37e3-4903-bdc1-6d2403e27cff@gulbrandsen.priv.no> In-Reply-To: <8e45ff5c-1173-4119-bd53-47194d0f056b@flaska.net> References: <0627dd7a-2203-40b6-b1c0-5b58613cbad8@gulbrandsen.priv.no> <8e45ff5c-1173-4119-bd53-47194d0f056b@flaska.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/DSRABHFiwY2Deho6LjOtrrnyEL0 Cc: imapext@ietf.org Subject: Re: [imapext] a very simple little imap extension, really quite magnificently simple X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 16:57:50 -0000 On Friday, April 25, 2014 3:46:57 PM CEST, Jan Kundr=C3=A1t wrote: >> This extension provides a way to avoid having to number the messages >> consecutively. > > Are there any other mail servers out there which actually have=20 > a problem with sequence numbers, besides GMail? Just wondering=20 > what is more probable -- GMail adding $code which makes this=20 > faster because it's a pain for their clients, or $developers=20 > updating N clients to make use of this extension. I expect it hits everyone who tries to have n million messages in a = single=20 mailbox. Storing MSNs persistently so you don't have to recompute at session = start=20 (and at the same time supporting simultaneous sessions) was simply too=20 difficult for me. >> command, and that it MUST NOT expect to receive an EXISTS response. > > How is the client supposed to notice that a new mail has=20 > arrived, then? I see it addressed later in the doc, but it=20 > startled me when I read this. So you learned it twenty lines later ;) >> While the client will still receive MSNs, it MUST NOT expect them to >> mean anything. (RFC editor: Remove this parenthesis. It seems to be >> easier to add support for UIDONLY by disregarding parsed MSNs than to >> change both the parser and the layer above, so I chose to leave dummy >> MSNs in.) > > I dislike this a lot. If the MSNs do not mean anything, just=20 > send a BAD and possibly even terminate the connection. Otherwise=20 > the garbage which is sent will only lead to further confusion of=20 > a non-compliant client. The client cannot send BAD. The server does send BAD when it receives a BAD from the client. >> The server MUST send the UID data item in all FETCH responses, >> including untagged FETCH responses. > > Which means that you effectively require QRESYNC and its=20 > VANISHED. Without that option it won't work. I see that you talk=20 > about that later in the text, but it should probably be a hard=20 > dep on 5162-bis. Disagree violently. 5162 is MUCH too complex to require all of it. > Given that this extension disables EXISTS, you need soemthing=20 > like ARRIVED [1]. This change on its own will complicate this=20 > dratf a lot. Why? What's wrong with UIDNEXT? > Also, what about STATUS and querying the offset of first=20 > message having a certain property through that? Or even STATUS=20 > EXISTS? I don't think that matters. If a client issues STATUS EXISTS, it kills any performance benefit it = got=20 by issuing ENABLE UIDONLY. That's a problem for the client, but not an=20 interop problem, so nothing that requires rules here. >> The server MUST respond with BAD to any command that uses an MSN, > > Add a requirement for [CLIENTBUG], please. We have these=20 > response codes for a reason. Do any other RFCs require CLIENTBUG? (Not suggest, or show in examples.=20 Require.) >> including a UID SEARCH command that uses MSNs. This search command >> would retrieve the UID of the last message: > > "...but the reference to the "last message" is specified as a MSN." > >> The server MUST NOT send EXISTS responses at any time (not even >> during SELECT/EXAMINE processing), and MUST send UIDNEXT when it >> would have sent EXISTS. > > And the client doesn't know how many messages are in the=20 > mailbox anymore. That's a rather radical change. I don't like it=20 > at all, it's a nice safeguard to detect broken sync and this=20 > extension takes it away. Not knowing how many messages there are is a necessary effect of this. = It's=20 not an accidental problem, it's necessary. The point of this is to stop counting messages. If you stop counting, = you=20 don't know (at least not precisely) how many you have. It's as simple as=20 that. >> For example: >>=20 >> C: a idle >> S: + >> S: * OK [UIDNEXT 10240902] next UID > > This *will* lead to race conditions. Basically, this extension=20 > makes an unspoken assumption on UIDs being assigned in a=20 > consecutive manner, without any gaps. Why do you think so? > I bet there is at least=20 > one server implementaiton which can sleep and will notice that=20 > two messages have arrived and the first of them got deleted. How=20 > do you convey that? * VANISHED 24134789 * OK [UIDNEXT 24135791] > I don't like mandating VANISHED without also requiring the rest=20 > of the QRESYNC functionality. I for one cannot imagine how to=20 > sync a mailbox at all in at least semi efficient manner when I=20 > cannot rely on UIDNEXT/EXISTS combination and without QRESYNC's=20 > availability at the same time. I can show you code that does that, OK? > This extension, as-is, opens up a few opportunities for client=20 > and server to go out of sync, and potentially could lead to loss=20 > of data. No, if you lose sync the worst that happens is that you send uid store=20 +flags \deleted for something that's already deleted. Remember: With this extension there are no MSNs any more. There's no way = to=20 store +flags on the wrong message. Arnt From nobody Fri Apr 25 10:07:49 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 379031A0680 for ; Fri, 25 Apr 2014 10:07:42 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id W96YFEQmY5DE for ; Fri, 25 Apr 2014 10:07:39 -0700 (PDT) Received: from strange.aox.org (strange.aox.org [80.244.248.170]) by ietfa.amsl.com (Postfix) with ESMTP id 26F1F1A0664 for ; Fri, 25 Apr 2014 10:07:39 -0700 (PDT) Received: from fri.gulbrandsen.priv.no (localhost [127.0.0.1]) by strange.aox.org (Postfix) with ESMTP id 378DBFA009A; Fri, 25 Apr 2014 17:07:32 +0000 (UTC) Received: from arnt@gulbrandsen.priv.no by fri.gulbrandsen.priv.no (Archiveopteryx 3.2.0) with esmtpsa id 1398445651-29782-29782/12/8; Fri, 25 Apr 2014 17:07:31 +0000 From: Arnt Gulbrandsen To: imapext@ietf.org Date: Fri, 25 Apr 2014 19:07:30 +0200 User-Agent: Trojita/v0.3.96-176-g713fcbb; Qt/4.8.4; X11; Linux; Ubuntu 13.10 Mime-Version: 1.0 Message-Id: <6c80b3a4-68c8-4d52-8069-eeb93b6d55b2@gulbrandsen.priv.no> In-Reply-To: <1398429768.12216.110316901.769BD1B9@webmail.messagingengine.com> References: <0627dd7a-2203-40b6-b1c0-5b58613cbad8@gulbrandsen.priv.no> <1398429768.12216.110316901.769BD1B9@webmail.messagingengine.com> Content-Type: text/plain; charset=utf-8; format=flowed Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/i5o0gBalicgN3nYpkOv5gC67bd8 Subject: Re: [imapext] a very simple little imap extension, really quite magnificently simple X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 17:07:42 -0000 On Friday, April 25, 2014 2:42:48 PM CEST, Bron Gondwana wrote: > It's a pity you can't use 0 as the MSN, but I don't see a problem with just > using 1, rather than some random 4-5 digit number. I don't see any advantage in 1. The strange number is recognizable; if you see that foo == 8214 in the debugger you have a fair chance of remembering that you recently implemented RFC8214 and realise that the problem you're chasing is related to that. Ditto google for "imap 8214". Arnt From nobody Fri Apr 25 10:29:19 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C310B1A06B8 for ; Fri, 25 Apr 2014 10:29:14 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.871 X-Spam-Level: X-Spam-Status: No, score=-1.871 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, MIME_8BIT_HEADER=0.3, MIME_QP_LONG_LINE=0.001, RP_MATCHES_RCVD=-0.272] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Dvfx_iqLAFhX for ; Fri, 25 Apr 2014 10:29:10 -0700 (PDT) Received: from latimerie.flaska.net (latimerie.flaska.net [IPv6:2a02:2b88:2:1::4a7:333]) by ietfa.amsl.com (Postfix) with ESMTP id 8D9E31A028B for ; Fri, 25 Apr 2014 10:29:09 -0700 (PDT) Received: by latimerie.flaska.net (Postfix, from userid 1000) id EE5E1615CE; Fri, 25 Apr 2014 19:28:56 +0200 (CEST) From: =?iso-8859-1?Q?Jan_Kundr=E1t?= To: Date: Fri, 25 Apr 2014 19:28:59 +0200 User-Agent: Trojita/v0.4.1-182-g7970633; Qt/4.8.5; X11; Linux; MIME-Version: 1.0 Message-ID: In-Reply-To: <7cb76610-37e3-4903-bdc1-6d2403e27cff@gulbrandsen.priv.no> References: <0627dd7a-2203-40b6-b1c0-5b58613cbad8@gulbrandsen.priv.no> <8e45ff5c-1173-4119-bd53-47194d0f056b@flaska.net> <7cb76610-37e3-4903-bdc1-6d2403e27cff@gulbrandsen.priv.no> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/CziR1BxK3QkAEhyyD95dw-TWh5I Subject: Re: [imapext] =?iso-8859-1?q?a_very_simple_little_imap_extension=2C_r?= =?iso-8859-1?q?eally_quite_magnificently_simple?= X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 17:29:15 -0000 On Friday, 25 April 2014 18:57:38 CEST, Arnt Gulbrandsen wrote: > Storing MSNs persistently so you don't have to recompute at=20 > session start (and at the same time supporting simultaneous=20 > sessions) was simply too difficult for me. Given that you already have to track UIDs, I find it hard that it is=20 expensive to take a look at the size of their set. > The client cannot send BAD. > > The server does send BAD when it receives a BAD from the client. The draft also says that the server should return a strange number when it=20= would otherwise return a MSN. I object to that. In my opinion the server=20 should signal this in a drastic way, including terminating the session if=20 everything else fails. Sending back a mock-up MSN is about as evil as one=20 could go in my book. >> Given that this extension disables EXISTS, you need soemthing=20 >> like ARRIVED [1]. This change on its own will complicate this=20 >> dratf a lot. > > Why? What's wrong with UIDNEXT? The fact that you can have gaps in UID range. The draft assumes that UIDs=20 are allocated by adding one the the previous UID. Your whole scheme falls=20 apart when this requirement does not hold. That requirement is not=20 documented anywhere in the draft. If the UIDs are not allocated consequently, and the highest UID right now=20 is 12345, what does the following mean: =09* OK [UIDNEXT 12500] guess it >> Also, what about STATUS and querying the offset of first=20 >> message having a certain property through that? Or even STATUS=20 >> EXISTS? > > I don't think that matters. > > If a client issues STATUS EXISTS, it kills any performance=20 > benefit it got by issuing ENABLE UIDONLY. That's a problem for=20 > the client, but not an interop problem, so nothing that requires=20 > rules here. So this is an extension whose benefit is immediately neutralized by a=20 client which wants to show the user the total number of messages. To me,=20 that sounds like a very basic operation. >>> The server MUST respond with BAD to any command that uses an MSN, >>=20 >> Add a requirement for [CLIENTBUG], please. We have these=20 >> response codes for a reason. > > Do any other RFCs require CLIENTBUG? (Not suggest, or show in=20 > examples. Require.) I'm not sure how that is relevant. This proposal is about designing an=20 extension which radically changes how IMAP operates. Why not at least=20 *mandate* a machine readable way of saying "hey client, you've just shot=20 yourself in the foot"? >> And the client doesn't know how many messages are in the=20 >> mailbox anymore. That's a rather radical change. I don't like=20 >> it at all, it's a nice safeguard to detect broken sync and this=20 >> extension takes it away. > > Not knowing how many messages there are is a necessary effect=20 > of this. It's not an accidental problem, it's necessary. > > The point of this is to stop counting messages. If you stop=20 > counting, you don't know (at least not precisely) how many you=20 > have. It's as simple as that. OK, at this point I cannot imagine how such a client is supposed to work=20 *at all*. Let's start with a blank state. The client knows nothing about=20 the mailbox. It opens it and presumably wants to show a list of messages to=20= the user. What is the client supposed to do now? What if it wants to show just a subset of the mailbox, perhaps last 1k=20 items (actually it doesn't have to be 1k, anything between 500 and 3000 is=20= probably fine -- it needs *something*, but not the full 4M mailbox). How to=20= achieve that? What happens when a mail arrives? How is a resynchronization after being offline for a while going to look=20 like? My answers and comments to the rest of your mail are very much dependant on=20= how you propose the client to behave. I can imagine that I have all sorts=20 of assumptions because I know how my client works. You might be coming from=20= a different background. So I'm interested in seeing how the proposed mode=20 of operation is supposed to actually work. I just don't get it now. Cheers, Jan --=20 Trojit=C3=A1, a fast Qt IMAP e-mail client -- http://trojita.flaska.net/ From nobody Fri Apr 25 13:45:36 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 42C4A1A06A8 for ; Fri, 25 Apr 2014 13:45:33 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UNfKuMc-HfKb for ; Fri, 25 Apr 2014 13:45:30 -0700 (PDT) Received: from strange.aox.org (strange.aox.org [IPv6:2001:4d88:100c::1]) by ietfa.amsl.com (Postfix) with ESMTP id 1AA531A06A9 for ; Fri, 25 Apr 2014 13:45:29 -0700 (PDT) Received: from fri.gulbrandsen.priv.no (localhost [127.0.0.1]) by strange.aox.org (Postfix) with ESMTP id 1357BFA009A; Fri, 25 Apr 2014 20:45:22 +0000 (UTC) Received: from arnt@gulbrandsen.priv.no by fri.gulbrandsen.priv.no (Archiveopteryx 3.2.0) with esmtpsa id 1398458721-29782-29782/12/9; Fri, 25 Apr 2014 20:45:21 +0000 From: Arnt Gulbrandsen To: imapext@ietf.org Date: Fri, 25 Apr 2014 22:45:21 +0200 User-Agent: Trojita/v0.3.96-176-g713fcbb; Qt/4.8.4; X11; Linux; Ubuntu 13.10 Mime-Version: 1.0 Message-Id: In-Reply-To: References: <0627dd7a-2203-40b6-b1c0-5b58613cbad8@gulbrandsen.priv.no> <8e45ff5c-1173-4119-bd53-47194d0f056b@flaska.net> <7cb76610-37e3-4903-bdc1-6d2403e27cff@gulbrandsen.priv.no> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/LICLzL02T22jiJHv4O2wNXr5k7w Subject: Re: [imapext] a very simple little imap extension, really quite magnificently simple X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 20:45:33 -0000 On Friday, April 25, 2014 7:28:59 PM CEST, Jan Kundr=C3=A1t wrote: > On Friday, 25 April 2014 18:57:38 CEST, Arnt Gulbrandsen wrote: >> Storing MSNs persistently so you don't have to recompute at=20 >> session start (and at the same time supporting simultaneous=20 >> sessions) was simply too difficult for me. > > Given that you already have to track UIDs, I find it hard that=20 > it is expensive to take a look at the size of their set. But I don't have to track UIDs. I just have to track the uidnext value = so I=20 can allocate new UIDs. > The draft also says that the server should return a strange=20 > number when it would otherwise return a MSN. I object to that. OK, how do you think an unsolicited fetch response should look? a) Not have a number after the leading asterisk. b) Have a number, but it should not be strange. > In my opinion the server should signal this in a drastic way,=20 > including terminating the session if everything else fails.=20 > Sending back a mock-up MSN is about as evil as one could go in=20 > my book. We're talking about unsolicited fetch responses here. Perhaps other = things,=20 there are dozens of IMAP extensions and I at least do not know all of = them=20 entirely. But practically speaking, unsolicited fetch responses. >> Why? What's wrong with UIDNEXT? > > The fact that you can have gaps in UID range. The draft assumes=20 > that UIDs are allocated by adding one the the previous UID. Your=20 > whole scheme falls apart when this requirement does not hold.=20 > That requirement is not documented anywhere in the draft. I really do not understand this. > If the UIDs are not allocated consequently, and the highest UID=20 > right now is 12345, what does the following mean: > > * OK [UIDNEXT 12500] guess it "There are <=3D5 new messages. If you want them, you can send uid fetch=20 12345:* and get the details you choose you want about all the new=20 messages". > So this is an extension whose benefit is immediately=20 > neutralized by a client which wants to show the user the total=20 > number of messages. To me, that sounds like a very basic=20 > operation. Oh? I once asked a few colleagues at a googleapps shop about that, and = none=20 of them had any idea about their total number of messages. "Oh, lots",=20 "I've never deleted anything", that was the sort of answer I got. None = told=20 me any number. One looked at his screen and told me how many per cent of=20 his quota he used. So I don't think that operation is basic or important for users of = _large_=20 mailboxes. >> Do any other RFCs require CLIENTBUG? (Not suggest, or show in=20 >> examples. Require.) > > I'm not sure how that is relevant. This proposal is about=20 > designing an extension which radically changes how IMAP=20 > operates. Why not at least *mandate* a machine readable way of=20 > saying "hey client, you've just shot yourself in the foot"? I can tell you why I didn't add anything like that in 5530. 5530, as you=20 may have noticed, only lists codes, it doesn't say that you have to use = any=20 code. If two codes apply, it doesn't say which one you must choose. That's because when I surveyed servers for 5530, I found a few instances=20 where two response codes applied. Two errors occured, and it wasn't = clear=20 that one response code ought to win over the other. So I decided to not=20 issue a rule on that. A server can let the first response code win, the=20 last one win, it doesn't matter. I think that also applies in this case. Suppose there's a command such = as=20 "uid search charset iso-8859-42 1 body foo". The charset specification = is=20 reasonable, but the 8859 committee hasn't issued number 42 yet, so that=20 causes one response code. The presence of the MSN causes another. The=20 syntax only permits one, so the server has to choose which one to send. = Do=20 you think that in this case, the MSN-related response code ought to win=20 over the charset-related code? Do you think it ought to win over EVERY=20 other code in ALL cases? I think it doesn't matter very much, so let the server do whatever comes=20 easiest. > OK, at this point I cannot imagine how such a client is=20 > supposed to work *at all*. Let's start with a blank state. The=20 > client knows nothing about the mailbox. It opens it and=20 > presumably wants to show a list of messages to the user. What is=20 > the client supposed to do now? Something "uid search unseen", "uid search or unseen within 604800", or=20 some variation. Most clients, of course, do "fetch 1:* ..." instead, which is okay, but=20 that approach means that this extension is not interesting for them. > What if it wants to show just a subset of the mailbox, perhaps=20 > last 1k items (actually it doesn't have to be 1k, anything=20 > between 500 and 3000 is probably fine -- it needs *something*,=20 > but not the full 4M mailbox). How to achieve that? "Show today's mail" is trivial, "show all the unread mail" is trivial,=20 "show the last five hundred messages" is difficult. > What happens when a mail arrives? Same as before. Think about it. Before, you got an EXISTS for two new messages. But then someone = expunges=20 one before the server processes your fetch, and you get one message = back.=20 With UIDONLY, you get a UIDNEXT indicating that two new UIDs have been=20 allocated. You issue a fetch, and get <=3D2 messages. There's no = difference=20 in principle. > How is a resynchronization after being offline for a while=20 > going to look like? Same as before. I'll send you a session transcript offlist. I may not get it done = tonight,=20 though. Arnt From nobody Fri Apr 25 15:42:21 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 301091A0414 for ; Fri, 25 Apr 2014 15:42:20 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.1 X-Spam-Level: X-Spam-Status: No, score=-2.1 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, J_CHICKENPOX_55=0.6, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id U5NkS4gpIVEO for ; Fri, 25 Apr 2014 15:42:17 -0700 (PDT) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) by ietfa.amsl.com (Postfix) with ESMTP id 03EED1A03EF for ; Fri, 25 Apr 2014 15:42:16 -0700 (PDT) Received: from compute4.internal (compute4.nyi.mail.srv.osa [10.202.2.44]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 19F4D21500 for ; Fri, 25 Apr 2014 18:42:08 -0400 (EDT) Received: from web2 ([10.202.2.212]) by compute4.internal (MEProxy); Fri, 25 Apr 2014 18:42:08 -0400 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=fastmail.fm; h= message-id:from:to:mime-version:content-transfer-encoding :content-type:in-reply-to:references:subject:date; s=mesmtp; bh= n8zdNxP9s0lASXcXMLcvOCXZb54=; b=hRuy4HgTyY4B4fsqZXOBLnUi93pogmNT d9qpdfgYIO5xlDH0jxBPmzO3Tbikwg56ZzF+RgEkAB73izGjgBIk3VitgHonolpn 6QIU76Gfzlc6G9SOzfRVrhwe7G+LPJRTsVgfsW4XSQG53Q11txcz9ojGybA7CLkm 1lJRhlMiWnI= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=message-id:from:to:mime-version :content-transfer-encoding:content-type:in-reply-to:references :subject:date; s=smtpout; bh=n8zdNxP9s0lASXcXMLcvOCXZb54=; b=lO4 MK0DokMTqvRsYZx++AT9u0SrF28sEy0f2ni4D24OWF3T9E0Fj3C1BfE4HFaMwUFK 1eLPXLM+QmI4t6EGLeGi0QD6FvwV3nbsQcNsDy5sEWqr8snTmf33f/KdQCwcGbWm s4+fIWwsyZCfqMwFx2+zPh89KeCuouepYwNKNHWE= Received: by web2.nyi.mail.srv.osa (Postfix, from userid 99) id EF907540102; Fri, 25 Apr 2014 18:42:07 -0400 (EDT) Message-Id: <1398465727.23560.110511065.7F835F58@webmail.messagingengine.com> X-Sasl-Enc: G1A8gA5wprmkoU4lBbhxHBHJPb9B0ZlZgnbYSaNUZUSH 1398465727 From: Bron Gondwana To: imapext@ietf.org MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="ISO-8859-1" X-Mailer: MessagingEngine.com Webmail Interface - ajax-aafac2c3 In-Reply-To: <7cb76610-37e3-4903-bdc1-6d2403e27cff@gulbrandsen.priv.no> References: <0627dd7a-2203-40b6-b1c0-5b58613cbad8@gulbrandsen.priv.no> <8e45ff5c-1173-4119-bd53-47194d0f056b@flaska.net> <7cb76610-37e3-4903-bdc1-6d2403e27cff@gulbrandsen.priv.no> Date: Sat, 26 Apr 2014 08:42:07 +1000 Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/aZkRq5Ja7vekqOns6UNcD2EkPDo Subject: Re: [imapext] a very simple little imap extension, really quite magnificently simple X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 22:42:20 -0000 On Sat, Apr 26, 2014, at 02:57 AM, Arnt Gulbrandsen wrote: > On Friday, April 25, 2014 3:46:57 PM CEST, Jan Kundr=E1t wrote: > >> This extension provides a way to avoid having to number the messages > >> consecutively. > > > > Are there any other mail servers out there which actually have=20 > > a problem with sequence numbers, besides GMail? Just wondering=20 > > what is more probable -- GMail adding $code which makes this=20 > > faster because it's a pain for their clients, or $developers=20 > > updating N clients to make use of this extension. >=20 > I expect it hits everyone who tries to have n million messages in a singl= e=20 > mailbox. >=20 > Storing MSNs persistently so you don't have to recompute at session start= =20 > (and at the same time supporting simultaneous sessions) was simply too=20 > difficult for me. Same for me in Cyrus. We recompute MSNs at SELECT time. We used to keep them just as offsets in the cyrus.index file and rewrite/rename on every expunge, but that has its own pain points (every single expunge causes a full index rewrite), and it meant that sessions working from on old index file had outdated information. CONDSTORE became totally bogus. So now we just write a flag for EXPUNGED and repack occasionally. > >> command, and that it MUST NOT expect to receive an EXISTS response. > > > > How is the client supposed to notice that a new mail has=20 > > arrived, then? I see it addressed later in the doc, but it=20 > > startled me when I read this. >=20 > So you learned it twenty lines later ;) Well, yeah - EXISTS on the other hand. We can keep track of EXISTS in Cyrus just fine without calculating MSNs, because we keep a counter in the index so that STATUS EXISTS doesn't suck - because most clients request this on a ton of mailboxes frequently. I'm not sure why not having MSNs necessitates removing EXISTS. Knowing how many messages are in the folder is, as pointed out by others, a very common thing in clients. So much so that we added a ton of code to also keep that figure easy to fetch when we merge messages into conversations. Both EXISTS and UNSEEN counts are used in our UI for every folder, for display and for calculating the size of the scroll area in the mailbox view. > >> While the client will still receive MSNs, it MUST NOT expect them to > >> mean anything. (RFC editor: Remove this parenthesis. It seems to be > >> easier to add support for UIDONLY by disregarding parsed MSNs than to > >> change both the parser and the layer above, so I chose to leave dummy > >> MSNs in.) > > > > I dislike this a lot. If the MSNs do not mean anything, just=20 > > send a BAD and possibly even terminate the connection. Otherwise=20 > > the garbage which is sent will only lead to further confusion of=20 > > a non-compliant client. >=20 > The client cannot send BAD. >=20 > The server does send BAD when it receives a BAD from the client. I think the confusion here was things like * 1 FETCH (UID 500 FLAGS (\Seen)) * 2 FETCH (UID 501 FLAGS ($Special)) ... honestly though, now that I look at it. Is there any reason not to just use the UID everywhere that MSN appears? I still really dislike the use of magic numbers. Either that or a string like NOMSN. Anyone sending ENABLE UIDONLY will already be aware that they need to change their code, and the nice thing about it really not being a number is that the client parser will either be correct or fail hard. > > Which means that you effectively require QRESYNC and its=20 > > VANISHED. Without that option it won't work. I see that you talk=20 > > about that later in the text, but it should probably be a hard=20 > > dep on 5162-bis. >=20 > Disagree violently. 5162 is MUCH too complex to require all of it. I didn't find it so hard once CONDSTORE existed. The hardest part is keeping track of the low-water-mark so you know if you can respond with full information or not. We use a field called DELETEDMODSEQ for this. But you don't need to do that. You can do a no-information version of it which always responds to a SELECT QRESYNC as if it's forgotten all the history of the mailbox. It's not too complex then. > > Given that this extension disables EXISTS, you need soemthing=20 > > like ARRIVED [1]. This change on its own will complicate this=20 > > dratf a lot. >=20 > Why? What's wrong with UIDNEXT? You still need to capture every create and expunge event on the entire mailbox so you can send VANISHED responses. At least in Cyrus, that's not really architecturally easy in the case that you've slept for a bit and the EXISTS count has changed by one but the UIDNEXT has jumped by 5. You need to scan back through the earlier file looking for which other message disappeared. It got a lot easier with CONDSTORE, because you just query for every record which changed since the MODSEQ of the last sync. That's how our internals work now - everything is built on MODSEQ tracking rather than positional mathematics. > > And the client doesn't know how many messages are in the=20 > > mailbox anymore. That's a rather radical change. I don't like it=20 > > at all, it's a nice safeguard to detect broken sync and this=20 > > extension takes it away. >=20 > Not knowing how many messages there are is a necessary effect of this. It= 's=20 > not an accidental problem, it's necessary. It's really not if you still support sending EXISTS counts. I'm surprised that they are expensive, while UIDs are being allocated. Unlike MSNs, they don't change between connections, so you can just keep a global cache per mailbox. Since you only allow UID commands, you can resync them immediately rather than having to avoid telling clients about expunges for a bit. > The point of this is to stop counting messages. If you stop counting, you= =20 > don't know (at least not precisely) how many you have. It's as simple as= =20 > that. I thought the point was to avoid having to enumerate messages to give them sequential IDs such that you can reference to any particular offset in the set. That's quite different from not counting at all. If you implement quotas, you're already updating a central value when any message is added or removed. Updating another central value at the same time doesn't strike me as hard (in Cyrus we now actually have a quota on number of messages in the mailbox as a settable field - along with a quota on number of sub- mailboxes and quotas on number and size of annotations... all the resource limits we used to hard-code on a per-user basis) So I do have a problem with losing EXISTS. It's purely the using MSNs as a way of accessing a particular "distance" through the mailbox that I'd like to lose. > > I don't like mandating VANISHED without also requiring the rest=20 > > of the QRESYNC functionality. I for one cannot imagine how to=20 > > sync a mailbox at all in at least semi efficient manner when I=20 > > cannot rely on UIDNEXT/EXISTS combination and without QRESYNC's=20 > > availability at the same time. >=20 > I can show you code that does that, OK? Real examples are always good :) > > This extension, as-is, opens up a few opportunities for client=20 > > and server to go out of sync, and potentially could lead to loss=20 > > of data. >=20 > No, if you lose sync the worst that happens is that you send uid store=20 > +flags \deleted for something that's already deleted. >=20 > Remember: With this extension there are no MSNs any more. There's no way = to=20 > store +flags on the wrong message. Agree here. I just can't see many clients getting the benefits of this wit= hout an EXISTS method. Users do like being able to see how many messages they h= ave in the UI. (aside: I'd love to see an RFC or errata saying that it's cool to call STAT= US on the currently selected mailbox, and you see the same counts that you would = see if you queried that data from the mailbox. It seems to be what the good serve= rs are already doing - and it matches user expectations and client author expectat= ions. We take advantage of it with a LIST RETURN STATUS that gets all the folder = counts and the folder list in a single call for our web UI) Bron. --=20 Bron Gondwana brong@fastmail.fm From nobody Fri Apr 25 18:39:34 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 720341A044C for ; Fri, 25 Apr 2014 18:39:32 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.55 X-Spam-Level: X-Spam-Status: No, score=-3.55 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HELO_MISMATCH_ORG=0.611, HOST_MISMATCH_COM=0.311, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.272] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id H4ZXXfSwk88d for ; Fri, 25 Apr 2014 18:39:27 -0700 (PDT) Received: from smtp.mozilla.org (mx2.corp.phx1.mozilla.com [63.245.216.70]) by ietfa.amsl.com (Postfix) with ESMTP id 335741A06FB for ; Fri, 25 Apr 2014 18:39:24 -0700 (PDT) Received: from [192.168.1.7] (pool-173-49-137-94.phlapa.fios.verizon.net [173.49.137.94]) (Authenticated sender: asuth@mozilla.com) by mx2.mail.corp.phx1.mozilla.com (Postfix) with ESMTPSA id DAB59F2A2D for ; Fri, 25 Apr 2014 18:39:17 -0700 (PDT) Message-ID: <535B0E45.60005@mozilla.com> Date: Fri, 25 Apr 2014 21:39:17 -0400 From: Andrew Sutherland User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: imapext@ietf.org References: <0627dd7a-2203-40b6-b1c0-5b58613cbad8@gulbrandsen.priv.no> <8e45ff5c-1173-4119-bd53-47194d0f056b@flaska.net> <7cb76610-37e3-4903-bdc1-6d2403e27cff@gulbrandsen.priv.no> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/9QRx7piws-E6IIFnK5eEmGL53Dw Subject: Re: [imapext] a very simple little imap extension, really quite magnificently simple X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Apr 2014 01:39:32 -0000 On 04/25/2014 01:28 PM, Jan Kundrát wrote: >> The point of this is to stop counting messages. If you stop counting, >> you don't know (at least not precisely) how many you have. It's as >> simple as that. > > OK, at this point I cannot imagine how such a client is supposed to > work *at all*. Let's start with a blank state. The client knows > nothing about the mailbox. It opens it and presumably wants to show a > list of messages to the user. What is the client supposed to do now? > > What if it wants to show just a subset of the mailbox, perhaps last 1k > items (actually it doesn't have to be 1k, anything between 500 and > 3000 is probably fine -- it needs *something*, but not the full 4M > mailbox). How to achieve that? The Firefox OS Gaia e-mail client uses date-based synchronization. Although we optimize for small folders, in a big folder we start by doing a "SEARCH NOT DELETED SINCE (3 days ago)". If we get too many back, we bisect our way down until we're talking only about a single day. We sync the headers. Then if we still want more messages, we move our time window back in time. We use various heuristics to grow/shrink the time window as we go for efficiency. We declare a folder entirely synchronized when we get to 1990. This is a nice approach to clients that only sync a subset of what's on the server since while there is usually a correlation between UID and INTERNALDATE, thanks to message moves and APPEND, it's not a sure thing. (Note: the awkwardness about SINCE/BEFORE and timezones and interactions with INTERNALDATE are not without their issues. We have a longstanding embarassing bug about this at https://bugzil.la/886534 that still remains to be addressed. There's additional high-level details and rationale at https://wiki.mozilla.org/Gaia/Email/Implementation/MailSynchronization if anyone is interested.) Andrew From nobody Mon Apr 28 01:52:12 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EFDE81A08B1 for ; Mon, 28 Apr 2014 01:52:09 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.598 X-Spam-Level: X-Spam-Status: No, score=0.598 tagged_above=-999 required=5 tests=[BAYES_40=-0.001, J_CHICKENPOX_47=0.6, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DPXVMN9zUmQ5 for ; Mon, 28 Apr 2014 01:52:08 -0700 (PDT) Received: from wursti.dovecot.fi (wursti.dovecot.fi [87.106.245.223]) by ietfa.amsl.com (Postfix) with ESMTP id 380841A0912 for ; Mon, 28 Apr 2014 01:52:08 -0700 (PDT) Received: from [192.168.10.103] (cs181255018.pp.htv.fi [82.181.255.18]) by wursti.dovecot.fi (Postfix) with ESMTPSA id 1D05C21C64 for ; Mon, 28 Apr 2014 10:52:05 +0200 (CEST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) From: Timo Sirainen In-Reply-To: <0627dd7a-2203-40b6-b1c0-5b58613cbad8@gulbrandsen.priv.no> Date: Mon, 28 Apr 2014 11:52:03 +0300 Content-Transfer-Encoding: quoted-printable Message-Id: <2162569C-16C8-4BBD-8734-E9918587E968@iki.fi> References: <0627dd7a-2203-40b6-b1c0-5b58613cbad8@gulbrandsen.priv.no> To: imapext@ietf.org X-Mailer: Apple Mail (2.1874) Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/VqKVU4msJhXm9fmRIUpoVzmXV6A Subject: Re: [imapext] a very simple little imap extension, really quite magnificently simple X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Apr 2014 08:52:10 -0000 On 25.4.2014, at 15.30, Arnt Gulbrandsen = wrote: > as some of you will have discovered, opening large mailboxes takes = real time. So I discussed that with Brandon and wrote a draft, really = short, perhaps it's actually shorter than the boilerplate. The gist: "If = a client runs ENABLE UIDONLY, then that client cannot later use any MSNs = in any commands, and if syntax forces the server needs to send an MSN, = then it just sends a phony number." There are a couple of details. I don't really understand the main point of this. Is this only about = reducing the amount of work for some server implementations? Clients = won't get any direct benefits from this, right? As for servers having to do MSN <-> UID mapping when mailbox is opened, = that's not really true either. You can do it lazily only as needed. As = Bron mentioned you can efficiently keep track of the total number of = messages in mailbox. After that, clients will typically work in one of = two ways: a) Either they FETCH 1:* FLAGS, in which case you're pretty much = building the MSN <-> UID mapping for free while returning the results, = just with the cost of some memory. b) They're only accessing somewhat recent messages, in which case you = can only build the MSN <-> UID mapping for the recent messages and = ignore the rest until they're needed (and they most likely won't be). I've been planning on implementing b) for Dovecot, but haven't yet. So = maybe there is something in it that I haven't thought about that would = cause trouble, but currently the main problem I see with it is that it = would require some extra development. (Maybe SORT+PARTIAL could be = problematic.) From nobody Mon Apr 28 02:08:34 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 40FBE1A098F for ; Mon, 28 Apr 2014 02:08:33 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.8 X-Spam-Level: X-Spam-Status: No, score=0.8 tagged_above=-999 required=5 tests=[BAYES_50=0.8] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id R7gKiS2rAXTH for ; Mon, 28 Apr 2014 02:08:30 -0700 (PDT) Received: from strange.aox.org (strange.aox.org [IPv6:2001:4d88:100c::1]) by ietfa.amsl.com (Postfix) with ESMTP id 4AC651A098A for ; Mon, 28 Apr 2014 02:08:30 -0700 (PDT) Received: from fri.gulbrandsen.priv.no (localhost [127.0.0.1]) by strange.aox.org (Postfix) with ESMTP id 65A6AFA0078; Mon, 28 Apr 2014 09:08:28 +0000 (UTC) Received: from arnt@gulbrandsen.priv.no by fri.gulbrandsen.priv.no (Archiveopteryx 3.2.0) with esmtpsa id 1398676107-29782-29782/11/27; Mon, 28 Apr 2014 09:08:27 +0000 From: Arnt Gulbrandsen To: imapext@ietf.org Date: Mon, 28 Apr 2014 11:08:26 +0200 User-Agent: Trojita/v0.3.96-176-g713fcbb; Qt/4.8.4; X11; Linux; Ubuntu 13.10 Mime-Version: 1.0 Message-Id: <702db6f9-7038-4b15-ab1d-60e4b1568233@gulbrandsen.priv.no> In-Reply-To: <1398465727.23560.110511065.7F835F58@webmail.messagingengine.com> References: <0627dd7a-2203-40b6-b1c0-5b58613cbad8@gulbrandsen.priv.no> <8e45ff5c-1173-4119-bd53-47194d0f056b@flaska.net> <7cb76610-37e3-4903-bdc1-6d2403e27cff@gulbrandsen.priv.no> <1398465727.23560.110511065.7F835F58@webmail.messagingengine.com> Content-Type: text/plain; charset=utf-8; format=flowed Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/CkQqytJ7Uugq2TtqEMPrbA5w1sc Subject: Re: [imapext] a very simple little imap extension, really quite magnificently simple X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Apr 2014 09:08:33 -0000 > Well, yeah - EXISTS on the other hand. We can keep track of EXISTS in Cyrus > just fine without calculating MSNs, because we keep a counter in the index > so that STATUS EXISTS doesn't suck - because most clients request this on a > ton of mailboxes frequently. > > I'm not sure why not having MSNs necessitates removing EXISTS. It depends on server implementation. If the server computes EXISTS from the MSN-UID map, then dropping that map and keeping EXISTS is real work. > Knowing how > many messages are in the folder is, as pointed out by others, a very common > thing in clients. Yes, just like breaking horribly on large mailboxes. I posit that when the process of optimising for large mailbox access involves removing EXISTS, and that therefore, it's not worth optimising _this_ extension for clients that display EXISTS. Clients that really want it can always send STATUS EXISTS. AFAICS, the clients I've tried that work well with a large mailbox do not display EXISTS. They don't display the mailbox at all, but rather some subset, such as "the unseen and recent mail". Mailbox for iphone is an example. If there is any total message count anywhere in the UI I haven't found it. I'd be happy to make EXISTS optional and say that the server must allow issuing STATUS on the selected mailbox. > I think the confusion here was things like > > * 1 FETCH (UID 500 FLAGS (\Seen)) > * 2 FETCH (UID 501 FLAGS ($Special)) > > ... honestly though, now that I look at it. Is there any reason not to > just use the UID everywhere that MSN appears? I still really dislike the > use of magic numbers. Either that or a string like NOMSN. Anyone sending > ENABLE UIDONLY will already be aware that they need to change their code, > and the nice thing about it really not being a number is that the client > parser will either be correct or fail hard. You mean like this? * 500 FETCH (UID 500 FLAGS (\Seen)) Compared to using a constant for the MSN, this has the advantage that MSN math almost continues to work, and the disadvantage that MSN math almost continues to work. Anyone who does "uid fetch x-25, x" will see the code continue to work provided the UIDs are mostly contiguous, which is often the case for large mailboxes. So do you prefer to flush out bugs quickly or allow buggy code to work after a fashion? My preference is to flush out bugs, but I do not feel strongly about this issue. > I didn't find it so hard once CONDSTORE existed. The hardest part is > keeping track of the low-water-mark so you know if you can respond with > full information or not. We use a field called DELETEDMODSEQ for this. > > But you don't need to do that. You can do a no-information version of > it which always responds to a SELECT QRESYNC as if it's forgotten all > the history of the mailbox. It's not too complex then. Ask me what I think about such shams sometime. >>> Given that this extension disables EXISTS, you need soemthing >>> like ARRIVED [1]. This change on its own will complicate this >>> dratf a lot. >> >> Why? What's wrong with UIDNEXT? > > You still need to capture every create and expunge event on the entire > mailbox so you can send VANISHED responses. At least in Cyrus, that's > not really architecturally easy in the case that you've slept for a bit > and the EXISTS count has changed by one but the UIDNEXT has jumped by 5. > You need to scan back through the earlier file looking for which other > message disappeared. I don't see what you're trying to say here. I can see that in your case, it's as easy to generated ARRIVED+VANISHED, but what does that have to do with whether clients need ARRIVED? > It got a lot easier with CONDSTORE, because you just query for every record > which changed since the MODSEQ of the last sync. That's how our internals > work now - everything is built on MODSEQ tracking rather than positional > mathematics. Yes, and? >> The point of this is to stop counting messages. If you stop counting, you >> don't know (at least not precisely) how many you have. It's as simple as >> that. > > I thought the point was to avoid having to enumerate messages to give them > sequential IDs such that you can reference to any particular offset in the > set. That's quite different from not counting at all. Strictly speaking, the point is to avoid O(n) behaviour in SELECT. I understand now that it's possible to do the counting at other times and thereby keep EXISTS while speeding up SELECT. But as I said above, I don't think the need for displaying mailbox size warrants doing that. (The cost is zero if you've already done the work, of course.) > So I do have a problem with losing EXISTS. It's purely the using MSNs as > a way of accessing a particular "distance" through the mailbox that I'd > like to lose. Well, the client can issue STATUS (EXISTS) if it wants... >>> I don't like mandating VANISHED without also requiring the rest >>> of the QRESYNC functionality. I for one cannot imagine how to >>> sync a mailbox at all in at least semi efficient manner when I >>> cannot rely on UIDNEXT/EXISTS combination and without QRESYNC's >>> availability at the same time. >> >> I can show you code that does that, OK? > > Real examples are always good :) I can't post that code, it's much too large for the list. But I'll add a commented session transcript as an appendix to the -01 draft. > Agree here. I just can't see many clients getting the benefits > of this without > an EXISTS method. Users do like being able to see how many > messages they have > in the UI. I've seen many complaints about gmail, but not that you can't see how many messages you have, so I suspect that users' desire to see that count disappears when the number grows incomprehensibly large. > (aside: I'd love to see an RFC or errata saying that it's cool > to call STATUS on > the currently selected mailbox, and you see the same counts > that you would see if > you queried that data from the mailbox. It seems to be what > the good servers are > already doing - and it matches user expectations and client > author expectations. > We take advantage of it with a LIST RETURN STATUS that gets all > the folder counts > and the folder list in a single call for our web UI) I'd be happy to add such a requirement here. Arnt From nobody Mon Apr 28 02:17:21 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 150AD1A0999 for ; Mon, 28 Apr 2014 02:17:20 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.3 X-Spam-Level: X-Spam-Status: No, score=-1.3 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, J_CHICKENPOX_47=0.6] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KdGVmN6p7JGX for ; Mon, 28 Apr 2014 02:17:18 -0700 (PDT) Received: from strange.aox.org (strange.aox.org [80.244.248.170]) by ietfa.amsl.com (Postfix) with ESMTP id BEE8F1A0998 for ; Mon, 28 Apr 2014 02:17:18 -0700 (PDT) Received: from fri.gulbrandsen.priv.no (localhost [127.0.0.1]) by strange.aox.org (Postfix) with ESMTP id 403ACFA0078; Mon, 28 Apr 2014 09:17:17 +0000 (UTC) Received: from arnt@gulbrandsen.priv.no by fri.gulbrandsen.priv.no (Archiveopteryx 3.2.0) with esmtpsa id 1398676636-29782-29782/11/28; Mon, 28 Apr 2014 09:17:16 +0000 From: Arnt Gulbrandsen To: Timo Sirainen Date: Mon, 28 Apr 2014 11:17:16 +0200 User-Agent: Trojita/v0.3.96-176-g713fcbb; Qt/4.8.4; X11; Linux; Ubuntu 13.10 Mime-Version: 1.0 Message-Id: <93d7557f-ae46-434f-9a7d-7b88e368477c@gulbrandsen.priv.no> In-Reply-To: <2162569C-16C8-4BBD-8734-E9918587E968@iki.fi> References: <0627dd7a-2203-40b6-b1c0-5b58613cbad8@gulbrandsen.priv.no> <2162569C-16C8-4BBD-8734-E9918587E968@iki.fi> Content-Type: text/plain; charset=utf-8; format=flowed Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/qWEW49f6S4FMU9ck5CySDzCLwR4 Cc: imapext@ietf.org Subject: Re: [imapext] a very simple little imap extension, really quite magnificently simple X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Apr 2014 09:17:20 -0000 On Monday, April 28, 2014 10:52:03 AM CEST, Timo Sirainen wrote: > I don't really understand the main point of this. Is this only > about reducing the amount of work for some server > implementations? Clients won't get any direct benefits from > this, right? The ability to open 10M mailboxes fast. Is that an advantage for the client, or merely for the user? > As for servers having to do MSN <-> UID mapping when mailbox is > opened, that's not really true either. You can do it lazily only > as needed. As Bron mentioned you can efficiently keep track of > the total number of messages in mailbox. After that, clients > will typically work in one of two ways: > > a) Either they FETCH 1:* FLAGS, in which case you're pretty > much building the MSN <-> UID mapping for free while returning > the results, just with the cost of some memory. This one is irrelevant. A client that does FETCH 1:* won't bother about a fast SELECT. > b) They're only accessing somewhat recent messages, in which > case you can only build the MSN <-> UID mapping for the recent > messages and ignore the rest until they're needed (and they most > likely won't be). I can't tell how do do this. Assume you have about two million messages in a mailbox, and you're about to send an unsolicited flag response for one that's neither the first or the last. How do you do it? > I've been planning on implementing b) for Dovecot, but haven't > yet. So maybe there is something in it that I haven't thought > about that would cause trouble, but currently the main problem I > see with it is that it would require some extra development. > (Maybe SORT+PARTIAL could be problematic.) Abhijit and I tried it and gave up after half a day. So yes, maybe it's possible, but after running into a few snags we gave up, and neither of us is stupid. Arnt From nobody Mon Apr 28 02:20:17 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 569571A08EB for ; Mon, 28 Apr 2014 02:20:15 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XEd1fSag29LE for ; Mon, 28 Apr 2014 02:20:14 -0700 (PDT) Received: from strange.aox.org (strange.aox.org [IPv6:2001:4d88:100c::1]) by ietfa.amsl.com (Postfix) with ESMTP id 385621A08CB for ; Mon, 28 Apr 2014 02:20:14 -0700 (PDT) Received: from fri.gulbrandsen.priv.no (localhost [127.0.0.1]) by strange.aox.org (Postfix) with ESMTP id 367CAFA0078; Mon, 28 Apr 2014 09:20:13 +0000 (UTC) Received: from arnt@gulbrandsen.priv.no by fri.gulbrandsen.priv.no (Archiveopteryx 3.2.0) with esmtpsa id 1398676812-29782-29782/11/29; Mon, 28 Apr 2014 09:20:12 +0000 From: Arnt Gulbrandsen To: imapext@ietf.org Date: Mon, 28 Apr 2014 11:20:12 +0200 User-Agent: Trojita/v0.3.96-176-g713fcbb; Qt/4.8.4; X11; Linux; Ubuntu 13.10 Mime-Version: 1.0 Message-Id: <9dd7e8ae-1dce-4cd3-a353-a942adb9e9b7@gulbrandsen.priv.no> In-Reply-To: <93d7557f-ae46-434f-9a7d-7b88e368477c@gulbrandsen.priv.no> References: <0627dd7a-2203-40b6-b1c0-5b58613cbad8@gulbrandsen.priv.no> <2162569C-16C8-4BBD-8734-E9918587E968@iki.fi> <93d7557f-ae46-434f-9a7d-7b88e368477c@gulbrandsen.priv.no> Content-Type: text/plain; charset=utf-8; format=flowed Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/sNUTCq1UVTmw9_Z84R2jAVDQOqE Subject: Re: [imapext] a very simple little imap extension, really quite magnificently simple X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Apr 2014 09:20:15 -0000 On Monday, April 28, 2014 11:17:16 AM CEST, Arnt Gulbrandsen wrote: > I can't tell how do do this. Oh, I see. You keep the total count persistently, and build the MSN-UID map in memory downwards. Right? Arnt From nobody Mon Apr 28 02:35:17 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2B7AE1A07CA for ; Mon, 28 Apr 2014 02:35:15 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.901 X-Spam-Level: X-Spam-Status: No, score=-1.901 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2bprjYOinmmK for ; Mon, 28 Apr 2014 02:35:13 -0700 (PDT) Received: from wursti.dovecot.fi (wursti.dovecot.fi [87.106.245.223]) by ietfa.amsl.com (Postfix) with ESMTP id 1B4D41A0786 for ; Mon, 28 Apr 2014 02:35:13 -0700 (PDT) Received: from [192.168.10.103] (cs181255018.pp.htv.fi [82.181.255.18]) by wursti.dovecot.fi (Postfix) with ESMTPSA id 498BF21C64; Mon, 28 Apr 2014 11:35:11 +0200 (CEST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) From: Timo Sirainen In-Reply-To: <93d7557f-ae46-434f-9a7d-7b88e368477c@gulbrandsen.priv.no> Date: Mon, 28 Apr 2014 12:35:09 +0300 Content-Transfer-Encoding: quoted-printable Message-Id: <72A4B461-76E8-43E0-96D1-38CFAC8BF48F@iki.fi> References: <0627dd7a-2203-40b6-b1c0-5b58613cbad8@gulbrandsen.priv.no> <2162569C-16C8-4BBD-8734-E9918587E968@iki.fi> <93d7557f-ae46-434f-9a7d-7b88e368477c@gulbrandsen.priv.no> To: Arnt Gulbrandsen X-Mailer: Apple Mail (2.1874) Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/Im3jZvy5Z3zdDweJPGt-1qYmdHA Cc: imapext@ietf.org Subject: Re: [imapext] a very simple little imap extension, really quite magnificently simple X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Apr 2014 09:35:15 -0000 On 28.4.2014, at 12.17, Arnt Gulbrandsen = wrote: >> b) They're only accessing somewhat recent messages, in which case you = can only build the MSN <-> UID mapping for the recent messages and = ignore the rest until they're needed (and they most likely won't be). >=20 > I can't tell how do do this. >=20 > Assume you have about two million messages in a mailbox, and you're = about to send an unsolicited flag response for one that's neither the = first or the last. How do you do it? Here's a relatively easy way: Have two index mappings, one containing = "old" messages and one containing "new" messages (every once in a while = moving some messages from new to old). For both you know the first and = the last UID and also the message count. As long as client is accessing = only the messages in the "new" index, you'll never need to load the = "old" index into memory or calculate its MSN <-> UID mapping, only the = "new" MSN <-> UID mapping. So the "new" mapping would usually be = somewhere between 100-10000 mails, which would likely contain all the = mails that most people/clients will access. From nobody Mon Apr 28 03:34:29 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5143F1A096E for ; Mon, 28 Apr 2014 03:34:28 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.1 X-Spam-Level: X-Spam-Status: No, score=-2.1 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, J_CHICKENPOX_16=0.6, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QeYijJEFXMUi for ; Mon, 28 Apr 2014 03:34:25 -0700 (PDT) Received: from out2-smtp.messagingengine.com (out2-smtp.messagingengine.com [66.111.4.26]) by ietfa.amsl.com (Postfix) with ESMTP id B17931A0976 for ; Mon, 28 Apr 2014 03:34:25 -0700 (PDT) Received: from compute1.internal (compute1.nyi.mail.srv.osa [10.202.2.41]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 6F0FE20C8E for ; Mon, 28 Apr 2014 06:34:24 -0400 (EDT) Received: from web2 ([10.202.2.212]) by compute1.internal (MEProxy); Mon, 28 Apr 2014 06:34:24 -0400 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=fastmail.fm; h= message-id:from:to:mime-version:content-transfer-encoding :content-type:in-reply-to:references:subject:date; s=mesmtp; bh= PfOjvRDyPycCjQFVZnuJb/tiC4c=; b=hK9Rm/E6gmrCy03ZTCxaGs1xxxGGgfrW b+bckPyj06+jyWmzSPVHaWvi/N6P9tWkjbEai6XhS/v+CiNqUKOUHf+CBU1VtdTv rZpBQBVQbABKRJ+wquyDpPmO1HM1wmPUhg9r9o7/Ao82PuU/2hBtdKE6IjX/hPGE ru1WmE87/Nw= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=message-id:from:to:mime-version :content-transfer-encoding:content-type:in-reply-to:references :subject:date; s=smtpout; bh=PfOjvRDyPycCjQFVZnuJb/tiC4c=; b=bFV CLiC2uW2eOECFMNW2rdOL1RZx7qxf3JRLTEIaL2V6qDKpNtEJEgxiX3Xr65/I31M gy0tBaYNUDArRZWtKNPC+ZWiY6ycJr392nGiUTkLfP3SRMyAwD2QTWSrSgjtk+Eu GbqdeBXVL4gbfjk7HA2rdHopiyzQLLUuRHBS2UjE= Received: by web2.nyi.mail.srv.osa (Postfix, from userid 99) id 0990E5403D2; Mon, 28 Apr 2014 06:34:24 -0400 (EDT) Message-Id: <1398681263.12000.111151785.341C921F@webmail.messagingengine.com> X-Sasl-Enc: XJ5E0b7dALoFumkKkAJVE51r8DKlwYwWxKDUD31x7gIc 1398681263 From: Bron Gondwana To: imapext@ietf.org MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain X-Mailer: MessagingEngine.com Webmail Interface - ajax-aafac2c3 In-Reply-To: <702db6f9-7038-4b15-ab1d-60e4b1568233@gulbrandsen.priv.no> References: <0627dd7a-2203-40b6-b1c0-5b58613cbad8@gulbrandsen.priv.no> <8e45ff5c-1173-4119-bd53-47194d0f056b@flaska.net> <7cb76610-37e3-4903-bdc1-6d2403e27cff@gulbrandsen.priv.no> <1398465727.23560.110511065.7F835F58@webmail.messagingengine.com> <702db6f9-7038-4b15-ab1d-60e4b1568233@gulbrandsen.priv.no> Date: Mon, 28 Apr 2014 20:34:23 +1000 Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/Npvb2Ry-U2q_FLvJwHoSKwCtW1Y Subject: Re: [imapext] a very simple little imap extension, really quite magnificently simple X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Apr 2014 10:34:28 -0000 On Mon, Apr 28, 2014, at 07:08 PM, Arnt Gulbrandsen wrote: > > Well, yeah - EXISTS on the other hand. We can keep track of EXISTS in > Cyrus > > just fine without calculating MSNs, because we keep a counter in the > index > > so that STATUS EXISTS doesn't suck - because most clients request this on > a > > ton of mailboxes frequently. > > > > I'm not sure why not having MSNs necessitates removing EXISTS. > > It depends on server implementation. If the server computes EXISTS from the > MSN-UID map, then dropping that map and keeping EXISTS is real work. True - as with anything, it's up to the server how it tracks things. Of course, if it does that, then STATUS is going to be a really expensive command. Lots of clients call STATUS too. You're shooting yourself in the foot not keeping an exists count unless you're really sure exactly who your clients are. > > But you don't need to do that. You can do a no-information version of > > it which always responds to a SELECT QRESYNC as if it's forgotten all > > the history of the mailbox. It's not too complex then. > > Ask me what I think about such shams sometime. Honestly, you have to suck that up at some point, unless you never forget anything ever. Hence the use of DELETEDMODSEQ in Cyrus. > > You still need to capture every create and expunge event on the entire > > mailbox so you can send VANISHED responses. At least in Cyrus, that's > > not really architecturally easy in the case that you've slept for a bit > > and the EXISTS count has changed by one but the UIDNEXT has jumped by 5. > > You need to scan back through the earlier file looking for which other > > message disappeared. > > I don't see what you're trying to say here. I'm saying that in the Cyrus case, just calculating which messages were VANISHED (if you didn't do it yourself) is O(n). Once you start optimising your standard for one particular implementation, you wind up with weirdness like not requiring EXISTS. I'd much rather not require VANISHED, particularly since you don't know if the client knows about this UID, so you can wind up sending a lot of VANISHED information that the client doesn't care about. > > I thought the point was to avoid having to enumerate messages to give > them > > sequential IDs such that you can reference to any particular offset in > the > > set. That's quite different from not counting at all. > > Strictly speaking, the point is to avoid O(n) behaviour in SELECT. Which you only have for EXISTS if you don't keep a count of how many items are in your mailbox. As I've already said before, you already need to do housekeeping when you receive or delete messages, otherwise you can't do quotas. You haven't answered that. > I understand now that it's possible to do the counting at other times and > thereby keep EXISTS while speeding up SELECT. But as I said above, I don't > think the need for displaying mailbox size warrants doing that. (The cost > is zero if you've already done the work, of course.) The cost of just about anything is zero if you've done the work. The problem with MSNs is that they are explicitly per connection, so you have no choice but to do the O(n) work for every connection. Even there, there are tricks you could probably do with with clever data structures to reduce the cost to O(number of disjoin expunges). > I've seen many complaints about gmail, but not that you can't see how many > messages you have, so I suspect that users' desire to see that count > disappears when the number grows incomprehensibly large. Could be. Could be that you don't get the same complaints as other people. http://webapps.stackexchange.com/questions/6262/is-there-a-way-to-display-the-number-of-messages-for-any-label-in-the-gmail-labe Bron. -- Bron Gondwana brong@fastmail.fm From nobody Mon Apr 28 04:33:52 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 239721A0972 for ; Mon, 28 Apr 2014 04:33:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yyZDPM5x_KSj for ; Mon, 28 Apr 2014 04:33:48 -0700 (PDT) Received: from strange.aox.org (strange.aox.org [80.244.248.170]) by ietfa.amsl.com (Postfix) with ESMTP id C66291A095F for ; Mon, 28 Apr 2014 04:33:48 -0700 (PDT) Received: from fri.gulbrandsen.priv.no (localhost [127.0.0.1]) by strange.aox.org (Postfix) with ESMTP id 36ECCFA0078; Mon, 28 Apr 2014 11:33:47 +0000 (UTC) Received: from arnt@gulbrandsen.priv.no by fri.gulbrandsen.priv.no (Archiveopteryx 3.2.0) with esmtpsa id 1398684826-29782-29782/11/33; Mon, 28 Apr 2014 11:33:46 +0000 From: Arnt Gulbrandsen To: imapext@ietf.org Date: Mon, 28 Apr 2014 13:33:45 +0200 User-Agent: Trojita/v0.3.96-176-g713fcbb; Qt/4.8.4; X11; Linux; Ubuntu 13.10 Mime-Version: 1.0 Message-Id: <98fe3222-bdb5-4390-9d8e-f25f5090019d@gulbrandsen.priv.no> In-Reply-To: <1398681263.12000.111151785.341C921F@webmail.messagingengine.com> References: <0627dd7a-2203-40b6-b1c0-5b58613cbad8@gulbrandsen.priv.no> <8e45ff5c-1173-4119-bd53-47194d0f056b@flaska.net> <7cb76610-37e3-4903-bdc1-6d2403e27cff@gulbrandsen.priv.no> <1398465727.23560.110511065.7F835F58@webmail.messagingengine.com> <702db6f9-7038-4b15-ab1d-60e4b1568233@gulbrandsen.priv.no> <1398681263.12000.111151785.341C921F@webmail.messagingengine.com> Content-Type: text/plain; charset=utf-8; format=flowed Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/-cQrGlI_Zr9t2yhBl-r8ZTho6qk Subject: Re: [imapext] a very simple little imap extension, really quite magnificently simple X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Apr 2014 11:33:50 -0000 On Monday, April 28, 2014 12:34:23 PM CEST, Bron Gondwana wrote: > True - as with anything, it's up to the server how it tracks > things. Of course, > if it does that, then STATUS is going to be a really expensive > command. Lots of > clients call STATUS too. You're shooting yourself in the foot > not keeping an > exists count unless you're really sure exactly who your clients are. O(n) is okay as long as n is small ;) I mean, if a client has ha habit of doing FETCH 1:* and STATUS (EXISTS), the STATUS will always be less painful. The problem only happens when clients that are optimised for small mailboxes and used with large ones. Fixing that require client change: Cache well, don't retrieve what you don't need, don't use MSNs, don't ever look at the entire mailbox, etc. Btw, the SE link you posted is like ESEARCH (RETURN COUNT) KEYWORD TODO in IMAP terms, it's not like STATUS/EXISTS. I agree that the size of such subsets is something people want to know. You may notice that all of the numbers in the question's example are small. People do care about numbers as long as they're within human range. Arnt From nobody Mon Apr 28 05:00:40 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AE1CB1A06D7 for ; Mon, 28 Apr 2014 05:00:37 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.7 X-Spam-Level: X-Spam-Status: No, score=-2.7 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tFkw6wReIS1N for ; Mon, 28 Apr 2014 05:00:35 -0700 (PDT) Received: from out2-smtp.messagingengine.com (out2-smtp.messagingengine.com [66.111.4.26]) by ietfa.amsl.com (Postfix) with ESMTP id 730791A0684 for ; Mon, 28 Apr 2014 05:00:35 -0700 (PDT) Received: from compute5.internal (compute5.nyi.mail.srv.osa [10.202.2.45]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 84A7720BEC for ; Mon, 28 Apr 2014 08:00:34 -0400 (EDT) Received: from web2 ([10.202.2.212]) by compute5.internal (MEProxy); Mon, 28 Apr 2014 08:00:34 -0400 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=fastmail.fm; h= message-id:from:to:mime-version:content-transfer-encoding :content-type:in-reply-to:references:subject:date; s=mesmtp; bh= GtvJDtnMhlQ7dFUXWRgQLrLGlzA=; b=mJaxcwD3KaiX7fpjzORP+iWCmAFRdMuc EDGZ6Zc+LmtKPP085Kb4M8yjWacMKWZzPj5gaH1SY5/Bi9mCchSc2bEwQrNaGXZQ LIAFV4O2BTSAghHf6AofypfGYQPQ+xNdzzqYw9AoN9dPCBY7slMuoTlptvX5iSFh n3rXhdQ7LH8= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=message-id:from:to:mime-version :content-transfer-encoding:content-type:in-reply-to:references :subject:date; s=smtpout; bh=GtvJDtnMhlQ7dFUXWRgQLrLGlzA=; b=Yfd Df2nqPeY+NnqXh6bwrJEODNh67/iBse0oXJP/iIw/xaXhUDQq21soWhittDTHcRB wVIEdG6pS5yAaCz86apcaYIdXziVDUPbr0FTy8ZNSSFcRBEjYUqGgveAKKM3qR0V Rt2TgVDyY89bUZZpCH/oSWyY7w3LFVxQHTk25eGM= Received: by web2.nyi.mail.srv.osa (Postfix, from userid 99) id 61EEA540541; Mon, 28 Apr 2014 08:00:34 -0400 (EDT) Message-Id: <1398686434.32073.111182897.54110B1C@webmail.messagingengine.com> X-Sasl-Enc: F/iXyK+ZkJT135K1tje7otHHrEZWM+MEPkZW2db72MA1 1398686434 From: Bron Gondwana To: imapext@ietf.org MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain X-Mailer: MessagingEngine.com Webmail Interface - ajax-aafac2c3 In-Reply-To: <98fe3222-bdb5-4390-9d8e-f25f5090019d@gulbrandsen.priv.no> References: <0627dd7a-2203-40b6-b1c0-5b58613cbad8@gulbrandsen.priv.no> <8e45ff5c-1173-4119-bd53-47194d0f056b@flaska.net> <7cb76610-37e3-4903-bdc1-6d2403e27cff@gulbrandsen.priv.no> <1398465727.23560.110511065.7F835F58@webmail.messagingengine.com> <702db6f9-7038-4b15-ab1d-60e4b1568233@gulbrandsen.priv.no> <1398681263.12000.111151785.341C921F@webmail.messagingengine.com> <98fe3222-bdb5-4390-9d8e-f25f5090019d@gulbrandsen.priv.no> Date: Mon, 28 Apr 2014 22:00:34 +1000 Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/1Hrq-akkjVyf1oOrlkln13h51M4 Subject: Re: [imapext] a very simple little imap extension, really quite magnificently simple X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Apr 2014 12:00:37 -0000 On Mon, Apr 28, 2014, at 09:33 PM, Arnt Gulbrandsen wrote: > On Monday, April 28, 2014 12:34:23 PM CEST, Bron Gondwana wrote: > > True - as with anything, it's up to the server how it tracks > > things. Of course, > > if it does that, then STATUS is going to be a really expensive > > command. Lots of > > clients call STATUS too. You're shooting yourself in the foot > > not keeping an > > exists count unless you're really sure exactly who your clients are. > > O(n) is okay as long as n is small ;) I mean, if a client has ha habit of > doing FETCH 1:* and STATUS (EXISTS), the STATUS will always be less > painful. We work on a theory of 1 million emails in a mailbox... that's the sizing data we used 5 years ago when we were deciding what was reasonable performance. Might need to revise that upwards in a couple of years. > The problem only happens when clients that are optimised for small > mailboxes and used with large ones. Fixing that require client change: > Cache well, don't retrieve what you don't need, don't use MSNs, don't ever > look at the entire mailbox, etc. Yeah - though I have to say Timo's "count down" idea is pretty nice. > Btw, the SE link you posted is like ESEARCH (RETURN COUNT) KEYWORD TODO in > IMAP terms, it's not like STATUS/EXISTS. I agree that the size of such > subsets is something people want to know. You may notice that all of the > numbers in the question's example are small. People do care about numbers > as long as they're within human range. So I'm going to throw this discussion over at Neil as well, our frontend developer, and see what we can do about making our interface happy about a 4 million+ message folder (see http://jmap.io/ - working on improving that and making our server actually talk it is next on our list after the calendaring features are done). Bron. -- Bron Gondwana brong@fastmail.fm From nobody Mon Apr 28 09:18:24 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B322A1A1F20 for ; Mon, 28 Apr 2014 09:18:22 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.351 X-Spam-Level: X-Spam-Status: No, score=-0.351 tagged_above=-999 required=5 tests=[BAYES_20=-0.001, MIME_8BIT_HEADER=0.3, MIME_QP_LONG_LINE=0.001, RP_MATCHES_RCVD=-0.651] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 83urUKkg0fkk for ; Mon, 28 Apr 2014 09:18:18 -0700 (PDT) Received: from latimerie.flaska.net (latimerie.flaska.net [IPv6:2a02:2b88:2:1::4a7:333]) by ietfa.amsl.com (Postfix) with ESMTP id 3D0DE1A07A0 for ; Mon, 28 Apr 2014 09:18:17 -0700 (PDT) Received: by latimerie.flaska.net (Postfix, from userid 1000) id 805D6619FA; Mon, 28 Apr 2014 18:18:12 +0200 (CEST) From: =?iso-8859-1?Q?Jan_Kundr=E1t?= To: Date: Mon, 28 Apr 2014 18:18:15 +0200 User-Agent: Trojita/v0.4.1-191-gaeca318; Qt/4.8.5; X11; Linux; MIME-Version: 1.0 Message-ID: In-Reply-To: References: <0627dd7a-2203-40b6-b1c0-5b58613cbad8@gulbrandsen.priv.no> <8e45ff5c-1173-4119-bd53-47194d0f056b@flaska.net> <7cb76610-37e3-4903-bdc1-6d2403e27cff@gulbrandsen.priv.no> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/7ZW05syttEOFlp5DMiNnYGjl33k Subject: Re: [imapext] =?iso-8859-1?q?a_very_simple_little_imap_extension=2C_r?= =?iso-8859-1?q?eally_quite_magnificently_simple?= X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Apr 2014 16:18:22 -0000 On Friday, 25 April 2014 22:45:21 CEST, Arnt Gulbrandsen wrote: > OK, how do you think an unsolicited fetch response should look? > > a) Not have a number after the leading asterisk. Yes, that's the option which I much prefer over using made-up numbers. >> * OK [UIDNEXT 12500] guess it > > "There are <=3D5 new messages. If you want them, you can send uid=20 > fetch 12345:* and get the details you choose you want about all=20 > the new messages". Fair enough. With <=3D5 new messages, a client could very well just request=20= all the ENVELOPEs and INTERNALDATEs and other informaiton it needs, and it=20= wouldn't be a big deal. However, what if the UIDNEXT jumped by some "big=20 enough" number -- say, a 1000? Unlike a change in EXISTS, the client=20 doesn't know what really happened. It only knows an upper bound on the=20 number of new arrivals. It could still mean that there's no new arrival at=20= all, or that a thousand of new messages arrived. It would be unwise to ask=20= for a thousand of envelopes at once. >> So this is an extension whose benefit is immediately=20 >> neutralized by a client which wants to show the user the total=20 >> number of messages. To me, that sounds like a very basic=20 >> operation. > > Oh? I once asked a few colleagues at a googleapps shop about=20 > that, and none of them had any idea about their total number of=20 > messages. "Oh, lots", "I've never deleted anything", that was=20 > the sort of answer I got. None told me any number. One looked at=20 > his screen and told me how many per cent of his quota he used. I wonder whether they don't know because they don't care, or whether it's=20 because because their only UI doesn't offer that information, and thus they=20= become trained not to care about that. GMail's UI doesn't offer a way of=20 sorting the messages, yet people still do stuff like sorting by recipient=20 or by mail size. "Not supported by GMail" doesn't equal "not useful". > I can tell you why I didn't add anything like that in 5530.=20 > 5530, as you may have noticed, only lists codes, it doesn't say=20 > that you have to use any code. If two codes apply, it doesn't=20 > say which one you must choose. > > That's because when I surveyed servers for 5530, I found a few=20 > instances where two response codes applied. Two errors occured,=20 > and it wasn't clear that one response code ought to win over the=20 > other. So I decided to not issue a rule on that. A server can=20 > let the first response code win, the last one win, it doesn't=20 > matter. > > I think that also applies in this case. Suppose there's a=20 > command such as "uid search charset iso-8859-42 1 body foo". The=20 > charset specification is reasonable, but the 8859 committee=20 > hasn't issued number 42 yet, so that causes one response code.=20 > The presence of the MSN causes another. The syntax only permits=20 > one, so the server has to choose which one to send. Do you think=20 > that in this case, the MSN-related response code ought to win=20 > over the charset-related code? Do you think it ought to win over=20 > EVERY other code in ALL cases? > > I think it doesn't matter very much, so let the server do=20 > whatever comes easiest. I think that "using a command which is explicitly forbidden in this=20 situation" should indeed trump any other error state, yes. If only because=20= it's trivial to check this condition, and also because it's an easy way of=20= telling the client developer that they screwed up -- something which I find=20= very valuable as a client developer. >> OK, at this point I cannot imagine how such a client is=20 >> supposed to work *at all*. Let's start with a blank state. The=20 >> client knows nothing about the mailbox. It opens it and=20 >> presumably wants to show a list of messages to the user. What=20 >> is the client supposed to do now? > > Something "uid search unseen", "uid search or unseen within=20 > 604800", or some variation. > > Most clients, of course, do "fetch 1:* ..." instead, which is=20 > okay, but that approach means that this extension is not=20 > interesting for them. As a (desktop) client developer, I want to show my user what e-mails she=20 has in her mailbox(es). The UI framework I use needs to know the amount of=20= items to be shown in a ListView/TreeView, otherwise I have to resort to=20 ugly bits like resizing the scrollbar handle when she scrolls too close to=20= the current boundary. So OK, I have an empty state, know nothing about the number of messages in=20= a mailbox. I just know that my viewport measures 832px, and my code can=20 calculate that I can fit 40 messages in there (these numbers are made-up).=20= I have no idea what messages are these, so I can make a rough guess that a=20= week of messages might approximate what I need to show on the screen, plus=20= some extra room for the scrollbar. So I request them: a01 UID SEARCH SINCE ... Three possibilities might happen: - I got a reasonable number of messages, say, 50 - 500. In this case=20 everything is good; I can show a list of messages to the user. - I got 10k UIDs back because the user opened a LKML archive. Oops. Anyway,=20= at least I don't have to go asking for more UIDs when they scroll back. - I got just two UIDs back, perhaps because it's a low traffic mailbox.=20 This means that I can only show two mails on my big screen. That's wrong --=20= just the week ago the mailbox might have had 200 messages delivered in=20 there. I have to expand the window I sync, which means that the client has=20= to wait for multiple network round trips. I might save some bandwidth, but=20= I pay a rather big cost in latency. The client will probably want to do some extra preload, if only to know=20 whether to include some platform-specific UI equivalent of "load more=20 messages" -- be it either a button, or something which actives upon show,=20 or just some scrollbar indicator. >> What if it wants to show just a subset of the mailbox, perhaps=20 >> last 1k items (actually it doesn't have to be 1k, anything=20 >> between 500 and 3000 is probably fine -- it needs *something*,=20 >> but not the full 4M mailbox). How to achieve that? > > "Show today's mail" is trivial, "show all the unread mail" is=20 > trivial, "show the last five hundred messages" is difficult. And that's exactly the point -- taking away EXISTS and MSNs makes it=20 impossible to page through the result set based on the number of messages.=20= The ability of UIs to show stuff to the user depens on the number of items=20= to be shown, not on the time interval covered by the N items, IMHO. >> What happens when a mail arrives? > > Same as before. Think about it. > > Before, you got an EXISTS for two new messages. But then=20 > someone expunges one before the server processes your fetch, and=20 > you get one message back. With UIDONLY, you get a UIDNEXT=20 > indicating that two new UIDs have been allocated. You issue a=20 > fetch, and get <=3D2 messages. There's no difference in principle. Apart to what I said at the beginning of the mail, this change also takes=20 away the possibility to double-check that both sides are synced.=20 Previously, a change of EXISTS without an appropriate EXPUNGE means that=20 someone screwed up, and it is easy to detect this and give up. When you=20 remove EXISTS, the notification changes to "Some number of new messages=20 might have arrived. The is no more than N new messages.", which is a step=20 backwards IMHO. (Yes, I'm still counting messages in my client and in my=20 mental model.) >> How is a resynchronization after being offline for a while=20 >> going to look like? > > Same as before. > > I'll send you a session transcript offlist. I may not get it=20 > done tonight, though. I think it should be a part of the RFC. I don't think I would be the only=20 one asking these questions. With kind regards, Jan --=20 Trojit=C3=A1, a fast Qt IMAP e-mail client -- http://trojita.flaska.net/ From nobody Mon Apr 28 16:58:56 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B949D1A8841 for ; Mon, 28 Apr 2014 16:58:54 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.552 X-Spam-Level: X-Spam-Status: No, score=-2.552 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-0.651, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Y3QcHxsre5tr for ; Mon, 28 Apr 2014 16:58:52 -0700 (PDT) Received: from smtp.qbik.com (smtp.qbik.com [210.55.214.35]) by ietfa.amsl.com (Postfix) with ESMTP id 6EE3F1A6F17 for ; Mon, 28 Apr 2014 16:58:52 -0700 (PDT) Received: From [192.168.0.38] (unverified [192.168.0.38]) by SMTP Server [192.168.0.1] (WinGate SMTP Receiver v8.2.1 (Build 4711)) with SMTP id <0020010897@smtp.qbik.com>; Tue, 29 Apr 2014 11:58:49 +1200 From: "Adrien de Croy" To: "Bron Gondwana" , "imapext@ietf.org" Date: Mon, 28 Apr 2014 23:58:49 +0000 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; format=flowed; charset=utf-8 In-Reply-To: <1398686434.32073.111182897.54110B1C@webmail.messagingengine.com> Message-Id: Mime-Version: 1.0 User-Agent: eM_Client/6.0.20154.0 Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/U5S3PAoX1VhFmEh_tlK-kdCtveg Subject: Re: [imapext] a very simple little imap extension, really quite magnificently simple X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Adrien de Croy List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Apr 2014 23:58:54 -0000 I'm finding it hard to get my head around 1M messages in a mailbox. This obviously isn't a person's mailbox, unless they have their email=20 address on every spammer's list and haven't deleted a message (including= =20 spam) since 1967. Why mess with the spec to handle a use-case that is meaningless? Unless this is some kind of use-IMAP-as-a-database-for-some-app kind of=20 scenario? ------ Original Message ------ From: "Bron Gondwana" To: "imapext@ietf.org" Sent: 29/04/2014 12:00:34 a.m. Subject: Re: [imapext] a very simple little imap extension, really quite= =20 magnificently simple >On Mon, Apr 28, 2014, at 09:33 PM, Arnt Gulbrandsen wrote: >> On Monday, April 28, 2014 12:34:23 PM CEST, Bron Gondwana wrote: >> > True - as with anything, it's up to the server how it tracks >> > things. Of course, >> > if it does that, then STATUS is going to be a really expensive >> > command. Lots of >> > clients call STATUS too. You're shooting yourself in the foot >> > not keeping an >> > exists count unless you're really sure exactly who your clients=20 >>are. >> >> O(n) is okay as long as n is small ;) I mean, if a client has ha=20 >>habit of >> doing FETCH 1:* and STATUS (EXISTS), the STATUS will always be less >> painful. > >We work on a theory of 1 million emails in a mailbox... that's the=20 >sizing >data we used 5 years ago when we were deciding what was reasonable >performance. Might need to revise that upwards in a couple of years. > >> The problem only happens when clients that are optimised for small >> mailboxes and used with large ones. Fixing that require client=20 >>change: >> Cache well, don't retrieve what you don't need, don't use MSNs, don't= =20 >>ever >> look at the entire mailbox, etc. > >Yeah - though I have to say Timo's "count down" idea is pretty nice. > >> Btw, the SE link you posted is like ESEARCH (RETURN COUNT) KEYWORD=20 >>TODO in >> IMAP terms, it's not like STATUS/EXISTS. I agree that the size of=20 >>such >> subsets is something people want to know. You may notice that all of= =20 >>the >> numbers in the question's example are small. People do care about=20 >>numbers >> as long as they're within human range. > >So I'm going to throw this discussion over at Neil as well, our=20 >frontend >developer, and see what we can do about making our interface happy=20 >about >a 4 million+ message folder (see http://jmap.io/ - working on improving >that and making our server actually talk it is next on our list after=20 >the >calendaring features are done). > >Bron. > >-- > Bron Gondwana > brong@fastmail.fm > >_______________________________________________ >imapext mailing list >imapext@ietf.org >https://www.ietf.org/mailman/listinfo/imapext From nobody Mon Apr 28 17:08:21 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7F6691A8841 for ; Mon, 28 Apr 2014 17:08:19 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.029 X-Spam-Level: X-Spam-Status: No, score=-2.029 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, RP_MATCHES_RCVD=-0.651, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7X2dVAWk_XGq for ; Mon, 28 Apr 2014 17:08:17 -0700 (PDT) Received: from mail-ig0-x231.google.com (mail-ig0-x231.google.com [IPv6:2607:f8b0:4001:c05::231]) by ietfa.amsl.com (Postfix) with ESMTP id F26981A6F17 for ; Mon, 28 Apr 2014 17:08:16 -0700 (PDT) Received: by mail-ig0-f177.google.com with SMTP id h3so5495729igd.10 for ; Mon, 28 Apr 2014 17:08:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=8y9SnpgGAqqJQfsftFEY8+FlgTw7S52zifK1BTVVXDU=; b=aYLpQ2reEaExPmjLNaKQWyOb5D6r+/2+BXRsTwOgct0f3y6wkDT2pCKP8Bwzv/zYvn er9rZRbekpSp7IhjY9aL8Lt+81r/wR80skksCFhzzmgZrSrSdJ3FVuJ6b5ehW2vIx7FT V+j/Q0H0apzMkVyjOQWV8C6My4CKNZYRoIL9fRI2kIQQmqKUGeTEJNCwu7fRJyGLYQxx P9vWAU6A7SSUq7LiLlsSCMS5klJUWC0qiTFn+oIIr2MBRqUR9quiJCMCz4HuchCyUfRm ATKf2FpYHOtwVEHAOaL9N8bdpGIDRHmVmjs3n95SAiE9CyLoejrjeLwrxE5vbut03WlU 9g9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=8y9SnpgGAqqJQfsftFEY8+FlgTw7S52zifK1BTVVXDU=; b=TT60/GK0CoClb5NxtaDyoWU/8ukmjgCVx1Mh6BXYMHgUNhcK+w/nB/KEf2ryi/b4Sn FIAqd5UwU2lDvkUAS1+mORkNVFtcSChE1V1J1iOutQ0Z2vBlYLpJJtVIkdjeUuky1zLK ee+hgmPIZT96/SEsuirNBj7waojApjC9NoUtEuo9tftF7i33BOWW/Au7xbyVPqmsNZyF GYk4sKEenbtnweuDnl7RWK/ogzW0ujVu3Y03sDe3yb6G6Mo3GIN5/HKRijg16IHk0ywE /CBgs9AEFmApWVVZd0BJVGHgSo2ALOZzTZLqtRr32IPMSSVb3tCKKti6MtBZHNBgeYBc pMwg== X-Gm-Message-State: ALoCoQl3W3QVShUrIEi8lsUvie1aNz7b0U2Y1/NWbk89UfJHU6XgnbVA0TQWx8IdciD5fVl6rkzQELuvbp8Z9ezlQ7bE9pXtmcijrKcAGuE4xKDBeY1ibVmVJ4daO8lNzuKEaDid+t+XtBCMePwpKGUJkkHOHbdkNB9K/8g2W3FYRg5sm1eeiaueOFP9XJjOEg8aD4Oxz/Mm MIME-Version: 1.0 X-Received: by 10.42.251.78 with SMTP id mr14mr25747096icb.47.1398730095988; Mon, 28 Apr 2014 17:08:15 -0700 (PDT) Received: by 10.64.77.201 with HTTP; Mon, 28 Apr 2014 17:08:15 -0700 (PDT) In-Reply-To: References: <1398686434.32073.111182897.54110B1C@webmail.messagingengine.com> Date: Mon, 28 Apr 2014 17:08:15 -0700 Message-ID: From: Brandon Long To: Adrien de Croy Content-Type: multipart/alternative; boundary=14dae93412858240fb04f8233a03 Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/nVVM-tICF_R6fz2rdOq7mTnztS0 Cc: Bron Gondwana , "imapext@ietf.org" Subject: Re: [imapext] a very simple little imap extension, really quite magnificently simple X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Apr 2014 00:08:19 -0000 --14dae93412858240fb04f8233a03 Content-Type: text/plain; charset=UTF-8 We have many many mailboxes over 1M, and I'm sure that's partially due to our "virtual" folders and the fact that there exists the "all mail" folder which is everything in the users entire account. In fact, we have accounts in the 10-50M range, though yes, the likelihood of those being "useful" accounts is arguable. They range from legal mail dumps to source accounts for support systems to who knows what. My work account here at google has 1.6M messages in it. And such accounts are just growing in size. Brandon On Mon, Apr 28, 2014 at 4:58 PM, Adrien de Croy wrote: > > I'm finding it hard to get my head around 1M messages in a mailbox. > > This obviously isn't a person's mailbox, unless they have their email > address on every spammer's list and haven't deleted a message (including > spam) since 1967. > > Why mess with the spec to handle a use-case that is meaningless? > > Unless this is some kind of use-IMAP-as-a-database-for-some-app kind of > scenario? > > > > > ------ Original Message ------ > From: "Bron Gondwana" > To: "imapext@ietf.org" > Sent: 29/04/2014 12:00:34 a.m. > Subject: Re: [imapext] a very simple little imap extension, really quite > magnificently simple > > On Mon, Apr 28, 2014, at 09:33 PM, Arnt Gulbrandsen wrote: >> >>> On Monday, April 28, 2014 12:34:23 PM CEST, Bron Gondwana wrote: >>> > True - as with anything, it's up to the server how it tracks >>> > things. Of course, >>> > if it does that, then STATUS is going to be a really expensive >>> > command. Lots of >>> > clients call STATUS too. You're shooting yourself in the foot >>> > not keeping an >>> > exists count unless you're really sure exactly who your clients are. >>> >>> O(n) is okay as long as n is small ;) I mean, if a client has ha habit >>> of >>> doing FETCH 1:* and STATUS (EXISTS), the STATUS will always be less >>> painful. >>> >> >> We work on a theory of 1 million emails in a mailbox... that's the sizing >> data we used 5 years ago when we were deciding what was reasonable >> performance. Might need to revise that upwards in a couple of years. >> >> The problem only happens when clients that are optimised for small >>> mailboxes and used with large ones. Fixing that require client change: >>> Cache well, don't retrieve what you don't need, don't use MSNs, don't >>> ever >>> look at the entire mailbox, etc. >>> >> >> Yeah - though I have to say Timo's "count down" idea is pretty nice. >> >> Btw, the SE link you posted is like ESEARCH (RETURN COUNT) KEYWORD TODO >>> in >>> IMAP terms, it's not like STATUS/EXISTS. I agree that the size of such >>> subsets is something people want to know. You may notice that all of the >>> numbers in the question's example are small. People do care about >>> numbers >>> as long as they're within human range. >>> >> >> So I'm going to throw this discussion over at Neil as well, our frontend >> developer, and see what we can do about making our interface happy about >> a 4 million+ message folder (see http://jmap.io/ - working on improving >> that and making our server actually talk it is next on our list after the >> calendaring features are done). >> >> Bron. >> >> -- >> Bron Gondwana >> brong@fastmail.fm >> >> _______________________________________________ >> imapext mailing list >> imapext@ietf.org >> https://www.ietf.org/mailman/listinfo/imapext >> > > _______________________________________________ > imapext mailing list > imapext@ietf.org > https://www.ietf.org/mailman/listinfo/imapext > --14dae93412858240fb04f8233a03 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
We have many many mailboxes over 1M, and I'm sure that= 's partially due to our "virtual" folders and the fact that t= here exists the "all mail" folder which is everything in the user= s entire account.

In fact, we have accounts in the 10-50M range, though yes, t= he likelihood of those being "useful" accounts is arguable. =C2= =A0They range from legal mail dumps to source accounts for support systems = to who knows what.

My work account here at google has 1.6M messages in it.=

And such accounts are just growing in size.
=

Brandon


On Mon, Apr 28, 2014 at 4:58 PM, Adrien de Croy = <= adrien@qbik.com> wrote:

I'm finding it hard to get my head around 1M messages in a mailbox.

This obviously isn't a person's mailbox, unless they have their ema= il address on every spammer's list and haven't deleted a message (i= ncluding spam) since 1967.

Why mess with the spec to handle a use-case that is meaningless?

Unless this is some kind of use-IMAP-as-a-database-for-some-app kind= of scenario?




------ Original Message ------
From: "Bron Gondwana" <brong@fastmail.fm>
To: "imapext@iet= f.org" <i= mapext@ietf.org>
Sent: 29/04/2014 12:00:34 a.m.
Subject: Re: [imapext] a very simple little imap extension, really quite ma= gnificently simple

On Mon, Apr 28, 2014, at 09:33 PM, Arnt Gulbrandsen wrote:
=C2=A0On Monday, April 28, 2014 12:34:23 PM CEST, Bron Gondwana wrote:
=C2=A0> True - as with anything, it's up to the server how it tracks=
=C2=A0> things. Of course,
=C2=A0> if it does that, then STATUS is going to be a really expensive =C2=A0> command. Lots of
=C2=A0> clients call STATUS too. You're shooting yourself in the foo= t
=C2=A0> not keeping an
=C2=A0> exists count unless you're really sure exactly who your clie= nts are.

=C2=A0O(n) is okay as long as n is small ;) I mean, if a client has ha habi= t of
=C2=A0doing FETCH 1:* and STATUS (EXISTS), the STATUS will always be less =C2=A0painful.

We work on a theory of 1 million emails in a mailbox... that's the sizi= ng
data we used 5 years ago when we were deciding what was reasonable
performance. Might need to revise that upwards in a couple of years.

=C2=A0The problem only happens when clients that are optimised for small =C2=A0mailboxes and used with large ones. Fixing that require client change= :
=C2=A0Cache well, don't retrieve what you don't need, don't use= MSNs, don't ever
=C2=A0look at the entire mailbox, etc.

Yeah - though I have to say Timo's "count down" idea is prett= y nice.

=C2=A0Btw, the SE link you posted is like ESEARCH (RETURN COUNT) KEYWORD TO= DO in
=C2=A0IMAP terms, it's not like STATUS/EXISTS. I agree that the size of= such
=C2=A0subsets is something people want to know. You may notice that all of = the
=C2=A0numbers in the question's example are small. People do care about= numbers
=C2=A0as long as they're within human range.

So I'm going to throw this discussion over at Neil as well, our fronten= d
developer, and see what we can do about making our interface happy about a 4 million+ message folder (see http://jmap.io/ - working on improving
that and making our server actually talk it is next on our list after the calendaring features are done).

Bron.

--
=C2=A0 Bron Gondwana
=C2=A0 brong@fastmai= l.fm

_______________________________________________
imapext mailing list
imapext@ietf.org<= br> https://www.ietf.org/mailman/listinfo/imapext

_______________________________________________
imapext mailing list
imapext@ietf.org<= br> https://www.ietf.org/mailman/listinfo/imapext

--14dae93412858240fb04f8233a03-- From nobody Mon Apr 28 17:12:12 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B0E501A8843 for ; Mon, 28 Apr 2014 17:12:09 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.551 X-Spam-Level: X-Spam-Status: No, score=-2.551 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RP_MATCHES_RCVD=-0.651, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5pm7VZCMivD3 for ; Mon, 28 Apr 2014 17:12:07 -0700 (PDT) Received: from smtp.qbik.com (smtp.qbik.com [210.55.214.35]) by ietfa.amsl.com (Postfix) with ESMTP id B3F271A6F17 for ; Mon, 28 Apr 2014 17:12:06 -0700 (PDT) Received: From [192.168.0.38] (unverified [192.168.0.38]) by SMTP Server [192.168.0.1] (WinGate SMTP Receiver v8.2.1 (Build 4711)) with SMTP id <0020010906@smtp.qbik.com>; Tue, 29 Apr 2014 12:12:05 +1200 From: "Adrien de Croy" To: "Brandon Long" Date: Tue, 29 Apr 2014 00:12:05 +0000 Content-Type: multipart/alternative; boundary="------=_MB3FF74D8E-30EE-4C47-A7E1-A5B461FDDCA6" In-Reply-To: Message-Id: Mime-Version: 1.0 User-Agent: eM_Client/6.0.20154.0 Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/0syfSP1yZI92DWpMpr-wJjJlGmI Cc: Bron Gondwana , "imapext@ietf.org" Subject: Re: [imapext] a very simple little imap extension, really quite magnificently simple X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Adrien de Croy List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Apr 2014 00:12:09 -0000 --------=_MB3FF74D8E-30EE-4C47-A7E1-A5B461FDDCA6 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; format=flowed; charset=utf-8 maybe it's time for some quota enforcement or forced archiving. How long would it take a person to make head or tail of 1M messages. =20 E.g. even only to decide what to keep or not. It is easier to just keep everything just in case, but there has to be=20 some sanity somewhere... ------ Original Message ------ From: "Brandon Long" To: "Adrien de Croy" Cc: "Bron Gondwana" ; "imapext@ietf.org"=20 Sent: 29/04/2014 12:08:15 p.m. Subject: Re: [imapext] a very simple little imap extension, really quite= =20 magnificently simple >We have many many mailboxes over 1M, and I'm sure that's partially due=20 >to our "virtual" folders and the fact that there exists the "all mail"=20 >folder which is everything in the users entire account. > >In fact, we have accounts in the 10-50M range, though yes, the=20 >likelihood of those being "useful" accounts is arguable. They range=20 >from legal mail dumps to source accounts for support systems to who=20 >knows what. > >My work account here at google has 1.6M messages in it. > >And such accounts are just growing in size. > >Brandon > > >On Mon, Apr 28, 2014 at 4:58 PM, Adrien de Croy =20 >wrote: >> >>I'm finding it hard to get my head around 1M messages in a mailbox. >> >>This obviously isn't a person's mailbox, unless they have their email=20 >>address on every spammer's list and haven't deleted a message=20 >>(including spam) since 1967. >> >>Why mess with the spec to handle a use-case that is meaningless? >> >>Unless this is some kind of use-IMAP-as-a-database-for-some-app kind=20 >>of scenario? >> >> >> >> >>------ Original Message ------ >>From: "Bron Gondwana" >>To: "imapext@ietf.org" >>Sent: 29/04/2014 12:00:34 a.m. >>Subject: Re: [imapext] a very simple little imap extension, really=20 >>quite magnificently simple >> >>>On Mon, Apr 28, 2014, at 09:33 PM, Arnt Gulbrandsen wrote: >>>> On Monday, April 28, 2014 12:34:23 PM CEST, Bron Gondwana wrote: >>>> > True - as with anything, it's up to the server how it tracks >>>> > things. Of course, >>>> > if it does that, then STATUS is going to be a really expensive >>>> > command. Lots of >>>> > clients call STATUS too. You're shooting yourself in the foot >>>> > not keeping an >>>> > exists count unless you're really sure exactly who your clients=20 >>>>are. >>>> >>>> O(n) is okay as long as n is small ;) I mean, if a client has ha=20 >>>>habit of >>>> doing FETCH 1:* and STATUS (EXISTS), the STATUS will always be less >>>> painful. >>> >>>We work on a theory of 1 million emails in a mailbox... that's the=20 >>>sizing >>>data we used 5 years ago when we were deciding what was reasonable >>>performance. Might need to revise that upwards in a couple of years. >>> >>>> The problem only happens when clients that are optimised for small >>>> mailboxes and used with large ones. Fixing that require client=20 >>>>change: >>>> Cache well, don't retrieve what you don't need, don't use MSNs,=20 >>>>don't ever >>>> look at the entire mailbox, etc. >>> >>>Yeah - though I have to say Timo's "count down" idea is pretty nice. >>> >>>> Btw, the SE link you posted is like ESEARCH (RETURN COUNT) KEYWORD= =20 >>>>TODO in >>>> IMAP terms, it's not like STATUS/EXISTS. I agree that the size of=20 >>>>such >>>> subsets is something people want to know. You may notice that all=20 >>>>of the >>>> numbers in the question's example are small. People do care about=20 >>>>numbers >>>> as long as they're within human range. >>> >>>So I'm going to throw this discussion over at Neil as well, our=20 >>>frontend >>>developer, and see what we can do about making our interface happy=20 >>>about >>>a 4 million+ message folder (see http://jmap.io/ - working on=20 >>>improving >>>that and making our server actually talk it is next on our list after= =20 >>>the >>>calendaring features are done). >>> >>>Bron. >>> >>>-- >>> Bron Gondwana >>> brong@fastmail.fm >>> >>>_______________________________________________ >>>imapext mailing list >>>imapext@ietf.org >>>https://www.ietf.org/mailman/listinfo/imapext >> >>_______________________________________________ >>imapext mailing list >>imapext@ietf.org >>https://www.ietf.org/mailman/listinfo/imapext > --------=_MB3FF74D8E-30EE-4C47-A7E1-A5B461FDDCA6 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8
 
maybe it's time for some quota enforcement or forced archiving.
 
How long would it take a person to make head or tail of 1M messages.&n= bsp; E.g. even only to decide what to keep or not.
 
It is easier to just keep everything just in case, but there has to= be some sanity somewhere...
 
 
------ Original Message ------
From: "Brandon Long" <blong@goo= gle.com>
To: "Adrien de Croy" <adrien@qbi= k.com>
Cc: "Bron Gondwana" <brong@fas= tmail.fm>; "imapext@ietf.org" <imapext@ietf.org>
Sent: 29/04/2014 12:08:15 p.m.
Subject: Re: [imapext] a very simple little imap extension, really = quite magnificently simple
 
We have many many mailboxes over 1M, and I'm sure that's = partially due to our "virtual" folders and the fact that there exists the= "all mail" folder which is everything in the users entire account.=20

In fact, we have accounts in the 10-50M range, though yes, the likelih= ood of those being "useful" accounts is arguable.  They range from = legal mail dumps to source accounts for support systems to who knows what.<= /DIV>

My work account here at google has 1.6M messages in it.

And such accounts are just growing in size.

Brandon


On Mon, Apr 28, 2014 at 4:58 PM, Adrien de Croy= <adrien@qbik.com= > wrote:

I'm finding it hard to get= my head around 1M messages in a mailbox.

This obviously isn't a = person's mailbox, unless they have their email address on every spammer's= list and haven't deleted a message (including spam) since 1967.

Why= mess with the spec to handle a use-case that is meaningless?

Unless= this is some kind of use-IMAP-as-a-database-for-some-app kind of= scenario?=20




------ Original Message ------
From: = "Bron Gondwana" <brong@fastmail.fm<= /A>>
To: "imapext@ietf.org"= <imapext@ietf.org>
Sent:= 29/04/2014 12:00:34 a.m.
Subject: Re: [imapext] a very simple little= imap extension, really quite magnificently simple

On Mon, Apr 28, 2014, at 09:33= PM, Arnt Gulbrandsen wrote:
 On Monday, April 28, 2014= 12:34:23 PM CEST, Bron Gondwana wrote:
 > True - as with anythi= ng, it's up to the server how it tracks
 > things. Of course, > if it does that, then STATUS is going to be a really expensive<= BR> > command. Lots of
 > clients call STATUS too. You'r= e shooting yourself in the foot
 > not keeping an
 >= exists count unless you're really sure exactly who your clients are.
 O(n) is okay as long as n is small ;) I mean, if a client has ha= habit of
 doing FETCH 1:* and STATUS (EXISTS), the STATUS will = always be less
 painful.

We work on a theory= of 1 million emails in a mailbox... that's the sizing
data we used 5= years ago when we were deciding what was reasonable
performance. Might= need to revise that upwards in a couple of years.

 The problem only happens= when clients that are optimised for small
 mailboxes and used with= large ones. Fixing that require client change:
 Cache well, don't= retrieve what you don't need, don't use MSNs, don't ever
 look = at the entire mailbox, etc.

Yeah - though I have to = say Timo's "count down" idea is pretty nice.

 Btw, the SE link you post= ed is like ESEARCH (RETURN COUNT) KEYWORD TODO in
 IMAP terms, it's= not like STATUS/EXISTS. I agree that the size of such
 subsets = is something people want to know. You may notice that all of the
 n= umbers in the question's example are small. People do care about numbers as long as they're within human range.

So I'm= going to throw this discussion over at Neil as well, our frontend
devel= oper, and see what we can do about making our interface happy about
a= 4 million+ message folder (see http://jmap.io/= - working on improving
that and making our server actually talk = it is next on our list after the
calendaring features are done).

= Bron.

--
  Bron Gondwana
  brong@fastmail.fm

______________________________<= /U>_________________
imapext mailing list
imapext@ietf.org
https://www.ietf.org/mailman/listinfo/imapext

_______________________________________________imapext mailing list
imapext@ietf.o= rg
https:/= /www.ietf.org/mailman/listinfo/imapext

--------=_MB3FF74D8E-30EE-4C47-A7E1-A5B461FDDCA6-- From nobody Mon Apr 28 17:18:55 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1EA921A8842 for ; Mon, 28 Apr 2014 17:18:54 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.699 X-Spam-Level: X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ndc68AXXiGMz for ; Mon, 28 Apr 2014 17:18:51 -0700 (PDT) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) by ietfa.amsl.com (Postfix) with ESMTP id 287751A6F17 for ; Mon, 28 Apr 2014 17:18:50 -0700 (PDT) Received: from compute5.internal (compute5.nyi.mail.srv.osa [10.202.2.45]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 5800620C65; Mon, 28 Apr 2014 20:18:42 -0400 (EDT) Received: from web6 ([10.202.2.216]) by compute5.internal (MEProxy); Mon, 28 Apr 2014 20:18:42 -0400 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=fastmail.fm; h= message-id:from:to:cc:mime-version:content-transfer-encoding :content-type:subject:date:in-reply-to:references; s=mesmtp; bh= iXfKPhX4w78q39qBSGiugdtkrvY=; b=GHCPMx0V1FT+CL9A2Mk01J27Vr3A0XOq XYxKhhceUYJdn8g7yYF//A1w08X+XNIKFbD0EA0aVCbiozVyRO6dfEbwlDDHEXiv vDy8jPUls/mfYQHCLwYPLOXK8UWKC/wlgujjkNRPH7t+ntC0kv3MmkA67hJdsg14 UgbpOBFfWjE= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=message-id:from:to:cc:mime-version :content-transfer-encoding:content-type:subject:date:in-reply-to :references; s=smtpout; bh=iXfKPhX4w78q39qBSGiugdtkrvY=; b=PC2ZB mFenUHH2jnGdlUMjmNGbuSzxuxVu5rW/D/iT601Xhy9JsO78A91sSe/tgko0TGEw yjGBLVoRtqM4CzAIz5QEJpuvCs7oENl36NFESwZiEj4JEJpyJ5GFB2U0PaL6NQUp luzi7n3nOaD/lqVOiXLe2xRsBy6a7Fff5f5zl0= Received: by web6.nyi.mail.srv.osa (Postfix, from userid 99) id 2DCAD283211; Mon, 28 Apr 2014 20:18:42 -0400 (EDT) Message-Id: <1398730722.14717.111466701.6F36AE10@webmail.messagingengine.com> X-Sasl-Enc: bFiGT9vcZ4SnoLkZsKW11o8BWl4dzMkIUgTkoACcczoF 1398730722 From: Bron Gondwana To: Adrien de Croy , Brandon Long MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: multipart/alternative; boundary="_----------=_1398730722147170"; charset="utf-8" X-Mailer: MessagingEngine.com Webmail Interface - ajax-aafac2c3 Date: Tue, 29 Apr 2014 10:18:42 +1000 In-Reply-To: References: Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/CTUaUg1kqPAGwW4kRn0SbE6OEd4 Cc: imapext@ietf.org Subject: Re: [imapext] a very simple little imap extension, really quite magnificently simple X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Apr 2014 00:18:54 -0000 This is a multi-part message in MIME format. --_----------=_1398730722147170 Content-Transfer-Encoding: 7bit Content-Type: text/plain With good search (something that Google has) you don't need to make head or tail of everything. You can search for what you need when you need it. The use-case makes sense to me. Now - personally I archive messages per-year, so each folder contains fewer than that. With cross-folder searching, it doesn't cost any more to find messages, but I can page through each folder a little faster. (I also filter each mailing list into a separate folder though... so I think I'm some kind of weird dinosaur) Bron. On Tue, Apr 29, 2014, at 10:12 AM, Adrien de Croy wrote: maybe it's time for some quota enforcement or forced archiving. How long would it take a person to make head or tail of 1M messages. E.g. even only to decide what to keep or not. It is easier to just keep everything just in case, but there has to be some sanity somewhere... ------ Original Message ------ From: "Brandon Long" <[1]blong@google.com> To: "Adrien de Croy" <[2]adrien@qbik.com> Cc: "Bron Gondwana" <[3]brong@fastmail.fm>; "imapext@ietf.org" <[4]imapext@ietf.org> Sent: 29/04/2014 12:08:15 p.m. Subject: Re: [imapext] a very simple little imap extension, really quite magnificently simple We have many many mailboxes over 1M, and I'm sure that's partially due to our "virtual" folders and the fact that there exists the "all mail" folder which is everything in the users entire account. In fact, we have accounts in the 10-50M range, though yes, the likelihood of those being "useful" accounts is arguable. They range from legal mail dumps to source accounts for support systems to who knows what. My work account here at google has 1.6M messages in it. And such accounts are just growing in size. Brandon On Mon, Apr 28, 2014 at 4:58 PM, Adrien de Croy <[5]adrien@qbik.com> wrote: I'm finding it hard to get my head around 1M messages in a mailbox. This obviously isn't a person's mailbox, unless they have their email address on every spammer's list and haven't deleted a message (including spam) since 1967. Why mess with the spec to handle a use-case that is meaningless? Unless this is some kind of use-IMAP-as-a-database-for-some-app kind of scenario? ------ Original Message ------ From: "Bron Gondwana" <[6]brong@fastmail.fm> To: "[7]imapext@ietf.org" <[8]imapext@ietf.org> Sent: 29/04/2014 12:00:34 a.m. Subject: Re: [imapext] a very simple little imap extension, really quite magnificently simple On Mon, Apr 28, 2014, at 09:33 PM, Arnt Gulbrandsen wrote: On Monday, April 28, 2014 12:34:23 PM CEST, Bron Gondwana wrote: > True - as with anything, it's up to the server how it tracks > things. Of course, > if it does that, then STATUS is going to be a really expensive > command. Lots of > clients call STATUS too. You're shooting yourself in the foot > not keeping an > exists count unless you're really sure exactly who your clients are. O(n) is okay as long as n is small ;) I mean, if a client has ha habit of doing FETCH 1:* and STATUS (EXISTS), the STATUS will always be less painful. We work on a theory of 1 million emails in a mailbox... that's the sizing data we used 5 years ago when we were deciding what was reasonable performance. Might need to revise that upwards in a couple of years. The problem only happens when clients that are optimised for small mailboxes and used with large ones. Fixing that require client change: Cache well, don't retrieve what you don't need, don't use MSNs, don't ever look at the entire mailbox, etc. Yeah - though I have to say Timo's "count down" idea is pretty nice. Btw, the SE link you posted is like ESEARCH (RETURN COUNT) KEYWORD TODO in IMAP terms, it's not like STATUS/EXISTS. I agree that the size of such subsets is something people want to know. You may notice that all of the numbers in the question's example are small. People do care about numbers as long as they're within human range. So I'm going to throw this discussion over at Neil as well, our frontend developer, and see what we can do about making our interface happy about a 4 million+ message folder (see [9]http://jmap.io/ - working on improving that and making our server actually talk it is next on our list after the calendaring features are done). Bron. -- Bron Gondwana [10]brong@fastmail.fm _______________________________________________ imapext mailing list [11]imapext@ietf.org [12]https://www.ietf.org/mailman/listinfo/imapext _______________________________________________ imapext mailing list [13]imapext@ietf.org [14]https://www.ietf.org/mailman/listinfo/imapext _______________________________________________ imapext mailing list [15]imapext@ietf.org [16]https://www.ietf.org/mailman/listinfo/imapext -- Bron Gondwana brong@fastmail.fm References 1. mailto:blong@google.com 2. mailto:adrien@qbik.com 3. mailto:brong@fastmail.fm 4. mailto:imapext@ietf.org 5. mailto:adrien@qbik.com 6. mailto:brong@fastmail.fm 7. mailto:imapext@ietf.org 8. mailto:imapext@ietf.org 9. http://jmap.io/ 10. mailto:brong@fastmail.fm 11. mailto:imapext@ietf.org 12. https://www.ietf.org/mailman/listinfo/imapext 13. mailto:imapext@ietf.org 14. https://www.ietf.org/mailman/listinfo/imapext 15. mailto:imapext@ietf.org 16. https://www.ietf.org/mailman/listinfo/imapext --_----------=_1398730722147170 Content-Transfer-Encoding: 7bit Content-Type: text/html
With good search (something that Google has) you don't need to make head or tail of everything.  You can search for what you need when you need it.  The use-case makes sense to me.
 
Now - personally I archive messages per-year, so each folder contains fewer than that.  With cross-folder searching, it doesn't cost any more to find messages, but I can page through each folder a little faster.
 
(I also filter each mailing list into a separate folder though... so I think I'm some kind of weird dinosaur)

Bron.
 
On Tue, Apr 29, 2014, at 10:12 AM, Adrien de Croy wrote:
 
maybe it's time for some quota enforcement or forced archiving.
 
How long would it take a person to make head or tail of 1M messages.  E.g. even only to decide what to keep or not.
 
It is easier to just keep everything just in case, but there has to be some sanity somewhere...
 
 
------ Original Message ------
From: "Brandon Long" <blong@google.com>
To: "Adrien de Croy" <adrien@qbik.com>
Cc: "Bron Gondwana" <brong@fastmail.fm>; "imapext@ietf.org" <imapext@ietf.org>
Sent: 29/04/2014 12:08:15 p.m.
Subject: Re: [imapext] a very simple little imap extension, really quite magnificently simple
 
We have many many mailboxes over 1M, and I'm sure that's partially due to our "virtual" folders and the fact that there exists the "all mail" folder which is everything in the users entire account.
 
In fact, we have accounts in the 10-50M range, though yes, the likelihood of those being "useful" accounts is arguable.  They range from legal mail dumps to source accounts for support systems to who knows what.
 
My work account here at google has 1.6M messages in it.
 
And such accounts are just growing in size.
 
Brandon
 
 
 
On Mon, Apr 28, 2014 at 4:58 PM, Adrien de Croy <adrien@qbik.com> wrote:
 
 
I'm finding it hard to get my head around 1M messages in a mailbox.
 
This obviously isn't a person's mailbox, unless they have their email address on every spammer's list and haven't deleted a message (including spam) since 1967.
 
Why mess with the spec to handle a use-case that is meaningless?
 
Unless this is some kind of use-IMAP-as-a-database-for-some-app kind of scenario?
 
 
 
 
------ Original Message ------
From: "Bron Gondwana" <brong@fastmail.fm>
Sent: 29/04/2014 12:00:34 a.m.
Subject: Re: [imapext] a very simple little imap extension, really quite magnificently simple
 
 
On Mon, Apr 28, 2014, at 09:33 PM, Arnt Gulbrandsen wrote:
 
 On Monday, April 28, 2014 12:34:23 PM CEST, Bron Gondwana wrote:
 > True - as with anything, it's up to the server how it tracks
 > things. Of course,
 > if it does that, then STATUS is going to be a really expensive
 > command. Lots of
 > clients call STATUS too. You're shooting yourself in the foot
 > not keeping an
 > exists count unless you're really sure exactly who your clients are.
 
 O(n) is okay as long as n is small ;) I mean, if a client has ha habit of
 doing FETCH 1:* and STATUS (EXISTS), the STATUS will always be less
 painful.
 
We work on a theory of 1 million emails in a mailbox... that's the sizing
data we used 5 years ago when we were deciding what was reasonable
performance. Might need to revise that upwards in a couple of years.
 
 
 The problem only happens when clients that are optimised for small
 mailboxes and used with large ones. Fixing that require client change:
 Cache well, don't retrieve what you don't need, don't use MSNs, don't ever
 look at the entire mailbox, etc.
 
Yeah - though I have to say Timo's "count down" idea is pretty nice.
 
 
 Btw, the SE link you posted is like ESEARCH (RETURN COUNT) KEYWORD TODO in
 IMAP terms, it's not like STATUS/EXISTS. I agree that the size of such
 subsets is something people want to know. You may notice that all of the
 numbers in the question's example are small. People do care about numbers
 as long as they're within human range.
 
So I'm going to throw this discussion over at Neil as well, our frontend
developer, and see what we can do about making our interface happy about
a 4 million+ message folder (see http://jmap.io/ - working on improving
that and making our server actually talk it is next on our list after the
calendaring features are done).
 
Bron.
 
--
  Bron Gondwana
 
_______________________________________________
imapext mailing list
 
_______________________________________________
imapext mailing list
 
_______________________________________________
imapext mailing list
 
--
Bron Gondwana
brong@fastmail.fm
 
--_----------=_1398730722147170-- From nobody Mon Apr 28 17:20:13 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B5AA71A8842 for ; Mon, 28 Apr 2014 17:20:02 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.551 X-Spam-Level: X-Spam-Status: No, score=-2.551 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-0.651] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id js_GbwB2Ji_h for ; Mon, 28 Apr 2014 17:20:00 -0700 (PDT) Received: from orthanc.ca (neo.orthanc.ca [IPv6:2607:f2f8:abf8::2]) by ietfa.amsl.com (Postfix) with ESMTP id 988CC1A6F17 for ; Mon, 28 Apr 2014 17:20:00 -0700 (PDT) Received: from [192.168.42.6] (d66-183-221-35.bchsia.telus.net [66.183.221.35]) (authenticated bits=0) by orthanc.ca (8.14.7/8.14.7) with ESMTP id s3T0JtUx030093 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Mon, 28 Apr 2014 17:19:57 -0700 (PDT) (envelope-from lyndon@orthanc.ca) Content-Type: multipart/signed; boundary="Apple-Mail=_181ED90F-69BE-491B-83DE-274A8D9EE128"; protocol="application/pgp-signature"; micalg=pgp-sha1 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) From: Lyndon Nerenberg In-Reply-To: Date: Mon, 28 Apr 2014 17:19:54 -0700 Message-Id: <1667DEF0-7487-41A3-9F22-0AF5464A3D6E@orthanc.ca> References: <1398686434.32073.111182897.54110B1C@webmail.messagingengine.com> To: Brandon Long X-Mailer: Apple Mail (2.1874) Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/Xyx25M9cIAmVgosnivIy8Ht5_0o Cc: "imapext@ietf.org" Subject: Re: [imapext] a very simple little imap extension, really quite magnificently simple X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Apr 2014 00:20:02 -0000 --Apple-Mail=_181ED90F-69BE-491B-83DE-274A8D9EE128 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii On Apr 28, 2014, at 5:08 PM, Brandon Long wrote: > My work account here at google has 1.6M messages in it. > > And such accounts are just growing in size. I have to ask: do you store all your computer files in a single directory? If not, why not. --lyndon P.S. I'm not asking rhetorically. --Apple-Mail=_181ED90F-69BE-491B-83DE-274A8D9EE128 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- iQIcBAEBAgAGBQJTXvAqAAoJEG8PnXiV/JnUh38P/iARndOMiWAaFgGs6e5RzFXH hlFbNMvnQyTWiCdB7/exjF3qXSOxjiRReRIaQZaGLXy5R6kmzBmDIHPMhglmSx4C Su9UGPNOeqYyKLFwgp8PGcPvAMCb3YDF2ndAOeMO44fYW6JdQiSj+O10gaFwHPXN uxNdIqIqcbBc4fMRMUTFryqN09av5ggZj7kLkBo0l73Dwx3GVc2NSfLL1wHNRC2p sugm2AvNs+HaF0oeepmnMEcLsfx+YbUpZ6U13zO4iHhlWqgCRGsk7utXj+SZV1mS Zz3w4KqO4aKcHSymJOM5eJ+7sIpW37tn32u/kk54BiOMZEUJQeoJzckynFZyaMHA g15wAB+i4t3Q7NfgfFf/Zu3fVVMZTTJXnZKBkAbtoqVZCIxp8Jen/tkLQt0bcFni dSv99S/57FQAEahVrs9wCdvFHazDT219EB+ia5RItvmEo23h2O4s+OaLzUflSgOC YpZzEqVFjEGO7i0dm2ohSfPKMsdEOmLmL3uySKOojCLcGOkesyIc9Lb5El9XvSU4 nQSgNP5AclPeC0YdPZZGjMxRX6ktlRlCQR49J6MD8lI89t6o910clUMvXxTAici1 zzZwHTwTVNs+bX8Tw/4iTxD9UPdjtU2C4deNkNPab8o7tkVpXsZdetX9UKgeDB65 V0TM61qr1wupy+HIp2HX =3W/W -----END PGP SIGNATURE----- --Apple-Mail=_181ED90F-69BE-491B-83DE-274A8D9EE128-- From nobody Mon Apr 28 18:16:12 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5FD7E1A886B for ; Mon, 28 Apr 2014 18:16:08 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.4 X-Spam-Level: X-Spam-Status: No, score=-0.4 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, MANGLED_MEN=2.3, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Jsk35rfnF73A for ; Mon, 28 Apr 2014 18:16:06 -0700 (PDT) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) by ietfa.amsl.com (Postfix) with ESMTP id 7B8941A8869 for ; Mon, 28 Apr 2014 18:16:06 -0700 (PDT) Received: from compute5.internal (compute5.nyi.mail.srv.osa [10.202.2.45]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 623A8215FD for ; Mon, 28 Apr 2014 21:16:05 -0400 (EDT) Received: from web2 ([10.202.2.212]) by compute5.internal (MEProxy); Mon, 28 Apr 2014 21:16:05 -0400 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=fastmail.fm; h= message-id:from:to:mime-version:content-transfer-encoding :content-type:in-reply-to:references:subject:date; s=mesmtp; bh= fbOL1HWY76Mb9X/pBs03VBFyAXo=; b=b6q5s+2h0Hlxl9pFCXxS3OVI+GRAHS62 IpsyBROjUzfH90s9r1stR+KBfNSxq3M5iYlZwR5mZutujn/z5oQjPlZzwlWJ/EP4 TCANxAiHkxWnrIv04jUrY59GW1Hx9VsnmORZn4LqaYGWcTglDQBkfd4cBcgZ0vWS XV+RI8KgjVU= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=message-id:from:to:mime-version :content-transfer-encoding:content-type:in-reply-to:references :subject:date; s=smtpout; bh=fbOL1HWY76Mb9X/pBs03VBFyAXo=; b=AcI WCbzPTTyWTrEyIM8miKGXggG5yv5S7R845EX6Td9QIOo/1/SJTDnhLGMKiruLYEj rAWyDMbFj7ikEjjUZ6QJ4WleoOX6f8BCcGWy+Kt4NxBmwHwYXAGHX0me8LS8hxah CZhMeqIKvrbQYJkc8GVw3w2JNpPDFrWnBsLqJUi0= Received: by web2.nyi.mail.srv.osa (Postfix, from userid 99) id 44015540552; Mon, 28 Apr 2014 21:16:05 -0400 (EDT) Message-Id: <1398734165.5670.111480533.0C776372@webmail.messagingengine.com> X-Sasl-Enc: CXX/Ccex93IBZQk7QnH0xFaN/StF17T/7r0ldAM9m84J 1398734165 From: Bron Gondwana To: imapext@ietf.org MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="ISO-8859-1" X-Mailer: MessagingEngine.com Webmail Interface - ajax-aafac2c3 In-Reply-To: References: <0627dd7a-2203-40b6-b1c0-5b58613cbad8@gulbrandsen.priv.no> <8e45ff5c-1173-4119-bd53-47194d0f056b@flaska.net> <7cb76610-37e3-4903-bdc1-6d2403e27cff@gulbrandsen.priv.no> Date: Tue, 29 Apr 2014 11:16:05 +1000 Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/7kcpTa_5bl_cqnG0R_Zt4cf_64o Subject: Re: [imapext] a very simple little imap extension, really quite magnificently simple X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Apr 2014 01:16:08 -0000 On Tue, Apr 29, 2014, at 02:18 AM, Jan Kundr=E1t wrote: > I wonder whether they don't know because they don't care, or whether it's= =20 > because because their only UI doesn't offer that information, and thus th= ey=20 > become trained not to care about that. GMail's UI doesn't offer a way of= =20 > sorting the messages, yet people still do stuff like sorting by recipient= =20 > or by mail size. "Not supported by GMail" doesn't equal "not useful". Yes indeed :) > >> OK, at this point I cannot imagine how such a client is=20 > >> supposed to work *at all*. Let's start with a blank state. The=20 > >> client knows nothing about the mailbox. It opens it and=20 > >> presumably wants to show a list of messages to the user. What=20 > >> is the client supposed to do now? > > > > Something "uid search unseen", "uid search or unseen within=20 > > 604800", or some variation. > > > > Most clients, of course, do "fetch 1:* ..." instead, which is=20 > > okay, but that approach means that this extension is not=20 > > interesting for them. >=20 > As a (desktop) client developer, I want to show my user what e-mails she= =20 > has in her mailbox(es). The UI framework I use needs to know the amount o= f=20 > items to be shown in a ListView/TreeView, otherwise I have to resort to=20 > ugly bits like resizing the scrollbar handle when she scrolls too close t= o=20 > the current boundary. >=20 > So OK, I have an empty state, know nothing about the number of messages i= n=20 > a mailbox. I just know that my viewport measures 832px, and my code can=20 > calculate that I can fit 40 messages in there (these numbers are made-up)= .=20 > I have no idea what messages are these, so I can make a rough guess that = a=20 > week of messages might approximate what I need to show on the screen, plu= s=20 > some extra room for the scrollbar. So I request them: >=20 > a01 UID SEARCH SINCE ... >=20 > Three possibilities might happen: >=20 > - I got a reasonable number of messages, say, 50 - 500. In this case=20 > everything is good; I can show a list of messages to the user. >=20 > - I got 10k UIDs back because the user opened a LKML archive. Oops. Anywa= y,=20 > at least I don't have to go asking for more UIDs when they scroll back. >=20 > - I got just two UIDs back, perhaps because it's a low traffic mailbox.=20 > This means that I can only show two mails on my big screen. That's wrong = --=20 > just the week ago the mailbox might have had 200 messages delivered in=20 > there. I have to expand the window I sync, which means that the client ha= s=20 > to wait for multiple network round trips. I might save some bandwidth, bu= t=20 > I pay a rather big cost in latency. >=20 > The client will probably want to do some extra preload, if only to know=20 > whether to include some platform-specific UI equivalent of "load more=20 > messages" -- be it either a button, or something which actives upon show,= =20 > or just some scrollbar indicator. This is exactly what we do with our web interface, and it's why we implemen= ted something similar to what's in the JMAP spec: http://www.jmap.io/#getmessagelist Which does offset and limit - so you can actually get a range within a sear= ch/sort rather than only a range by MSN, which isn't very useful if you want your m= essages sorted as well (particularly in a folder which has had messages moved in fr= om various locations, so the sort order isn't consistent with the UID/MSNs) You'll note that our own API also includes a way to say "give me N messages= in a window around this UID within the sorted output" - obviously if you're half= way down the list, you need to materialise it at least enough to calculate the = offset of that named messageid - and if it's missing, that means loading the entire mailbox. That's pretty rare though - only happens if you're paging through= at the same time as you expunge a message. I think this is a much more useful abstraction than MSNs, because MSNs don'= t give you any sorting ability. On the negative side, it means that messages can = both appear and disappear further back in the list - unlike MSNs where new messa= ges only ever appear at the top. On the positive side, we support searches across m= ultiple folders with full sorting and searching and limiting, causing low latency a= nd low bandwidth use to get exactly what the user wants onto their screens. > > "Show today's mail" is trivial, "show all the unread mail" is=20 > > trivial, "show the last five hundred messages" is difficult. >=20 > And that's exactly the point -- taking away EXISTS and MSNs makes it=20 > impossible to page through the result set based on the number of messages= .=20 > The ability of UIs to show stuff to the user depens on the number of item= s=20 > to be shown, not on the time interval covered by the N items, IMHO. The last 500 messages is less interesting if they can only be the last 500 messages appended to this mailbox, regardless of moves between mailboxes, out-of-order delivery, etc. In theory it's mostly OK, but you can miss messages that way if you do something like copy the entire contents of one folder into another in batches and they're out of order. Bron. --=20 Bron Gondwana brong@fastmail.fm From nobody Mon Apr 28 22:04:51 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E04C51A08AD for ; Mon, 28 Apr 2014 22:04:42 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.029 X-Spam-Level: X-Spam-Status: No, score=-2.029 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, RP_MATCHES_RCVD=-0.651, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yXLd95xnhOAY for ; Mon, 28 Apr 2014 22:04:41 -0700 (PDT) Received: from mail-ig0-x22f.google.com (mail-ig0-x22f.google.com [IPv6:2607:f8b0:4001:c05::22f]) by ietfa.amsl.com (Postfix) with ESMTP id 8BF0E1A8836 for ; Mon, 28 Apr 2014 22:04:35 -0700 (PDT) Received: by mail-ig0-f175.google.com with SMTP id h3so5672263igd.14 for ; Mon, 28 Apr 2014 22:04:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=xmji/y8+BhNhmza+YNsKBvGuxmeT2t+QYdliMXBSCfk=; b=H6/52hNolmtxfAKFPwBSBhutPjFJtEv5dVDyvRQkfJ8ZPu8mwrw0wRa1HGtk4T1YHh HU4UNxbFIw4U8B8FYLPIkql8cPWu54TR9o2EBkalMaNaUld9Fv0+QU35Vk9jUKlHMeCq MLlGHVAN8F0Dd9RiiQuM6pO38w7iNIh4uCaIZQviqSp8uZ69gtPQqV7I0b0AhkKgNb3S siuVDbUsR8+k0F71mRMLMOykWlfkcr+eh8ssNx1j33J92TyZF3e2FMM/rL40yM8Qw+rB l3Nsd0IT+1Q4aPxQdUnY05Kg4FTU+Cy6rawGtGSJyqZsVobL62yWGF5RQ1QXdE3SEpAb DL5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=xmji/y8+BhNhmza+YNsKBvGuxmeT2t+QYdliMXBSCfk=; b=Gc75uaoiZKv1QBm7njoK2M4vCUs1gVCulkssLJlDO/ZtIBnhV4UeL3cfhJ5/agJBgx MYD60R/a23FH9tcFswsi3GNXrB/S5fbIIV7aWoZTIkYtsviHjqEixe59wdbhtqlNyc14 LnCjquVPWjRrER8++AcR0oBhdpCGuAVdSo9yETv/TzAqTr2d/gkbzoQwxRvuDNhJ3lnW Vj/6UNp9PDLNnnwIFROKimBG0FtPbY72Vux5fUjYKQwgJzV06kI7Ptqfa6Ykuded39lt wzG0dSrMlGQ6nyTMcP4+1Tr8nVgr9egES3kHGtbq2KKH4yqvvBA3p+sfOhqMLyuXbYEp +pMA== X-Gm-Message-State: ALoCoQm5f4QMplSG12OQysZ3R67D/SmID66wRWNCZIHkeDryMxe403Y4YVClBC7Bhscb4xt65IMzAgcdXSzZRnsvqzCW1LmIM2TpMbXK9Q6zcLtLjcHDBXiU0aQroKThmNwWRmrErBp5hROj7VaBv+ehRyy69gqgOkDtI2DfIuVUTqvNWh1nuuCEiQPV6vf3rf1cgI4IBFgo MIME-Version: 1.0 X-Received: by 10.50.143.34 with SMTP id sb2mr28188665igb.48.1398747874451; Mon, 28 Apr 2014 22:04:34 -0700 (PDT) Received: by 10.64.77.201 with HTTP; Mon, 28 Apr 2014 22:04:34 -0700 (PDT) In-Reply-To: <1667DEF0-7487-41A3-9F22-0AF5464A3D6E@orthanc.ca> References: <1398686434.32073.111182897.54110B1C@webmail.messagingengine.com> <1667DEF0-7487-41A3-9F22-0AF5464A3D6E@orthanc.ca> Date: Mon, 28 Apr 2014 22:04:34 -0700 Message-ID: From: Brandon Long To: Lyndon Nerenberg Content-Type: multipart/alternative; boundary=001a1135effe301b4304f8275ebb Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/Tk5r_aYNWtWokRY9kATChDD-Zt0 Cc: "imapext@ietf.org" Subject: Re: [imapext] a very simple little imap extension, really quite magnificently simple X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Apr 2014 05:04:43 -0000 --001a1135effe301b4304f8275ebb Content-Type: text/plain; charset=UTF-8 No, I don't store all of my files in a single directory, but neither is my mail. I have probably a dozen labels, but one of those labels is "all mail", which all Gmail accounts have, and that's exposed as a folder. And that is becoming more commonly used by some IMAP clients, since it allows for a more "Gmail-like" experience to treat the mailbox as a single bag of messages with different labels (keywords) on the messages. But really, folders are rarely that useful when one has this much mail. If you want a file-system example, the Google codebase is basically a single repository with a deep hierarchical structure.. and I use our search index over that data more often than trying to find something in the structure. I'm unclear what you're really asking, however. Yes, Gmail has several design decisions which mean that we may be hitting these limits earlier than others, and I certainly don't want to force IMAP to change just to support us. But I think that the current MSN design implies an O(n) design for IMAP, and n is just going to keep increasing for everyone. I think that this is a useful discussion for that reason. Do we want to argue whether or not users should have that many messages? That seems like a losing argument, users do what users do. Is it really worth users' effort to decide what to keep from 10 years ago? That seems like a waste of effort on their part, so its not a surprise that they choose not to. There are clearly other options available to us, but I think this is a useful thought exercise to see if it would be possible to just remove the impediment entirely, instead of trying to work around it. Brandon On Mon, Apr 28, 2014 at 5:19 PM, Lyndon Nerenberg wrote: > > On Apr 28, 2014, at 5:08 PM, Brandon Long wrote: > > > My work account here at google has 1.6M messages in it. > > > > And such accounts are just growing in size. > > I have to ask: do you store all your computer files in a single directory? > > If not, why not. > > --lyndon > > P.S. I'm not asking rhetorically. > > --001a1135effe301b4304f8275ebb Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
No, I don't store all of my files in a single director= y, but neither is my mail.

I have probably a dozen label= s, but one of those labels is "all mail", which all Gmail account= s have, and that's exposed as a folder. =C2=A0And that is becoming more= commonly used by some IMAP clients, since it allows for a more "Gmail= -like" experience to treat the mailbox as a single bag of messages wit= h different labels (keywords) on the messages.

But really, folders are rarely that useful when one has= this much mail. =C2=A0If you want a file-system example, the Google codeba= se is basically a single repository with a deep hierarchical structure.. an= d I use our search index over that data more often than trying to find some= thing in the structure.

I'm unclear what you're really asking, however.= =C2=A0Yes, Gmail has several design decisions which mean that we may be hi= tting these limits earlier than others, and I certainly don't want to f= orce IMAP to change just to support us. =C2=A0But I think that the current = MSN design implies an O(n) design for IMAP, and n is just going to keep inc= reasing for everyone.

I think that this is a useful discussion for that reaso= n.

Do we want to argue whether or not users should= have that many messages? =C2=A0That seems like a losing argument, users do= what users do. =C2=A0Is it really worth users' effort to decide what t= o keep from 10 years ago? =C2=A0That seems like a waste of effort on their = part, so its not a surprise that they choose not to.

There are clearly other options available to us, but I = think this is a useful thought exercise to see if it would be possible to j= ust remove the impediment entirely, instead of trying to work around it.

Brandon




On Mon, Apr 28, 201= 4 at 5:19 PM, Lyndon Nerenberg <lyndon@orthanc.ca> wrote:

On Apr 28, 2014, at 5:08 PM, Brandon Long <blong@google.com> wrote:

> My work account here at google has 1.6M messages in it.
>
> And such accounts are just growing in size.

I have to ask: do you store all your computer files in a single direc= tory?

If not, why not.

--lyndon

P.S. =C2=A0I'm not asking rhetorically.


--001a1135effe301b4304f8275ebb-- From nobody Tue Apr 29 00:43:24 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2EECC1A0740 for ; Tue, 29 Apr 2014 00:43:23 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id iDQ_olAgnjtf for ; Tue, 29 Apr 2014 00:43:21 -0700 (PDT) Received: from strange.aox.org (strange.aox.org [IPv6:2001:4d88:100c::1]) by ietfa.amsl.com (Postfix) with ESMTP id B37661A0453 for ; Tue, 29 Apr 2014 00:43:21 -0700 (PDT) Received: from fri.gulbrandsen.priv.no (localhost [127.0.0.1]) by strange.aox.org (Postfix) with ESMTP id 52537FA0078; Tue, 29 Apr 2014 07:43:19 +0000 (UTC) Received: from arnt@gulbrandsen.priv.no by fri.gulbrandsen.priv.no (Archiveopteryx 3.2.0) with esmtpsa id 1398757398-26819-26819/11/5; Tue, 29 Apr 2014 07:43:18 +0000 From: Arnt Gulbrandsen To: imapext@ietf.org Date: Tue, 29 Apr 2014 09:43:17 +0200 User-Agent: Trojita/v0.3.96-176-g713fcbb; Qt/4.8.4; X11; Linux; Ubuntu 13.10 Mime-Version: 1.0 Message-Id: In-Reply-To: References: <0627dd7a-2203-40b6-b1c0-5b58613cbad8@gulbrandsen.priv.no> <8e45ff5c-1173-4119-bd53-47194d0f056b@flaska.net> <7cb76610-37e3-4903-bdc1-6d2403e27cff@gulbrandsen.priv.no> Content-Type: text/plain; charset=utf-8; format=flowed Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/rQT4Lvi0N7gGakbWxr8jsZI6L2E Subject: [imapext] A digression X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Apr 2014 07:43:23 -0000 I may not feel strongly about the number. But I do feel strongly about this mode of decision-making: NN wrote: > Yes, that's the option which I much prefer over using made-up numbers. "I prefer". Not even the shadow of a rationale. Can we please try to use arguments? Arnt From nobody Tue Apr 29 08:00:26 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C59CF1A0935 for ; Tue, 29 Apr 2014 08:00:22 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.151 X-Spam-Level: X-Spam-Status: No, score=-1.151 tagged_above=-999 required=5 tests=[BAYES_05=-0.5, RP_MATCHES_RCVD=-0.651] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id m71_8YUhtlUM for ; Tue, 29 Apr 2014 08:00:20 -0700 (PDT) Received: from daboo.name (daboo.name [173.13.55.49]) by ietfa.amsl.com (Postfix) with ESMTP id 147081A0936 for ; Tue, 29 Apr 2014 08:00:20 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by daboo.name (Postfix) with ESMTP id 0B3D364AAFF6; Tue, 29 Apr 2014 11:00:19 -0400 (EDT) X-Virus-Scanned: amavisd-new at example.com Received: from daboo.name ([127.0.0.1]) by localhost (daboo.name [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wuDbAt_Z19Tk; Tue, 29 Apr 2014 11:00:15 -0400 (EDT) Received: from [17.212.155.73] (unknown [17.212.155.73]) by daboo.name (Postfix) with ESMTPSA id 6B77364AAFDF; Tue, 29 Apr 2014 11:00:15 -0400 (EDT) Date: Tue, 29 Apr 2014 08:00:12 -0700 From: Cyrus Daboo To: Arnt Gulbrandsen , imapext@ietf.org Message-ID: <053B37DA4BB43818D19C907C@cyrus.local> In-Reply-To: <6c80b3a4-68c8-4d52-8069-eeb93b6d55b2@gulbrandsen.priv.no> References: <0627dd7a-2203-40b6-b1c0-5b58613cbad8@gulbrandsen.priv.no> <1398429768.12216.110316901.769BD1B9@webmail.messagingengine.com> <6c80b3a4-68c8-4d52-8069-eeb93b6d55b2@gulbrandsen.priv.no> X-Mailer: Mulberry/4.1.0b1 (Mac OS X) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline; size=806 Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/31e2Dk51SsVpCt_AxaHA2pAkrK0 Subject: Re: [imapext] a very simple little imap extension, really quite magnificently simple X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Apr 2014 15:00:22 -0000 Hi Arnt, --On April 25, 2014 at 7:07:30 PM +0200 Arnt Gulbrandsen wrote: >> It's a pity you can't use 0 as the MSN, but I don't see a problem with > just >> using 1, rather than some random 4-5 digit number. > > I don't see any advantage in 1. The strange number is recognizable; if > you see that foo == 8214 in the debugger you have a fair chance of > remembering that you recently implemented RFC8214 and realise that the > problem you're chasing is related to that. Ditto google for "imap 8214". Since the use of this extension is an explicit agreement between client and server, why not modify the sequence syntax to allow zero for this one specific case (or some other indicator like ".")? Then we can stop arguing about what special number to use. -- Cyrus Daboo From nobody Tue Apr 29 09:04:04 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8FA6E1A0909 for ; Tue, 29 Apr 2014 09:04:01 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.25 X-Spam-Level: X-Spam-Status: No, score=-2.25 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, MIME_8BIT_HEADER=0.3, MIME_QP_LONG_LINE=0.001, RP_MATCHES_RCVD=-0.651] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8z7Z0NaktI7j for ; Tue, 29 Apr 2014 09:03:59 -0700 (PDT) Received: from latimerie.flaska.net (latimerie.flaska.net [IPv6:2a02:2b88:2:1::4a7:333]) by ietfa.amsl.com (Postfix) with ESMTP id 14FFE1A0905 for ; Tue, 29 Apr 2014 09:03:58 -0700 (PDT) Received: by latimerie.flaska.net (Postfix, from userid 1000) id EC58861B81; Tue, 29 Apr 2014 18:03:52 +0200 (CEST) From: =?iso-8859-1?Q?Jan_Kundr=E1t?= To: Date: Tue, 29 Apr 2014 18:03:50 +0200 User-Agent: Trojita/v0.4.1-191-gaeca318; Qt/4.8.5; X11; Linux; MIME-Version: 1.0 Message-ID: In-Reply-To: References: <0627dd7a-2203-40b6-b1c0-5b58613cbad8@gulbrandsen.priv.no> <8e45ff5c-1173-4119-bd53-47194d0f056b@flaska.net> <7cb76610-37e3-4903-bdc1-6d2403e27cff@gulbrandsen.priv.no> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/cFBl5P1GqqJFHDBV_AVJVxr_aMI Subject: Re: [imapext] A digression X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Apr 2014 16:04:01 -0000 On Tuesday, 29 April 2014 09:43:17 CEST, Arnt Gulbrandsen wrote: >> Yes, that's the option which I much prefer over using made-up numbers. > > "I prefer". Not even the shadow of a rationale. > > Can we please try to use arguments? The main argument is clarity, of course. Using a made-up number (or a=20 random one, a magic constant, or whatever which happens to be a number) can=20= be interpretted as an actual number, and will lead to confusion. I cannot=20 explain this better, sorry -- to me, this is completely intuitive. Changing=20= the syntax to use anything but a number prevents accidental=20 misinterpretation. I'm surprised that I'm asked to provide a rationale for=20= that. What is your rationale for using a made-up numeric constant instead of=20 using something like "NOMSN", or even just "* FETCH"? With kind regards, Jan --=20 Trojit=C3=A1, a fast Qt IMAP e-mail client -- http://trojita.flaska.net/ From nobody Tue Apr 29 09:59:19 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E173E1A092C for ; Tue, 29 Apr 2014 09:59:17 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EecDKDsb-_I7 for ; Tue, 29 Apr 2014 09:59:15 -0700 (PDT) Received: from strange.aox.org (strange.aox.org [80.244.248.170]) by ietfa.amsl.com (Postfix) with ESMTP id 825301A0912 for ; Tue, 29 Apr 2014 09:59:15 -0700 (PDT) Received: from fri.gulbrandsen.priv.no (localhost [127.0.0.1]) by strange.aox.org (Postfix) with ESMTP id 60707F8C001; Tue, 29 Apr 2014 16:59:13 +0000 (UTC) Received: from arnt@gulbrandsen.priv.no by fri.gulbrandsen.priv.no (Archiveopteryx 3.2.0) with esmtpsa id 1398790752-26819-26819/11/18; Tue, 29 Apr 2014 16:59:12 +0000 From: Arnt Gulbrandsen To: imapext@ietf.org Date: Tue, 29 Apr 2014 18:59:12 +0200 User-Agent: Trojita/v0.3.96-176-g713fcbb; Qt/4.8.4; X11; Linux; Ubuntu 13.10 Mime-Version: 1.0 Message-Id: <13c1b793-90fb-4fd0-816f-95c4fbae0ec0@gulbrandsen.priv.no> In-Reply-To: <053B37DA4BB43818D19C907C@cyrus.local> References: <0627dd7a-2203-40b6-b1c0-5b58613cbad8@gulbrandsen.priv.no> <1398429768.12216.110316901.769BD1B9@webmail.messagingengine.com> <6c80b3a4-68c8-4d52-8069-eeb93b6d55b2@gulbrandsen.priv.no> <053B37DA4BB43818D19C907C@cyrus.local> Content-Type: text/plain; charset=utf-8; format=flowed Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/r2jjbmEFUxsvVPN5Ibfs8o1dXik Subject: Re: [imapext] a very simple little imap extension, really quite magnificently simple X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Apr 2014 16:59:18 -0000 On Tuesday, April 29, 2014 5:00:12 PM CEST, Cyrus Daboo wrote: > Since the use of this extension is an explicit agreement > between client and server, why not modify the sequence syntax to > allow zero for this one specific case (or some other indicator > like ".")? Then we can stop arguing about what special number to > use. Now that you say it, why have a rule at all? The client is not permitted to care about the number, so telling the server what to send is superfluous. Arnt From nobody Tue Apr 29 10:49:51 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E52DB1A0948 for ; Tue, 29 Apr 2014 10:49:44 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.551 X-Spam-Level: X-Spam-Status: No, score=-2.551 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-0.651] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id iuqOpPVFW6mr for ; Tue, 29 Apr 2014 10:49:40 -0700 (PDT) Received: from daboo.name (daboo.name [173.13.55.49]) by ietfa.amsl.com (Postfix) with ESMTP id C701A1A0947 for ; Tue, 29 Apr 2014 10:49:40 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by daboo.name (Postfix) with ESMTP id 9B0DB64AEE19; Tue, 29 Apr 2014 13:49:39 -0400 (EDT) X-Virus-Scanned: amavisd-new at example.com Received: from daboo.name ([127.0.0.1]) by localhost (daboo.name [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id j6kZZshjmRx2; Tue, 29 Apr 2014 13:49:39 -0400 (EDT) Received: from [17.212.155.73] (unknown [17.212.155.73]) by daboo.name (Postfix) with ESMTPSA id 899E164AEE0C; Tue, 29 Apr 2014 13:49:38 -0400 (EDT) Date: Tue, 29 Apr 2014 10:49:35 -0700 From: Cyrus Daboo To: Arnt Gulbrandsen , imapext@ietf.org Message-ID: <03908659E6C341753EECB4CC@cyrus.local> In-Reply-To: <13c1b793-90fb-4fd0-816f-95c4fbae0ec0@gulbrandsen.priv.no> References: <0627dd7a-2203-40b6-b1c0-5b58613cbad8@gulbrandsen.priv.no> <1398429768.12216.110316901.769BD1B9@webmail.messagingengine.com> <6c80b3a4-68c8-4d52-8069-eeb93b6d55b2@gulbrandsen.priv.no> <053B37DA4BB43818D19C907C@cyrus.local> <13c1b793-90fb-4fd0-816f-95c4fbae0ec0@gulbrandsen.priv.no> X-Mailer: Mulberry/4.1.0b1 (Mac OS X) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline; size=735 Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/p6My_Y6Dc9RVbiJwCAwXxo01bAE Subject: Re: [imapext] a very simple little imap extension, really quite magnificently simple X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Apr 2014 17:49:45 -0000 Hi Arnt, --On April 29, 2014 at 6:59:12 PM +0200 Arnt Gulbrandsen wrote: >> Since the use of this extension is an explicit agreement >> between client and server, why not modify the sequence syntax to >> allow zero for this one specific case (or some other indicator >> like ".")? Then we can stop arguing about what special number to >> use. > > Now that you say it, why have a rule at all? The client is not permitted > to care about the number, so telling the server what to send is > superfluous. Well just from a debugging standpoint where you may only have been given a partial view of what is happening, being able to recognise that this state is present or not would be handy. -- Cyrus Daboo From nobody Tue Apr 29 13:21:22 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C70EE1A098F for ; Tue, 29 Apr 2014 13:21:18 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 068hBgf0K8cT for ; Tue, 29 Apr 2014 13:21:17 -0700 (PDT) Received: from strange.aox.org (strange.aox.org [IPv6:2001:4d88:100c::1]) by ietfa.amsl.com (Postfix) with ESMTP id F152C1A0986 for ; Tue, 29 Apr 2014 13:21:16 -0700 (PDT) Received: from fri.gulbrandsen.priv.no (localhost [127.0.0.1]) by strange.aox.org (Postfix) with ESMTP id 36E3BFA015E; Tue, 29 Apr 2014 20:21:14 +0000 (UTC) Received: from arnt@gulbrandsen.priv.no by fri.gulbrandsen.priv.no (Archiveopteryx 3.2.0) with esmtpsa id 1398802872-26819-26819/11/19; Tue, 29 Apr 2014 20:21:12 +0000 From: Arnt Gulbrandsen To: imapext@ietf.org Date: Tue, 29 Apr 2014 22:21:12 +0200 User-Agent: Trojita/v0.3.96-176-g713fcbb; Qt/4.8.4; X11; Linux; Ubuntu 13.10 Mime-Version: 1.0 Message-Id: In-Reply-To: References: <0627dd7a-2203-40b6-b1c0-5b58613cbad8@gulbrandsen.priv.no> <8e45ff5c-1173-4119-bd53-47194d0f056b@flaska.net> <7cb76610-37e3-4903-bdc1-6d2403e27cff@gulbrandsen.priv.no> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/CAON05cRJ89feGgNuUvKaw2R7nE Subject: Re: [imapext] A digression X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Apr 2014 20:21:19 -0000 On Tuesday, April 29, 2014 6:03:50 PM CEST, Jan Kundr=C3=A1t wrote: > The main argument is clarity, of course. Using a made-up number=20 > (or a random one, a magic constant, or whatever which happens to=20 > be a number) can be interpretted as an actual number, and will=20 > lead to confusion. I cannot explain this better, sorry -- to me,=20 > this is completely intuitive. Changing the syntax to use=20 > anything but a number prevents accidental misinterpretation. I'm=20 > surprised that I'm asked to provide a rationale for that. > > What is your rationale for using a made-up numeric constant=20 > instead of using something like "NOMSN", or even just "* FETCH"? Simplicity. Ease of implementation. Testability. This extension is an optimisation. For this reason, it's reasonable to=20 expect that clients do not change their user interface based on its=20 presence. So if you want to implement this without making a (partly)=20 separate user interface, you parse and ignore MSNs sent by unextended=20 servers. So, since clients then know to parse and ignore, I want to leverage that=20 ability. This means that supporting the extension requires zero changes = to=20 the client parser, and zero extra unit tests. Arnt From nobody Tue Apr 29 14:58:37 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0ABDB1A09A9 for ; Tue, 29 Apr 2014 14:58:36 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.25 X-Spam-Level: X-Spam-Status: No, score=-2.25 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, MIME_8BIT_HEADER=0.3, MIME_QP_LONG_LINE=0.001, RP_MATCHES_RCVD=-0.651] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HdSs_BLLRi95 for ; Tue, 29 Apr 2014 14:58:32 -0700 (PDT) Received: from latimerie.flaska.net (latimerie.flaska.net [IPv6:2a02:2b88:2:1::4a7:333]) by ietfa.amsl.com (Postfix) with ESMTP id AD7111A09DD for ; Tue, 29 Apr 2014 14:58:27 -0700 (PDT) Received: by latimerie.flaska.net (Postfix, from userid 1000) id 3E97A61B81; Tue, 29 Apr 2014 23:58:20 +0200 (CEST) From: =?iso-8859-1?Q?Jan_Kundr=E1t?= To: Date: Tue, 29 Apr 2014 23:58:22 +0200 User-Agent: Trojita/v0.4.1-191-gaeca318; Qt/4.8.5; X11; Linux; MIME-Version: 1.0 Message-ID: <46dce38d-40ae-476b-b1bf-aaea7055668d@flaska.net> In-Reply-To: References: <0627dd7a-2203-40b6-b1c0-5b58613cbad8@gulbrandsen.priv.no> <8e45ff5c-1173-4119-bd53-47194d0f056b@flaska.net> <7cb76610-37e3-4903-bdc1-6d2403e27cff@gulbrandsen.priv.no> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/Xe6jl_4vo6c2Qdf0xhrlluDeqzA Subject: Re: [imapext] A digression X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Apr 2014 21:58:36 -0000 On Tuesday, 29 April 2014 22:21:12 CEST, Arnt Gulbrandsen wrote: > This extension is an optimisation. For this reason, it's=20 > reasonable to expect that clients do not change their user=20 > interface based on its presence. So if you want to implement=20 > this without making a (partly) separate user interface, you=20 > parse and ignore MSNs sent by unextended servers. Wait -- what has the user interface to do with the way how MSNs are parsed=20= or handled? There is plenty of MUAs which can show POP3, IMAP4 and local=20 mailbox/maildir/... accounts using the same UI. Surely a change not to use=20= MSNs does not require any change to the UI. > So, since clients then know to parse and ignore, I want to=20 > leverage that ability. This means that supporting the extension=20 > requires zero changes to the client parser, and zero extra unit=20 > tests. You probably mean zero additional unit tests for the parser, but definitely=20= some additional unit tests for that part of code which speaks IMAP and=20 models the remote server's state. That's true. On the other hand, you now have a protocol which contains superfluous=20 tokens with no useful value; in fact, these tokens look like something=20 which is well known to a client developer, yet actually forbidden to be=20 used in this situation. I consider this to be a very confusing feature of a=20= protocol. Based on the reactions on this ML, anybody who commented except=20 you agrees with that point of view. > Simplicity. I'll grant you that if you refer to the fact that the IMAP parser doesn't=20 have to be changed. That's true. It's also true that the amount of changes=20= needed in the other parts of a client is at least one order of magnitude=20 higher. > Ease of implementation. No matter which of these two responses formats were used, both would=20 require roughly the same amount of work in my client. I'm willing to believe that there's at least one client out there where=20 your proposal would be simpler to implement. However, having made-up data=20 in a response is such an antipattern in my opinion that it completely=20 trumps this possible benefit. > Testability. The above is why I pretty violently disagree with "testability" being one=20 of the reasons in favor of retaining these fake MSNs. For the reasons=20 stated above, I actually consider the inclusion of fake MSNs to actively=20 harm testability. You have the same responses on the wire which now require=20= completely different processing. Something which previously mandated a=20 lookup based on the MSN must now be handled by the now-made-mandatory UID,=20= and the MSN must be suddenly ignored. In Trojita, this would mean a change=20= in the way FETCH responses are handled (after they are already parsed), and=20= I surely would add multiple test cases for that. With kind regards, Jan --=20 Trojit=C3=A1, a fast Qt IMAP e-mail client -- http://trojita.flaska.net/ From nobody Tue Apr 29 17:17:58 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 133C21A0989 for ; Tue, 29 Apr 2014 17:17:57 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.552 X-Spam-Level: X-Spam-Status: No, score=-2.552 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-0.651, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4iE35deAFery for ; Tue, 29 Apr 2014 17:17:55 -0700 (PDT) Received: from smtp.qbik.com (smtp.qbik.com [210.55.214.35]) by ietfa.amsl.com (Postfix) with ESMTP id 24C831A0290 for ; Tue, 29 Apr 2014 17:17:55 -0700 (PDT) Received: From [192.168.0.38] (unverified [192.168.0.38]) by SMTP Server [192.168.0.1] (WinGate SMTP Receiver v8.2.1 (Build 4711)) with SMTP id <0020011993@smtp.qbik.com>; Wed, 30 Apr 2014 12:17:50 +1200 From: "Adrien de Croy" To: "Arnt Gulbrandsen" , "imapext@ietf.org" Date: Wed, 30 Apr 2014 00:17:50 +0000 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; format=flowed; charset=utf-8 In-Reply-To: <13c1b793-90fb-4fd0-816f-95c4fbae0ec0@gulbrandsen.priv.no> Message-Id: Mime-Version: 1.0 User-Agent: eM_Client/6.0.20154.0 Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/i1WhezgZZUMhxv2hOKB-cgrRCYU Subject: Re: [imapext] a very simple little imap extension, really quite magnificently simple X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Adrien de Croy List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Apr 2014 00:17:57 -0000 why even send the number then if it must be discarded? Adrien ------ Original Message ------ From: "Arnt Gulbrandsen" To: "imapext@ietf.org" Sent: 30/04/2014 4:59:12 a.m. Subject: Re: [imapext] a very simple little imap extension, really quite= =20 magnificently simple >On Tuesday, April 29, 2014 5:00:12 PM CEST, Cyrus Daboo wrote: >>Since the use of this extension is an explicit agreement between=20 >>client and server, why not modify the sequence syntax to allow zero=20 >>for this one specific case (or some other indicator like ".")? Then we= =20 >>can stop arguing about what special number to use. > >Now that you say it, why have a rule at all? The client is not=20 >permitted to care about the number, so telling the server what to send=20 >is superfluous. > >Arnt > >_______________________________________________ >imapext mailing list >imapext@ietf.org >https://www.ietf.org/mailman/listinfo/imapext From nobody Tue Apr 29 21:48:51 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BD4281A0A7F for ; Tue, 29 Apr 2014 21:48:43 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.9 X-Spam-Level: X-Spam-Status: No, score=-0.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, J_BACKHAIR_33=1] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pGqGyIMta1Lr for ; Tue, 29 Apr 2014 21:48:42 -0700 (PDT) Received: from qmta13.emeryville.ca.mail.comcast.net (qmta13.emeryville.ca.mail.comcast.net [IPv6:2001:558:fe2d:44:76:96:27:243]) by ietfa.amsl.com (Postfix) with ESMTP id C38F91A08A8 for ; Tue, 29 Apr 2014 21:48:42 -0700 (PDT) Received: from omta24.emeryville.ca.mail.comcast.net ([76.96.30.92]) by qmta13.emeryville.ca.mail.comcast.net with comcast id w4nV1n0061zF43QAD4ohwp; Wed, 30 Apr 2014 04:48:41 +0000 Received: from bigworm.curecanti.org ([67.176.83.228]) by omta24.emeryville.ca.mail.comcast.net with comcast id w4of1n00E4vagWz8k4ogho; Wed, 30 Apr 2014 04:48:40 +0000 Received: from bigworm.curecanti.org (localhost.localdomain [IPv6:::1]) (Authenticated sender: slusarz) by bigworm.curecanti.org (Postfix) with ESMTPSA id C5BF96806A for ; Tue, 29 Apr 2014 22:48:38 -0600 (MDT) Received: from dangerzone.curecanti.org (dangerzone.curecanti.org [172.18.5.4]) by bigworm.curecanti.org (Horde Framework) with HTTP; Tue, 29 Apr 2014 22:48:38 -0600 Date: Tue, 29 Apr 2014 22:48:38 -0600 Message-ID: <20140429224838.Horde.OsVkA_jsCMci0JRqRAxEiw5@bigworm.curecanti.org> From: Michael M Slusarz To: imapext@ietf.org References: <0627dd7a-2203-40b6-b1c0-5b58613cbad8@gulbrandsen.priv.no> <8e45ff5c-1173-4119-bd53-47194d0f056b@flaska.net> <7cb76610-37e3-4903-bdc1-6d2403e27cff@gulbrandsen.priv.no> In-Reply-To: User-Agent: Internet Messaging Program (IMP) H5 (6.2.0-git) Accept-Language: en X-Originating-IP: 172.18.5.4 X-Remote-Browser: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:28.0) Gecko/20100101 Firefox/28.0 Content-Type: text/plain; charset=UTF-8; format=flowed; DelSp=Yes MIME-Version: 1.0 Content-Disposition: inline Content-Transfer-Encoding: 8bit DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=comcast.net; s=q20140121; t=1398833321; bh=LX8WkWqaFVyNXRI28qNgNQE9j36uR2362zXWLTPrHfU=; h=Received:Received:Received:Received:Date:Message-ID:From:To: Subject:Content-Type:MIME-Version; b=Z0hB0xKLGHe+qHtsFbjIu75QttE40TneK40u0XHwnVy+lda7C0CXyquYOtQpuypS2 s9X4Qps3qR634hCifMxaPuTzgPizPdeAaPmtMauQLVIFEoYNPP9HDmj2xIg6PpjrEw ntm4BJL3DhrQLL3UdYqaCQbbSXO3KW/JbrZkPgamviWhfRovLrSnSAl9u7a5gPZrDe gUDk7pCzXa6S+mvjNELGAOZ5ZjhUrem9rR54aY09kRD/RP43yPDa8ACrxzQGwLznEp KO/bY6pa1Yb4MOvIvWpwJ1BO+Bpj0rQcdsz3Ef1OnjR+RzWKVPx/0O80ifSCV1ClRT bw2PEz24aqOuA== Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/LlWXA13yqELqhrB35DELB9Yo-bg Subject: Re: [imapext] A digression X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Apr 2014 04:48:43 -0000 Quoting Arnt Gulbrandsen : > On Tuesday, April 29, 2014 6:03:50 PM CEST, Jan Kundrát wrote: >> The main argument is clarity, of course. Using a made-up number (or >> a random one, a magic constant, or whatever which happens to be a >> number) can be interpretted as an actual number, and will lead to >> confusion. I cannot explain this better, sorry -- to me, this is >> completely intuitive. Changing the syntax to use anything but a >> number prevents accidental misinterpretation. I'm surprised that >> I'm asked to provide a rationale for that. >> >> What is your rationale for using a made-up numeric constant instead >> of using something like "NOMSN", or even just "* FETCH"? > > Simplicity. Ease of implementation. Testability. > > This extension is an optimisation. For this reason, it's reasonable > to expect that clients do not change their user interface based on > its presence. So if you want to implement this without making a > (partly) separate user interface, you parse and ignore MSNs sent by > unextended servers. > > So, since clients then know to parse and ignore, I want to leverage > that ability. This means that supporting the extension requires zero > changes to the client parser, and zero extra unit tests. I agree with this. Especially since, as written, the RFC is focused on server optimization not client optimization. A client that is optimized to not use MSNs is already configured to ignore these values anyway, so you're not asking anything of client authors. I don't see the need for yet another FETCH response, when the desire is not to add to the return some piece of data that was available before but instead to mask what already exists. That's akin to creating an extension that identifies which ENVELOPE fields to return. Keeps documentation nice and tidy also, with a single trivial ABNF change (adding the extension name). '1' should be used instead of "RFC Value" for the placeholder. (Would be nice to use something like 0, but that would require modification of ABNF which, as I stated above, doesn't seem useful.) I see no discernable difference in ease of debugging for something like this: * 1 FETCH (UID 100 [...]) * 1 FETCH (UID 101 [...]) * 1 FETCH (UID 102 [...]) versus: * 8000 FETCH (UID 100 [...]) * 8000 FETCH (UID 101 [...]) * 8000 FETCH (UID 102 [...]) I would assume a developer isn't going to necessarily remember that they implemented "RFC 8000" 6 months after the fact, so seeing "8000" is much less important than the fact they are seeing duplicate MSN<->UID mappings. Might as well use a single-digit number to reduce network size as much as possible. Depending on buffering and # of messages returned, might save a packet or two of data. '1' is as arbitrary as any number. michael ___________________________________ Michael Slusarz [slusarz@horde.org] From nobody Tue Apr 29 22:01:12 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 45CE61A2130 for ; Tue, 29 Apr 2014 22:01:09 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.8 X-Spam-Level: X-Spam-Status: No, score=0.8 tagged_above=-999 required=5 tests=[BAYES_50=0.8, DKIM_SIGNED=0.1, DKIM_VALID=-0.1] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TP8Gv3_u84va for ; Tue, 29 Apr 2014 22:01:07 -0700 (PDT) Received: from qmta13.emeryville.ca.mail.comcast.net (qmta13.emeryville.ca.mail.comcast.net [IPv6:2001:558:fe2d:44:76:96:27:243]) by ietfa.amsl.com (Postfix) with ESMTP id E0B861A212D for ; Tue, 29 Apr 2014 22:01:07 -0700 (PDT) Received: from omta12.emeryville.ca.mail.comcast.net ([76.96.30.44]) by qmta13.emeryville.ca.mail.comcast.net with comcast id w50p1n0030x6nqcAD516hB; Wed, 30 Apr 2014 05:01:06 +0000 Received: from bigworm.curecanti.org ([67.176.83.228]) by omta12.emeryville.ca.mail.comcast.net with comcast id w5151n00B4vagWz8Y515Dt; Wed, 30 Apr 2014 05:01:06 +0000 Received: from bigworm.curecanti.org (localhost.localdomain [IPv6:::1]) (Authenticated sender: slusarz) by bigworm.curecanti.org (Postfix) with ESMTPSA id BFE736806A for ; Tue, 29 Apr 2014 23:00:34 -0600 (MDT) Received: from dangerzone.curecanti.org (dangerzone.curecanti.org [172.18.5.4]) by bigworm.curecanti.org (Horde Framework) with HTTP; Tue, 29 Apr 2014 23:00:34 -0600 Date: Tue, 29 Apr 2014 23:00:34 -0600 Message-ID: <20140429230034.Horde.vakEpWqsZ_jn07xtCtsnUg2@bigworm.curecanti.org> From: Michael M Slusarz To: imapext@ietf.org References: <1398686434.32073.111182897.54110B1C@webmail.messagingengine.com> <1667DEF0-7487-41A3-9F22-0AF5464A3D6E@orthanc.ca> In-Reply-To: User-Agent: Internet Messaging Program (IMP) H5 (6.2.0-git) Accept-Language: en X-Originating-IP: 172.18.5.4 X-Remote-Browser: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:28.0) Gecko/20100101 Firefox/28.0 Content-Type: text/plain; charset=UTF-8; format=flowed; DelSp=Yes MIME-Version: 1.0 Content-Disposition: inline DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=comcast.net; s=q20140121; t=1398834066; bh=Z4DZ6yaYHCYZsI2chbrQXnWpOHo8OwF72p0FvIckHk8=; h=Received:Received:Received:Received:Date:Message-ID:From:To: Subject:Content-Type:MIME-Version; b=fUG44iSxMIRSDo/feEpBZ9XDjgHbVo1b4IGbla9d0IMVyb3jI8GWMuTxOwonJJkt1 NJ9+9qO4qG74eIUZin/xlehpRKQmDjvbF7cGfsQovpM9VySGIzlDYauJvV9icG3fh5 pQKD3+1KNthCViDOU4On0mGW4Qe9Z2jS7P5EgaN4AlcEp5fjMK8DhT8Wr3dLVhBFao gpUZiAAO7Ho0Z1s4DhtqP0On2QTSKray3bbH/5wlDLCX4UW8mX9+tw41/Ij7rO5N4A zxjeICR/rE8TfXC+LP+SI82tJZb9W5lCpX+eR+ozTmZh8Cr9jO8zt2Yo8+lPK5Kzlf apo5rBLlkgxnQ== Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/dURHA0VSQtPklSAqmqOtlbtXIDs Subject: Re: [imapext] a very simple little imap extension, really quite magnificently simple X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Apr 2014 05:01:09 -0000 Quoting Brandon Long : > Do we want to argue whether or not users should have that many messages? > That seems like a losing argument, users do what users do. Is it really > worth users' effort to decide what to keep from 10 years ago? That seems > like a waste of effort on their part, so its not a surprise that they > choose not to. I'm not even seeing why the # of messages in a mailbox is terribly relevant for this discussion. If you have 10,000 users with 100 messages each, I would guess you are still going to get a slight performance advantage from indicating to the server you don't need MSNs. So it's a huge win for giant mailboxes and a small win for smaller mailboxes. Yay! Worst case scenario: if a server thinks it is more trouble than its worth, it can continue to send MSNs as it currently does and the client will continue to ignore those MSNs as it currently does and we are at the status quo, with the exception of a single 'ENABLE NOUID' command. Maybe that answers the "what MSN to return" question - a server should be able to send whatever it wants as a MSN. Maybe it is too difficult in a server's code to change the MSN to a certain defined value without major refactoring, so it should feel free to send whatever it wants in the interim although recommended goal is to send the smallest payload possible. michael ___________________________________ Michael Slusarz [slusarz@horde.org] From nobody Tue Apr 29 22:04:13 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 65CDF1A2130 for ; Tue, 29 Apr 2014 22:04:07 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Kw-no6dYSPv7 for ; Tue, 29 Apr 2014 22:04:05 -0700 (PDT) Received: from qmta04.emeryville.ca.mail.comcast.net (qmta04.emeryville.ca.mail.comcast.net [IPv6:2001:558:fe2d:43:76:96:30:40]) by ietfa.amsl.com (Postfix) with ESMTP id 962A81A212D for ; Tue, 29 Apr 2014 22:04:05 -0700 (PDT) Received: from omta11.emeryville.ca.mail.comcast.net ([76.96.30.36]) by qmta04.emeryville.ca.mail.comcast.net with comcast id w4yN1n0020mlR8UA4544CG; Wed, 30 Apr 2014 05:04:04 +0000 Received: from bigworm.curecanti.org ([67.176.83.228]) by omta11.emeryville.ca.mail.comcast.net with comcast id w5431n00B4vagWz8X543FK; Wed, 30 Apr 2014 05:04:04 +0000 Received: from bigworm.curecanti.org (localhost.localdomain [IPv6:::1]) (Authenticated sender: slusarz) by bigworm.curecanti.org (Postfix) with ESMTPSA id EFBEB6806A for ; Tue, 29 Apr 2014 23:03:32 -0600 (MDT) Received: from dangerzone.curecanti.org (dangerzone.curecanti.org [172.18.5.4]) by bigworm.curecanti.org (Horde Framework) with HTTP; Tue, 29 Apr 2014 23:03:32 -0600 Date: Tue, 29 Apr 2014 23:03:32 -0600 Message-ID: <20140429230332.Horde.Xq5XAsNjP9lH3thrUxmo-Q1@bigworm.curecanti.org> From: Michael M Slusarz To: imapext@ietf.org References: <1398686434.32073.111182897.54110B1C@webmail.messagingengine.com> <1667DEF0-7487-41A3-9F22-0AF5464A3D6E@orthanc.ca> <20140429230034.Horde.vakEpWqsZ_jn07xtCtsnUg2@bigworm.curecanti.org> In-Reply-To: <20140429230034.Horde.vakEpWqsZ_jn07xtCtsnUg2@bigworm.curecanti.org> User-Agent: Internet Messaging Program (IMP) H5 (6.2.0-git) Accept-Language: en X-Originating-IP: 172.18.5.4 X-Remote-Browser: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:28.0) Gecko/20100101 Firefox/28.0 Content-Type: text/plain; charset=UTF-8; format=flowed; DelSp=Yes MIME-Version: 1.0 Content-Disposition: inline DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=comcast.net; s=q20140121; t=1398834244; bh=0bNhuEIPxFwYPrCzGkpGJ/2F5OIiBZMExHvxkGo6Ubw=; h=Received:Received:Received:Received:Date:Message-ID:From:To: Subject:Content-Type:MIME-Version; b=hNTD4I15Eb1qiGGA02s/a044Adr4B32G1gHFLcxcIqr1yNTbG9yYp7I8S0QJ3x/2d t2812KtlrO09s1l/M1dAHdxLgvdB8bZUIT1HlyGIutM/cZfyjndKjP0ZdsU9dT9TR3 Jy2JVr/6s4oPekiku/PDX2E4b8TmQ6JGicRih4xJ/DFSFQXr7HDP0M0MEIwzsmIUeF CL1drm5P+IuV/LgvQZLPh+R7A5ODUDRfG9aZYsZKMAqgCXEhDNyRpjAGKS1Ty0JGan C5+UoUWCuZb/zpiaV9iuzVhFsRcJnT/e1uKBKuVf0diW2ieqWDYincRPyqpsGg+PT2 lN+lLt/0Z8CWg== Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/Ctchjwe-7M1_qRZMgFZWoYdMjGk Subject: Re: [imapext] a very simple little imap extension, really quite magnificently simple X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Apr 2014 05:04:07 -0000 Quoting Michael M Slusarz : > Worst case scenario: if a server thinks it is more trouble than its > worth, it can continue to send MSNs as it currently does and the > client will continue to ignore those MSNs as it currently does and > we are at the status quo, with the exception of a single 'ENABLE > NOUID' command. "ENABLE NOUID" should obviously be "ENABLE NOMSN". michael ___________________________________ Michael Slusarz [slusarz@horde.org] From nobody Wed Apr 30 00:25:35 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B10CD1A6EFB for ; Wed, 30 Apr 2014 00:25:29 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5zHMzA2266ZJ for ; Wed, 30 Apr 2014 00:25:26 -0700 (PDT) Received: from strange.aox.org (strange.aox.org [IPv6:2001:4d88:100c::1]) by ietfa.amsl.com (Postfix) with ESMTP id 10F481A6EF9 for ; Wed, 30 Apr 2014 00:25:25 -0700 (PDT) Received: from fri.gulbrandsen.priv.no (localhost [127.0.0.1]) by strange.aox.org (Postfix) with ESMTP id 5518AFA00CC; Wed, 30 Apr 2014 07:25:23 +0000 (UTC) Received: from arnt@gulbrandsen.priv.no by fri.gulbrandsen.priv.no (Archiveopteryx 3.2.0) with esmtpsa id 1398842722-26819-26819/11/21; Wed, 30 Apr 2014 07:25:22 +0000 From: Arnt Gulbrandsen To: imapext@ietf.org Date: Wed, 30 Apr 2014 09:25:21 +0200 User-Agent: Trojita/v0.3.96-176-g713fcbb; Qt/4.8.4; X11; Linux; Ubuntu 13.10 Mime-Version: 1.0 Message-Id: <999f2794-3fcf-4973-9619-cd69a9f5c193@gulbrandsen.priv.no> In-Reply-To: <20140429230034.Horde.vakEpWqsZ_jn07xtCtsnUg2@bigworm.curecanti.org> References: <1398686434.32073.111182897.54110B1C@webmail.messagingengine.com> <1667DEF0-7487-41A3-9F22-0AF5464A3D6E@orthanc.ca> <20140429230034.Horde.vakEpWqsZ_jn07xtCtsnUg2@bigworm.curecanti.org> Content-Type: text/plain; charset=utf-8; format=flowed Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/-Ei0GQrMzQUtSnRy3gKKI3EA4NI Subject: Re: [imapext] a very simple little imap extension, really quite magnificently simple X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Apr 2014 07:25:29 -0000 On Wednesday, April 30, 2014 7:00:34 AM CEST, Michael M Slusarz wrote: > Maybe that answers the "what MSN to return" question - a server > should be able to send whatever it wants as a MSN. I changed to this, for a different reason. Suppose the document said "the server MUST send blah" and a server actually sends something different. Then nothing bad happens. Therefore, the MUST is pointless, so I deleted it. Arnt From nobody Wed Apr 30 00:33:36 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9DE7A1A6EED for ; Wed, 30 Apr 2014 00:33:35 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Dlal08dGeRv1 for ; Wed, 30 Apr 2014 00:33:32 -0700 (PDT) Received: from strange.aox.org (strange.aox.org [80.244.248.170]) by ietfa.amsl.com (Postfix) with ESMTP id 6B7591A00D7 for ; Wed, 30 Apr 2014 00:33:32 -0700 (PDT) Received: from fri.gulbrandsen.priv.no (localhost [127.0.0.1]) by strange.aox.org (Postfix) with ESMTP id 4929CFA015E; Wed, 30 Apr 2014 07:33:30 +0000 (UTC) Received: from arnt@gulbrandsen.priv.no by fri.gulbrandsen.priv.no (Archiveopteryx 3.2.0) with esmtpsa id 1398843209-26819-26819/11/22; Wed, 30 Apr 2014 07:33:29 +0000 From: Arnt Gulbrandsen To: Adrien de Croy Date: Wed, 30 Apr 2014 09:33:28 +0200 User-Agent: Trojita/v0.3.96-176-g713fcbb; Qt/4.8.4; X11; Linux; Ubuntu 13.10 Mime-Version: 1.0 Message-Id: <4840c22b-75ac-40e9-b55b-d14e9de36837@gulbrandsen.priv.no> In-Reply-To: References: <0627dd7a-2203-40b6-b1c0-5b58613cbad8@gulbrandsen.priv.no> <1398429768.12216.110316901.769BD1B9@webmail.messagingengine.com> <6c80b3a4-68c8-4d52-8069-eeb93b6d55b2@gulbrandsen.priv.no> <053B37DA4BB43818D19C907C@cyrus.local> <13c1b793-90fb-4fd0-816f-95c4fbae0ec0@gulbrandsen.priv.no> Content-Type: text/plain; charset=utf-8; format=flowed Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/XXNALfPnGhyc57sIsQUvBWsPnMo Cc: imapext@ietf.org Subject: Re: [imapext] a very simple little imap extension, really quite magnificently simple X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Apr 2014 07:33:35 -0000 On Wednesday, April 30, 2014 2:17:50 AM CEST, Adrien de Croy wrote: > why even send the number then if it must be discarded? I answered this in the draft itself and twice so far on the list, but here goes again: An implementing client has to parse-and-ignore if it wants to work with unextended servers and present the same UI independent of whether the server offers UIDONLY. Therefore, if extended servers send nonsense numbers, the client needs do _nothing_ conditionally on whether the server supports UIDONLY. The parser layer requires no change, there isn't a single if(serverHas("uidonly")) anywhere, and this extension requires no extra unit tests. Arnt From nobody Wed Apr 30 02:34:27 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D50F31A6F1F for ; Wed, 30 Apr 2014 02:34:16 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.552 X-Spam-Level: X-Spam-Status: No, score=-2.552 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-0.651, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aH3nASPKaPby for ; Wed, 30 Apr 2014 02:34:08 -0700 (PDT) Received: from smtp.qbik.com (smtp.qbik.com [210.55.214.35]) by ietfa.amsl.com (Postfix) with ESMTP id E09AE1A6EF8 for ; Wed, 30 Apr 2014 02:34:07 -0700 (PDT) Received: From [192.168.0.38] (unverified [192.168.0.38]) by SMTP Server [192.168.0.1] (WinGate SMTP Receiver v8.2.1 (Build 4711)) with SMTP id <0020012408@smtp.qbik.com>; Wed, 30 Apr 2014 21:34:04 +1200 From: "Adrien de Croy" To: "Arnt Gulbrandsen" Date: Wed, 30 Apr 2014 09:34:04 +0000 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; format=flowed; charset=utf-8 In-Reply-To: <4840c22b-75ac-40e9-b55b-d14e9de36837@gulbrandsen.priv.no> Message-Id: Mime-Version: 1.0 User-Agent: eM_Client/6.0.20154.0 Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/JG43gYpwAStYy5fQEOZmKy4maBQ Cc: "imapext@ietf.org" Subject: Re: [imapext] a very simple little imap extension, really quite magnificently simple X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Adrien de Croy List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Apr 2014 09:34:17 -0000 I don't think that's a good reason. If we adopted that for all extensions where would we end up? Isn't it=20 just laziness on the part of the coder? We leave the MSN in the=20 messages and it becomes garbage on the wire. Since when do we start=20 deliberately sending garbage as part of protocol? There are plenty of extensions which modify parsing. Imagine if we'd=20 taken this approach? Half the bits on the wire might be garbage. Or=20 maybe we never really wanted to strip something out before so this is=20 the first time for this. ------ Original Message ------ From: "Arnt Gulbrandsen" To: "Adrien de Croy" Cc: "imapext@ietf.org" Sent: 30/04/2014 7:33:28 p.m. Subject: Re: [imapext] a very simple little imap extension, really quite= =20 magnificently simple >On Wednesday, April 30, 2014 2:17:50 AM CEST, Adrien de Croy wrote: >>why even send the number then if it must be discarded? > >I answered this in the draft itself and twice so far on the list, but=20 >here goes again: > >An implementing client has to parse-and-ignore if it wants to work with= =20 >unextended servers and present the same UI independent of whether the=20 >server offers UIDONLY. Therefore, if extended servers send nonsense=20 >numbers, the client needs do _nothing_ conditionally on whether the=20 >server supports UIDONLY. The parser layer requires no change, there=20 >isn't a single if(serverHas("uidonly")) anywhere, and this extension=20 >requires no extra unit tests. > >Arnt > From nobody Wed Apr 30 02:48:28 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DEE3E1A6F33 for ; Wed, 30 Apr 2014 02:48:19 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qqVbynCDH_K1 for ; Wed, 30 Apr 2014 02:48:14 -0700 (PDT) Received: from strange.aox.org (strange.aox.org [80.244.248.170]) by ietfa.amsl.com (Postfix) with ESMTP id 54E941A0729 for ; Wed, 30 Apr 2014 02:48:14 -0700 (PDT) Received: from fri.gulbrandsen.priv.no (localhost [127.0.0.1]) by strange.aox.org (Postfix) with ESMTP id 2C631FA015E; Wed, 30 Apr 2014 09:48:12 +0000 (UTC) Received: from arnt@gulbrandsen.priv.no by fri.gulbrandsen.priv.no (Archiveopteryx 3.2.0) with esmtpsa id 1398851290-26819-26819/11/25; Wed, 30 Apr 2014 09:48:10 +0000 From: Arnt Gulbrandsen To: Adrien de Croy Date: Wed, 30 Apr 2014 11:48:10 +0200 User-Agent: Trojita/v0.3.96-176-g713fcbb; Qt/4.8.4; X11; Linux; Ubuntu 13.10 Mime-Version: 1.0 Message-Id: <4a2d2d29-e9fc-445e-9b3f-ffbb28771517@gulbrandsen.priv.no> In-Reply-To: References: <0627dd7a-2203-40b6-b1c0-5b58613cbad8@gulbrandsen.priv.no> <1398429768.12216.110316901.769BD1B9@webmail.messagingengine.com> <6c80b3a4-68c8-4d52-8069-eeb93b6d55b2@gulbrandsen.priv.no> <053B37DA4BB43818D19C907C@cyrus.local> <13c1b793-90fb-4fd0-816f-95c4fbae0ec0@gulbrandsen.priv.no> <4840c22b-75ac-40e9-b55b-d14e9de36837@gulbrandsen.priv.no> Content-Type: text/plain; charset=utf-8; format=flowed Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/DjceBk1O5bRWeEynrWgTarek5mc Cc: imapext@ietf.org Subject: Re: [imapext] a very simple little imap extension, really quite magnificently simple X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Apr 2014 09:48:20 -0000 On Wednesday, April 30, 2014 11:34:04 AM CEST, Adrien de Croy wrote: > I don't think that's a good reason. > > If we adopted that for all extensions where would we end up? > Isn't it just laziness on the part of the coder? We leave the > MSN in the messages and it becomes garbage on the wire. Since > when do we start deliberately sending garbage as part of > protocol? Decades ago? Most IMAP servers send a capability list before the one that matters when the client uses starttls. Many email messages include a timezone name in the date header field as well as the numeric timezone, decades after the change to numbers. Many IMAP clients send STATUS commands so often that most of them return the exact same information as before. And so on. There are lots of little examples of waste. Arnt From nobody Wed Apr 30 03:26:30 2014 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 493D31A0714 for ; Wed, 30 Apr 2014 03:26:19 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.25 X-Spam-Level: X-Spam-Status: No, score=-2.25 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, MIME_8BIT_HEADER=0.3, MIME_QP_LONG_LINE=0.001, RP_MATCHES_RCVD=-0.651] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qsvyIeKB88Vb for ; Wed, 30 Apr 2014 03:26:13 -0700 (PDT) Received: from latimerie.flaska.net (latimerie.flaska.net [IPv6:2a02:2b88:2:1::4a7:333]) by ietfa.amsl.com (Postfix) with ESMTP id 545DB1A0118 for ; Wed, 30 Apr 2014 03:26:13 -0700 (PDT) Received: by latimerie.flaska.net (Postfix, from userid 1000) id 5E4C0619FA; Wed, 30 Apr 2014 12:26:06 +0200 (CEST) From: =?iso-8859-1?Q?Jan_Kundr=E1t?= To: Date: Wed, 30 Apr 2014 12:26:04 +0200 User-Agent: Trojita/v0.4.1-191-gaeca318; Qt/4.8.5; X11; Linux; MIME-Version: 1.0 Message-ID: In-Reply-To: <4a2d2d29-e9fc-445e-9b3f-ffbb28771517@gulbrandsen.priv.no> References: <0627dd7a-2203-40b6-b1c0-5b58613cbad8@gulbrandsen.priv.no> <1398429768.12216.110316901.769BD1B9@webmail.messagingengine.com> <6c80b3a4-68c8-4d52-8069-eeb93b6d55b2@gulbrandsen.priv.no> <053B37DA4BB43818D19C907C@cyrus.local> <13c1b793-90fb-4fd0-816f-95c4fbae0ec0@gulbrandsen.priv.no> <4840c22b-75ac-40e9-b55b-d14e9de36837@gulbrandsen.priv.no> <4a2d2d29-e9fc-445e-9b3f-ffbb28771517@gulbrandsen.priv.no> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable Archived-At: http://mailarchive.ietf.org/arch/msg/imapext/6eTAC3wfTXDF7M3CFaqSf6IiCH4 Subject: Re: [imapext] =?iso-8859-1?q?a_very_simple_little_imap_extension=2C_r?= =?iso-8859-1?q?eally_quite_magnificently_simple?= X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Apr 2014 10:26:19 -0000 On Wednesday, 30 April 2014 11:48:10 CEST, Arnt Gulbrandsen wrote: > Most IMAP servers send a capability list before the one that=20 > matters when the client uses starttls. The reason for that is that the server might not know whether the client is=20= going to issue STARTTLS. In this case, it's an optimization which saves one=20= round trip for a client which needs CAPABILITY, but doesn't want STARTTLS.=20= It's also needed to see whether the server actually speaks IMAP4rev1. Unlike this proposal, the CAPABILITY does not contain conflicting=20 information. Duplicate !=3D conflicting. > Many email messages=20 > include a timezone name in the date header field as well as the=20 > numeric timezone, decades after the change to numbers. The syntax does not mandate sending the TZ name, so it is a different=20 situation. For example. the message you sent includes just the numeric=20 offset. Also note that the TZ name and the numeric offset usually match. There was=20= no mandate to use "UTC" and add a numeric offset of -0545 in there. > Many IMAP=20 > clients send STATUS commands so often that most of them return=20 > the exact same information as before. Which is because many IMAP servers do not support NOTIFY. The returned=20 information is indeed a repetition of what was sent before, but the client=20= cannot know that in advance, and there is no conflicting information in=20 what is sent. Cheers, Jan --=20 Trojit=C3=A1, a fast Qt IMAP e-mail client -- http://trojita.flaska.net/