Subject: Re: [Jmap] Destroying mailboxes: how this affects messages

From: Neil Jenkins <[email-masked]>
Date: Tue, 05 Sep 2017 20:15:40 +1000

Yes, there's no need for an All Mail mailbox; if we wanted to go this
route, we just relax the requirement that a message must be in at least
one mailbox and allow messages to exist that aren't in any mailboxes.
The main trouble with this is such a message would be completely
invisible via IMAP, so mandating this behaviour would be very awkward to
implement if you want to keep compatibility with both (which I reiterate
is, I believe, vital for smooth migration). And if it's optional that
just adds extra complexity and still leaves the problem of what to do in
systems that don't allow zero mailboxes.
Regards,
Neil.

PS Benoît: your MUA seems to have set neither an In-Reply-To nor a
   References header, so your reply did not thread. Would be great if
   you could fix that.

On Tue, 5 Sep 2017, at 06:45 PM, Benoît TELLIER wrote:
> Hi,

> At Linagora, we have a "All mail" mailbox displayed in our front-end.
> We simply get this using a GetMessageList request (hence the mailbox
> do not exist, we can say the "all mail" mailbox is virtual).> 
> It solves the "All Mail" problem without enforcing any server specific
> behaviour, just requires a little bit smarter client.> 

> Cheers.

> -- Tellier Benoit  Software engineer dedicated to OpenPaaS at Linagora
> PMC of the Apache JAMES project VIE in Vietnam
> https://twitter.com/AwesomePaaS
> https://medium.com/linagora-engineering  (I'm proud to tell you this
> e-mail is JAMES and OpenPaaS powered)> On Sep 4, 2017 7:41 PM, from [email-masked]
>> On Sep 4, 2017, at 12:54 AM, Neil Jenkins  wrote:
>> > Opinions on the right action here?
>> 
>> Have a mailbox called "all" that every message is always in. Or, as
>> you say, make it an error to delete a non-empty mailbox. But I prefer
>> "all" because enforcing that restriction could be incompatible with a
>> label-based inbox paradigm, where we would expect labels to
>> potentially be quite ephemeral.>> _______________________________________________
>> Jmap mailing list
>> [email-masked]
>> https://www.ietf.org/mailman/listinfo/jmap

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Tue, 5 Sep 2017 12:07:40 +0100

Neil Jenkins writes:
> if we wanted to 
> go this route, we just relax the requirement that a message must 
> be in at least one mailbox and allow messages to exist that 
> aren't in any mailboxes.

Because of this requirement, (some) mailboxes are (individually or 
collectively) containers. IMO it's a fine requirement. Fits users' mental 
model of how gmail works, and that's the MSB.

Saying "mailboxes can be deleted only if that doesn't require deleting 
messages" is a restatement of the same requirement, not something new. It 
implies that if clients want to delete mailboxes with messages, they have 
to do work O(mailboxes+messages), but isn't that OK? Does anything justify 
striving for sublinear performance there? I don't see anything.

I'll probably resume lurking now. I've said my piece.

Arnt

---

From: Ted Lemon <[email-masked]>
Date: Tue, 5 Sep 2017 09:01:02 -0400

Hm.   Well, this has been a clarifying discussion for me—I realize that the reason I'm arguing in the direction I am isn't quite what I expressed.   Let me state the problem from my perspective, understanding that my perspective isn't necessarily correct, but just where I am coming from.

From my perspective the data store is a collection of messages, plus a collection of labels.   Each label is a list of message IDs, plus a label name.   We can talk about "all messages," but it's not really a label—it's just an artifact of the data store.   "All messages" is the list of all message IDs in the data store, and can be constructed automatically if the index is lost.   This is really important to me—I want to be able to reconstruct the data store if the index is lost.

Similarly each label is both the list of message IDs (the index) and also the label as it's stored on the message.   If these indices are lost, they can be reconstructed by scanning the data store.   If part of the data store is lost, the data store can still be reconstructed along with its indices.

Also, in my way of thinking, "deleting" a message is something you do only if the message is literally garbage, e.g. spam.   Otherwise it remains in the data store even if it's no longer presented to the end user.   This makes sense for auditability; the opposite might make sense for user privacy, so I can see where that focus might be what is preferred by providers who are selling mail to non-enterprise, non-government end users.

The point is that with this model, the idea that deleting a label would delete the message is completely artificial.   And that's why I'm reacting the way I am to this proposal.   This proposal is addressing a very particular use case.

The other reason I am reacting the way I am is that I've been burned so many times by IMAP's model of "mailbox as container" that I really want to make that go away.   There is no reason at all to tie the concept of "label" and "container" together in this way.    It's totally artificial, just an artifact of history, of how IMAP used to work.   As Google has demonstrated, it's perfectly possible to run a data store that works the way I'm describing, and have that appear to the end user as an IMAP server.   So there is no need, for the goal of compatibility, to have this artificial association between "present in at least one label" and "not deleted".

It would be no trouble for me to implement this particular paradigm on top of my data store; the problem is that it doesn't make sense to me to do so.   I don't want this behavior—it's not consistent with my use model.   To use the old X11 motto, this is policy, not mechanism.

---

From: Neil Jenkins <[email-masked]>
Date: Mon, 11 Sep 2017 11:55:15 +1000

On Tue, 5 Sep 2017, at 11:01 PM, Ted Lemon wrote:
> From my perspective the data store is a collection of messages, plus a
> collection of labels.
This is a perfectly reasonable model, and in designing JMAP I have
tried to allow servers to support this behaviour with the one slight
quirk that all messages must be in at least one mailbox (have at least
one "label").
The transition from IMAP to JMAP cannot happen overnight and providing a
smooth transition path is really important to give it a chance of
happening at all. If you mandate in JMAP that messages MUST be allowed
to exist without any mailboxes, then you have the situation where users
can see some mail only if they connect to their server with a JMAP
client, and not if they use an IMAP client (unless you start mapping
them in via a virtual mailbox, which is doable but has significant
quirks and fundamentally I think requiring you to change your IMAP
implementation if you want to add JMAP as well is likely to be a barrier
to adoption).
If you made it *optional* for servers to allow 0-mailbox messages, then
you're increasing complexity for the spec and clients, and you still
have the problem of what to do when deleting a mailbox for servers that
opt not to allow 0-mailbox messages.
I think that the current spec of returning an error, thus forcing the
client to deal with it if there are existing messages in the mailbox you
are trying to destroy, seems to be the least problematic solution.
Does anyone else want to weigh in with an opinion on this?

Neil.

---

From: Ted Lemon <[email-masked]>
Date: Sun, 10 Sep 2017 22:05:30 -0400

> Does anyone else want to weigh in with an opinion on this?

Acknowledging that you were not asking me to weigh in again, I just want to clarify that I do not think that "messages with zero labels" are necessary.   I do think that an "all" mailbox is sufficient to address this issue: every message is always labeled "all," or, if you are using a JMAP skin over an IMAP server, then every message is always in the "all" mailbox as well as whatever other mailboxes, if any, it may be in.

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Mon, 11 Sep 2017 20:56:23 +0100

> Does anyone else want to weigh in with an opinion on this?

I'll weigh in with another.

It's a minor advantage if the text chosen permits moving forward with 
zero-mailbox messages at some future time (e.g. when world peace breaks 
out, people stop using pop3 or both of the above).

At this time, imap compatibility is an effective requirement. But that may 
not remain true — lots of new clients are gmail-only now, a few jmap-only 
clients could change the picture in a handful of years.

You've seen the rule I proposed. I think that permits moving forward.

Arnt

---

From: Stan Kalisch <[email-masked]>
Date: Mon, 11 Sep 2017 16:36:34 -0400

For my two cents, I was going to propose roughly the same thing, so +1.

Stan

> On Sep 11, 2017, at 3:56 PM, Arnt Gulbrandsen <[email-masked]> wrote:
> 
> Neil Jenkins writes:
> It's a minor advantage if the text chosen permits moving forward with zero-mailbox messages at some future time (e.g. when world peace breaks out, people stop using pop3 or both of the above).
> 
> At this time, imap compatibility is an effective requirement. But that may not remain true — lots of new clients are gmail-only now, a few jmap-only clients could change the picture in a handful of years.
> 
> _______________________________________________
> Jmap mailing list
> [email-masked]