Subject: [Jmap] Adding the Message::isForwarded property

From: Benoit Tellier <[email-masked]>
Date: Mon, 3 Apr 2017 16:13:24 +0700

Hello,

At Linagora, we tend to consider **forward** information as important
for the email we care about.

Today, it is not part of the RFC-3501 spec, and many IMAP
implementations handle it with the de-facto standard $Forwarded flag.

This implicit standard is a bad thing, and we truly would like the JMAP
mail protocol to do this right. To be right, it should be explicit.

We then propose this pull request:

It reproduces the behavior of **answered** feature:

 - Adds a **isForwarded** message property
 - Adds a mechanism for automatically marking messages as forwarded upon
sending emails
 - Clarifies interactions between isForwarded and threads
 - Makes isForwarded searchable

Does this proposal make sense to you?

Best regards,

Benoit Tellier
-----------------------------
Software engineer at Linagora
PMC on Apache JAMES

Le 01/04/2017 à 06:23, Dave Crocker a écrit :
> G'day,
>
> The working group meeting discussion about a static message, dynamic
> annotation, etc., resonated with a variety of similar discussions I've
> been around over the years (dating back to the mid-1970.)
>
> A simpler version equates the constructs of message and document, as
> two views of the same thing.  (Ie, Document with attributes; Message
> with a body.)
>
> The essence is to consider the nature and relationship of the objects,
> possibly permitting different semantics for the same set of objects,
> according to different applications or roles.
>
> That is, there can be a variety of constituent objects that are
> associated and can be viewed according to different semantics (or
> views)...  So a message, a document, a calendar entry, a series of
> comments, etc.  Each object has associated processing rules (eg,
> static vs. editable vs. executable; constrained choice of values;
> organization into folders or other schemas...)
>
> An environment like this can  be powerful and very appealing.  The
> challenge tends to be staying practical:  With no effort at all it
> devolves into an abstract computer science exercise.  Some of that is
> an efficiency issue(*) but I think it's mostly about the human
> manageability for design and operations.
>
> Based on both the years of commercial use and the public commentary
> about the performance, I've no doubt the fastmail system does not
> suffer these downsides.  But it's a potential that this re-casting
> through the IETF could easily suffer.
>
> I'm posting this note partly because I think it would exciting to
> produce specs that permit a degree of flexibility that such an
> approach permits, but also wanted to cite the dangers.
>
> At the moment, I'm guessing there needs to be a small number of basic
> object types and a small number of 'relationship' types that define
> the association between objects.  These could then be combined into
> higher-order, formal organizations/semantics the define an application
> semantic (mail, calendar, whatever.)
>
>
> d/
>
> (*) A system I did in 1977 has a little bit of this and the extremely
> pure design produced impressively horrible performance.
>

---

From: Benoit Tellier <[email-masked]>
Date: Mon, 3 Apr 2017 16:15:50 +0700

Link to the pull request: https://github.com/jmapio/jmap/pull/58

Le 03/04/2017 à 16:13, Benoit Tellier a écrit :
> _______________________________________________
> Jmap mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/jmap

---

From: "Chris Newman" <[email-masked]>
Date: Mon, 03 Apr 2017 10:10:21 -0700

IMAP $Forwarded is a registered keyword and thus a fully supported part 
of IMAP:

   https://www.iana.org/assignments/imap-keywords/imap-keywords.xhtml#imap-keywords-1

There's no need for JMAP to define all these registered keywords; it 
only needs to reference the registry.

		= Chris

---

From: Alexey Melnikov <[email-masked]>
Date: Mon, 3 Apr 2017 18:43:09 +0100

On 03/04/2017 18:10, Chris Newman wrote:

> IMAP $Forwarded is a registered keyword and thus a fully supported 
> part of IMAP:
>
Agreed. And at the Chicago face-to-face meeting I've raised the issue of 
lack of generic access to IMAP keywords in JMAP. I believe there was 
room agreement to fix it in a generic way.

---

From: Benoit Tellier <[email-masked]>
Date: Tue, 4 Apr 2017 17:08:31 +0700

|Hi,
|

|Thanks for your answers. How do you plan to access such registered
keyword, if it is not made somehow a message property?|

|As a JMAP user I would like this to be very easily accessible from the
Message object, and also it to be searchable.
|

|In the case of $Forwarded I think it needs to be consistent with the
reply feature, for automatically marked as forwarded, and for threads.|

|Cheers,
|

|Benoit Tellier|

Le 04/04/2017 à 00:43, Alexey Melnikov a écrit :
> Agreed. And at the Chicago face-to-face meeting I've raised the issue
> of lack of generic access to IMAP keywords in JMAP. I believe there
> was room agreement to fix it in a generic way.
>
>>
>>         = Chris
>>

---

From: Alexey Melnikov <[email-masked]>
Date: Tue, 4 Apr 2017 15:00:28 +0100

Hi Benoit,

> keyword, if it is not made somehow a message property?|
|
It has to be a message properly. For example "keywords" which contains 
an array of other IMAP keywords as strings. I have other applications 
that need to access to other keywords, some of which are listed on 
<https://www.iana.org/assignments/imap-keywords/imap-keywords.xhtml>|
>
> |As a JMAP user I would like this to be very easily accessible from 
> the Message object, and also it to be searchable.
> |
>
|
Agreed.

|
> ||
|
I wouldn't mind having some extra logic for forwarded messages, if the 
group can agree on that.

Best Regards,
Alexey

|

---

From: Benoit Tellier <[email-masked]>
Date: Tue, 4 Apr 2017 21:32:28 +0700

Hi Alexey,

Thanks for your answer.

> <https://www.iana.org/assignments/imap-keywords/imap-keywords.xhtml>|
|I agree that a **keyword** property is cool.

However, I would go a bit further. Rather than creating a message
property having a special meaning for the Forwarded feature, I would
make it a top-level message property. That is why I proposed to add a
**isForwarded** message property.
|
>>>> There's no need for JMAP to define all these registered keywords;
>>>> it only needs to reference the registry.
>>> Agreed. And at the Chicago face-to-face meeting I've raised the
>>> issue of lack of generic access to IMAP keywords in JMAP. I believe
>>> there was room agreement to fix it in a generic way.
>>>
>>>>
>>>>         = Chris
>>>>

---

From: "Chris Newman" <[email-masked]>
Date: Tue, 04 Apr 2017 12:25:14 -0700

> <https://www.iana.org/assignments/imap-keywords/imap-keywords.xhtml>|

As a general principle, I'm opposed to gratuitous changes between IMAP 
and JMAP (or Submit and JMAP). So a message has to have flags/keywords 
as a property and $Forwarded is a keyword no different from other 
keywords in how it is accessed.

>> the Message object, and also it to be searchable.

+1. IMAP flags/keywords are easy to access and searchable via IMAP. Same 
should be true via JMAP.

> I wouldn't mind having some extra logic for forwarded messages, if the 

I'm supportive of including flag setting logic for $Forwarded, $MDNSent 
and \Answered in JMAP to the extent JMAP provides forward, MDN and reply 
functionality. Our JMAP-equivalent IMAP/Submit proxy has flag setting 
logic for $MDNSent and \Answered. The omission of support for $Forwarded 
is likely because it was defined later than the other two.

		- Chris

---

From: Neil Jenkins <[email-masked]>
Date: Wed, 05 Apr 2017 00:36:45 -0500

The existing *isFoo* properties on the Message object correspond to
IMAP system flags, so I think they are OK to be special, but since we
know we need generic access to IMAP user flags I agree we should just
do that and use the IANA registry rather than adding a special
*isForwarded* flag.

My suggestion would be to add a *keywords* property to the Message
object, which corresponds to the list of IMAP user flags. There is a
question of what type this should be. An array is one option, but this
has two downsides:
 1.  The server must sort it to ensure a consistent representation is
     returned (probably just by ASCII code point, and for compatibility
     with IMAP the flags *will* have to be ASCII). This then leads to
     the question of whether the server accepts an unsorted array from
     the client when updating it (probably yes, but then it's returning
     something that's semantically different at the JSON layer to what
     was set, to represent something semantically the same at the
     application layer).
 2. It's much harder to define patch semantics for an array. In
    CalConnect[1], we've been working on a new JSON representation for
    event data, and there are often many "complex" properties on the
    event (e.g. set of alarms). Over time we've moved to having
    everything as a map with an id as the key, rather than an array, so
    that we can more easily patch event objects.
So I would advocate for making the *keywords* property type Object, with
the keywords as the keys, each mapping to a Boolean true as the value.
This avoids the above two issues, although at the expense of having an
irrelevant "value" in the map for each keyword.

To enable searching I would add the following properties to the
MessageList FilterCondition:
 * hasKeyword: `String|null`
 * notKeyword: `String|null`

With regards to the special server behaviour for the $Forwarded keyword,
I think we can define this without adding a *forwardingMessageId*
property. We can just define that the server should look for the X-Forwarded-Message-
Id header on the message, and then search for message(s) with this
RFC5322 message id, and add the $Forwarded keyword to any it finds.

The current *inReplyToMessageId* property on the Message object is
actually problematic, because it's meant to be the JMAP message id,
which can change if the original message is delivered and/or deleted
after the reply is fetched, but the property is *meant* to be immutable.

I think we can actually just drop the *inReplyToMessageId* property
altogether. Instead, clients can simply set the In-Reply-To and
References headers directly, and the server can use the In-Reply-To
header to search for the message(s) to set as replied, just like with
$Forwarded.

Neil.

Links:

  1. https://www.calconnect.org/

---

From: Neil Jenkins <[email-masked]>
Date: Wed, 05 Apr 2017 00:43:33 -0500

> devolves into an abstract computer science exercise.

This is the key here. While mailboxes, keywords and annotations could be
viewed as conceptually the same thing and so combined into one perfectly
abstract construct, I strongly believe that to do so would be a mistake.
It would make it harder for client authors to understand and use, harder
to implement efficiently on the server, and harder to map backwards
compatibility to IMAP.

Neil.

---

From: Ted Lemon <[email-masked]>
Date: Wed, 5 Apr 2017 05:21:27 -0700

On Apr 4, 2017, at 10:43 PM, Neil Jenkins <[email-masked]> wrote:
> This is the key here. While mailboxes, keywords and annotations could be viewed as conceptually the same thing and so combined into one perfectly abstract construct, I strongly believe that to do so would be a mistake. It would make it harder for client authors to understand and use, harder to implement efficiently on the server, and harder to map backwards compatibility to IMAP.

I think that the reason to have keywords as a special type of thing is so that you can define standard keywords.   Otherwise they don't seem any different than labels/mailboxes.   I think both types of object have essentially the same desired set of semantics: "give me all the things that match this label, quickly."  "tell me what things with this label have changed/appeared/disappeared."

The reason I pushed back on annotations in the meeting is that I think that annotations are actually likely to be more structured objects than simply "things I need to get done by Friday."   "Things I need to get done by friday" is not a static object: it's a verb with modifiers.   If you treat it as if it's just an annotation, I think that you have an overly limited solution to that problem.   The reason I reacted to that specific example at the microphone the way I did is that this sort of ad hoc label that mail programs currently offer is actually a great example of a missed opportunity: something that could really be a kick-ass feature instead winds up being just another way for me to do bookkeeping, and just another thing for me to remember.

If a thing is a "thing I should get done by Friday," there should be some code running that's helping me to arrange to actually get it done by Friday, not just a notice for me to read on Saturday and feel guilty about.

And an annotation at least as described during the conversation in the meeting last week would not be sufficient for this.   Sure, you could hang more stuff off it to make it work, or have something that cleverly parses these annotations and does something smart with them, but it's all way too sloppy and ad hoc to be useful.

So while I agree that everything isn't the same object, and there may be some value to having mailbox tags be different than annotation tags, I think that an annotation really ought to be a first-class object that can be linked to a message.   The link probably has to be two-way: give me all the messages with this annotation, and give me all the annotations on this message.

Of course, the response at the mic was that a message could have multiple annotations, each one with its own name, and not all downloaded to the client all the time.   I don't disagree with this.   And if indeed each annotation has a name, maybe that's enough structure.   But it doesn't feel right to me.   If there are two objects marked "get done by Friday," I think the question "list objects marked get done by Friday" is a really obvious one.

---

From: Alexey Melnikov <[email-masked]>
Date: Wed, 5 Apr 2017 16:16:27 +0100

Hi Neil,

On 05/04/2017 06:36, Neil Jenkins wrote:
> The existing /isFoo/ properties on the Message object correspond to 
> /isForwarded/ flag.
> My suggestion would be to add a /keywords/ property to the Message 
>     with IMAP the flags /will/ have to be ASCII). This then leads to
>     CalConnect <https://www.calconnect.org/>, we've been working on a
>     new JSON representation for event data, and there are often many
>     "complex" properties on the event (e.g. set of alarms). Over time
>     we've moved to having everything as a map with an id as the key,
>     rather than an array, so that we can more easily patch event objects.
>

Sorting seems sensible to me.
> So I would advocate for making the /keywords/ property type Object, 
> with the keywords as the keys, each mapping to a Boolean true as the 
> value. This avoids the above two issues, although at the expense of 
> having an irrelevant "value" in the map for each keyword.
That would be fine.

>   * notKeyword: `String|null`
Ok.

> With regards to the special server behaviour for the $Forwarded 
> keyword, I think we can define this without adding a 
> /forwardingMessageId/ property. We can just define that the server 
> should look for the X-Forwarded-Message-Id header on the message, and 
> then search for message(s) with this RFC5322 message id, and add the 
> $Forwarded keyword to any it finds.
Is X-Forwarded-Message-Id registered and which implementations are known 
to add it?

> The current /inReplyToMessageId/ property on the Message object is 
> after the reply is fetched, but the property is /meant/ to be immutable.
>
> I think we can actually just drop the /inReplyToMessageId/ property

---

From: "Adrien de Croy" <[email-masked]>
Date: Wed, 05 Apr 2017 20:51:35 +0000

Is there no equivalent to set?  E.g. a sorted container whose keys are 
also the values?

Saves the meaningless value issue.

Adrien

------ Original Message ------
From: "Neil Jenkins" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Sent: 5/04/2017 5:36:45 PM
Subject: Re: [Jmap] Adding the Message::isForwarded property

>The existing isFoo properties on the Message object correspond to IMAP 
>system flags, so I think they are OK to be special, but since we know 
>we need generic access to IMAP user flags I agree we should just do 
>that and use the IANA registry rather than adding a special isForwarded 
>flag.
>
>My suggestion would be to add a keywords property to the Message 
>The server must sort it to ensure a consistent representation is 
>returned (probably just by ASCII code point, and for compatibility with 
>IMAP the flags will have to be ASCII). This then leads to the question 
>of whether the server accepts an unsorted array from the client when 
>updating it (probably yes, but then it's returning something that's 
>semantically different at the JSON layer to what was set, to represent 
>something semantically the same at the application layer).
>It's much harder to define patch semantics for an array. In CalConnect 
><https://www.calconnect.org/>, we've been working on a new JSON 
>representation for event data, and there are often many "complex" 
>properties on the event (e.g. set of alarms). Over time we've moved to 
>having everything as a map with an id as the key, rather than an array, 
>so that we can more easily patch event objects.
>So I would advocate for making the keywords property type Object, with 
>hasKeyword: `String|null`
>notKeyword: `String|null`
>
>forwardingMessageId property. We can just define that the server should 
>look for the X-Forwarded-Message-Id header on the message, and then 
>search for message(s) with this RFC5322 message id, and add the 
>The current inReplyToMessageId property on the Message object is 
>after the reply is fetched, but the property is meant to be immutable.
>
>I think we can actually just drop the inReplyToMessageId property

---

From: Neil Jenkins <[email-masked]>
Date: Wed, 05 Apr 2017 18:13:48 -0500

On Wed, 5 Apr 2017, at 10:16 AM, Alexey Melnikov wrote:

> Is X-Forwarded-Message-Id registered and which implementations are
> known to add it?

Good question. I took this from Benoit's proposed changes, since we
would need *some* header for the server to be able to automatically set
the $Forwarded flag. Googling suggests this was introduced by
Thunderbird[1] and I don't see any evidence that it is registered
anywhere or implemented in other clients.

Anyone have further information on this, or know of a more standardised
alternative? The concept does seem useful to me.

> Is there no equivalent to set?  E.g. a sorted container whose keys are

No, JSON does not have a built-in "set" representation, hence the choice
between array or object representation.

Neil.

Links:

  1. https://bugzilla.mozilla.org/show_bug.cgi?id=583587

---

From: Benoit Tellier <[email-masked]>
Date: Fri, 7 Apr 2017 16:56:44 +0700

Hi Neil,

I took the X-Forwarded-Message-Id from Thunderbird. I did not find more
generic way to convey this information in the headers of the mail.
However I still found this header useful. Hence my proposal.

Otherwise, I definitely agree with having keywords as part of JMAP spec.
I believe we can add mentions about the different keywords defined by
https://www.iana.org/assignments/imap-keywords/imap-keywords.xhtml as
part of JMAP spec.

Would you like me to update my proposal during the following week, to
match the discussion status?

I will include:

 - Keyword field on Messages (Map of keyword to true)

 - GetMessageList logic for keywords

 - The logic about marking messages as $Forwarded when sending them. I
will also include changes about dropping */inReplyToMessageId*./

/  - /The//changes that this implies on Threads (algorithm and order in
display)

Cheers,

Benoit

Le 06/04/2017 à 06:13, Neil Jenkins a écrit :
> would need /some/ header for the server to be able to automatically
> set the $Forwarded flag. Googling suggests this was introduced by
> Thunderbird <https://bugzilla.mozilla.org/show_bug.cgi?id=583587> and
> I don't see any evidence that it is registered anywhere or implemented
> in other clients.
>
> Anyone have further information on this, or know of a more
> standardised alternative? The concept does seem useful to me.
>
>     Is there no equivalent to set?  E.g. a sorted container whose keys
>     are also the values?
>
>
> No, JSON does not have a built-in "set" representation, hence the
> choice between array or object representation.
>
> Neil.
>
>

---

From: Neil Jhaveri <[email-masked]>
Date: Mon, 10 Apr 2017 00:08:59 +0000

On Apr 4, 2017, at 7:32 AM, Benoit Tellier <[email-masked]<mailto:[email-masked]>> wrote:

Hi Alexey,

Thanks for your answer.

Hi Benoit,

Hi,

Thanks for your answers. How do you plan to access such registered keyword, if it is not made somehow a message property?

It has to be a message properly. For example "keywords" which contains an array of other IMAP keywords as strings. I have other applications that need to access to other keywords, some of which are listed on <https://www.iana.org/assignments/imap-keywords/imap-keywords.xhtml><https://www.iana.org/assignments/imap-keywords/imap-keywords.xhtml>
I agree that a **keyword** property is cool.

However, I would go a bit further. Rather than creating a message property having a special meaning for the Forwarded feature, I would make it a top-level message property. That is why I proposed to add a **isForwarded** message property.

I seem to be on the minority side here, but I personally do see the appeal in **isForwarded** being a first-class property. This is a core part of many modern client UI's showing a list of messages. If I had never used IMAP before, it would definitely seem weird and incongruent to me that answered is a property, but forwarded is a keyword, and it would probably cause some momentary confusion trying to figure out how to know if a message was forwarded or not.

I also see the dilemma of having to play catch-up with every new registered keyword becoming a supplemental property.

If there does end up being more support for this, I could see an argument for drawing a line in the sand on the most important keywords, and maybe upgrading $Junk/$NotJunk and $Phishing into first-class properties too. Other keywords like $SubmitPending and $Submitted seem to have their use cases handled quite differently in JMAP with the Outbox concept, so could be left alone in a raw keywords property.

I think it’s a stylistic question, though, and not a functional one. Consensus so far seems to be against this, and I think either way is probably fine, but I do personally support the more OOP-like first-class property, where it makes sense and is cleaner to the newcomer.

As a JMAP user I would like this to be very easily accessible from the Message object, and also it to be searchable.

Agreed.

In the case of $Forwarded I think it needs to be consistent with the reply feature, for automatically marked as forwarded, and for threads.

I wouldn't mind having some extra logic for forwarded messages, if the group can agree on that.

Best Regards,
Alexey

Cheers,

Benoit Tellier

IMAP $Forwarded is a registered keyword and thus a fully supported part of IMAP:

There's no need for JMAP to define all these registered keywords; it only needs to reference the registry.
Agreed. And at the Chicago face-to-face meeting I've raised the issue of lack of generic access to IMAP keywords in JMAP. I believe there was room agreement to fix it in a generic way.

        = Chris

Hello,

Best regards,

Benoit Tellier

d/

_______________________________________________
Jmap mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/jmap

_______________________________________________
Jmap mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/jmap

_______________________________________________
Jmap mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/jmap

---

From: Neil Jenkins <[email-masked]>
Date: Mon, 10 Apr 2017 17:44:01 +1000

On Mon, 10 Apr 2017, at 10:08 AM, Neil Jhaveri wrote:

> I seem to be on the minority side here, but I personally do see the
> appeal in **isForwarded** being a first-class property. This is a core
> part of many modern client UI's showing a list of messages. If I had
> never used IMAP before, it would definitely seem weird and incongruent
> to me that answered is a property, but forwarded is a keyword,

This is a good point. From a pure conceptual view it *is* weird and
inconsistent.

> If there does end up being more support for this, I could see an
> argument for drawing a line in the sand on the most important
> keywords, and maybe upgrading $Junk/$NotJunk and $Phishing into first-
> class properties too. Other keywords like $SubmitPending and
> $Submitted seem to have their use cases handled quite differently in
> JMAP with the Outbox concept, so could be left alone in a raw keywords
> property.

Yes, I think this is worth considering. It provides a better interface
to this functionality. The main downside is when a new IANA keyword is
registered, at first it would just appear in the keywords property. Then
in the next JMAP spec version you would probably have it added as a
message property (and therefore disappear from the keywords array), and
so you would potentially need client code to handle both paths.
Although, maybe not if the JMAP spec revision could happen reasonably
quickly (and arguably it's a way of knowing the server does something
with this flag, which is the whole point of them, so you never need to
handle the keywords option).

Neil.

---

From: "Chris Newman" <[email-masked]>
Date: Mon, 10 Apr 2017 16:38:02 -0700

On 9 Apr 2017, at 17:08, Neil Jhaveri wrote:
> to me that answered is a property, but forwarded is a keyword, and it 
> would probably cause some momentary confusion trying to figure out how 
> to know if a message was forwarded or not.

System flags and keywords have largely identical syntax and behavior in 
the IMAP data model and are all searchable.

So the fact that JMAP is introducing a special data model for system 
flags that's different from the syntax for keywords seems odd to me. Why 
can't JMAP treat all flags/keywords as peers in the data model; the way 
IMAP does?

The only significant difference between flags and keywords in the IMAP 
base spec is that servers are not required to support keywords. One of 
the design flaws in IMAP is it's too permissive of server variations 
that make it harder to write clients. When the majority of deployed IMAP 
servers support a feature (providing support for a minimum number of 
extensible keywords being an example), I'm supportive of making JMAP 
require that feature. So adding a clause that requires JMAP servers to 
support at least X extensible keywords (maybe X=32 or X=64) is a change 
I would support.

> keywords, and maybe upgrading $Junk/$NotJunk and $Phishing into 
> first-class properties too. Other keywords like $SubmitPending and 

 From my viewpoint, differences between the IMAP data model and JMAP 
data model are added complexity to the email infrastructure unless 
there's a justification for making a change. So I'd prefer to leave the 
line between system flags and registered keywords where it is unless 
there's a compelling reason to change it. Otherwise we have to modify 
the keyword registry to distinguish ones that are treated differently in 
JMAP from IMAP.

> I think it’s a stylistic question, though, and not a functional one. 
> Consensus so far seems to be against this, and I think either way is 
> probably fine, but I do personally support the more OOP-like 
> first-class property, where it makes sense and is cleaner to the 
> newcomer.

The extensibility of a data model is one of the most important parts of 
a protocol's design. I consider JMAP severely deficient if its 
extensibility model for flags/keywords is inferior to the one in IMAP. I 
also consider JMAP deficient if it can't largely share the extension 
registries for IMAP keywords/metadata/annotations and the extension 
registry for the SMTP Submission protocol.

		- Chris

---

From: "HANSEN, TONY L" <[email-masked]>
Date: Tue, 11 Apr 2017 18:09:34 +0000

I support everything that Chris says here.

	Tony Hansen

On 4/10/17, 7:38 PM, "Jmap on behalf of Chris Newman" <[email-masked] on behalf of [email-masked]> wrote:

    

    

    
    		- Chris

---

From: Neil Jenkins <[email-masked]>
Date: Wed, 19 Apr 2017 15:57:32 +1000

The initial JMAP design was only interested in the system flags, so
having the separate properties made sense. But given we want to add full
support for custom IMAP flags (and reuse the IANA registry), I think
having the system ones in a different format is going to end up causing
pain down the line. We could make properties for the keywords currently
in the IANA registry, but this would clearly cause headaches as more are
standardised down the line. So I agree we should change to just a single
"keywords" property, and reuse the keyword names from IMAP.
This means we would remove the "isUnread", "isFlagged", "isAnswered" and
"isDraft" properties from the Message object and introduce a single
"keywords" property (with respectively "\Seen" (flipped meaning),
"\Flagged", "\Answered" and "\Draft" keywords, as per IMAP).
The main downside is I think  is this is slightly less accessible for
newcomers who haven't been exposed to IMAP, but this is probably
outweighed by the benefits already discussed.
Do we have consensus on this? The last few messages all seem to be in
favour of making this change. If so, I'm happy to update the spec and
call this issue closed.
In terms of representation, do we agree on an Object type, with the
keywords as the keys (and a boolean `true` as the value (as discussed in
a previous email in this thread)?
Neil.

---

From: "Adrien de Croy" <[email-masked]>
Date: Wed, 19 Apr 2017 06:20:17 +0000

Hi

one thing that has bugged me for a long time about IMAP is its basic 
verbosity.

We protocol log IMAP here, and the IMAP logs are a lot bigger than any 
others even though a lot less work is done.

Even trimming the completely redundant \ from the start of a flag 
keyword would save a bunch of space and transfer bandwidth.

Are we going to make any effort to make the JMAP more efficient or will 
it be equally or even more bloated?

Will there be a requirement for compression support?

I don't know why we would stick with the same keywords for the flags as 
IMAP uses.  I wonder if any IMAP implementors are storing these flags in 
textual format anyway.  We certainly aren't.

Regards

Adrien

------ Original Message ------
From: "Neil Jenkins" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Sent: 19/04/2017 5:57:32 PM
Subject: Re: [Jmap] Adding the Message::isForwarded property

>having the separate properties made sense. But given we want to add 
>full support for custom IMAP flags (and reuse the IANA registry), I 
>think having the system ones in a different format is going to end up 
>causing pain down the line. We could make properties for the keywords 
>currently in the IANA registry, but this would clearly cause headaches 
>as more are standardised down the line. So I agree we should change to 
>just a single "keywords" property, and reuse the keyword names from 
>IMAP.
>
>This means we would remove the "isUnread", "isFlagged", "isAnswered" 
>and "isDraft" properties from the Message object and introduce a single 
>keywords as the keys (and a boolean `true` as the value (as discussed 
>in a previous email in this thread)?
>
>Neil.

---

From: Neil Jenkins <[email-masked]>
Date: Wed, 19 Apr 2017 16:51:13 +1000

> keyword would save a bunch of space and transfer bandwidth.
The main issue would be namespace conflicts. While it would be a very
odd and confusing thing to do, someone may have defined an IMAP keyword
called "Flagged" for example, so if you drop the \ from the beginning of
system flags you could end up with a conflict between the system and
user keyword names.
> Will there be a requirement for compression support?

Since it goes over HTTP, any standard compression supported by the HTTP
server can be negotiated. I would expect all servers to support gzip at
a minimum. This reduces the data size in responses from the server
considerably, and effectively negates any difference between "\Flagged"
and "Flagged".
Neil.

---

From: Alexey Melnikov <[email-masked]>
Date: Wed, 19 Apr 2017 08:42:35 +0100

Hi,

> On 19 Apr 2017, at 07:20, Adrien de Croy <[email-masked]> wrote:
> 
> I don't know why we would stick with the same keywords for the flags as IMAP uses.  I wonder if any IMAP implementors are storing these flags in textual format anyway.  

If a server supports arbitrary IMAP keywords, it has to. Multiple implementations do.

> We certainly aren't.

---

From: Neil Jhaveri <[email-masked]>
Date: Thu, 27 Apr 2017 22:07:32 +0000

On Apr 18, 2017, at 10:57 PM, Neil Jenkins <[email-masked]<mailto:[email-masked]>> wrote:

This means we would remove the "isUnread", "isFlagged", "isAnswered" and "isDraft" properties from the Message object and introduce a single "keywords" property (with respectively "\Seen" (flipped meaning), "\Flagged", "\Answered" and "\Draft" keywords, as per IMAP).

The main downside is I think is this is slightly less accessible for newcomers who haven't been exposed to IMAP, but this is probably outweighed by the benefits already discussed.

Given the discussion above, this makes sense to me, too. My original motivation for supporting an isForwarded property was to eliminate the incongruence between answered and forwarded. Going back to a keywords list achieves that.

---

From: Neil Jenkins <[email-masked]>
Date: Fri, 28 Apr 2017 06:22:02 +0000

On Thu, 27 Apr 2017, at 10:07 PM, Neil Jhaveri wrote:
> Given the discussion above, this makes sense to me, too. My original
> motivation for supporting an isForwarded property was to eliminate the
> incongruence between answered and forwarded. Going back to a keywords
> list achieves that.
OK, I think we have consensus on this. I've made a pull request on
GitHub with the changes[1] for people to review. Two interesting points:
Firstly, I think we should map the 4 \System flags to $System in JMAP.
Using backslash is a real pain because it is an escape character in
JSON, so examples in the spec are confusing if nothing else! Changing it
to $ would make the system keywords consistent with the IANA registered
keywords, which I believe will be less confusing for developers coming
in without an IMAP background. The only issue would be what to do if
someone has added "$Flagged" etc. user keywords on an IMAP server. This
seems unlikely to be much of a real-world occurrence to me, on the basis
it would have been very confusing, but I think we can just specify that
these keywords are not visible over JMAP.
Secondly, matching the setting of the \Answered keyword, I've added a
section that the server SHOULD automatically set the $Forwarded keyword
if appropriate on send (the request that started this whole thread!).
This is looking at the X-Forwarded-Message-Id header though, which as
mentioned earlier in the thread is not a standard (it seems it was first
introduced by Thunderbird).
Any thoughts on either of these, or the change as a whole?

Neil.

Links:

  1. https://github.com/jmapio/jmap/pull/61/files