Subject: [EAI] I-D Action: draft-ietf-eai-5738bis-05.txt

From: [email-masked]
Date: Sat, 14 Jul 2012 03:30:18 -0700

A New Internet-Draft is available from the on-line Internet-Drafts directories.
 This draft is a work item of the Email Address Internationalization Working Group of the IETF.

	Title           : IMAP Support for UTF-8
	Author(s)       : Pete Resnick
                          Chris Newman
                          Sean Shen
	Filename        : draft-ietf-eai-5738bis-05.txt
	Pages           : 9
	Date            : 2012-07-14

Abstract:
   This specification extends the Internet Message Access Protocol
   version 4rev1 (IMAP4rev1) to support UTF-8 encoded international
   characters in user names, mail addresses and message headers.  This
   specification replaces RFC 5738.

The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-eai-5738bis

There's also a htmlized version available at:
http://tools.ietf.org/html/draft-ietf-eai-5738bis-05

A diff from previous version is available at:
http://tools.ietf.org/rfcdiff?url2=draft-ietf-eai-5738bis-05

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Sat, 14 Jul 2012 13:08:02 +0200

Now the ability to send Utf8 in quoted strings is completely gone. I an sorry about that. While it is not critical to correctness, I think it is important to ease of implementation.

A client that sent select commands using printf no longer can do that: the printf format has to change a lot and formally speaking the client has to wait on a literal.

Alexey, can you suggest a syntax change to allow utf8 in quited-strings? (i am afk, sorry.)

Arnt

---

From: Alexey Melnikov <[email-masked]>
Date: Mon, 16 Jul 2012 16:02:27 +0100

On 14/07/2012 12:08, Arnt Gulbrandsen wrote:
>
> Now the ability to send Utf8 in quoted strings is completely gone. I 
> an sorry about that. While it is not critical to correctness, I think 
> it is important to ease of implementation.
>
> A client that sent select commands using printf no longer can do that: 
> the printf format has to change a lot and formally speaking the client 
> has to wait on a literal.
>
Here is the proposed change:

3.  UTF8=ACCEPT IMAP Capability

    The "UTF8=ACCEPT" capability indicates that the server supports the
    ability to open mailboxes containing internationalized messages with
    SELECT and EXAMINE, and UTF-8 responses from the LIST and LSUB
    commands.

    A client MUST use the "ENABLE UTF8=ACCEPT" command (defined in
    [RFC5161]) to indicate to the server that the client accepts UTF-8

Insert "in" here

    quoted-strings.

Add back the Section 3.1 (slightly modified):

3.1.  UTF-8 in IMAP Quoted Strings

    The IMAP4rev1 [RFC3501] base specification forbids the use of 8-bit
    characters in atoms or quoted strings.  Thus, a UTF-8 string can only
    be sent as a literal.  This can be inconvenient from a coding
    standpoint, and unless the server offers IMAP4 non-synchronizing
    literals [RFC2088], this requires an extra round trip for each UTF-8
    string sent by the client.  When the IMAP server advertises the
    "UTF8=ACCEPT" capability, it informs the client that it supports
    UTF-8 in quoted-strings with the following syntax:

         quoted        =/ DQUOTE *uQUOTED-CHAR DQUOTE
                ; QUOTED-CHAR is not modified, as it will affect
                ; other RFC 3501 ABNF non terminal.

         uQUOTED-CHAR  = QUOTED-CHAR / UTF8-2 / UTF8-3 / UTF8-4

         UTF8-2        =   <Defined in Section 4 of RFC3629>

         UTF8-3        =   <Defined in Section 4 of RFC3629>

         UTF8-4        =   <Defined in Section 4 of RFC3629>

    When this extended quoting mechanism is used by the client, then the 
server
    MUST reject octet sequences with the high bit set that fail to comply
    with the formal syntax in [RFC3629] with a BAD response.

    The IMAP server MUST NOT send UTF-8 in quoted strings to the client 
unless
    the client has indicated support for that syntax by using the "ENABLE
    UTF8=ACCEPT" command.

    If the server advertises the "UTF8=ACCEPT" capability, the client MAY
    use extended quoted syntax with any IMAP argument that permits a string
    (including astring and nstring).  However, if characters outside the
    US-ASCII repertoire are used in an inappropriate place, the results
    would be the same as if other syntactically valid but semantically
    invalid characters were used.  Specific cases where UTF-8 characters
    are permitted or not permitted are described in the following
    paragraphs.

    All IMAP servers that advertise the "UTF8=ACCEPT" capability SHOULD
    accept UTF-8 in mailbox names, and those that also support the
    "Mailbox International Naming Convention" described in RFC 3501,
    Section 5.1.3 MUST accept utf8-quoted mailbox names and convert them
    to the appropriate internal format.  Mailbox names MUST comply with
    the Net-Unicode Definition (Section 2 of [RFC5198]) with the specific
    exception that they MUST NOT contain control characters (0000-001F,
    0080-009F), delete (007F), line separator (2028), or paragraph
    separator (2029).

    An IMAP client MUST NOT issue a SEARCH command that uses a mixture of
    UTF-8 in quoted strings and a SEARCH CHARSET other than UTF-8. If an 
IMAP
    server receives such a SEARCH command, it SHOULD reject the command
    with a BAD response (due to the conflicting charset labels).

(And add back the removed references: RFC 2088 and RFC 5198)

4.  IMAP UTF8 Append Data Extension

  [...]

         utf8-literal   = "UTF8" SP "(" literal8 ")"

It might be worth adding here:

          literal8 = <Defined in RFC 4466>

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Mon, 16 Jul 2012 17:16:42 +0200

> Here is the proposed change:

I like it, except one thing.

>     with a BAD response (due to the conflicting charset labels).

I implemented that, it wasn't too bad. But I don't really like it. I 
would feel happier with either a simpler rule like "an IMAP client MUST 
NOT issue a SEARCH command using CHARSET after enabling blah" or not 
mentioning CHARSET at all.

The rule as proposed is neither very simple nor very good.

Arnt

---

From: Alexey Melnikov <[email-masked]>
Date: Mon, 16 Jul 2012 16:31:47 +0100

On 16/07/2012 16:16, Arnt Gulbrandsen wrote:
> would feel happier with either a simpler rule like "an IMAP client 
> MUST NOT issue a SEARCH command using CHARSET after enabling blah" or 
> not mentioning CHARSET at all.

I don't like the latter. I think the default is US-ASCII, so we need to 
say that we are changing the default if UTF-8 is used in quoted strings 
or something like that.

> The rule as proposed is neither very simple nor very good.

Please explain, I think this is a very simple check.

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Mon, 16 Jul 2012 17:51:46 +0200

> Please explain, I think this is a very simple check.

It's not related to actual needs. Good checks prevent breakage, that 
check prevents nothing at all. The server has to parse and handle 
CHARSET with any and all values, unless it's willing to turn away 
clients that do not send the ENABLE.

Since the server has the code to do that, why not let all the clients 
use it?

IMO CHARSET is an offensive mess and should be taken out and shot. But 
if we're going to keep it we should keep it. The rule you're suggesting 
is neither here nor there. Think about it: You're suggesting that both d 
and e are legal, but not f:

    a enable utf8=accept
    b login alexey foo
    c select inbox
    d search charset iso64610 subject "bl}b{rsyltet|y"
    e search subject "blåbærsyltetøy"
    f search charset iso64610 subject "bl}b{rsyltet|y" text "blåbærsyltetøy"

I'm saying that if d is legal after a, so is f, and that f is illegal 
after a, then so is d.

(Yes, the example looks a lot worse if you use latin1 or viscii or 
something, with ugly literals and blah. The point is unchanged.)

Arnt