From william.w.fisher@gmail.com Fri Oct 7 12:00:49 2016 Return-Path: X-Original-To: precis@ietfa.amsl.com Delivered-To: precis@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 79107129742 for ; Fri, 7 Oct 2016 12:00:48 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.699 X-Spam-Level: X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DvZL3LeXKf_N for ; Fri, 7 Oct 2016 12:00:46 -0700 (PDT) Received: from mail-io0-x230.google.com (mail-io0-x230.google.com [IPv6:2607:f8b0:4001:c06::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 79D8012980A for ; Fri, 7 Oct 2016 12:00:32 -0700 (PDT) Received: by mail-io0-x230.google.com with SMTP id r30so56352796ioi.1 for ; Fri, 07 Oct 2016 12:00:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to; bh=wVbC5oujD7snyKWviVMUSNBVUj62H187dQJb48pIzo4=; b=DAoWsxIisCk/3ONaNwWakoVslyJJRrOCGcLeWqpwYmVbdTASjPzKyYaT3QmX3bjFIw VKVrQ1GnxnzQa6OC3FTI19JMBlOEJSH6+oXT3ksKXy4dCNgReDp/UgigllETBtHffoV2 sAKn4YinOHHqUKmGTDuOOVTGbyzXn4QYzUUkRXN6/2bAhdYyYrVkGIoNdrAfuOCRNFJ4 H59ShdTbT2NE/BE4bezxN1V9/2tbz+dL3+dTMwQuo2d7g0TrmlhUkHjLLais88m9pQyf UX+ly0opGzn/lsNg6tfj3h4dbiK2E6ZtlsZCuuEOg3dLH/F+Zdij3nni5N/gE7ilkc8I 7cxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=wVbC5oujD7snyKWviVMUSNBVUj62H187dQJb48pIzo4=; b=MhSghZObqPqKwiYBY0K0kqi7ec6niMgjGBcT9aypmHt+Ku6foWApfMsXTxW4y3UM5b GpCGsI9JQ9GwnXNnN8kzHsCOVwDbISypLA8dn3QmCs47t2leuCl1432afyatpTY85vdD Y5IQOroZmv9e8+yi7ENAZ++z0eZ6tinzEzhTEXc/J7JkmZI7TS+jvrMoS0HGX1i+ltnP waJPkDyi4/Y6UtRM0J4jxQ1c2gvH/pLeZFVKZcCLRPNZqyBlxhRwgn+1yOHEoCADeZj1 DbVolRmAm1KDKi3Sp1/9TltGgAdzNP+ZyZG2XpEmehO9TAu1wheYaDxCQPBxZpo7VhKe uOdg== X-Gm-Message-State: AA6/9RnSPobVQjEeENJ1NsioeA4QqEE7AqZir9LkY5zo80Gm2DPAqin1fAeys3KbRLR6DhydPnvHlfYXb7A1Fg== X-Received: by 10.107.129.69 with SMTP id c66mr25056038iod.111.1475866831484; Fri, 07 Oct 2016 12:00:31 -0700 (PDT) MIME-Version: 1.0 Received: by 10.50.95.42 with HTTP; Fri, 7 Oct 2016 12:00:31 -0700 (PDT) From: William Fisher Date: Fri, 7 Oct 2016 12:00:31 -0700 Message-ID: To: precis@ietf.org Content-Type: multipart/alternative; boundary=001a113f9b0c39fd17053e4b06c0 Archived-At: Subject: [precis] PRECIS bidi rule for usernames X-BeenThere: precis@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Preparation and Comparison of Internationalized Strings List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Oct 2016 19:11:40 -0000 --001a113f9b0c39fd17053e4b06c0 Content-Type: text/plain; charset=UTF-8 I've been working on a python implementation of the PRECIS specification (https:github.com/byllyfish/precis-i18n). The Username profiles apply the "bidi rule". Directionality Rule: Applications MUST apply the "Bidi Rule" defined in [RFC5893] to strings that contain right-to-left characters (i.e., each of the six conditions of the Bidi Rule must be satisfied). Can someone please clarify whether: A. The "Bidi rule" is ONLY applied to strings that contain right-to-left characters. B. The "Bidi rule" is applied to ALL strings. If the answer is A, then "Juliet+" is a valid username. If the answer is B, "Juliet+" is not a valid username because it violates the Bidi rule. I think the answer is probably A. However, this implies that right-to-left usernames can never begin or end with ASCII punctuation. Thanks, Bill --001a113f9b0c39fd17053e4b06c0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
I've been working on a python implementation of the PR= ECIS specification (https:github.com/byllyfish/precis-i18n).

The Username profiles appl= y the "bidi rule".=C2=A0

=C2=A0 =C2=A0 =C2=A0 =C2=A0Direc= tionality Rule: Applications MUST apply the "Bidi Rule"
=C2=A0= =C2=A0 =C2=A0 =C2=A0defined in [RFC5893] to strings that contain right-to-= left
=C2=A0 =C2=A0 =C2=A0 =C2=A0characters (i.e., each of the six condit= ions of the Bidi Rule
=C2=A0 =C2=A0 =C2=A0 =C2=A0must be satisfied).
=

Can someone please clarify whether:
A. The "Bidi rule" is ONLY applied to strings that co= ntain right-to-left characters.
B. The "Bidi rule" is a= pplied to ALL strings.

If the answer is A, then &q= uot;Juliet+" is a valid username.=C2=A0

If th= e answer is B, "Juliet+" is not a valid username because it viola= tes the Bidi rule.

I think the answer is probably = A. However, this implies that right-to-left usernames can never begin or en= d with ASCII punctuation.

Thanks,
Bill
--001a113f9b0c39fd17053e4b06c0-- From nobody Fri Oct 7 12:29:40 2016 Return-Path: X-Original-To: precis@ietfa.amsl.com Delivered-To: precis@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9DA461295FE for ; Fri, 7 Oct 2016 12:29:38 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.701 X-Spam-Level: X-Spam-Status: No, score=-2.701 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=samwhited.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id H3MF_WaUyBLS for ; Fri, 7 Oct 2016 12:29:37 -0700 (PDT) Received: from mail-qt0-x22d.google.com (mail-qt0-x22d.google.com [IPv6:2607:f8b0:400d:c0d::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 07A60129530 for ; Fri, 7 Oct 2016 12:29:37 -0700 (PDT) Received: by mail-qt0-x22d.google.com with SMTP id f6so25714482qtd.2 for ; Fri, 07 Oct 2016 12:29:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samwhited.com; s=swgoo; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=TqYYOeJgTyfVrHQpmzx4e0PUxioJc0tC1ETUo69Cq4s=; b=H7cqO6boae47Hr5uAOQuwph2dC/UjeNEuA+UsGQpIr7OJPcw7g86BeZkAXqYmFDNbc zGu0KP9ZwZzw1sokblGkj6/glLc7mpcTFO0d2Ce1KCAzIcDIjx0r4i3vASBk+m8Ah2C7 BraCvQSD6AwAGEwOuEPIBRbbnBoBNJ5VdSxcg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=TqYYOeJgTyfVrHQpmzx4e0PUxioJc0tC1ETUo69Cq4s=; b=FV+0JeZUJlbSsNuxr5XJyWbw1/BhG9z470tHSvqS7NPEz4GCAG5OGhzYJ0kkXiMvPE fWQNOXCaH44Yf8pIsVlg3PEt4JOoLcJhre31TTCWNC6sV0vTpiTjVGqCC4R9XDJPX3I4 CgDXfrSwhcM7mHK5EQyZjAjo0L8ce8zVgVT0tKVrf/ChKLX1VFuwS6cL5DuWwtMfbKL4 5ZWN6G+f+6CW46vnD7R1phJBQcSYlRDXPnhY9q5WZeV5oNtnIdTqr0LapzbDlJvFEOV5 9v61La0S4bdMviTFqb0rSCksWm4tO84chfkccQxYRxVjJb4E7QBfOh1LGbX8/ON5iS7L wXfQ== X-Gm-Message-State: AA6/9Rnpi5mE3+8vtfGK5zJ6zhqMOBDwjLb8UsQMTovNtV46DgShh96t70Tk9oGY0Q/dyhArgTA8HBAowYCkVA== X-Received: by 10.200.37.177 with SMTP id e46mr21550264qte.95.1475868576023; Fri, 07 Oct 2016 12:29:36 -0700 (PDT) MIME-Version: 1.0 Received: by 10.55.15.166 with HTTP; Fri, 7 Oct 2016 12:28:55 -0700 (PDT) X-Originating-IP: [72.48.156.244] In-Reply-To: References: From: Sam Whited Date: Fri, 7 Oct 2016 14:28:55 -0500 Message-ID: To: William Fisher Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Archived-At: Cc: precis@ietf.org Subject: Re: [precis] PRECIS bidi rule for usernames X-BeenThere: precis@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Preparation and Comparison of Internationalized Strings List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Oct 2016 19:29:39 -0000 On Fri, Oct 7, 2016 at 2:00 PM, William Fisher wrote: > I've been working on a python implementation of the PRECIS specification > (https:github.com/byllyfish/precis-i18n). Great timing; I was just this moment looking at your library and considering including it in some interoperability tests for a set of test vectors I'm working on. > Can someone please clarify whether: > > A. The "Bidi rule" is ONLY applied to strings that contain right-to-left > characters. > B. The "Bidi rule" is applied to ALL strings. The recent draft-ietf-precis-7613bis-03 clarifies this: Directionality Rule: Apply the "Bidi Rule" defined in [RFC5893] to strings that contain right-to-left characters (i.e., each of the six conditions of the Bidi Rule must be satisfied); for strings that do not contain right-to-left characters, there is no special processing for directionality. I was apparently confused about this before too, I'm applying Bidi all the time in the Go implementation. I'll be sure to add a test for this in the test vectors when I publish them. I'm actually not convinced this is the correct behavior though; it seems confusing to me that usernames with RTL characters couldn't end with punctuation, but strings with them could. This violates the principal of least suprise. Maybe Peter (CCed) can clarify if I'm misunderstanding something? =E2=80=94Sam --=20 Sam Whited pub 4096R/54083AE104EA7AD3 From nobody Sun Oct 9 20:12:58 2016 Return-Path: X-Original-To: precis@ietfa.amsl.com Delivered-To: precis@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 699BA1295CB for ; Sun, 9 Oct 2016 20:12:56 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.898 X-Spam-Level: X-Spam-Status: No, score=-4.898 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-2.996, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yVuvuaPi-BfF for ; Sun, 9 Oct 2016 20:12:54 -0700 (PDT) Received: from stpeter.im (mailhost.stpeter.im [207.210.219.225]) by ietfa.amsl.com (Postfix) with ESMTP id C0F14129468 for ; Sun, 9 Oct 2016 20:12:54 -0700 (PDT) Received: from aither.local (unknown [76.25.4.24]) (Authenticated sender: stpeter) by stpeter.im (Postfix) with ESMTPSA id 19E4F40325; Sun, 9 Oct 2016 21:15:29 -0600 (MDT) To: Sam Whited , William Fisher References: From: Peter Saint-Andre Message-ID: Date: Sun, 9 Oct 2016 21:12:52 -0600 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Archived-At: Cc: precis@ietf.org Subject: Re: [precis] PRECIS bidi rule for usernames X-BeenThere: precis@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Preparation and Comparison of Internationalized Strings List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Oct 2016 03:12:56 -0000 On 10/7/16 1:28 PM, Sam Whited wrote: > On Fri, Oct 7, 2016 at 2:00 PM, William Fisher > wrote: >> I've been working on a python implementation of the PRECIS specification >> (https:github.com/byllyfish/precis-i18n). > > Great timing; I was just this moment looking at your library and > considering including it in some interoperability tests for a set of > test vectors I'm working on. > > >> Can someone please clarify whether: >> >> A. The "Bidi rule" is ONLY applied to strings that contain right-to-left >> characters. >> B. The "Bidi rule" is applied to ALL strings. > > The recent draft-ietf-precis-7613bis-03 clarifies this: > > Directionality Rule: Apply the "Bidi Rule" defined in [RFC5893] > to strings that contain right-to-left characters (i.e., each of > the six conditions of the Bidi Rule must be satisfied); for > strings that do not contain right-to-left characters, there is no > special processing for directionality. > > I was apparently confused about this before too, I'm applying Bidi all > the time in the Go implementation. I'll be sure to add a test for this > in the test vectors when I publish them. > > I'm actually not convinced this is the correct behavior though; it > seems confusing to me that usernames with RTL characters couldn't end > with punctuation, but strings with them could. There are plenty of RTL punctuation characters (e.g., U+05BE), and those are allowed in RTL strings (even as the last character). RFC 5893 says that an RTL string must not have an LTR character at the end, with the result that an RTL string cannot end in "."; this helps to prevent confusion in the typical presentation of domain names (which is the target use case for RFC 5893). > This violates the > principal of least suprise. It's perhaps not advisable for you and I to speculate about what might surprise a user whose native language is represented in a right-to-left script. Peter From nobody Sun Oct 9 21:27:26 2016 Return-Path: X-Original-To: precis@ietfa.amsl.com Delivered-To: precis@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BB7651295D8 for ; Sun, 9 Oct 2016 21:27:24 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.701 X-Spam-Level: X-Spam-Status: No, score=-2.701 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=samwhited.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7eQ9SSkqOY4n for ; Sun, 9 Oct 2016 21:27:23 -0700 (PDT) Received: from mail-qk0-x22a.google.com (mail-qk0-x22a.google.com [IPv6:2607:f8b0:400d:c09::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4675F129452 for ; Sun, 9 Oct 2016 21:27:23 -0700 (PDT) Received: by mail-qk0-x22a.google.com with SMTP id n189so90015131qke.0 for ; Sun, 09 Oct 2016 21:27:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samwhited.com; s=swgoo; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=7YDoeoYYg68P7HShDgvmAP9Yh7odpd01SDv/ioql2RM=; b=znkzHSIXj9YTvyQmO1BJOsS2yYq2x8CWjyxZ9+9Zp4PrIMVd8t2tvaBl+ZpmZB0ITO oInNiXtwzP+X5Xaq47PYV2Wog2L8sZnhxRKxSCHC0QFcmzEZkwMXIkwHKJVXChLBMWck lr2VyYRN0FJKX1aydSVxrPu47lKD9dRfIF7Ng= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=7YDoeoYYg68P7HShDgvmAP9Yh7odpd01SDv/ioql2RM=; b=L62PVtb23Uwg7Njh3NDnM2FabGMt0rzUnkpntKTd0vvDd5rKCLLvAp3Ju287GLChXI R6UzRrbdt1/Gw0Md5uJjRIejLiby804qcFMqMocBvlBBF7TqQnt3ZTp3Vm077IlCrDs3 7qbFD1oF6DZN1fVo7ZWlPi+AWHsZO7TPGolFpCINEXhU06kcbAMlsVKeHX/lzkocFLeC n4K0+hvnzqD3EIUX8n0VmyAx4u4H6/9uPS2mIuvSMTVrJU1OS5EXK9DCznmF4Bh19vgb ACV6GNklFhCr+25Ed7CIKFPQEkRJFpXf9u9nheGtetqDHcqya9tsJ79RSTwIAigqPxQq LSzw== X-Gm-Message-State: AA6/9RmevF/2noSZfs7Il+ssJXP5AB3pZKQ6dmtpM/dyG0QqT1Au4+pqTyefakS20EM8mhz7ZzibKdDd0/ugMw== X-Received: by 10.55.188.193 with SMTP id m184mr29059006qkf.129.1476073642338; Sun, 09 Oct 2016 21:27:22 -0700 (PDT) MIME-Version: 1.0 Received: by 10.55.15.166 with HTTP; Sun, 9 Oct 2016 21:26:41 -0700 (PDT) X-Originating-IP: [2605:a601:1166:8b00:b774:1488:5007:efa9] In-Reply-To: References: From: Sam Whited Date: Sun, 9 Oct 2016 23:26:41 -0500 Message-ID: To: Peter Saint-Andre Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Archived-At: Cc: precis@ietf.org Subject: Re: [precis] PRECIS bidi rule for usernames X-BeenThere: precis@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Preparation and Comparison of Internationalized Strings List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Oct 2016 04:27:25 -0000 On Sun, Oct 9, 2016 at 10:12 PM, Peter Saint-Andre wro= te: > There are plenty of RTL punctuation characters (e.g., U+05BE), and those = are > allowed in RTL strings (even as the last character). RFC 5893 says that a= n > RTL string must not have an LTR character at the end, with the result tha= t > an RTL string cannot end in "."; this helps to prevent confusion in the > typical presentation of domain names (which is the target use case for RF= C > 5893). That makes sense; thanks for the clarification. =E2=80=94Sam --=20 Sam Whited pub 4096R/54083AE104EA7AD3 From nobody Wed Oct 12 12:56:26 2016 Return-Path: X-Original-To: precis@ietfa.amsl.com Delivered-To: precis@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CA16312958F for ; Wed, 12 Oct 2016 12:56:24 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.699 X-Spam-Level: X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id y_q4iJiAWOo1 for ; Wed, 12 Oct 2016 12:56:23 -0700 (PDT) Received: from mail-io0-x233.google.com (mail-io0-x233.google.com [IPv6:2607:f8b0:4001:c06::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 58C0C12958D for ; Wed, 12 Oct 2016 12:56:23 -0700 (PDT) Received: by mail-io0-x233.google.com with SMTP id q192so62517776iod.0 for ; Wed, 12 Oct 2016 12:56:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to; bh=qDBap/ULZgS21zpskreFtBkLH2ad3IijRNY65kdCkEo=; b=0TdJmzJ5av+bU6GO/ok2mbnBCbcZpTPZsCjJb6vcANZXdUucAPTN92UJmRzf3U4IbM BcAIcChrRf5ToeaHY/2mBKmUW//a79fGEeBp89ZE81d+raeAzP8Wo40ys9dJFR1ZINEL irZ6s4y7Yc3VzPdddFchhtzmGABX7unxURbLcbZUBlms5rpRymtbPioVaWaqiRPLV4zJ KfS4bHE8JIAS3ugFcPus6FaK1Xl8LS/skEQUUXDaGP4I/tYUFRZCckddQB09URw9FyFb FQK7Sta2lTzdJOPzIrZjnpHlGrXCR+cnbAvZ3Fe1FttEjkC0/tcUl3RTicIeI2o6AaY3 QtxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=qDBap/ULZgS21zpskreFtBkLH2ad3IijRNY65kdCkEo=; b=csZY85ETsRTeq1RgN/3s54doQgtbatKn4i/NjDwvwKOQr3AzU00nZhRBFJNfM54eum 8WzLGRJIUqcuUS6++gZpJVSvinYP46zsvWdJcpD8uWB9qpQkIt7NGNfu60q4E3d/NEtw kr41bZe8ynF+1bLFEEAqwpQuqtWPVsd0zw93bIPoYiZa9xgMEaYKoAwP0LnpbllUxWPL JKY4ouHDx+tMvHqhLWyb86ja40Zk+mEKMlr3psdpkyAaqpj1TBr1QP03wgXAwUHEDhV7 Ba/zRTd8T7wNGBJkoGeHaUqDCNYPlyBsYyyk2S1GG2C7zzkA6s+cXHBZs0wKpq/tMoVS knMw== X-Gm-Message-State: AA6/9Rmc9cRmTPRVYxm1+lHwlDvs6py5ar0vnp/De20P2Mx/0ch1zGhU6KpUxLMmRgis6GPbqaik4zuDmk58qw== X-Received: by 10.107.18.228 with SMTP id 97mr3132915ios.41.1476302182295; Wed, 12 Oct 2016 12:56:22 -0700 (PDT) MIME-Version: 1.0 Received: by 10.50.95.42 with HTTP; Wed, 12 Oct 2016 12:56:21 -0700 (PDT) From: William Fisher Date: Wed, 12 Oct 2016 12:56:21 -0700 Message-ID: To: precis@ietf.org Content-Type: multipart/alternative; boundary=001a113fbb4028360a053eb063ab Archived-At: Subject: [precis] Enforcement as an Idempotent operation X-BeenThere: precis@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Preparation and Comparison of Internationalized Strings List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Oct 2016 19:56:25 -0000 --001a113fbb4028360a053eb063ab Content-Type: text/plain; charset=UTF-8 Should enforcing a string using PRECIS be idempotent? If I apply the enforce operation to a string twice, should I get the same result as applying it just once? The nickname profile is NOT idempotent for some inputs. 1. Certain characters are NFKC normalized to sequences with ASCII spaces. This can lead to nicknames that begin with a space or contain adjacent interior spaces that are removed if you apply the nickname profile again. U+00A8 => U+0020 U+0308 => U+0308 2. Some characters can be further case mapped after NFKC normalization. U+1F11 => (K) => (k) I also noticed that the RFC 7700 has case-mapping defined only when comparing nicknames. I thought this was confusing. I didn't understand why username is split into two profiles (CasePreserved and CaseMapped), but nickname is not. If not all PRECIS profiles are idempotent, it would help to mention this in the IANA Profile registry, e.g. Idempotent: No. As an implementer, I would prefer profiles that are idempotent. Regards, Bill --001a113fbb4028360a053eb063ab Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Should enforcing a string using PRECIS be idempotent?=C2= =A0 If I apply the enforce operation to a string twice, should I get the sa= me result as applying it just once?=C2=A0

The nickn= ame profile is NOT idempotent for some inputs.

1. Ce= rtain characters are NFKC normalized to sequences with ASCII spaces. This c= an lead to nicknames that begin with a space or contain adjacent interior s= paces that are removed if you apply the nickname profile again.
=

=C2=A0 U+00A8 =C2=A0=3D> =C2=A0U+0020 U+0308 = =C2=A0=3D> =C2=A0U+0308

=C2=A02. Some characters can be further case mapped aft= er NFKC normalization.

=C2=A0 U+1F11 =C2=A0=3D>= (K) =3D> (k)

I also noticed that the RFC 7700 has case-mapping defin= ed only when comparing nicknames.=C2=A0 I thought this was confusing. I did= n't understand why username is split into two profiles (CasePreserved a= nd CaseMapped), but nickname is not.=C2=A0

If not = all PRECIS profiles are idempotent, it would help to mention this in the IA= NA Profile registry, e.g.

=C2=A0 =C2=A0Idempotent:= =C2=A0No.

As an implementer, I would prefer p= rofiles that are idempotent.

Regards,
Bi= ll
--001a113fbb4028360a053eb063ab-- From nobody Wed Oct 12 21:03:44 2016 Return-Path: X-Original-To: precis@ietfa.amsl.com Delivered-To: precis@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id ED439129818 for ; Wed, 12 Oct 2016 21:03:42 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.898 X-Spam-Level: X-Spam-Status: No, score=-4.898 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-2.996, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2VSDHZhtRZrc for ; Wed, 12 Oct 2016 21:03:41 -0700 (PDT) Received: from stpeter.im (mailhost.stpeter.im [207.210.219.225]) by ietfa.amsl.com (Postfix) with ESMTP id 4D4B6129811 for ; Wed, 12 Oct 2016 21:03:41 -0700 (PDT) Received: from aither.local (unknown [76.25.4.24]) (Authenticated sender: stpeter) by stpeter.im (Postfix) with ESMTPSA id 715084032A; Wed, 12 Oct 2016 22:06:27 -0600 (MDT) To: precis@ietf.org References: From: Peter Saint-Andre Message-ID: Date: Wed, 12 Oct 2016 22:03:29 -0600 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Archived-At: Subject: Re: [precis] Enforcement as an Idempotent operation X-BeenThere: precis@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Preparation and Comparison of Internationalized Strings List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Oct 2016 04:03:43 -0000 On 10/12/16 1:56 PM, William Fisher wrote: > Should enforcing a string using PRECIS be idempotent? As far as I know, that was not a design criterion for PRECIS. Naturally, it might be a desirable property nonetheless. > If I apply the > enforce operation to a string twice, should I get the same result as > applying it just once? > > The nickname profile is NOT idempotent for some inputs. > > 1. Certain characters are NFKC normalized to sequences with ASCII > spaces. This can lead to nicknames that begin with a space or contain > adjacent interior spaces that are removed if you apply the nickname > profile again. > > U+00A8 => U+0020 U+0308 => U+0308 That's a good example. One could argue that the leading/trailing space and adjacent interior space rules are application-specific and don't really belong in the nickname profile (indeed, I seem to recall a message to the WG about that years ago). I've been on the fence about that several times. > 2. Some characters can be further case mapped after NFKC normalization. > > U+1F11 => (K) => (k) It's not clear to me that U+1F11 has the problem you describe; perhaps could you sketch it out further? > I also noticed that the RFC 7700 has case-mapping defined only when > comparing nicknames. I thought this was confusing. I didn't understand > why username is split into two profiles (CasePreserved and CaseMapped), > but nickname is not. We try really hard not to multiply profiles beyond necessity. In this instance, we deemed acceptable not to apply the case mapping rule for enforcement (e.g., "StPeter" is a fine nickname) but would like to avoid nicknames in the same address space (e.g., a chatroom) that differ only by case because that would be confusing (e.g., "StPeter" and "stpeter"). As you suggest, we could have accomplished the same result by defining two separate profiles. > If not all PRECIS profiles are idempotent, it would help to mention this > in the IANA Profile registry, e.g. > > Idempotent: No. > > As an implementer, I would prefer profiles that are idempotent. Thanks for your input. Personally I will think about it further and post again after I do so. Peter From nobody Thu Oct 13 12:33:19 2016 Return-Path: X-Original-To: precis@ietfa.amsl.com Delivered-To: precis@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8C03B12963C for ; Thu, 13 Oct 2016 12:33:18 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.7 X-Spam-Level: X-Spam-Status: No, score=-2.7 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id At-pEMHlLgYy for ; Thu, 13 Oct 2016 12:33:16 -0700 (PDT) Received: from mail-io0-x22d.google.com (mail-io0-x22d.google.com [IPv6:2607:f8b0:4001:c06::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 030F8129622 for ; Thu, 13 Oct 2016 12:33:15 -0700 (PDT) Received: by mail-io0-x22d.google.com with SMTP id j37so97236516ioo.3 for ; Thu, 13 Oct 2016 12:33:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=Exm3uaAUtCtqtNRdX3QRHarXoAuEB8tOJSoyfasNdX4=; b=yVyXkscXVKxjFRQaunxntDUVXeJA0sgIGy8vSPhgisCE3GkMv3T5+CtXYDijiBcGAP DwPjp1fv+nIzsV5Ondc7RF8C60w0bHJ34OJUC+cNtPPTuEaGRlsnm6B1ZRDl5vQzQ3AF axO2fPCgxxrFmY6jLnqZ/BK4+pcJ5+rwJwUVWZsE3OhhOb0fmXJ+o6iKuW9nVjvT0TzD xHqknkY/Az/2YWS56rLuHXZJKfLO+TWoGqY68Ed4cA2rutsc/8aP/QhljEk6Q6iZP6l1 ok59U7tTsavC1/E0SzmrPHXqGZX12LmnlMMwUQNSptMqLbba45NS1jEDNlGeUvynNpMV lhLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=Exm3uaAUtCtqtNRdX3QRHarXoAuEB8tOJSoyfasNdX4=; b=GQslTcp5kotPuZx9s2opW4AkjfDfznSux1Y6sMpjUFftcsrbkk2xuBaIe+ezkJbVrO ifJB1MLdOodsbMP5rh0mi6mrBNGVonyayYCUbgtGe/fe1i0niJ5ACGwxsxBDTNd0Oth+ Q+OXzYrzq1AkznxkvggyMPQr85FalT9QTFsHzgRZ/atEMM5zgRxGyE9ZIY79lm+rs7Sw uW0m3f+9DmnPy7t0PdrinRovyZEVzsTcNE6igGmGSgiCbN2vJwqc1EF781mvViAl93pn gw8FW+1u1GK01sz1L2mdQomqzgcjv9kBZ8mlfTvcBiLPVWqViNUSJQi7m+iMV/fsCge+ iMpg== X-Gm-Message-State: AA6/9RmMBEqBtDjV9ahvJIU6k9xSZidIp0BNcJfpxsqci18eZEjs2U4gFnzwQgXS6OyhA83RHn2IEvujYAhYCg== X-Received: by 10.107.20.199 with SMTP id 190mr8710168iou.214.1476387195211; Thu, 13 Oct 2016 12:33:15 -0700 (PDT) MIME-Version: 1.0 Received: by 10.50.95.42 with HTTP; Thu, 13 Oct 2016 12:33:14 -0700 (PDT) In-Reply-To: References: From: William Fisher Date: Thu, 13 Oct 2016 12:33:14 -0700 Message-ID: To: Peter Saint-Andre Content-Type: text/plain; charset=UTF-8 Archived-At: Cc: precis@ietf.org Subject: Re: [precis] Enforcement as an Idempotent operation X-BeenThere: precis@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Preparation and Comparison of Internationalized Strings List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Oct 2016 19:33:18 -0000 On Wed, Oct 12, 2016 at 9:03 PM, Peter Saint-Andre wrote: > It's not clear to me that U+1F11 has the problem you describe; perhaps could you sketch it out further? Oops, that should be U+0001F11A. The full example is: "\U0001f11aevin" => "(K)evin" => "(k)evin" I wrote a program to categorize characters that are not idempotent under Nickname "ToLower" (ignoring white space). The numbers are the same for Unicode 6.3, 8.0 and 9.0. { '': 467, '': 90, '': 35, '': 27, '': 4 } The following two characters also appear to fail the idempotent test. The initial decompositions do not begin with '<'. \u03d3 GREEK UPSILON WITH ACUTE AND HOOK SYMBOL \u03d4 GREEK UPSILON WITH DIAERESIS AND HOOK SYMBOL > Thanks for your input. Personally I will think about it further and post again after I do so. To me, the problem is to take untrusted input, validate it using specified rules, and transform it into a stable, unambiguous format. I'm still learning more about Unicode. Is there a reason that the case mapping rule has to be applied *before* the normalization rule? The order appears to make a difference for NFKC. I suppose the Nickname "comparison" profile could re-apply the case mapping rule after the normalization rule? Thanks, -Bill