From nobody Tue Aug 27 10:48:32 2019 Return-Path: X-Original-To: ltru@ietfa.amsl.com Delivered-To: ltru@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7A6361200C7 for ; Tue, 27 Aug 2019 10:48:30 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.897 X-Spam-Level: X-Spam-Status: No, score=-1.897 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 03SGakpd5rz1 for ; Tue, 27 Aug 2019 10:48:29 -0700 (PDT) Received: from p3plwbeout03-04.prod.phx3.secureserver.net (p3plsmtp03-04-2.prod.phx3.secureserver.net [72.167.218.216]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4AF83120045 for ; Tue, 27 Aug 2019 10:48:29 -0700 (PDT) Received: from p3plgemwbe03-04.prod.phx3.secureserver.net ([72.167.218.132]) by :WBEOUT: with SMTP id 2fZVizHq8zvR72fZViHNbR; Tue, 27 Aug 2019 10:47:57 -0700 X-SID: 2fZVizHq8zvR7 Received: (qmail 25828 invoked by uid 99); 27 Aug 2019 17:47:57 -0000 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" X-Originating-IP: 208.51.143.189 User-Agent: Workspace Webmail 6.9.59 Message-Id: <20190827104755.665a7a7059d7ee80bb4d670165c8327d.0f79efb126.wbe@email03.godaddy.com> From: "Doug Ewell" To: ltru@ietf.org Date: Tue, 27 Aug 2019 10:47:55 -0700 Mime-Version: 1.0 X-CMAE-Envelope: MS4wfJtXrSDpW7hjocOyIW24C5xx2tN7doqWf8Q9hGg3XfJJUTYKBfxLGJTOeSO+jkjiljxHv+DdSGXi9Ra6kopE43FIz8yqitlBynWTKC5XZXDBtX2IW6oQ BseAtxS1Dse0irfLJ05sbaxXhw5TlwUCzkh05BXZxqWjdpAkXmGfAZwe8lIJfBe/Ne6jyYi20Y1ueuIaB28m0N16dOCyxIqTPhw= Archived-At: Subject: Re: [Ltru] Mail regarding draft-ietf-ltru-4646bis and draft-ietf-ltru-matching X-BeenThere: ltru@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Language Tag Registry Update working group discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Aug 2019 17:48:31 -0000 On July 27, Florian Rivoal wrote:=0A =0A> However, RFC5646 Section 4.5, whi= ch defines canonicalization, only=0A> does so for language tags, not for la= nguage ranges. Presumably, the=0A> process is largely the same, with wildca= rds in the language subtag=0A> being preserved, and I suppose wildcards in = other subtags would likely=0A> be dropped. But as it stands, that seems und= efined.=0A =0AI think you are on the right track by assuming that ranges ar= e=0Acanonicalized just like tags, with asterisks left alone.=0A =0AIt's not= very likely that most LTRU participants will be eager to start=0Aup a new = IETF project to update 5646 for something like this. Best to go=0Awith your= assumption.=0A =0A> Also, while giving recommendations about canonicalizat= ion for the=0A> purpose of filtering, it would seem useful to mention (and = possibly to=0A> recommend) canonicalizing to the "extlang form". The defini= tion of the=0A> extlang form itself (in RFC5646 Section 4.5) mentions that= it is=0A> useful for matching and selecting, but that information isn't re= layed=0A> anywhere RFC4647.=0A =0AAt the time these documents were written,= there was a strong sentiment=0Aaround de-emphasizing extlangs in general. = It's good to know that=0Athere's a real-world use case for using them here.= Again, it's unlikely=0Athat people will want to rev 4647 for this.=0A =0A-= -=0ADoug Ewell | Thornton, CO, US | ewellic.org=0A From nobody Wed Aug 28 02:56:51 2019 Return-Path: X-Original-To: ltru@ietfa.amsl.com Delivered-To: ltru@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DAFD91200B4 for ; Wed, 28 Aug 2019 02:56:49 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.7 X-Spam-Level: X-Spam-Status: No, score=-2.7 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=rivoal.net header.b=gPHVdwwe; dkim=pass (2048-bit key) header.d=messagingengine.com header.b=RS1rQnoB Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id t1sN4wPEcVEa for ; Wed, 28 Aug 2019 02:56:47 -0700 (PDT) Received: from out2-smtp.messagingengine.com (out2-smtp.messagingengine.com [66.111.4.26]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 44514120096 for ; Wed, 28 Aug 2019 02:56:47 -0700 (PDT) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id 6A80621F15; Wed, 28 Aug 2019 05:56:45 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Wed, 28 Aug 2019 05:56:45 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivoal.net; h= content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; s=fm1; bh=b xWb0JVBYSK/E0KMfWvQIo68gRSz1/EU7ItvaNy26Pg=; b=gPHVdwweLHO+Xrrjw v33C0k5OpphC27kjPXbvffXUamYnyIE6YTcvU+NyC00HI2AbEBEbfAba4bIY5EOl vsO/CDGR/bewf+OVtwT4kGZYCeBlENsxbsg3r7+E+w7N2c2B+/pu2bhbkTMDI+AC b5FGCXOJWYPUxWBdsuTuJbJ/DbssoZ5iUz4bd/WS+HIFx1u57IHm+a+6Qy+V+rUO /TmjVw9mXO63UXRBcQQ3kiNtC8YtpUeOO4RJ/ENnlpCTLCpmRQQS3JQaS/4VmV4K s1w946QNy7gq+oAMJa4mI6WO6gc62/Y1kQZWCZaw7csc+0ACkG2U2uS19bIburFT fvGBg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; bh=bxWb0JVBYSK/E0KMfWvQIo68gRSz1/EU7ItvaNy26 Pg=; b=RS1rQnoBPHZ6YPJisUJ6zzfVOKrzPsKloSUsHC/AcNKeinlHoDDGKC8vG 5r086JzfXxsVZp5VX6Ejl9es6kiAVynER80BR9DwMUdf4KPX1oHOo71XTjLbydqQ o8I6Olq2BaRahFKFeZsvsZWlWq3UaVO4wb5xKGylfMBqoz5F1xAR1f2yjT1VQ6PX YUx4BPSn2r4bJmwpGkr4sAkrD5HIgYfxEmqlethAqj5TfjSArIqUpoQdKBkwFRKd qRFh4sxx9sfeO20gF8gxNCylYUW5XogemjzXRuI/3Rx44eZgTWc5hHJbWIt6Tfm+ pbJ74/Ocl/G/Qekzfqf1vp6gy72Hw== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduvddrudeitddgvddvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurheptggguffhjgffgffkfhfvofesthhqmhdthhdtjeenucfhrhhomhephfhlohhr ihgrnhcutfhivhhorghluceofhhlohhrihgrnhesrhhivhhorghlrdhnvghtqeenucffoh hmrghinhepihgrnhgrrdhorhhgnecukfhppedukeefrdejiedrhedrvddtnecurfgrrhgr mhepmhgrihhlfhhrohhmpehflhhorhhirghnsehrihhvohgrlhdrnhgvthenucevlhhush htvghrufhiiigvpedt X-ME-Proxy: Received: from [192.168.1.3] (ab005020.dynamic.ppp.asahi-net.or.jp [183.76.5.20]) by mail.messagingengine.com (Postfix) with ESMTPA id F0A7CD6005D; Wed, 28 Aug 2019 05:56:43 -0400 (EDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) From: Florian Rivoal In-Reply-To: <20190827104755.665a7a7059d7ee80bb4d670165c8327d.0f79efb126.wbe@email03.godaddy.com> Date: Wed, 28 Aug 2019 18:56:40 +0900 Cc: ltru@ietf.org Content-Transfer-Encoding: quoted-printable Message-Id: <910CB6C8-9F66-4255-B149-B146DA8E5695@rivoal.net> References: <20190827104755.665a7a7059d7ee80bb4d670165c8327d.0f79efb126.wbe@email03.godaddy.com> To: Doug Ewell X-Mailer: Apple Mail (2.3445.104.11) Archived-At: Subject: Re: [Ltru] Mail regarding draft-ietf-ltru-4646bis and draft-ietf-ltru-matching X-BeenThere: ltru@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Language Tag Registry Update working group discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Aug 2019 09:56:50 -0000 > On Aug 28, 2019, at 2:47, Doug Ewell wrote: >=20 > On July 27, Florian Rivoal wrote: >=20 >> However, RFC5646 Section 4.5, which defines canonicalization, only >> does so for language tags, not for language ranges. Presumably, the >> process is largely the same, with wildcards in the language subtag >> being preserved, and I suppose wildcards in other subtags would = likely >> be dropped. But as it stands, that seems undefined. >=20 > I think you are on the right track by assuming that ranges are > canonicalized just like tags, with asterisks left alone. Thanks for confirming. > It's not very likely that most LTRU participants will be eager to = start > up a new IETF project to update 5646 for something like this. Best to = go > with your assumption. >=20 >> Also, while giving recommendations about canonicalization for the >> purpose of filtering, it would seem useful to mention (and possibly = to >> recommend) canonicalizing to the "extlang form". The definition of = the >> extlang form itself (in RFC5646 Section 4.5) mentions that it is >> useful for matching and selecting, but that information isn't relayed >> anywhere RFC4647. >=20 > At the time these documents were written, there was a strong sentiment > around de-emphasizing extlangs in general. It's good to know that > there's a real-world use case for using them here. Again, it's = unlikely > that people will want to rev 4647 for this. The use case is CSS selectors, when writing rules for typography/styling = in a document. On the one hand, the document gets marked up which part = of it are in which language. On the other side, the style sheet = describes which part of the document must be styled which way, and can = make that styling dependent on the language. The need for normalization comes from the fact that stylesheet authoring = and document authoring are not coordinated in the general case, so a = stylesheet author cannot know, generally speaking, if a document will be = marked up with, for example, zh-yue or yue. The stylesheet author is = then faced with two options, both unattractive for different reasons: * use the deprecated tag: it's more likely to be found in existing = documents due to being older. The first downside is that it doesn't = always work. The second one is that this slows down adoption of the = newer preferred tag, as document authors wanting to be compatible with = existing stylesheets will keep on using the deprecated one as well for = compatibility, and we get into a vicious cycle of everybody continuing = to use the deprecated variant. * Use both the deprecated and the preferred tag in the stylesheet's = selector. This works, but it means that stylesheet authors need to be = aware of, and manually replicate the information in = https://www.iana.org/assignments/language-subtag-registry/language-subtag-= registry. Asking people to manually do what software could isn't great, = as they tend not to, or to do it with bugs, or to not update when the = registry updates, etc. So it seems preferable, given that this correspondence is maintained in = a neatly usable format, to have CSS renderers deal with the = correspondence between deprecated and preferred tags by way of = canonicalizing to the extlang form and doing the selector matching on = that. In the long run, both document authors and stylesheet authors should use = the preferred tag without the extlang prefix, and the canonicalization = to extang form will be invisible to them. But even if some don't, = everything works. =E2=80=94Florian= From nobody Wed Aug 28 09:52:21 2019 Return-Path: X-Original-To: ltru@ietfa.amsl.com Delivered-To: ltru@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4326C120804 for ; Wed, 28 Aug 2019 09:52:19 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.897 X-Spam-Level: X-Spam-Status: No, score=-1.897 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id j8Q9kAYFCwM7 for ; Wed, 28 Aug 2019 09:52:17 -0700 (PDT) Received: from p3plwbeout03-02.prod.phx3.secureserver.net (p3plsmtp03-02-2.prod.phx3.secureserver.net [72.167.218.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 9324B120803 for ; Wed, 28 Aug 2019 09:52:17 -0700 (PDT) Received: from p3plgemwbe03-04.prod.phx3.secureserver.net ([72.167.218.132]) by :WBEOUT: with SMTP id 31AgiyZKKS4QU31AgiNfoc; Wed, 28 Aug 2019 09:51:46 -0700 X-SID: 31AgiyZKKS4QU Received: (qmail 195313 invoked by uid 99); 28 Aug 2019 16:51:46 -0000 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" X-Originating-IP: 208.51.143.189 User-Agent: Workspace Webmail 6.9.59 Message-Id: <20190828095144.665a7a7059d7ee80bb4d670165c8327d.1a210b4b97.wbe@email03.godaddy.com> From: "Doug Ewell" To: "Florian Rivoal" Cc: ltru@ietf.org Date: Wed, 28 Aug 2019 09:51:44 -0700 Mime-Version: 1.0 X-CMAE-Envelope: MS4wfB+0byMQ2vYl3d9949YBox7dMhHyObV3otQFT9OdZUMSNhKM48mxzMJZOCDm6DHZI96v/dXVk313VtwhKBrV7j4/GHoFLovpLIVkqS5dL7ooEKVf2VWA /mKS8Qv2EUTW1/L5VEWga4EjiBdDCqMxH6zEbuWyqe9Kczdqb7lU3RULDPCBCEkT7cRU91doHQT6q9ZE19+bRi7qlwLrSP7Sf/0= Archived-At: Subject: Re: [Ltru] Mail regarding draft-ietf-ltru-4646bis and draft-ietf-ltru-matching X-BeenThere: ltru@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Language Tag Registry Update working group discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Aug 2019 16:52:19 -0000 Florian Rivoal wrote:=0A =0A> The stylesheet author is then faced with two = options, both=0A> unattractive for different reasons:=0A>=0A> * use the dep= recated tag: [...]=0A>=0A> * Use both the deprecated and the preferred tag = in the stylesheet's=0A> selector. [...]=0A =0AExtlang subtags, and tags lik= e "zh-yue" that use them, aren't=0Adeprecated. Extlang subtags have a Prefe= rred-Value, but they don't have=0Aa Deprecated value. While this is a subtl= e distinction, "deprecated" is=0Aa term of art in BCP 47. It might be bette= r to use a more unwieldy term=0Alike "non-preferred."=0A =0ASection 3.1.7 s= ays:=0A =0A"For records of type 'extlang', the 'Preferred-Value' field appe= ars=0Awithout a corresponding 'Deprecated' field. An implementation MAY=0A= ignore these preferred value mappings, although if it ignores the=0Amapping= , it SHOULD do so consistently. It SHOULD also treat the=0A'Preferred-Valu= e' as equivalent to the mapped item. For example, the=0Atags "zh-yue-Hant-= HK" and "yue-Hant-HK" are semantically equivalent and=0Aought to be treated= as if they were the same tag."=0A =0AIt does go on to say:=0A =0A"The 'Pre= ferred-Value' field in subtag records of type "extlang" also=0Acontains an = "extended language range". This allows the subtag to be=0Adeprecated in fa= vor of either a single primary language subtag or a new=0Alanguage-extlang = sequence."=0A =0Abut this is confusing to me, as I'm not sure who does the = "deprecating"=0Ahere.=0A =0A--=0ADoug Ewell | Thornton, CO, US | ewellic.or= g=0A From nobody Wed Aug 28 13:34:02 2019 Return-Path: X-Original-To: ltru@ietfa.amsl.com Delivered-To: ltru@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 700A012004C for ; Wed, 28 Aug 2019 13:33:54 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.898 X-Spam-Level: X-Spam-Status: No, score=-1.898 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=ccil-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eUqPKUjS2isR for ; Wed, 28 Aug 2019 13:33:52 -0700 (PDT) Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 7D0EB120059 for ; Wed, 28 Aug 2019 13:33:52 -0700 (PDT) Received: by mail-wm1-x32c.google.com with SMTP id d16so1450543wme.2 for ; Wed, 28 Aug 2019 13:33:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ccil-org.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=IByt3XGji7oC4p4+IPF3Ot5c1gqtqVVnz8r0diG3PSI=; b=xSwnkDVW39TBp8jdu5J7xViG5C2fKcnbPnI1TDn2+PpxfSZS6yTNE8QPVyp0aFTP/Z ob3pOpBixn/bIhR3ZevEOm3n7djTyOCwz9hfil2VsFeUHYIwGtfj8keKF+mlL1LuMVwV 4tRdChrU0PnNkWfMmeSP8HxTGJGy208pNAsejSY38m/yW17K8ty2bRwUM3KMryhgOW7H gsZS/xlvRnwVoxXWFQF3MWgPQmBVMgXOOn5tAWP4Y9rDz+pUZAireG05AHEH0lMTT+un e5UHMHQe1xLfxdfkZtORcE91vO/o0qM57vKsM4rjWWLtq8gw4RI4YHMBP+q2lhd1qEmX FNVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=IByt3XGji7oC4p4+IPF3Ot5c1gqtqVVnz8r0diG3PSI=; b=pJ7nZWYqI74svXYXQXofX5sJyiPmfejWVK0m2QX5+1atgy3OYunuADuAj8lnt1MPjZ FHnIis0NeFkJTHR7Zeki89ubJv+4FHsvZsqpKyYE4qwTWtkygviitKCQ3oUerXYBu6a8 6K//VoF/QqflW0R+JfU1mjX6rV+aW7t1rwEruEbaEZfVHoLcCEAlv0QLxemVLeOw4iKE FH9E34yQq/o4pmqOeCF9EV6BkeZj2YUBWZRX1uCXHh4p+dWCiUkB+BpczRPshtCQHJ/+ 4QW6lbcdLeQ4PbJ21nOPjTW3ZaNvR7YbT1Mdv/WmdbWzEZWWbVmkrugC1RngICnXhBmh nvKQ== X-Gm-Message-State: APjAAAUpviUyfFql+CzXeeGUub9UQHoBKKdTWiYJ8nP0GneNiiBVDoEQ pLbGTefwtk4xomf9CjOpaxtpEwqlHmny9TIiWo9qcUkVRl4= X-Google-Smtp-Source: APXvYqxLRSw71ECkU4Bz+M53dilF8N5v016vL23FLkCZqVF5PlRc1A2KYuYX0CkhwYnP8KfB1sfJW7fwEZrI6D3WJL4= X-Received: by 2002:a1c:9c45:: with SMTP id f66mr7211467wme.38.1567024430805; Wed, 28 Aug 2019 13:33:50 -0700 (PDT) MIME-Version: 1.0 References: <20190828095144.665a7a7059d7ee80bb4d670165c8327d.1a210b4b97.wbe@email03.godaddy.com> In-Reply-To: <20190828095144.665a7a7059d7ee80bb4d670165c8327d.1a210b4b97.wbe@email03.godaddy.com> From: John Cowan Date: Wed, 28 Aug 2019 16:33:38 -0400 Message-ID: To: Doug Ewell Cc: Florian Rivoal , ltru@ietf.org Content-Type: multipart/alternative; boundary="0000000000008d92070591334e07" Archived-At: Subject: Re: [Ltru] Mail regarding draft-ietf-ltru-4646bis and draft-ietf-ltru-matching X-BeenThere: ltru@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Language Tag Registry Update working group discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Aug 2019 20:33:54 -0000 --0000000000008d92070591334e07 Content-Type: text/plain; charset="UTF-8" On Wed, Aug 28, 2019 at 12:52 PM Doug Ewell wrote: "The 'Preferred-Value' field in subtag records of type "extlang" also > contains an "extended language range". This allows the subtag to be > deprecated in favor of either a single primary language subtag or a new > language-extlang sequence." > I think this means that if SIL decides to change a primary language tag that is also an extlang tag, we can deprecate the old extlang tag as well. Another scenario would be that they decide some language doesn't belong in its macrolanguage, and we need to deprecate the extlang tag while leaving the primary language tag alone. Hopefully neither of these will ever happen. John Cowan http://vrici.lojban.org/~cowan cowan@ccil.org "Any legal document draws most of its meaning from context. A telegram that says 'SELL HUNDRED THOUSAND SHARES IBM SHORT' (only 190 bits in 5-bit Baudot code plus appropriate headers) is as good a legal document as any, even sans digital signature." --me --0000000000008d92070591334e07 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Wed, Aug 28, 2019 at 12:52 PM Doug= Ewell <doug@ewellic.org> wro= te:

"The 'Preferred-Value' fie= ld in subtag records of type "extlang" also
contains an "extended language range".=C2=A0 This allows the subt= ag to be
deprecated in favor of either a single primary language subtag or a new
language-extlang sequence."

I thin= k this means that if SIL decides to change a primary language tag that is a= lso an extlang tag, we can deprecate the old extlang tag as well.=C2=A0 Ano= ther scenario would be that they decide some language doesn't belong in= its macrolanguage, and we need to deprecate the extlang tag while leaving = the primary language tag alone.=C2=A0 =C2=A0Hopefully neither of these will= ever happen.


John Cowan =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0http://v= rici.lojban.org/~cowan =C2=A0 =C2=A0 =C2=A0 =C2=A0cowan@ccil.org
=C2=A0 =C2=A0 "Any legal document d= raws most of its meaning from context.=C2=A0 A telegram
=C2=A0 =C2=A0 th= at says 'SELL HUNDRED THOUSAND SHARES IBM SHORT' (only 190 bits in<= br>=C2=A0 =C2=A0 5-bit Baudot code plus appropriate headers) is as good a l= egal document
=C2=A0 =C2=A0 as any, even sans digital signature." -= -me

--0000000000008d92070591334e07-- From nobody Wed Aug 28 23:19:41 2019 Return-Path: X-Original-To: ltru@ietfa.amsl.com Delivered-To: ltru@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0345A1201E5 for ; Wed, 28 Aug 2019 23:19:40 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.388 X-Spam-Level: X-Spam-Status: No, score=-1.388 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.249, FREEMAIL_FROM=0.001, FROM_EXCESS_BASE64=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.249, HTML_FONT_FACE_BAD=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_KAM_HTML_FONT_INVALID=0.01] autolearn=no autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=macchiato-com.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2IGlw4NDy9WV for ; Wed, 28 Aug 2019 23:19:36 -0700 (PDT) Received: from mail-ot1-x333.google.com (mail-ot1-x333.google.com [IPv6:2607:f8b0:4864:20::333]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6A3D1120020 for ; Wed, 28 Aug 2019 23:19:36 -0700 (PDT) Received: by mail-ot1-x333.google.com with SMTP id 100so2346239otn.2 for ; Wed, 28 Aug 2019 23:19:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=macchiato-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=d4vJSUeEcU8ZpBUz5kqJ7JBo7dovjtlqihDQxPCi4ps=; b=jBvgl3+bKknzkmCKoWGAnN02shkuujHZht8QK4jbRdxAyNVlYA1u989N7mNuDyYWTa Gzk4mLrVMhFUNDl6VppMDTucup2NEnuwGW0z6iQDIO/3tj9HsX8iH4SJ7VmWUTUVStJA KiWm7byV1ujx3s7F3yvoEwUH5LXnLgt2NGZBKwlaI3/d32L2Ayhlp+1agnY2RLyzy1C9 bZr2wBJT1+BKaYELs5tj4FjG166O6ASBirhNU/QL6X7ymId48SQHePtDvhGoID/zjvFp hMDqUCDehMx8UgWo+fSIYkzIU2x6N+0ysbD+qAtOpxY+mDMIjaOd/Zn+XBUedQPv9K8U 27Fw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=d4vJSUeEcU8ZpBUz5kqJ7JBo7dovjtlqihDQxPCi4ps=; b=MfeNLwvRt8gWqtDG2HowSXka8NaArv8/YcxkalY7Pqzny+6KXv2sd/8eT6sQtar3dy VRW0OLmshFeCg5MxPnhJO3rq+VQzrthytggaBXIwj65i6/nnof+FRwnhb9NXp/3mF3qX zRZHbdE6+dv4NuljnxaY+oJ2kX4XF+N9o5QkFyUe+CF9QvbSfg1aVD4WzQyD3KdbsndM IX57zgqSuiAsv14miFnFwYK8K1abCygpYvwVINI0TlR5DMDGkTfffR1zvSngTScVF72y KJEaLCratC312lY0zb210j4GlGAWRXGd8i1VmqQH1wiMNrXfUScUw6yEP1QLbQwv2Apr 9cLg== X-Gm-Message-State: APjAAAVMObClSEZJ2I1BwSd/7hqnKb0m1TE2VCxKKu84IfeIAJm9by3d KJsyJSij+NR+b28A/H/0tDn3TRinAbvJbjucMo3hzsfp X-Google-Smtp-Source: APXvYqwy/r4mtons8+OeJd17gHzYzbRZ9w7/sx8gBvqeFy2XPoY22VcG7dzgpPzxGZ92FDOOArm9MeZRIhPXS9Fd/1A= X-Received: by 2002:a9d:909:: with SMTP id 9mr6468867otp.261.1567059575351; Wed, 28 Aug 2019 23:19:35 -0700 (PDT) MIME-Version: 1.0 References: <20190827104755.665a7a7059d7ee80bb4d670165c8327d.0f79efb126.wbe@email03.godaddy.com> <910CB6C8-9F66-4255-B149-B146DA8E5695@rivoal.net> In-Reply-To: <910CB6C8-9F66-4255-B149-B146DA8E5695@rivoal.net> From: =?UTF-8?B?TWFyayBEYXZpcyDimJXvuI8=?= Date: Thu, 29 Aug 2019 08:19:24 +0200 Message-ID: To: Florian Rivoal Cc: Doug Ewell , LTRU Working Group Content-Type: multipart/alternative; boundary="00000000000054aed605913b7d29" Archived-At: Subject: Re: [Ltru] Mail regarding draft-ietf-ltru-4646bis and draft-ietf-ltru-matching X-BeenThere: ltru@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Language Tag Registry Update working group discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Aug 2019 06:19:40 -0000 --00000000000054aed605913b7d29 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Canonicalizing to the extlang form has a number of disadvantages, and I would recommend strongly against it. Don't have time to discuss now, will see about next week. Mark On Wed, Aug 28, 2019 at 11:56 AM Florian Rivoal wrote: > > > > On Aug 28, 2019, at 2:47, Doug Ewell wrote: > > > > On July 27, Florian Rivoal wrote: > > > >> However, RFC5646 Section 4.5, which defines canonicalization, only > >> does so for language tags, not for language ranges. Presumably, the > >> process is largely the same, with wildcards in the language subtag > >> being preserved, and I suppose wildcards in other subtags would likely > >> be dropped. But as it stands, that seems undefined. > > > > I think you are on the right track by assuming that ranges are > > canonicalized just like tags, with asterisks left alone. > > Thanks for confirming. > > > It's not very likely that most LTRU participants will be eager to start > > up a new IETF project to update 5646 for something like this. Best to g= o > > with your assumption. > > > >> Also, while giving recommendations about canonicalization for the > >> purpose of filtering, it would seem useful to mention (and possibly to > >> recommend) canonicalizing to the "extlang form". The definition of the > >> extlang form itself (in RFC5646 Section 4.5) mentions that it is > >> useful for matching and selecting, but that information isn't relayed > >> anywhere RFC4647. > > > > At the time these documents were written, there was a strong sentiment > > around de-emphasizing extlangs in general. It's good to know that > > there's a real-world use case for using them here. Again, it's unlikely > > that people will want to rev 4647 for this. > > The use case is CSS selectors, when writing rules for typography/styling > in a document. On the one hand, the document gets marked up which part of > it are in which language. On the other side, the style sheet describes > which part of the document must be styled which way, and can make that > styling dependent on the language. > > The need for normalization comes from the fact that stylesheet authoring > and document authoring are not coordinated in the general case, so a > stylesheet author cannot know, generally speaking, if a document will be > marked up with, for example, zh-yue or yue. The stylesheet author is then > faced with two options, both unattractive for different reasons: > * use the deprecated tag: it's more likely to be found in existing > documents due to being older. The first downside is that it doesn't alway= s > work. The second one is that this slows down adoption of the newer > preferred tag, as document authors wanting to be compatible with existing > stylesheets will keep on using the deprecated one as well for > compatibility, and we get into a vicious cycle of everybody continuing to > use the deprecated variant. > > * Use both the deprecated and the preferred tag in the stylesheet's > selector. This works, but it means that stylesheet authors need to be awa= re > of, and manually replicate the information in > https://www.iana.org/assignments/language-subtag-registry/language-subtag= -registry. > Asking people to manually do what software could isn't great, as they ten= d > not to, or to do it with bugs, or to not update when the registry updates= , > etc. > > So it seems preferable, given that this correspondence is maintained in a > neatly usable format, to have CSS renderers deal with the correspondence > between deprecated and preferred tags by way of canonicalizing to the > extlang form and doing the selector matching on that. > > In the long run, both document authors and stylesheet authors should use > the preferred tag without the extlang prefix, and the canonicalization to > extang form will be invisible to them. But even if some don't, everything > works. > > =E2=80=94Florian > _______________________________________________ > Ltru mailing list > Ltru@ietf.org > https://www.ietf.org/mailman/listinfo/ltru > --00000000000054aed605913b7d29 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Canonicalizing to the extlang form has a number of disadvant= ages, and I would recommend strongly against it. Don't have time to dis= cuss now, will see about next week.

<= div>
Mark
=
=


On Wed, Aug 28, 2019 at 11:56 AM Florian Rivoal <<= a href=3D"mailto:florian@rivoal.net">florian@rivoal.net> wrote:
<= /div>


> On Aug 28, 2019, at 2:47, Doug Ewell <doug@ewellic.org> wrote:
>
> On July 27, Florian Rivoal wrote:
>
>> However, RFC5646 Section 4.5, which defines canonicalization, only=
>> does so for language tags, not for language ranges. Presumably, th= e
>> process is largely the same, with wildcards in the language subtag=
>> being preserved, and I suppose wildcards in other subtags would li= kely
>> be dropped. But as it stands, that seems undefined.
>
> I think you are on the right track by assuming that ranges are
> canonicalized just like tags, with asterisks left alone.

Thanks for confirming.

> It's not very likely that most LTRU participants will be eager to = start
> up a new IETF project to update 5646 for something like this. Best to = go
> with your assumption.
>
>> Also, while giving recommendations about canonicalization for the<= br> >> purpose of filtering, it would seem useful to mention (and possibl= y to
>> recommend) canonicalizing to the "extlang form". The def= inition of the
>> extlang form itself (in=C2=A0 RFC5646 Section 4.5) mentions that i= t is
>> useful for matching and selecting, but that information isn't = relayed
>> anywhere RFC4647.
>
> At the time these documents were written, there was a strong sentiment=
> around de-emphasizing extlangs in general. It's good to know that<= br> > there's a real-world use case for using them here. Again, it's= unlikely
> that people will want to rev 4647 for this.

The use case is CSS selectors, when writing rules for typography/styling in= a document. On the one hand, the document gets marked up which part of it = are in which language. On the other side, the style sheet describes which p= art of the document must be styled which way, and can make that styling dep= endent on the language.

The need for normalization comes from the fact that stylesheet authoring an= d document authoring are not coordinated in the general case, so a styleshe= et author cannot know, generally speaking, if a document will be marked up = with, for example, zh-yue or yue. The stylesheet author is then faced with = two options, both unattractive for different reasons:
* use the deprecated tag: it's more likely to be found in existing docu= ments due to being older. The first downside is that it doesn't always = work. The second one is that this slows down adoption of the newer preferre= d tag, as document authors wanting to be compatible with existing styleshee= ts will keep on using the deprecated one as well for compatibility, and we = get into a vicious cycle of everybody continuing to use the deprecated vari= ant.

* Use both the deprecated and the preferred tag in the stylesheet's sel= ector. This works, but it means that stylesheet authors need to be aware of= , and manually replicate the information in https://www.iana.org/assignments/language-subtag-= registry/language-subtag-registry. Asking people to manually do what so= ftware could isn't great, as they tend not to, or to do it with bugs, o= r to not update when the registry updates, etc.

So it seems preferable, given that this correspondence is maintained in a n= eatly usable format, to have CSS renderers deal with the correspondence bet= ween deprecated and preferred tags by way of canonicalizing to the extlang = form and doing the selector matching on that.

In the long run, both document authors and stylesheet authors should use th= e preferred tag without the extlang prefix, and the canonicalization to ext= ang form will be invisible to them. But even if some don't, everything = works.

=E2=80=94Florian
_______________________________________________
Ltru mailing list
Ltru@ietf.org
https://www.ietf.org/mailman/listinfo/ltru
--00000000000054aed605913b7d29-- From nobody Thu Aug 29 02:00:26 2019 Return-Path: X-Original-To: ltru@ietfa.amsl.com Delivered-To: ltru@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 78C87120103 for ; Thu, 29 Aug 2019 02:00:23 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.701 X-Spam-Level: X-Spam-Status: No, score=-2.701 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=rivoal.net header.b=YQ3s75gE; dkim=pass (2048-bit key) header.d=messagingengine.com header.b=Cdnzvq10 Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id oFSLaKtl_gD4 for ; Thu, 29 Aug 2019 02:00:21 -0700 (PDT) Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id DFECA120020 for ; Thu, 29 Aug 2019 02:00:20 -0700 (PDT) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id C60F521EC3; Thu, 29 Aug 2019 05:00:19 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Thu, 29 Aug 2019 05:00:19 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivoal.net; h= content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; s=fm1; bh=o 6Doyjy2JxIh1Ipn5I/KKlOZwLEwdWjOhSjgcfx4ZiA=; b=YQ3s75gEAuA1ayVwZ vgwgt6U+wWw/BZtmMHl1RhdhXU/BFmQnV72W0UWJYER1h32xbkHOSOaCh87qBohC FzG59SJmiCdel/kXwB8kCg+9A0juu276i0LrSqYnVUWgpY64GYB5MWZ7lSifPPjP sQaeMuaow+9GPF3euPCCiU8/0bA8PAQ3fRrUZn+FhujUDT51D0joPml3PC9o+9Cs 9qftLrce0rT3r9PGMg4OiyiW0FH4zJirUaZN4eIKuw1BBz7p1jQeSS4X7lG7MMdX +6YRIUTS1bXGwiEM056P62LVxC18MHPDHm9AHwTZnCmu2NT9j6C4sYXFT4RauvFy +HiWw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; bh=o6Doyjy2JxIh1Ipn5I/KKlOZwLEwdWjOhSjgcfx4Z iA=; b=Cdnzvq10DGEDRwYcJTIFaUfoW8EIeJk0uB5zG/FH99+exBz87Xxw1loie 4JO/VhCKEhLuijWvunc/lA5d0gW7q/K8ptQVJlsnM36As93A/bJzc8+kM24ZLMRC 6g69q/Eewv+o4o5S4TDYzl1UXDPn0trVap7ks30qsC4UoxtNehPHCin7jqgeLB37 pUuv+Sfhm3b159DukCbUwVPHDPZiBWOhBi6PMYUslWNtVqWdz2hnUIvS1gayBb8O vcNxoFmmpyNcEZcuQmbiT5wv3DI6KrNlEycXihnAPg+G11szK7op0zINxJ6J7k1h 0XUNVcriODn/3QvpigzYqtyQxMHRQ== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduvddrudeivddgtdekucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurheptggguffhjgffgffkfhfvofesthhqmhdthhdtjeenucfhrhhomhephfhlohhr ihgrnhcutfhivhhorghluceofhhlohhrihgrnhesrhhivhhorghlrdhnvghtqeenucffoh hmrghinhepihgvthhfrdhorhhgpdhirghnrgdrohhrghenucfkphepudekfedrjeeirdeh rddvtdenucfrrghrrghmpehmrghilhhfrhhomhepfhhlohhrihgrnhesrhhivhhorghlrd hnvghtnecuvehluhhsthgvrhfuihiivgeptd X-ME-Proxy: Received: from [192.168.1.3] (ab005020.dynamic.ppp.asahi-net.or.jp [183.76.5.20]) by mail.messagingengine.com (Postfix) with ESMTPA id E4168D60065; Thu, 29 Aug 2019 05:00:17 -0400 (EDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) From: Florian Rivoal In-Reply-To: Date: Thu, 29 Aug 2019 18:00:14 +0900 Cc: Doug Ewell , LTRU Working Group Content-Transfer-Encoding: quoted-printable Message-Id: <73BE5AF7-0C62-425F-834E-8759628D2C5F@rivoal.net> References: <20190827104755.665a7a7059d7ee80bb4d670165c8327d.0f79efb126.wbe@email03.godaddy.com> <910CB6C8-9F66-4255-B149-B146DA8E5695@rivoal.net> To: =?utf-8?B?TWFyayBEYXZpcyDimJXvuI8=?= X-Mailer: Apple Mail (2.3445.104.11) Archived-At: Subject: Re: [Ltru] Mail regarding draft-ietf-ltru-4646bis and draft-ietf-ltru-matching X-BeenThere: ltru@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Language Tag Registry Update working group discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Aug 2019 09:00:24 -0000 > On Aug 29, 2019, at 15:19, Mark Davis =E2=98=95=EF=B8=8F = wrote: >=20 > Canonicalizing to the extlang form has a number of disadvantages, and = I would recommend strongly against it. Don't have time to discuss now, = will see about next week. Very interested in what you have to say about this. In case that influences what you have to say, note that what I intend to = do is not to store the canonicalized-to-extlang form anywhere. It would = only be for internal processing: when performing an extended filtering = operation, where it is unknown whether the ranges and tags are in = extlang form or not, canonicalize both to extlang form do the extended = filtering operation on that. That way, a zh-*-Hant selector would match both a zh-yue-Hant document = element and a yue-Hant one. =E2=80=94Florian > On Wed, Aug 28, 2019 at 11:56 AM Florian Rivoal = wrote: >=20 >=20 > > On Aug 28, 2019, at 2:47, Doug Ewell wrote: > >=20 > > On July 27, Florian Rivoal wrote: > >=20 > >> However, RFC5646 Section 4.5, which defines canonicalization, only > >> does so for language tags, not for language ranges. Presumably, the > >> process is largely the same, with wildcards in the language subtag > >> being preserved, and I suppose wildcards in other subtags would = likely > >> be dropped. But as it stands, that seems undefined. > >=20 > > I think you are on the right track by assuming that ranges are > > canonicalized just like tags, with asterisks left alone. >=20 > Thanks for confirming. >=20 > > It's not very likely that most LTRU participants will be eager to = start > > up a new IETF project to update 5646 for something like this. Best = to go > > with your assumption. > >=20 > >> Also, while giving recommendations about canonicalization for the > >> purpose of filtering, it would seem useful to mention (and possibly = to > >> recommend) canonicalizing to the "extlang form". The definition of = the > >> extlang form itself (in RFC5646 Section 4.5) mentions that it is > >> useful for matching and selecting, but that information isn't = relayed > >> anywhere RFC4647. > >=20 > > At the time these documents were written, there was a strong = sentiment > > around de-emphasizing extlangs in general. It's good to know that > > there's a real-world use case for using them here. Again, it's = unlikely > > that people will want to rev 4647 for this. >=20 > The use case is CSS selectors, when writing rules for = typography/styling in a document. On the one hand, the document gets = marked up which part of it are in which language. On the other side, the = style sheet describes which part of the document must be styled which = way, and can make that styling dependent on the language. >=20 > The need for normalization comes from the fact that stylesheet = authoring and document authoring are not coordinated in the general = case, so a stylesheet author cannot know, generally speaking, if a = document will be marked up with, for example, zh-yue or yue. The = stylesheet author is then faced with two options, both unattractive for = different reasons: > * use the deprecated tag: it's more likely to be found in existing = documents due to being older. The first downside is that it doesn't = always work. The second one is that this slows down adoption of the = newer preferred tag, as document authors wanting to be compatible with = existing stylesheets will keep on using the deprecated one as well for = compatibility, and we get into a vicious cycle of everybody continuing = to use the deprecated variant. >=20 > * Use both the deprecated and the preferred tag in the stylesheet's = selector. This works, but it means that stylesheet authors need to be = aware of, and manually replicate the information in = https://www.iana.org/assignments/language-subtag-registry/language-subtag-= registry. Asking people to manually do what software could isn't great, = as they tend not to, or to do it with bugs, or to not update when the = registry updates, etc. >=20 > So it seems preferable, given that this correspondence is maintained = in a neatly usable format, to have CSS renderers deal with the = correspondence between deprecated and preferred tags by way of = canonicalizing to the extlang form and doing the selector matching on = that. >=20 > In the long run, both document authors and stylesheet authors should = use the preferred tag without the extlang prefix, and the = canonicalization to extang form will be invisible to them. But even if = some don't, everything works. >=20 > =E2=80=94Florian > _______________________________________________ > Ltru mailing list > Ltru@ietf.org > https://www.ietf.org/mailman/listinfo/ltru From nobody Thu Aug 29 06:44:27 2019 Return-Path: X-Original-To: ltru@ietfa.amsl.com Delivered-To: ltru@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 84CF3120071 for ; Thu, 29 Aug 2019 06:44:25 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.899 X-Spam-Level: X-Spam-Status: No, score=-1.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=ccil-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cAhtyG5hlscb for ; Thu, 29 Aug 2019 06:44:23 -0700 (PDT) Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 9CAA412004A for ; Thu, 29 Aug 2019 06:44:23 -0700 (PDT) Received: by mail-wr1-x42e.google.com with SMTP id y19so3507900wrd.3 for ; Thu, 29 Aug 2019 06:44:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ccil-org.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=ksjunRe4AYtvtZu5H//czyf1oBM5IsPN4ien4Os6Jnw=; b=sRcrgv9ijbOw99AVnBAehqqU5fiwJr1TcIMD5oH5pJ9ouiGFhRQ+0iPhCFReSOR+rh 4jfSST5DEX+rMwdN0DKukF/yDuUhJR2w78X5N1rrkcJVo/8XL4PrN32SSpTJ7qJpxudJ tZhdqd7Z2+eQkNYuoNktdYNt2hko1x++E6pbUmmoXkuEmV6DOx/MiU8BjYcjU559sQwa nFrM298TJVuyha/Qc1p4UBzvqPiNW7EZCemh2Ofghg+orkSrw+imNxnLAQaXHSkSPds3 W8JmzL7K6gstT69UldLImBvqY0rCAUHrYq6LHRjadA/8SaxpXEFkrwZQs0eR1ugHF7vw p6lA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=ksjunRe4AYtvtZu5H//czyf1oBM5IsPN4ien4Os6Jnw=; b=rn+luF6G/AZTHbhKA9GzWqNcYHjbWPq/C2Q7EsSq/37ShHU8rLiJk6v2EUF/R1hjNW tLJN1nzVeGJh5zyObF0B+DfpQPv47EI8mco2nZ4v7zEz7TY8tS546uv7aiIY1TSPaOtV St5rYk75hm7M7Qq6g/wOKkCsLKufhEJ2mfE0uId1aQCzOKc9DDCqGgLqHJ1SOHXe0ZFy I+M3N28PFyMth0HEomZHD2URIYQVnEGTPFNt9bJgTPEGG1uCh5+7Ev2ex1ww+CwhTWhA vwtuS8gmWzfqyNDPstygncEQFrVw0GHy6+oc5yRkbWZ6SsiNfkcK/lR+wa4Bbxr+3Jh2 oyOA== X-Gm-Message-State: APjAAAXO5GXSd/J/0CaQ8Yve4X9y0UIaHRN5FPOIFlVC9hjhsbg0MQFU VxayxhZH2DHJiOA8TwWQ42hbKhQr/5DFgr4Tv9ax0w== X-Google-Smtp-Source: APXvYqyJV/2/pKD8lROscwBgT6RVUlgNsfs3qyxA4FuvoCNYK9O98HSQGJs+IYEBzh6rRWNus3vGNmYwAS/HvtHVeWU= X-Received: by 2002:adf:ea03:: with SMTP id q3mr11351279wrm.219.1567086262153; Thu, 29 Aug 2019 06:44:22 -0700 (PDT) MIME-Version: 1.0 References: <20190827104755.665a7a7059d7ee80bb4d670165c8327d.0f79efb126.wbe@email03.godaddy.com> <910CB6C8-9F66-4255-B149-B146DA8E5695@rivoal.net> <73BE5AF7-0C62-425F-834E-8759628D2C5F@rivoal.net> In-Reply-To: <73BE5AF7-0C62-425F-834E-8759628D2C5F@rivoal.net> From: John Cowan Date: Thu, 29 Aug 2019 09:44:09 -0400 Message-ID: To: Florian Rivoal Cc: =?UTF-8?B?TWFyayBEYXZpcyDimJXvuI8=?= , LTRU Working Group , Doug Ewell Content-Type: multipart/alternative; boundary="000000000000fd089a059141b35b" Archived-At: Subject: Re: [Ltru] Mail regarding draft-ietf-ltru-4646bis and draft-ietf-ltru-matching X-BeenThere: ltru@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Language Tag Registry Update working group discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Aug 2019 13:44:26 -0000 --000000000000fd089a059141b35b Content-Type: text/plain; charset="UTF-8" On Thu, Aug 29, 2019 at 5:00 AM Florian Rivoal wrote: > > In case that influences what you have to say, note that what I intend to > do is not to store the canonicalized-to-extlang form anywhere. It would > only be for internal processing: when performing an extended filtering > operation, where it is unknown whether the ranges and tags are in extlang > form or not, canonicalize both to extlang form do the extended filtering > operation on that. > In that case you can equally canonicalize away from the extlang form as toward it. I recommend that. John Cowan http://vrici.lojban.org/~cowan cowan@ccil.org "Not to know The Smiths is not to know K.X.U." --K.X.U. --000000000000fd089a059141b35b Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Thu, Aug 29, 2019 at 5:00 AM Flori= an Rivoal <florian@rivoal.net&= gt; wrote:

In case that influences what you have to say, note that what I intend to do= is not to store the canonicalized-to-extlang form anywhere. It would only = be for internal processing: when performing an extended filtering operation= , where it is unknown whether the ranges and tags are in extlang form or no= t, canonicalize both to extlang form do the extended filtering operation on= that.

In that case you can equally can= onicalize away from the extlang form as toward it.=C2=A0 I recommend that.<= /div>


John Cowan =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0http://vrici.lojban.or= g/~cowan =C2=A0 =C2=A0 =C2=A0 =C2=A0c= owan@ccil.org
=C2=A0 =C2=A0 =C2=A0 =C2=A0 "Not to know The Smit= hs is not to know K.X.U." =C2=A0--K.X.U.

--000000000000fd089a059141b35b-- From nobody Thu Aug 29 11:15:38 2019 Return-Path: X-Original-To: ltru@ietfa.amsl.com Delivered-To: ltru@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AC244120AF9 for ; Thu, 29 Aug 2019 11:15:36 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dhXfQ1t-ebLF for ; Thu, 29 Aug 2019 11:15:35 -0700 (PDT) Received: from isaac.sophia.w3.org (isaac.sophia.w3.org [193.51.208.85]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E5A68120AF4 for ; Thu, 29 Aug 2019 11:15:34 -0700 (PDT) Received: from cpc120240-heme13-2-0-cust18.9-1.cable.virginm.net ([86.2.169.19] helo=Richards-MacBook-Pro.local) by isaac.sophia.w3.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.89) (envelope-from ) id 1i3OxG-0001lZ-Rd; Thu, 29 Aug 2019 18:15:30 +0000 To: John Cowan , Florian Rivoal Cc: LTRU Working Group , Doug Ewell References: <20190827104755.665a7a7059d7ee80bb4d670165c8327d.0f79efb126.wbe@email03.godaddy.com> <910CB6C8-9F66-4255-B149-B146DA8E5695@rivoal.net> <73BE5AF7-0C62-425F-834E-8759628D2C5F@rivoal.net> From: r12a Message-ID: <149ea9d5-a035-3f0d-95a2-b9206f1e5587@w3.org> Date: Thu, 29 Aug 2019 19:15:30 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-GB Content-Transfer-Encoding: 8bit Archived-At: Subject: Re: [Ltru] Mail regarding draft-ietf-ltru-4646bis and draft-ietf-ltru-matching X-BeenThere: ltru@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Language Tag Registry Update working group discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Aug 2019 18:15:37 -0000 On 29/08/2019 14:44, John Cowan wrote: > On Thu, Aug 29, 2019 at 5:00 AM Florian Rivoal > wrote: > > > In case that influences what you have to say, note that what I > intend to do is not to store the canonicalized-to-extlang form > anywhere. It would only be for internal processing: when performing > an extended filtering operation, where it is unknown whether the > ranges and tags are in extlang form or not, canonicalize both to > extlang form do the extended filtering operation on that. > > > In that case you can equally canonicalize away from the extlang form as > toward it.  I recommend that. ... which is what i was thinking too. ri From nobody Thu Aug 29 18:08:49 2019 Return-Path: X-Original-To: ltru@ietfa.amsl.com Delivered-To: ltru@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9DAC0120C61 for ; Thu, 29 Aug 2019 18:08:42 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.7 X-Spam-Level: X-Spam-Status: No, score=-2.7 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=rivoal.net header.b=a4uywvUy; dkim=pass (2048-bit key) header.d=messagingengine.com header.b=Cq4Fa+B1 Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 38b6G7BgjFPs for ; Thu, 29 Aug 2019 18:08:40 -0700 (PDT) Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 52893120C64 for ; Thu, 29 Aug 2019 18:08:40 -0700 (PDT) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id 9A59F21B5A; Thu, 29 Aug 2019 21:08:39 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Thu, 29 Aug 2019 21:08:39 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivoal.net; h= content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; s=fm1; bh=6 ZV/rvNl7pUvK6xtdxNYhAMSmtx3KEqloL1zTOAnKvQ=; b=a4uywvUyp2YnCKuut hNH/1k5QVdty5QJXkoT/H7NA77pRNe/V78I4Aa0h3FHH2DRjuO5MoVw0VG8sWh1D NLdkD6wGddY4ndKmkOr83vGsS0xC69Jq4YSsH80MWQgi7Dea/RogoIZOzdy9rT+h 5dGVTC2S8z0014HuUpxRYIaVTpkWOGNc/KBr4MpxyvNAEEk0QzBO7orOc6MRkakF UUykWGajupZweY01qJDv7Hgdcv5f9k3HiicvZGjk2vny3BbxKRpuVgo0CHPV9WQm IcGIQQ9Lc35b+xVQfdf5D2tsQMDOl918wb9Pva27UX22FAodUNmwjW41P3ye96pc 27djQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; bh=6ZV/rvNl7pUvK6xtdxNYhAMSmtx3KEqloL1zTOAnK vQ=; b=Cq4Fa+B1M9Hmzi3jV5TXDglElrrQsJE5oOOG5pdc5RKqjXWh4Xmisznbd 78T1NPQEPdJ9jT3h3I+wbxLIKGWiBmpgqJpd9TvXgVvXvXv3YKFU8T1xESxSW9s1 60/nrzsUGNJCcHY020CSttEpSlLcMf5Se8VPkjQa71rqvatYqK3ZyFqmsPgCi6xq csKAjUEUznMu5vRMby77nO70Rh0lihPLQzc9ERt17nWlOYmJB1MDX5vI/vDS4fmz kkoJ4YeuFX2L6/SBGXm+I9BRAtuL4PgIHtWcz/R1bAqNYdr5SjXAoDe6MuO0+2yW +W/0eTZq5hVYLywfLQWTu/QoC+Rfg== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduvddrudeifedgfeelucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurheptggguffhjgffgffkfhfvofesthhqmhdthhdtjeenucfhrhhomhephfhlohhr ihgrnhcutfhivhhorghluceofhhlohhrihgrnhesrhhivhhorghlrdhnvghtqeenucfkph epudekfedrjeeirdehrddvtdenucfrrghrrghmpehmrghilhhfrhhomhepfhhlohhrihgr nhesrhhivhhorghlrdhnvghtnecuvehluhhsthgvrhfuihiivgeptd X-ME-Proxy: Received: from [192.168.1.3] (ab005020.dynamic.ppp.asahi-net.or.jp [183.76.5.20]) by mail.messagingengine.com (Postfix) with ESMTPA id 88B9B80059; Thu, 29 Aug 2019 21:08:37 -0400 (EDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) From: Florian Rivoal In-Reply-To: Date: Fri, 30 Aug 2019 10:08:33 +0900 Cc: =?utf-8?B?TWFyayBEYXZpcyDimJXvuI8=?= , LTRU Working Group , Doug Ewell Content-Transfer-Encoding: quoted-printable Message-Id: <94B0FC03-B793-43BB-B864-38A306E6B5CA@rivoal.net> References: <20190827104755.665a7a7059d7ee80bb4d670165c8327d.0f79efb126.wbe@email03.godaddy.com> <910CB6C8-9F66-4255-B149-B146DA8E5695@rivoal.net> <73BE5AF7-0C62-425F-834E-8759628D2C5F@rivoal.net> To: John Cowan X-Mailer: Apple Mail (2.3445.104.11) Archived-At: Subject: Re: [Ltru] Mail regarding draft-ietf-ltru-4646bis and draft-ietf-ltru-matching X-BeenThere: ltru@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Language Tag Registry Update working group discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Aug 2019 01:08:48 -0000 > On Aug 29, 2019, at 22:44, John Cowan wrote: >=20 >=20 >=20 > On Thu, Aug 29, 2019 at 5:00 AM Florian Rivoal = wrote: >=20 > In case that influences what you have to say, note that what I intend = to do is not to store the canonicalized-to-extlang form anywhere. It = would only be for internal processing: when performing an extended = filtering operation, where it is unknown whether the ranges and tags are = in extlang form or not, canonicalize both to extlang form do the = extended filtering operation on that. >=20 > In that case you can equally canonicalize away from the extlang form = as toward it. I recommend that. Can you? Let's say you want to match (using extended filtering) the zh range = against documents that may contain the zh-yue or yue tags (and possibly = other zh-cmn, zh-hakka, zh, zh-HK=E2=80=A6). This could be something a = typesetter wants to do to use a particular font and set of line breaking = rules for any chunk of Chinese (in the broad sense) text. If we canonicalize to extlang form:=20 zh -> zh zh-yue -> zh-yue yue -> zh-yue Therefore, the zh range will match both the documents that contained = zh-yue or yue. This is what I want. If we canonicalize away from extlang form:=20 zh -> zh zh-yue -> yue yue -> yue Therefore, the zh range will match neither documents that contained = zh-yue nor yue. This is not what I want, and is worse than not = canonicalizing at all. So it seems to me that no, we cannot canonicalize away from the extlang = form and get the same results. If the extended filtering operation did something smart with = macrolanguages, then I wouldn't need canonicalization at all, but it = doesn't, so I feel I need to canonicalize, and as described above, only = canonicalization to extlang actually seems to help. Am I missing something? =E2=80=94Florian From nobody Thu Aug 29 18:14:36 2019 Return-Path: X-Original-To: ltru@ietfa.amsl.com Delivered-To: ltru@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 263721201C6 for ; Thu, 29 Aug 2019 18:14:35 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.7 X-Spam-Level: X-Spam-Status: No, score=-2.7 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=rivoal.net header.b=QnQ3sL1o; dkim=pass (2048-bit key) header.d=messagingengine.com header.b=yzlard3r Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 94beuytLRHKs for ; Thu, 29 Aug 2019 18:14:33 -0700 (PDT) Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D780512091B for ; Thu, 29 Aug 2019 18:14:32 -0700 (PDT) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id 168D321D6E; Thu, 29 Aug 2019 21:14:32 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Thu, 29 Aug 2019 21:14:32 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivoal.net; h= content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; s=fm1; bh=D MFGgZ5NAZpbGBabtB3oLrlE5Sl8Wmhyu0iJAzGNN8A=; b=QnQ3sL1oIL5XJb7M3 SOej9KZPhGalSGy03ZWxeJMO023CDIrGQVNT5APgP8/g60puPtOEkiWBy1YeAzjv S9YpR2S1K+u4Pf97D0n/DeKPnxgQVQRtShFd3TgMyCusuU5v4Nx7AwjGPadYQVzf jE16I4bGX+0U6f75aGPVK+6oJAjSESJXEKpN/3wucvc/SnBOYpHkesmZyrn6jRf1 Ve7g47B8i3uh04di08H4y/Np3iW5AZ16snt9tBNHuJ+C0ijkW1o5qib0sSmsVjv7 nWO1W/9DXt8mJHN8lK8rCRT3OaWCYWY1NfEtyOWSblsAdEGN8MOoOUvQrQlJyqKo c+LSw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; bh=DMFGgZ5NAZpbGBabtB3oLrlE5Sl8Wmhyu0iJAzGNN 8A=; b=yzlard3rCseJ8QeIVd2YfpZEu8klcmIXr5kqkuFLAZGTjao385/FG7CnF YSPGN6bpXWzaMhfXMApzbOLCwrth0wHpMr7Ui9f+Rg53l1HRZ1qbv55kl+4LDAfe /9mge3YLYijRcij7CuCUlDfeLnjCmkFsTexQ8Q/AaXcznZNd+MkAWKrq/FSQbo0D 1zBc6cvABeKieTc+iI8TM8HjrfqD1JKd22K9dbLmoFbXS8VAaW6w0u66yfPt/IHQ kcWvfpI4J39NkjiTA3LjDRHBxt0sHIql2NEOFgOnO1JbPeY6bah+6bfDj94iQ+BK 2j+vAqcJ48zJFFRdK9nIOZxNtkpfA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduvddrudeifedggedtucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurheptggguffhjgffgffkfhfvofesthhqmhdthhdtjeenucfhrhhomhephfhlohhr ihgrnhcutfhivhhorghluceofhhlohhrihgrnhesrhhivhhorghlrdhnvghtqeenucfkph epudekfedrjeeirdehrddvtdenucfrrghrrghmpehmrghilhhfrhhomhepfhhlohhrihgr nhesrhhivhhorghlrdhnvghtnecuvehluhhsthgvrhfuihiivgeptd X-ME-Proxy: Received: from [192.168.1.3] (ab005020.dynamic.ppp.asahi-net.or.jp [183.76.5.20]) by mail.messagingengine.com (Postfix) with ESMTPA id 750F280060; Thu, 29 Aug 2019 21:14:30 -0400 (EDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) From: Florian Rivoal In-Reply-To: <94B0FC03-B793-43BB-B864-38A306E6B5CA@rivoal.net> Date: Fri, 30 Aug 2019 10:14:28 +0900 Cc: LTRU Working Group , Doug Ewell Content-Transfer-Encoding: quoted-printable Message-Id: References: <20190827104755.665a7a7059d7ee80bb4d670165c8327d.0f79efb126.wbe@email03.godaddy.com> <910CB6C8-9F66-4255-B149-B146DA8E5695@rivoal.net> <73BE5AF7-0C62-425F-834E-8759628D2C5F@rivoal.net> <94B0FC03-B793-43BB-B864-38A306E6B5CA@rivoal.net> To: John Cowan X-Mailer: Apple Mail (2.3445.104.11) Archived-At: Subject: Re: [Ltru] Mail regarding draft-ietf-ltru-4646bis and draft-ietf-ltru-matching X-BeenThere: ltru@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Language Tag Registry Update working group discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Aug 2019 01:14:35 -0000 > On Aug 30, 2019, at 10:08, Florian Rivoal wrote: >=20 >=20 >=20 >> On Aug 29, 2019, at 22:44, John Cowan wrote: >>=20 >>=20 >>=20 >> On Thu, Aug 29, 2019 at 5:00 AM Florian Rivoal = wrote: >>=20 >> In case that influences what you have to say, note that what I intend = to do is not to store the canonicalized-to-extlang form anywhere. It = would only be for internal processing: when performing an extended = filtering operation, where it is unknown whether the ranges and tags are = in extlang form or not, canonicalize both to extlang form do the = extended filtering operation on that. >>=20 >> In that case you can equally canonicalize away from the extlang form = as toward it. I recommend that. >=20 > Can you? >=20 > Let's say you want to match (using extended filtering) the zh range = against documents that may contain the zh-yue or yue tags (and possibly = other zh-cmn, zh-hakka, zh, zh-HK=E2=80=A6). This could be something a = typesetter wants to do to use a particular font and set of line breaking = rules for any chunk of Chinese (in the broad sense) text. >=20 > If we canonicalize to extlang form:=20 > zh -> zh > zh-yue -> zh-yue > yue -> zh-yue > Therefore, the zh range will match both the documents that contained = zh-yue or yue. This is what I want. >=20 > If we canonicalize away from extlang form:=20 > zh -> zh > zh-yue -> yue > yue -> yue > Therefore, the zh range will match neither documents that contained = zh-yue nor yue. This is not what I want, and is worse than not = canonicalizing at all. >=20 > So it seems to me that no, we cannot canonicalize away from the = extlang form and get the same results. >=20 > If the extended filtering operation did something smart with = macrolanguages, then I wouldn't need canonicalization at all, but it = doesn't, so I feel I need to canonicalize, and as described above, only = canonicalization to extlang actually seems to help. >=20 > Am I missing something? >=20 > =E2=80=94Florian Sorry for not including that in the previous message, but to give = another example, if I want to use the no range to match any of: no, = no-bok, no-nyn, nb, or nn, canonicalization to extlang form works, and = canonicalization away from it doesn't. =E2=80=94Florian= From nobody Thu Aug 29 19:17:37 2019 Return-Path: X-Original-To: ltru@ietfa.amsl.com Delivered-To: ltru@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 281B4120116 for ; Thu, 29 Aug 2019 19:17:35 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.898 X-Spam-Level: X-Spam-Status: No, score=-1.898 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=ccil-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gMK2n1kXDIKe for ; Thu, 29 Aug 2019 19:17:32 -0700 (PDT) Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com [IPv6:2a00:1450:4864:20::32a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 7D4811200E5 for ; Thu, 29 Aug 2019 19:17:32 -0700 (PDT) Received: by mail-wm1-x32a.google.com with SMTP id y135so3311476wmc.1 for ; Thu, 29 Aug 2019 19:17:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ccil-org.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=H+s2WgIooqhd1D7MPLRCmGNhFw83yrenoogTdNIsYS8=; b=ct+1FXomSQx9mUHVgRKzC8e8n0S6PIpu3hKRSOddu6PlXD9eeSi0ktjwKktfYXiw+J 2X2YyOTwuhDeug2exnhId8uKlUyXnSH+cH7ngOjvNu5hugskZP4n2QZERa+dFdozMkki cyOXZ6O24gJvhLmTEWrJI2LmjyXqVVKVJIHcUYvjv66qHuRu1wdJwC5jGHu8pWXDhS2d JK+NtoBaJ6QK3i91N04OV+NQsUMjWiECOYhVUpA+KLA4qub3SO0ME3+91ANeYUw2EWt+ w3rY6HpJN4h/LnU24j74tyOUH0xVlFrveOk8oFXz5l4xDcez5vkCfNbxL/dh1GquO74D xt7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=H+s2WgIooqhd1D7MPLRCmGNhFw83yrenoogTdNIsYS8=; b=n+N4I8hWGp3gqdR3bVoYCkiHbcZZR6qpaMWkBaplqi8w5P5re21Jq9639B/gcV8oJy UHkFlPZmbnZCNVGAjOsfp18LvoxwbgXpGxeNL9Hq/Usx35Yi3sJIgvaSmTFw3dWDao4v REHvp9xeeFU5UIAuE1QoRWaWMcwfIbuIaDIkT1iKz2pV6zsRvfhQ5LPUfhmoy2O3+K1j lEoTO+jdRppJizhizCQukUNoMyGXfyL9wk17qZJlK8tL47UYJQ93XMkKqDLpLsBijC+D Pvt0drXyLS7qsSFCvQ84yfN0Xfmtvhc9SwAc7Skty55tjylnBpMnQt5jxUc8SSVbu8AZ GR9g== X-Gm-Message-State: APjAAAU3eK54WymFmuTrFEPNTiw8ysy/uXZiOkaHcwX6sKUJMrva4Hr0 N/6ocVy3pCX5e64+CIQ/vFkojEobaxdLLusTCmbCUzuY X-Google-Smtp-Source: APXvYqw2Gm1vA+/rhmckrsduEmUaWHmcwWDuvxzejDC7srB+PvQsg1d+CKVgnfSP15YfJKOGpSCAJ3Mo9DnMgvkhwQA= X-Received: by 2002:a1c:96c6:: with SMTP id y189mr8458218wmd.160.1567131450932; Thu, 29 Aug 2019 19:17:30 -0700 (PDT) MIME-Version: 1.0 References: <20190827104755.665a7a7059d7ee80bb4d670165c8327d.0f79efb126.wbe@email03.godaddy.com> <910CB6C8-9F66-4255-B149-B146DA8E5695@rivoal.net> <73BE5AF7-0C62-425F-834E-8759628D2C5F@rivoal.net> <94B0FC03-B793-43BB-B864-38A306E6B5CA@rivoal.net> In-Reply-To: From: John Cowan Date: Thu, 29 Aug 2019 22:17:19 -0400 Message-ID: To: Florian Rivoal Cc: LTRU Working Group , Doug Ewell Content-Type: multipart/alternative; boundary="0000000000007312a405914c398f" Archived-At: Subject: Re: [Ltru] Mail regarding draft-ietf-ltru-4646bis and draft-ietf-ltru-matching X-BeenThere: ltru@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Language Tag Registry Update working group discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Aug 2019 02:17:35 -0000 --0000000000007312a405914c398f Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Got it. Yes, with extended filtering, you need to canonicalize toward extlangs. I thought you were talking about simple equality matching after canonicalization. Note that almost all macrolanguages do not have extlang tags; the languages can be identified by the macrolanguage or by the individual language tag, but not both. Indeed, only ar, kok, lv, ms, sw, uz, zh have extlang tags subordinated to them; sgn does too but is not a macrolanguage. On Thu, Aug 29, 2019 at 9:14 PM Florian Rivoal wrote: > > > > On Aug 30, 2019, at 10:08, Florian Rivoal wrote: > > > > > > > >> On Aug 29, 2019, at 22:44, John Cowan wrote: > >> > >> > >> > >> On Thu, Aug 29, 2019 at 5:00 AM Florian Rivoal > wrote: > >> > >> In case that influences what you have to say, note that what I intend > to do is not to store the canonicalized-to-extlang form anywhere. It woul= d > only be for internal processing: when performing an extended filtering > operation, where it is unknown whether the ranges and tags are in extlang > form or not, canonicalize both to extlang form do the extended filtering > operation on that. > >> > >> In that case you can equally canonicalize away from the extlang form a= s > toward it. I recommend that. > > > > Can you? > > > > Let's say you want to match (using extended filtering) the zh range > against documents that may contain the zh-yue or yue tags (and possibly > other zh-cmn, zh-hakka, zh, zh-HK=E2=80=A6). This could be something a ty= pesetter > wants to do to use a particular font and set of line breaking rules for a= ny > chunk of Chinese (in the broad sense) text. > > > > If we canonicalize to extlang form: > > zh -> zh > > zh-yue -> zh-yue > > yue -> zh-yue > > Therefore, the zh range will match both the documents that contained > zh-yue or yue. This is what I want. > > > > If we canonicalize away from extlang form: > > zh -> zh > > zh-yue -> yue > > yue -> yue > > Therefore, the zh range will match neither documents that contained > zh-yue nor yue. This is not what I want, and is worse than not > canonicalizing at all. > > > > So it seems to me that no, we cannot canonicalize away from the extlang > form and get the same results. > > > > If the extended filtering operation did something smart with > macrolanguages, then I wouldn't need canonicalization at all, but it > doesn't, so I feel I need to canonicalize, and as described above, only > canonicalization to extlang actually seems to help. > > > > Am I missing something? > > > > =E2=80=94Florian > > Sorry for not including that in the previous message, but to give another > example, if I want to use the no range to match any of: no, no-bok, no-ny= n, > nb, or nn, canonicalization to extlang form works, and canonicalization > away from it doesn't. > > =E2=80=94Florian --0000000000007312a405914c398f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Got it. Yes, with extended filtering, you need to canonica= lize toward extlangs.=C2=A0 I thought you were talking about simple equalit= y matching after canonicalization.

Note that almost = all macrolanguages do not have extlang tags; the languages can be identifie= d by the macrolanguage or by the individual language tag, but not both.=C2= =A0 Indeed, only ar, kok, lv, ms, sw, uz, zh have extlang tags subordinated= to them; sgn does too but is not a macrolanguage.




On Thu, Aug 29, 2019 at 9:14 PM Florian Rivoal &l= t;florian@rivoal.net> wrote:


> On Aug 30, 2019, at 10:08, Florian Rivoal <florian@rivoal.net> wrote:
>
>
>
>> On Aug 29, 2019, at 22:44, John Cowan <cowan@ccil.org> wrote:
>>
>>
>>
>> On Thu, Aug 29, 2019 at 5:00 AM Florian Rivoal <florian@rivoal.net> wrote:<= br> >>
>> In case that influences what you have to say, note that what I int= end to do is not to store the canonicalized-to-extlang form anywhere. It wo= uld only be for internal processing: when performing an extended filtering = operation, where it is unknown whether the ranges and tags are in extlang f= orm or not, canonicalize both to extlang form do the extended filtering ope= ration on that.
>>
>> In that case you can equally canonicalize away from the extlang fo= rm as toward it.=C2=A0 I recommend that.
>
> Can you?
>
> Let's say you want to match (using extended filtering) the zh rang= e against documents that may contain the zh-yue or yue tags (and possibly o= ther zh-cmn, zh-hakka, zh, zh-HK=E2=80=A6). This could be something a types= etter wants to do to use a particular font and set of line breaking rules f= or any chunk of Chinese (in the broad sense) text.
>
> If we canonicalize to extlang form:
>=C2=A0 zh -> zh
>=C2=A0 zh-yue -> zh-yue
>=C2=A0 yue -> zh-yue
> Therefore, the zh range will match both the documents that contained z= h-yue or yue. This is what I want.
>
> If we canonicalize away from extlang form:
>=C2=A0 zh -> zh
>=C2=A0 zh-yue -> yue
>=C2=A0 yue -> yue
> Therefore, the zh range will match neither documents that contained zh= -yue nor yue. This is not what I want, and is worse than not canonicalizing= at all.
>
> So it seems to me that no, we cannot canonicalize away from the extlan= g form and get the same results.
>
> If the extended filtering operation did something smart with macrolang= uages, then I wouldn't need canonicalization at all, but it doesn't= , so I feel I need to canonicalize, and as described above, only canonicali= zation to extlang actually seems to help.
>
> Am I missing something?
>
> =E2=80=94Florian

Sorry for not including that in the previous message, but to give another e= xample, if I want to use the no range to match any of: no, no-bok, no-nyn, = nb, or nn, canonicalization to extlang form works, and canonicalization awa= y from it doesn't.

=E2=80=94Florian
--0000000000007312a405914c398f-- From nobody Thu Aug 29 20:55:29 2019 Return-Path: X-Original-To: ltru@ietfa.amsl.com Delivered-To: ltru@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3F94112024E for ; Thu, 29 Aug 2019 20:55:27 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.699 X-Spam-Level: X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=rivoal.net header.b=giYAEaQZ; dkim=pass (2048-bit key) header.d=messagingengine.com header.b=lC41eBmt Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Td6S_-bX919c for ; Thu, 29 Aug 2019 20:55:24 -0700 (PDT) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A9524120236 for ; Thu, 29 Aug 2019 20:55:24 -0700 (PDT) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id 9F1F421B74; Thu, 29 Aug 2019 23:55:23 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Thu, 29 Aug 2019 23:55:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivoal.net; h= from:message-id:content-type:mime-version:subject:date :in-reply-to:cc:to:references; s=fm1; bh=BeP9oPqrhHBemwkAL42JCs7 t6NqbhRMSaJIhMcMN0QQ=; b=giYAEaQZILwgzz/eaQ9LGJuu6/mAJ1XNYcMCxzK C7luEuy0dweJdlhwd8N6j/t0Uc06Nx1kq/f9qI1U86hgmkF2+N9Lp/FkuuMUjWkh qw/uNYCoNNJ8zB4m108BMixAqDYNpokLSewRVKWnpZpskWyY3U9Hk/yzabdrM/aU TPIcT0AMIMEh+X/0aygh0V/7nmmkWyekw5dopAhJ7dDhtktuY/cwPhcbR6N6jySW uLfZYo1ewTuWq80RbQcaJ40XFlMz4FSvWeV2yEDvz8KqaoBJ6AvquelUJ1Jb09O/ mC4aTX71DSUeNAtkVBqnab8qhNeXd0JgGIMzusJ81tQVv/g== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=BeP9oP qrhHBemwkAL42JCs7t6NqbhRMSaJIhMcMN0QQ=; b=lC41eBmtEhQ4i6gi0gpWK/ g3ETe8WIf1UzkTJVnwfAYdYQzDb3WHS+wtcmJuQIKYglcGihkXpfSgexRZAsYfwk q8a29YAJAHblzU60IN3ilW/2WMtS2x8Me7JKPL6eZYfmAm8ZONYuJ8sfTnI97+Z3 Ew/qSWh5bchSh1r4jvn5SYWVCH5Nz+6b5/f64TqWgCNtjBEqQo6UAkDTrgcetvWc T2RRCHDK0PCL7IHi48ouVnaG+BmRVNE5IpJImG9pRttvnhAA7UMuoofZ9aQtGBr1 vU1NbnE2cKwLtkCAesYEmMUUyStrIqQz0TOT+J4Wfr5FkzUZ4op6YW6uI7nMcPsA == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduvddrudeifedgjeegucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffktgggufffjgfvfhfosegrtdhmrehhtdejnecuhfhrohhmpefhlhhorhhi rghnucftihhvohgrlhcuoehflhhorhhirghnsehrihhvohgrlhdrnhgvtheqnecukfhppe dukeefrdejiedrhedrvddtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehflhhorhhirghn sehrihhvohgrlhdrnhgvthenucevlhhushhtvghrufhiiigvpedt X-ME-Proxy: Received: from [192.168.1.3] (ab005020.dynamic.ppp.asahi-net.or.jp [183.76.5.20]) by mail.messagingengine.com (Postfix) with ESMTPA id 96F36D6005D; Thu, 29 Aug 2019 23:55:21 -0400 (EDT) From: Florian Rivoal Message-Id: <3B2C913C-DE8F-43E2-9819-55DD349EF0BA@rivoal.net> Content-Type: multipart/alternative; boundary="Apple-Mail=_E062A478-AE77-4408-9CEB-59E908C20DB2" Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Date: Fri, 30 Aug 2019 12:55:17 +0900 In-Reply-To: Cc: LTRU Working Group , Doug Ewell To: John Cowan References: <20190827104755.665a7a7059d7ee80bb4d670165c8327d.0f79efb126.wbe@email03.godaddy.com> <910CB6C8-9F66-4255-B149-B146DA8E5695@rivoal.net> <73BE5AF7-0C62-425F-834E-8759628D2C5F@rivoal.net> <94B0FC03-B793-43BB-B864-38A306E6B5CA@rivoal.net> X-Mailer: Apple Mail (2.3445.104.11) Archived-At: Subject: Re: [Ltru] Mail regarding draft-ietf-ltru-4646bis and draft-ietf-ltru-matching X-BeenThere: ltru@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Language Tag Registry Update working group discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Aug 2019 03:55:27 -0000 --Apple-Mail=_E062A478-AE77-4408-9CEB-59E908C20DB2 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Aug 30, 2019, at 11:17, John Cowan wrote: >=20 > Got it. Yes, with extended filtering, you need to canonicalize toward = extlangs. I thought you were talking about simple equality matching = after canonicalization. >=20 > Note that almost all macrolanguages do not have extlang tags; the = languages can be identified by the macrolanguage or by the individual = language tag, but not both. Indeed, only ar, kok, lv, ms, sw, uz, zh = have extlang tags subordinated to them; Ah, indeed, I had failed to notice that. That doesn't make what I am = proposing to do wrong, but it does limit its usefulness to a subset of = the macrolanguage families. So my Chinese example would work, but my = Norwegian one wouldn't. Is there a reason for the lack of extlang on these macrolanguages that = don't have it? inferring the equivalent information by inverting the = "Prefered value" information doesn't sound hard, but if it wasn't done, = presumably there's a reason. > sgn does too but is not a macrolanguage. It doesn't sound terribly useful to me to do the extlang = canonicalization for the sake of sgn, but neither does it sound harmful. =E2=80=94Florian >=20 > On Thu, Aug 29, 2019 at 9:14 PM Florian Rivoal > wrote: >=20 >=20 > > On Aug 30, 2019, at 10:08, Florian Rivoal > wrote: > >=20 > >=20 > >=20 > >> On Aug 29, 2019, at 22:44, John Cowan > wrote: > >>=20 > >>=20 > >>=20 > >> On Thu, Aug 29, 2019 at 5:00 AM Florian Rivoal > wrote: > >>=20 > >> In case that influences what you have to say, note that what I = intend to do is not to store the canonicalized-to-extlang form anywhere. = It would only be for internal processing: when performing an extended = filtering operation, where it is unknown whether the ranges and tags are = in extlang form or not, canonicalize both to extlang form do the = extended filtering operation on that. > >>=20 > >> In that case you can equally canonicalize away from the extlang = form as toward it. I recommend that. > >=20 > > Can you? > >=20 > > Let's say you want to match (using extended filtering) the zh range = against documents that may contain the zh-yue or yue tags (and possibly = other zh-cmn, zh-hakka, zh, zh-HK=E2=80=A6). This could be something a = typesetter wants to do to use a particular font and set of line breaking = rules for any chunk of Chinese (in the broad sense) text. > >=20 > > If we canonicalize to extlang form:=20 > > zh -> zh > > zh-yue -> zh-yue > > yue -> zh-yue > > Therefore, the zh range will match both the documents that contained = zh-yue or yue. This is what I want. > >=20 > > If we canonicalize away from extlang form:=20 > > zh -> zh > > zh-yue -> yue > > yue -> yue > > Therefore, the zh range will match neither documents that contained = zh-yue nor yue. This is not what I want, and is worse than not = canonicalizing at all. > >=20 > > So it seems to me that no, we cannot canonicalize away from the = extlang form and get the same results. > >=20 > > If the extended filtering operation did something smart with = macrolanguages, then I wouldn't need canonicalization at all, but it = doesn't, so I feel I need to canonicalize, and as described above, only = canonicalization to extlang actually seems to help. > >=20 > > Am I missing something? > >=20 > > =E2=80=94Florian >=20 > Sorry for not including that in the previous message, but to give = another example, if I want to use the no range to match any of: no, = no-bok, no-nyn, nb, or nn, canonicalization to extlang form works, and = canonicalization away from it doesn't. >=20 > =E2=80=94Florian --Apple-Mail=_E062A478-AE77-4408-9CEB-59E908C20DB2 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8

On Aug 30, 2019, at 11:17, John Cowan <cowan@ccil.org> = wrote:

Got it. Yes, with extended filtering, you need to = canonicalize toward extlangs.  I thought you were talking about = simple equality matching after canonicalization.

Note that almost all = macrolanguages do not have extlang tags; the languages can be identified = by the macrolanguage or by the individual language tag, but not = both.  Indeed, only ar, kok, lv, ms, sw, uz, zh have extlang tags = subordinated to them;

Ah, indeed, I had failed to notice that. That = doesn't make what I am proposing to do wrong, but it does limit its = usefulness to a subset of the macrolanguage families. So my Chinese = example would work, but my Norwegian one wouldn't.

Is there a reason for the lack of extlang on these = macrolanguages that don't have it? inferring the equivalent information = by inverting the "Prefered value" information doesn't sound hard, but if = it wasn't done, presumably there's a reason.

sgn does too but is not a = macrolanguage.

It doesn't sound terribly useful to me to do the = extlang canonicalization for the sake of sgn, but neither does it sound = harmful.

=E2=80=94Florian


On Thu, Aug 29, 2019 at 9:14 PM Florian Rivoal = <florian@rivoal.net> wrote:


> On Aug 30, 2019, at 10:08, Florian Rivoal <florian@rivoal.net> wrote:
>
>
>
>> On Aug 29, 2019, at 22:44, John Cowan <cowan@ccil.org> wrote:
>>
>>
>>
>> On Thu, Aug 29, 2019 at 5:00 AM Florian Rivoal <florian@rivoal.net> wrote:
>>
>> In case that influences what you have to say, note that what I = intend to do is not to store the canonicalized-to-extlang form anywhere. = It would only be for internal processing: when performing an extended = filtering operation, where it is unknown whether the ranges and tags are = in extlang form or not, canonicalize both to extlang form do the = extended filtering operation on that.
>>
>> In that case you can equally canonicalize away from the extlang = form as toward it.  I recommend that.
>
> Can you?
>
> Let's say you want to match (using extended filtering) the zh range = against documents that may contain the zh-yue or yue tags (and possibly = other zh-cmn, zh-hakka, zh, zh-HK=E2=80=A6). This could be something a = typesetter wants to do to use a particular font and set of line breaking = rules for any chunk of Chinese (in the broad sense) text.
>
> If we canonicalize to extlang form:
>  zh -> zh
>  zh-yue -> zh-yue
>  yue -> zh-yue
> Therefore, the zh range will match both the documents that = contained zh-yue or yue. This is what I want.
>
> If we canonicalize away from extlang form:
>  zh -> zh
>  zh-yue -> yue
>  yue -> yue
> Therefore, the zh range will match neither documents that contained = zh-yue nor yue. This is not what I want, and is worse than not = canonicalizing at all.
>
> So it seems to me that no, we cannot canonicalize away from the = extlang form and get the same results.
>
> If the extended filtering operation did something smart with = macrolanguages, then I wouldn't need canonicalization at all, but it = doesn't, so I feel I need to canonicalize, and as described above, only = canonicalization to extlang actually seems to help.
>
> Am I missing something?
>
> =E2=80=94Florian

Sorry for not including that in the previous message, but to give = another example, if I want to use the no range to match any of: no, = no-bok, no-nyn, nb, or nn, canonicalization to extlang form works, and = canonicalization away from it doesn't.

=E2=80=94Florian

= --Apple-Mail=_E062A478-AE77-4408-9CEB-59E908C20DB2-- From nobody Thu Aug 29 21:06:52 2019 Return-Path: X-Original-To: ltru@ietfa.amsl.com Delivered-To: ltru@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F197312024E for ; Thu, 29 Aug 2019 21:06:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.898 X-Spam-Level: X-Spam-Status: No, score=-1.898 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=ccil-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OmFhKWW4bfRe for ; Thu, 29 Aug 2019 21:06:48 -0700 (PDT) Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1AAF2120241 for ; Thu, 29 Aug 2019 21:06:48 -0700 (PDT) Received: by mail-wr1-x42c.google.com with SMTP id s18so5529428wrn.1 for ; Thu, 29 Aug 2019 21:06:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ccil-org.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=l0b7XRcd1J/PbKJvtSPz0Q+bTWSrjpefnXB7enfmDoE=; b=AUDci+Ud+BWzYGgD1RVpNFQQonuZdgov0/mmWf1+ownSKb+vMT74RxCb4TdzICEuTQ iIuqfYKhGd8jUbhGTolhmv6C8iJ4Syi/3ENBlQ6Q3ypbkgk7v+ow9CSIVfc7elcMnCu/ DSmJ6hUMmYpy1eziMpm6Oh24fv4F1wQrHccO59oIbG1YndOUMrKgrdnxNDFJre8Qd3+q q9p5jgn3oyMiRLqq8zg7/CxBCyx5kNDuNsNN3R7QHgdL4R6iBxgekSh92ASYDgLQjH0a KIwDXdZtu67NgWi+OddZ8va4UCkEmViMKf9FsRbK82IW0sXkd5PtHFO+KHsc99cY4WHN gFBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=l0b7XRcd1J/PbKJvtSPz0Q+bTWSrjpefnXB7enfmDoE=; b=VUJi8efiVfdD4x2YBeGbJPHnQEUAXT1sI0qsy+eFRGgaY64NcnZdsZi6CsbZL9alg9 m3Tkldio/+r96Gfhwuk956gZjmaP8oSfloD40HAOR/7MaWA2ZyKIJzX5yOp4AwMhpprf VCuj99er2ZNw660/TkLo3hF/YgXh3Y6C0wXcPelZIeppWqqB+kCu3chA4JvVXNdPq4Q7 oQIGp6W3DiMDnVYdqd+kuSKRtJSHNG/HwMmhmOXjXb7xmaQhmxLsj8fBZtrB6PI7bk0K t6xWUsd4liAOeu61ck4KU63/mEvq2KWSCekzVGKuU0dKvT2dEQlwXr70Ggc69sMyQFy3 sXJw== X-Gm-Message-State: APjAAAWjaOt6o3EaGwMbyya7dJgnV8IlfWDkMfoYRa5PuKy3bP/7njPG H1/Jr//AKLawo0H+NMZJpQ7QvqVVQSEp2Qq3NCWpg0E0 X-Google-Smtp-Source: APXvYqz3mPe7YnC0KlMjCj+Vk29/E90qoKkZZOv4yNpEbdGClou+T9YO4HC3o1Yo2DLtWTGxpzlCJxZBOXU3tWBNw2c= X-Received: by 2002:a5d:678d:: with SMTP id v13mr7593212wru.133.1567138005482; Thu, 29 Aug 2019 21:06:45 -0700 (PDT) MIME-Version: 1.0 References: <20190827104755.665a7a7059d7ee80bb4d670165c8327d.0f79efb126.wbe@email03.godaddy.com> <910CB6C8-9F66-4255-B149-B146DA8E5695@rivoal.net> <73BE5AF7-0C62-425F-834E-8759628D2C5F@rivoal.net> <94B0FC03-B793-43BB-B864-38A306E6B5CA@rivoal.net> <3B2C913C-DE8F-43E2-9819-55DD349EF0BA@rivoal.net> In-Reply-To: <3B2C913C-DE8F-43E2-9819-55DD349EF0BA@rivoal.net> From: John Cowan Date: Fri, 30 Aug 2019 00:06:34 -0400 Message-ID: To: Florian Rivoal Cc: LTRU Working Group , Doug Ewell Content-Type: multipart/alternative; boundary="00000000000021a1c905914dc022" Archived-At: Subject: Re: [Ltru] Mail regarding draft-ietf-ltru-4646bis and draft-ietf-ltru-matching X-BeenThere: ltru@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Language Tag Registry Update working group discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Aug 2019 04:06:51 -0000 --00000000000021a1c905914dc022 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable It was only done for macrolanguages that already had compound tags like zh-yue, plus a few others that seemed likely to be treated similiarly, often because they only a single written form between them. Aymara is a macrolanguage with two individual languages, Central and Southern, but only Central has a written form and is an official language (of Bolivia). If you don't care about the diffference, you can use ay, or you can use ayc or ayr to distinguish, but nobody has to worry about abstracting over them. On Thu, Aug 29, 2019 at 11:55 PM Florian Rivoal wrote: > > > On Aug 30, 2019, at 11:17, John Cowan wrote: > > Got it. Yes, with extended filtering, you need to canonicalize toward > extlangs. I thought you were talking about simple equality matching afte= r > canonicalization. > > Note that almost all macrolanguages do not have extlang tags; the > languages can be identified by the macrolanguage or by the individual > language tag, but not both. Indeed, only ar, kok, lv, ms, sw, uz, zh hav= e > extlang tags subordinated to them; > > > Ah, indeed, I had failed to notice that. That doesn't make what I am > proposing to do wrong, but it does limit its usefulness to a subset of th= e > macrolanguage families. So my Chinese example would work, but my Norwegia= n > one wouldn't. > > Is there a reason for the lack of extlang on these macrolanguages that > don't have it? inferring the equivalent information by inverting the > "Prefered value" information doesn't sound hard, but if it wasn't done, > presumably there's a reason. > > sgn does too but is not a macrolanguage. > > > It doesn't sound terribly useful to me to do the extlang canonicalization > for the sake of sgn, but neither does it sound harmful. > > =E2=80=94Florian > > > On Thu, Aug 29, 2019 at 9:14 PM Florian Rivoal wrote= : > >> >> >> > On Aug 30, 2019, at 10:08, Florian Rivoal wrote: >> > >> > >> > >> >> On Aug 29, 2019, at 22:44, John Cowan wrote: >> >> >> >> >> >> >> >> On Thu, Aug 29, 2019 at 5:00 AM Florian Rivoal >> wrote: >> >> >> >> In case that influences what you have to say, note that what I intend >> to do is not to store the canonicalized-to-extlang form anywhere. It wou= ld >> only be for internal processing: when performing an extended filtering >> operation, where it is unknown whether the ranges and tags are in extlan= g >> form or not, canonicalize both to extlang form do the extended filtering >> operation on that. >> >> >> >> In that case you can equally canonicalize away from the extlang form >> as toward it. I recommend that. >> > >> > Can you? >> > >> > Let's say you want to match (using extended filtering) the zh range >> against documents that may contain the zh-yue or yue tags (and possibly >> other zh-cmn, zh-hakka, zh, zh-HK=E2=80=A6). This could be something a t= ypesetter >> wants to do to use a particular font and set of line breaking rules for = any >> chunk of Chinese (in the broad sense) text. >> > >> > If we canonicalize to extlang form: >> > zh -> zh >> > zh-yue -> zh-yue >> > yue -> zh-yue >> > Therefore, the zh range will match both the documents that contained >> zh-yue or yue. This is what I want. >> > >> > If we canonicalize away from extlang form: >> > zh -> zh >> > zh-yue -> yue >> > yue -> yue >> > Therefore, the zh range will match neither documents that contained >> zh-yue nor yue. This is not what I want, and is worse than not >> canonicalizing at all. >> > >> > So it seems to me that no, we cannot canonicalize away from the extlan= g >> form and get the same results. >> > >> > If the extended filtering operation did something smart with >> macrolanguages, then I wouldn't need canonicalization at all, but it >> doesn't, so I feel I need to canonicalize, and as described above, only >> canonicalization to extlang actually seems to help. >> > >> > Am I missing something? >> > >> > =E2=80=94Florian >> >> Sorry for not including that in the previous message, but to give anothe= r >> example, if I want to use the no range to match any of: no, no-bok, no-n= yn, >> nb, or nn, canonicalization to extlang form works, and canonicalization >> away from it doesn't. >> >> =E2=80=94Florian > > > --00000000000021a1c905914dc022 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
It was only done for macrolanguages that already had compo= und tags like zh-yue, plus a few others that seemed likely to be treated si= miliarly, often because they only a single written form between them.=C2=A0= Aymara is a macrolanguage with two individual languages, Central and South= ern, but only Central has a written form and is an official language (of Bo= livia).=C2=A0 If you don't care about the diffference, you can use ay, = or you can use ayc or ayr to distinguish, but nobody has to worry about abs= tracting over them.



On Thu, Aug 29, 2019 at 11= :55 PM Florian Rivoal <florian@riv= oal.net> wrote:


On Aug 30, 2019, at 11:17, John Cowan <cowan@ccil.org> wrote:
Got it. Yes, with extended filtering, you need to canonicalize t= oward extlangs.=C2=A0 I thought you were talking about simple equality matc= hing after canonicalization.

Note that almost all ma= crolanguages do not have extlang tags; the languages can be identified by t= he macrolanguage or by the individual language tag, but not both.=C2=A0 Ind= eed, only ar, kok, lv, ms, sw, uz, zh have extlang tags subordinated to the= m;

Ah, indeed, I had fail= ed to notice that. That doesn't make what I am proposing to do wrong, b= ut it does limit its usefulness to a subset of the macrolanguage families. = So my Chinese example would work, but my Norwegian one wouldn't.
<= div>
Is there a reason for the lack of extlang on these macro= languages that don't have it? inferring the equivalent information by i= nverting the "Prefered value" information doesn't sound hard,= but if it wasn't done, presumably there's a reason.

sgn does too but is not a m= acrolanguage.

It doesn= 9;t sound terribly useful to me to do the extlang canonicalization for the = sake of sgn, but neither does it sound harmful.

=E2=80= =94Florian


On Thu, Aug 29, 2019 at 9= :14 PM Florian Rivoal <florian@rivoal.net> wrote:


> On Aug 30, 2019, at 10:08, Florian Rivoal <florian@rivoal.net> wrote:
>
>
>
>> On Aug 29, 2019, at 22:44, John Cowan <cowan@ccil.org> wrote:
>>
>>
>>
>> On Thu, Aug 29, 2019 at 5:00 AM Florian Rivoal <florian@rivoal.net> wrote:<= br> >>
>> In case that influences what you have to say, note that what I int= end to do is not to store the canonicalized-to-extlang form anywhere. It wo= uld only be for internal processing: when performing an extended filtering = operation, where it is unknown whether the ranges and tags are in extlang f= orm or not, canonicalize both to extlang form do the extended filtering ope= ration on that.
>>
>> In that case you can equally canonicalize away from the extlang fo= rm as toward it.=C2=A0 I recommend that.
>
> Can you?
>
> Let's say you want to match (using extended filtering) the zh rang= e against documents that may contain the zh-yue or yue tags (and possibly o= ther zh-cmn, zh-hakka, zh, zh-HK=E2=80=A6). This could be something a types= etter wants to do to use a particular font and set of line breaking rules f= or any chunk of Chinese (in the broad sense) text.
>
> If we canonicalize to extlang form:
>=C2=A0 zh -> zh
>=C2=A0 zh-yue -> zh-yue
>=C2=A0 yue -> zh-yue
> Therefore, the zh range will match both the documents that contained z= h-yue or yue. This is what I want.
>
> If we canonicalize away from extlang form:
>=C2=A0 zh -> zh
>=C2=A0 zh-yue -> yue
>=C2=A0 yue -> yue
> Therefore, the zh range will match neither documents that contained zh= -yue nor yue. This is not what I want, and is worse than not canonicalizing= at all.
>
> So it seems to me that no, we cannot canonicalize away from the extlan= g form and get the same results.
>
> If the extended filtering operation did something smart with macrolang= uages, then I wouldn't need canonicalization at all, but it doesn't= , so I feel I need to canonicalize, and as described above, only canonicali= zation to extlang actually seems to help.
>
> Am I missing something?
>
> =E2=80=94Florian

Sorry for not including that in the previous message, but to give another e= xample, if I want to use the no range to match any of: no, no-bok, no-nyn, = nb, or nn, canonicalization to extlang form works, and canonicalization awa= y from it doesn't.

=E2=80=94Florian

--00000000000021a1c905914dc022-- From nobody Thu Aug 29 21:31:10 2019 Return-Path: X-Original-To: ltru@ietfa.amsl.com Delivered-To: ltru@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DE85C120C89 for ; Thu, 29 Aug 2019 21:31:08 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.7 X-Spam-Level: X-Spam-Status: No, score=-2.7 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=rivoal.net header.b=PclQPC0q; dkim=pass (2048-bit key) header.d=messagingengine.com header.b=YMnIy1v/ Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3kzKhKrYpx7I for ; Thu, 29 Aug 2019 21:31:07 -0700 (PDT) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 96B99120C87 for ; Thu, 29 Aug 2019 21:31:07 -0700 (PDT) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id D5F4021C1B; Fri, 30 Aug 2019 00:31:06 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Fri, 30 Aug 2019 00:31:06 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivoal.net; h= content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; s=fm1; bh=o mi7Sk1hRm/9UoEPhdVQYGmUNHDLVX10wMJ4XSlVW+4=; b=PclQPC0qYzreA8Est DhTSElB4tlQXzE5Y98WX/shNrAGPnhcmn2MXkCoN4dbFpz2PG+j4/ZfJ64weZ5uS QFw8znbN6DIbKfcL9R6UGvjzDHt9GXAwlVdvbw6BMOuSqpjgKCYerNSN7ulCItNn ummEg4KJMajzxV4Ha4HE8nNrJjuJA260nA2n0i+bnhzXEMws0lPmcTjYlXFPq7nx EhkwE5X4Eo2MkeRx37XN4bhe36I2jmzAsS7FBSv6nd9OzpmPB9ItRfNAqjiRE4hc 2r0vRlfFE45vSWGiARnSRM2G8STGLBbUh1cWJj9PAsAiKu75vn21CyLEiyv86D1n XtllA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; bh=omi7Sk1hRm/9UoEPhdVQYGmUNHDLVX10wMJ4XSlVW +4=; b=YMnIy1v/88+MBZe6rnE0u/WoYiA68VV4kw/oB3hTG67dxeiZYqhNV9VoB U8nAN0/OOBnAzFx0hlfBgUrkHFUNtuAdkBMRI8z39iHEYZ9r/13S0uzMbnXX1nb7 p4XbK7hLSGB408jIKUJdfE9DtmhG7GiAl2bY65ypmrkM0P7Gb1CF9oMgDibyb8Y9 Zy7V6VYekWO503OQgYvc7ZEPHwbj4Ndkx2IC4INQvke6UZ4Xo27jGYsI3NYwFDu3 I+hml7XgjYnxbSbZSopybRPjw0i4e09cZrMnlUYENtKRUcidrp9dyw5HYCOH6+7H G0rbc1YelR6Fq/VXSd1mgJyf1LmPA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduvddrudeifedgkedvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurheptggguffhjgffgffkfhfvofesthhqmhdthhdtjeenucfhrhhomhephfhlohhr ihgrnhcutfhivhhorghluceofhhlohhrihgrnhesrhhivhhorghlrdhnvghtqeenucfkph epudekfedrjeeirdehrddvtdenucfrrghrrghmpehmrghilhhfrhhomhepfhhlohhrihgr nhesrhhivhhorghlrdhnvghtnecuvehluhhsthgvrhfuihiivgeptd X-ME-Proxy: Received: from [192.168.1.3] (ab005020.dynamic.ppp.asahi-net.or.jp [183.76.5.20]) by mail.messagingengine.com (Postfix) with ESMTPA id 2CE3280059; Fri, 30 Aug 2019 00:31:04 -0400 (EDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) From: Florian Rivoal In-Reply-To: Date: Fri, 30 Aug 2019 13:31:02 +0900 Cc: LTRU Working Group , Doug Ewell Content-Transfer-Encoding: quoted-printable Message-Id: <08B7B3B9-F19B-4D4C-AEC6-FD9429BAC192@rivoal.net> References: <20190827104755.665a7a7059d7ee80bb4d670165c8327d.0f79efb126.wbe@email03.godaddy.com> <910CB6C8-9F66-4255-B149-B146DA8E5695@rivoal.net> <73BE5AF7-0C62-425F-834E-8759628D2C5F@rivoal.net> <94B0FC03-B793-43BB-B864-38A306E6B5CA@rivoal.net> <3B2C913C-DE8F-43E2-9819-55DD349EF0BA@rivoal.net> To: John Cowan X-Mailer: Apple Mail (2.3445.104.11) Archived-At: Subject: Re: [Ltru] Mail regarding draft-ietf-ltru-4646bis and draft-ietf-ltru-matching X-BeenThere: ltru@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Language Tag Registry Update working group discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Aug 2019 04:31:09 -0000 > On Aug 30, 2019, at 13:06, John Cowan wrote: >=20 > It was only done for macrolanguages that already had compound tags = like zh-yue, plus a few others that seemed likely to be treated = similiarly, often because they only a single written form between them. = Aymara is a macrolanguage with two individual languages, Central and = Southern, but only Central has a written form and is an official = language (of Bolivia). If you don't care about the diffference, you can = use ay, or you can use ayc or ayr to distinguish, but nobody has to = worry about abstracting over them. Not sure I get it. nb is the preferred form of grandfathered+deprecated no-bok, nn of = no-nyn, and no is marked as a macrolanguage. Why do they have not = extlang? =E2=80=94Florian= From nobody Fri Aug 30 06:13:27 2019 Return-Path: X-Original-To: ltru@ietfa.amsl.com Delivered-To: ltru@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CD09D12003F for ; Fri, 30 Aug 2019 06:13:20 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.898 X-Spam-Level: X-Spam-Status: No, score=-1.898 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=ccil-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EpgGl7BXz37t for ; Fri, 30 Aug 2019 06:13:18 -0700 (PDT) Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6190B120125 for ; Fri, 30 Aug 2019 06:13:18 -0700 (PDT) Received: by mail-wm1-x333.google.com with SMTP id l2so7413240wmg.0 for ; Fri, 30 Aug 2019 06:13:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ccil-org.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=KBi14ppdK4aQeIb0zSNLbDVD7t52fJ7/ByYp/iNkgSM=; b=g/5gbmp8HByZUOtB3o3kYm2/uS0Y6HQJxiK9lkx+atOZdjhhNnWqU/ktvGk5Qbvtqi numJEQD5ad7pFt66MrhSXm0tETOqDPF+n15aFERTtK0kxts0fiUpHkBDpI6Pq71jlXOa gUvuhdw2Lj+KXDvdpnBXmWGfVwARdXG27pLAF4Ky/b4/zxMuTSyLRivEDMQ8pfcqk1io J7WDpi9SoFKqPqGjB6Zq1RQZnPS3n7lRDC4aP+qXDv5RoeNFvqzdaBrotGmNkKB1j1ak N4BWeH0H3YJnNOUiQ7e27IhBfInLE+JOheOCHN8yqSJ4zhTyy2Wdn64bql0Ryx7iHyss qXjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=KBi14ppdK4aQeIb0zSNLbDVD7t52fJ7/ByYp/iNkgSM=; b=sramgFnNqSbW0tUN3HdDgdv2pxiwgm/4Q9iC9ivBByNrGBOklRyt1rI7Nwm90LVeJf KktwgffbvyD435Z9EAQIab7qA9DTyEutAHaWYxN+yr5lQaFrMlwLQ3Aq2RLjPDfsZ6TG cJc8KUd9LB6iL3O+pyC5tnooJ+dxzoNKEiuFRySOGSHZo3j3Zfrl4RlTTr8w8P5nFYq6 mV/vB4k8AiSdWp/krWdOk9R8lpRb6e9ncUaLqMdcR7X2CtdIAHh4qjn6IgqGprSi8aHC +wjVpOFCqtQQcZsPZfqwRZO7wCx7P4s8hizquOLfKEEMJ3F+xwj45zAL0iZFn0n1sJ1Z 1FZA== X-Gm-Message-State: APjAAAVirv+r5MEjw+TDlrBwq5Gu4E8PHm7fSyh00L2PMmZ20JPAnx/r M0U0q15LJ96i0DAPQZgKGFR1D8M5JyQErsodNOIaaevDVBQ= X-Google-Smtp-Source: APXvYqzcuRbOuzmrdYdCiamPxZHU7VQ2dN55MUJJbHG2WAw39A6EnFTEAuRGH6GoO04bkcS4oaH2ZHRZW3CErgW5zps= X-Received: by 2002:a1c:96c6:: with SMTP id y189mr11633789wmd.160.1567170796736; Fri, 30 Aug 2019 06:13:16 -0700 (PDT) MIME-Version: 1.0 References: <20190827104755.665a7a7059d7ee80bb4d670165c8327d.0f79efb126.wbe@email03.godaddy.com> <910CB6C8-9F66-4255-B149-B146DA8E5695@rivoal.net> <73BE5AF7-0C62-425F-834E-8759628D2C5F@rivoal.net> <94B0FC03-B793-43BB-B864-38A306E6B5CA@rivoal.net> <3B2C913C-DE8F-43E2-9819-55DD349EF0BA@rivoal.net> <08B7B3B9-F19B-4D4C-AEC6-FD9429BAC192@rivoal.net> In-Reply-To: <08B7B3B9-F19B-4D4C-AEC6-FD9429BAC192@rivoal.net> From: John Cowan Date: Fri, 30 Aug 2019 09:13:06 -0400 Message-ID: To: Florian Rivoal Cc: LTRU Working Group , Doug Ewell Content-Type: multipart/alternative; boundary="000000000000a466540591556274" Archived-At: Subject: Re: [Ltru] Mail regarding draft-ietf-ltru-4646bis and draft-ietf-ltru-matching X-BeenThere: ltru@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Language Tag Registry Update working group discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Aug 2019 13:13:26 -0000 --000000000000a466540591556274 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable The written forms of Norwegian don't align well with particular spoken forms, unlike the national varieties of English. In Norway there is no standard spoken language: you speak your own dialect and write either Nynorsk (10-15%) or Bokm=C3=A5l. So the written form is unambiguously nb o= r nn, and the spoken form is unambiguously no. When people tag a written form as no, they usually mean Bokm=C3=A5l, but that's not the kind of thing that extended filtering was meant to handle. A young Russian worker travels to Norway, right after the Bolshevik Revolution. He enters a pub and finds all the occupants quarreling about something in a flaming temper. He asks one of them: =E2=80=9CDoes this mean= the revolution is starting in Norway, too?=E2=80=9D =E2=80=93 =E2=80=9COh no=E2= =80=9D, says a Norwegian; =E2=80=9Cwe are just arguing about how to *spell* the word.=E2=80=9D (It's in fact _revolusjon_ in both.) On Fri, Aug 30, 2019 at 12:31 AM Florian Rivoal wrote: > > > > On Aug 30, 2019, at 13:06, John Cowan wrote: > > > > It was only done for macrolanguages that already had compound tags like > zh-yue, plus a few others that seemed likely to be treated similiarly, > often because they only a single written form between them. Aymara is a > macrolanguage with two individual languages, Central and Southern, but on= ly > Central has a written form and is an official language (of Bolivia). If > you don't care about the diffference, you can use ay, or you can use ayc = or > ayr to distinguish, but nobody has to worry about abstracting over them. > > Not sure I get it. > > nb is the preferred form of grandfathered+deprecated no-bok, nn of no-nyn= , > and no is marked as a macrolanguage. Why do they have not extlang? > > =E2=80=94Florian --000000000000a466540591556274 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
The written forms of Norwegian don't align well with p= articular spoken forms, unlike the national varieties of English.=C2=A0 In = Norway there is no standard spoken language: you speak your own dialect and= write either Nynorsk (10-15%) or Bokm=C3=A5l.=C2=A0 So the written form is= unambiguously nb or nn, and the spoken form is unambiguously no.=C2=A0 Whe= n people tag a written form as no, they usually mean Bokm=C3=A5l, but that&= #39;s not the kind of thing that extended filtering was meant to handle.
A young Russian worker travels to Norway, right after the Bolshevik R= evolution. He enters a pub and finds all the occupants quarreling about som= ething in a flaming temper. He asks one of them: =E2=80=9CDoes this mean th= e revolution is starting in Norway, too?=E2=80=9D =E2=80=93 =E2=80=9COh no= =E2=80=9D, says a Norwegian; =E2=80=9Cwe are just arguing about how to *spe= ll* the word.=E2=80=9D =C2=A0 (It's in fact _revolusjon_ in both.)

On Fri, Aug 30, 2019 at 12:31 AM Florian Rivoal <florian@rivoal.net> wrote:


> On Aug 30, 2019, at 13:06, John Cowan <cowan@ccil.org> wrote:
>
> It was only done for macrolanguages that already had compound tags lik= e zh-yue, plus a few others that seemed likely to be treated similiarly, of= ten because they only a single written form between them.=C2=A0 Aymara is a= macrolanguage with two individual languages, Central and Southern, but onl= y Central has a written form and is an official language (of Bolivia).=C2= =A0 If you don't care about the diffference, you can use ay, or you can= use ayc or ayr to distinguish, but nobody has to worry about abstracting o= ver them.

Not sure I get it.

nb is the preferred form of grandfathered+deprecated no-bok, nn of no-nyn, = and no is marked as a macrolanguage. Why do they have not extlang?

=E2=80=94Florian
--000000000000a466540591556274-- From nobody Fri Aug 30 15:12:04 2019 Return-Path: X-Original-To: ltru@ietfa.amsl.com Delivered-To: ltru@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9B94D1209C9 for ; Fri, 30 Aug 2019 15:12:02 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.897 X-Spam-Level: X-Spam-Status: No, score=-1.897 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xWhOV7Day3UG for ; Fri, 30 Aug 2019 15:12:01 -0700 (PDT) Received: from p3plwbeout03-01.prod.phx3.secureserver.net (p3plsmtp03-01-2.prod.phx3.secureserver.net [72.167.218.213]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 992651209CB for ; Fri, 30 Aug 2019 15:12:01 -0700 (PDT) Received: from p3plgemwbe03-05.prod.phx3.secureserver.net ([72.167.218.133]) by :WBEOUT: with SMTP id 3p7CiDolHHysj3p7Ciqzeb; Fri, 30 Aug 2019 15:11:30 -0700 X-SID: 3p7CiDolHHysj Received: (qmail 11861 invoked by uid 99); 30 Aug 2019 22:11:30 -0000 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" X-Originating-IP: 208.51.143.189 User-Agent: Workspace Webmail 6.9.59 Message-Id: <20190830151128.665a7a7059d7ee80bb4d670165c8327d.54f6e1b154.wbe@email03.godaddy.com> From: "Doug Ewell" To: "Florian Rivoal" , "John Cowan" Cc: "LTRU Working Group" Date: Fri, 30 Aug 2019 15:11:28 -0700 Mime-Version: 1.0 X-CMAE-Envelope: MS4wfJLI57vM4ON9BTisl5ALkH0rmY1XnmvrwCHUMMjt0/vguvYqgoPqnHBLUdFsz7Ofs7hW/qmss4T6SD2/M2HHncxPBNemrDfWB6X68ZIsDcMF+cBBuVXn 1f6VTCyiU7xJdRwJZXhsZck9z/+4Re9LbtwQLBRMKdR76P+9SM17ic0mtbrhOrqJQ18XsIr7KfJPAqw+EhrWzdaG6DREFTop4lI= Archived-At: Subject: Re: [Ltru] Mail regarding draft-ietf-ltru-4646bis and draft-ietf-ltru-matching X-BeenThere: ltru@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Language Tag Registry Update working group discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Aug 2019 22:12:03 -0000 Florian Rivoal wrote:=0A =0A> nb is the preferred form of grandfathered+de= precated no-bok, nn of=0A> no-nyn, and no is marked as a macrolanguage. Why= do they have not=0A> extlang?=0A =0AOne possible reason, although it might= not be "the" reason =E2=80=94 I would=0Ahave to look into the mail archive= s to see if this was raised =E2=80=94 was=0Athat extlangs and their corresp= onding primary language subtags have to=0Abe identical, and extlangs can't = be two letters long, because then they=0Awould be region subtags.=0A =0AIt = might be worth noting that "no-bok" and "no-nyn" aren't real=0Aextlangs, de= spite looking like them. They were created under the RFC=0A1766/3066 whole-= tag registration model, back in 1995, before ISO 639=0Aadded code elements = [nb] and [nn] in 2000. That's why they are=0Agrandfathered (because the tag= s don't mean what breaking them apart into=0Asubtags would indicate) and de= precated (because there are regular=0Asubtags that are preferred).=0A =0A--= =0ADoug Ewell | Thornton, CO, US | ewellic.org=0A