Subject: [saag] IDNA2008 and PKIX certificates

From: Nikos Mavrogiannopoulos <[email-masked]>
Date: Wed, 23 Nov 2016 08:19:45 +0100

Hi,
 RFC5280 and its update (6818), reference IDNA2003 (rfc3490) for
storing internationalized DNS names. However, IDNA2003 is already
obsolete standard (it seems it was already deprecated when RFC6818 was
published [0]) and in practice phased out. What is the current best
practice on internationalized names with certificates?

Is it transparently switch to IDNA2008 (rfc5890), and let software
figure out the reverse mappings to utf8 somehow?

Or is it store UTF-8 dns names on the certificate, and let the software
comparing DNS names do any mapping it deems necessary prior to
comparison?

regards,
Nikos

[0]. https://www.ietf.org/mail-archive/web/pkix/current/msg28386.html

PS. I'm resending to saag due to no re-actions in ietf-pkix list

---

From: Viktor Dukhovni <[email-masked]>
Date: Wed, 23 Nov 2016 10:22:57 -0500

> On Nov 23, 2016, at 2:19 AM, Nikos Mavrogiannopoulos <[email-masked]> wrote:
> 
> Hi,

The DNS subjectAltName in certificates is of type IA5String, which
only supports ASCII, not UTF-8.  RFC 6125 covers this topic, the
content expected by correctly implemented verifiers is A-labels.

Therefore, certificates must contain IDNA encoded hostnames
in the DNS subjectAltName, and verifiers connecting to UTF-8
hostnames must convert them to A-label form in order to construct
the reference identifier that is checked against the certificate:

   https://tools.ietf.org/html/rfc6125#section-6.4.2

   6.4.2.  Checking of Internationalized Domain Names

   If the DNS domain name portion of a reference identifier is an
   internationalized domain name, then an implementation MUST convert
   any U-labels [IDNA-DEFS] in the domain name to A-labels before
   checking the domain name.  In accordance with [IDNA-PROTO], A-labels
   MUST be compared as case-insensitive ASCII.  Each label MUST match in
   order for the domain names to be considered to match, except as
   supplemented by the rule about checking of wildcard labels
   (Section 6.4.3; but see also Section 7.2 regarding wildcards in
   internationalized domain names).

This is implemented (e.g.) in Postfix and the next release (3.2
in ~January 2017) is moving from IDNA2008 transitional to full
IDNA2008 which changes the encoding of German eszet (ß) and a
couple of other glyphs to match current registry practice.

FWIW, this is also documented in OpenSSL (I can't cite this as
an authoritative source of course, since I added that text):

   https://www.openssl.org/docs/man1.1.0/crypto/X509_check_ip_asc.html
   https://www.openssl.org/docs/man1.1.0/crypto/X509_check_host.html

   Per section 6.4.2 of RFC 6125, name values representing international
   domain names must be given in A-label form. ...

   (same text in both manpages)

I am in the process of adding DANE support to the TLS stack in
Haskell, and incidentally adding similar commentary to the
documentation there.

-- 
	Viktor.

---

From: Viktor Dukhovni <[email-masked]>
Date: Wed, 23 Nov 2016 12:55:35 -0500

> On Nov 23, 2016, at 10:22 AM, Viktor Dukhovni <[email-masked]> wrote:
> 
>> 

One more comment.  It seems that perhaps you were thinking that
verifiers would have to convert A-label IDNA names from certificates
*back* to UTF-8 for comparison with the reference identifier.

That would not be right, instead the verifier needs to convert a UTF-8
name to A-labels to compare against the certificate.

One reason this is needed, is that UTF-8 names can represent the "."
between domain name labels in 4 different ways (lifted the Haskell
DANE survey code I am working on):

    -- Besides U+002E (full stop) IDNA allows DNS labels to be
    -- separated by any of the Unicode variants U+3002 (ideographic
    -- full stop), U+FF0E (fullwidth full stop), and U+FF61
    -- (halfwidth ideographic full stop).

    dots :: [Char]
    dots = map chr [0x002E, 0x3002, 0xFF0E, 0xFF61]

All of these map to just 0x002E in the A-label form.  So for
at least (and other reasons) the mapping from the UTF-8
representation to the A-label form is many-to-one and is
thus not reversible.

The A-label form is the `canonical' form of the name for
equality comparison.

-- 
	Viktor.

---

From: Andrew Sullivan <[email-masked]>
Date: Wed, 23 Nov 2016 14:10:56 -0500

Hi,

> representation to the A-label form is many-to-one and is

Just a bit of pedantry, but possibly useful for those looking for the
relevant information.

Strictly speaking, IDNA does not "allow" the mapping in question.
IDNA works on labels, not on domain names.  There are _lots_ of ways
that domain names could be represented in a system.  This includes
encoding U-labels as non-Unicode text encodings (such as ISO 8859-* or
Shift JIS), using separator characters other than U+002E, representing
the labels in NFD, and so on.

RFC 5895 (the "mappings" document) specifies some mappings that could
be done to input that appears to be a domain name prior to processing
the input as a series of U-labels under IDNA.  One of those mappings
contemplates U+3002 (but not other code points) as a separator
character.

UTS#46 is a Unicode Technical Standard that was arguably published to
address some dissatisfacton on the part of UTC with the IETF
consensus.  In section 2.3, it specifies that a "label" is a substring
of a "domain name", and labels are separated by label-separators which
are the list you posted.  (I note that this specification of "domain
name" and "label" is actually at odds with STD 13.  UTS#46 claims that
it uses the term "domain name" as it is used in RFC 3490, though I'm
not totally convinced that's correct.  There's anyway a certain bitter
irony in this, because the somewhat imprecise use of "domain name" in
3490 is one of the things that IDNA2008 -- which obsoletes 3490 -- was
trying to fix.)  Many implementations use UTS#46, however, for
mapping, so it is probably wise to treat those characters as potential
separators of labels and map them into U+002E before continuing with
processing.  None of those characters is an accepable character in a
U-label, so there shouldn't be a problem.

Remember, also, that the dots in domain names aren't part of the wire
protocol. They only appear in presentation format.  DNS libraries know
what to do with the dots, so mapping to U+002E ought to be safe.

Best regards,

A

-- 
Andrew Sullivan
[email-masked]

---

From: Viktor Dukhovni <[email-masked]>
Date: Wed, 23 Nov 2016 15:46:20 -0500

> On Nov 23, 2016, at 2:10 PM, Andrew Sullivan <[email-masked]> wrote:
> 
>>    dots :: [Char]

Thanks for the clarification, appreciated!

> that domain names could be represented in a system. [...]

Yes, I was aware of this, sorry if the original language gave anyone
a contrary impression.  [ Indeed in that same Haskell DANE survey
code I am working on, the IDNA library encodes one label at a time,
and is not based UTS#46.  That library unfortunately implements IDNA2003,
and a Haskell IDNA2008 remains to be implemented.  So at present I still
get the wrong (transitional) encoding of "eszet" et. al.]

> contemplates U+3002 (but not other code points) as a separator

Yes, I got the separator list list from:

   https://tools.ietf.org/html/rfc3490#section-3.1

   3.1 Requirements

   IDNA conformance means adherence to the following four requirements:

   1) Whenever dots are used as label separators, the following
      characters MUST be recognized as dots: U+002E (full stop), U+3002
      (ideographic full stop), U+FF0E (fullwidth full stop), U+FF61
      (halfwidth ideographic full stop).

Which is what made it into Postfix, which, by the way, like many other
IDNA applications now uses UTS#46 via "icu4c".  I see that indeed the
IDNA2008 list of "dots" is shorter:

   https://tools.ietf.org/html/rfc5895#page-5

   4.  [IDNA2008protocol] is specified such that the protocol acts on
       the individual labels of the domain name.  If an implementation
       of this mapping is also performing the step of separation of the
       parts of a domain name into labels by using the FULL STOP
       character (U+002E), the IDEOGRAPHIC FULL STOP character (U+3002)
       can be mapped to the FULL STOP before label separation occurs.
       There are other characters that are used as "full stops" that one
       could consider mapping as label separators, but their use as such
       has not been investigated thoroughly.  This step was chosen
       because some input mechanisms do not allow the user to easily
       enter proper label separators.  Only the IDEOGRAPHIC FULL STOP
       character (U+3002) is added in this mapping because the authors
       have not fully investigated the applicability of other characters
       and the environments where they should and should not be
       considered domain name label separators.

Given that Postfix added EAI support in 2014, we should perhaps have
referenced 5895 and not 3490, but that's mostly moot, since icu4c handles
the conversion of non-ASCII domain names, and we only process the special
3490-style dots for ".example.com" sub-domain patterns, which would otherwise
be rejected by icu4c.

> not totally convinced that's correct.

:-)

> There's anyway a certain bitter
> processing.

Yep, running code plays a strong role in defining practice.  [ Perhaps if
IDNA2008 had come with a portable C implementation, more applications
would have conformed to IDNA2008 where it differs from UTS#46... ]

-- 
	Viktor.

---

From: Nikos Mavrogiannopoulos <[email-masked]>
Date: Fri, 25 Nov 2016 09:10:32 +0100

> content expected by correctly implemented verifiers is A-labels.

RFC5280 is specific (section 7.2) on how internationalized names are
handled and does not reference RFC6125 (nor RFC6125 updates RFC5280).
Even though RFC6125 links to IDNA2008 (RFC5890), RFC5280 references
IDNA2003 (RFC3490), and is very specific on how to store (ToAscii) and
present (ToUnicode). The RFC6125 instructions are largerly vague and
do not cover the reverse mapping of names (ToUnicode) for example.

> couple of other glyphs to match current registry practice.

That's my point. It seems a logical thing to do, and I was also
planning to do that for gnutls, but doing it violates RFC5280. The
main problem that I see with the automatic upgrade to IDNA2008 is that
the reverse is not easy to achieve, i.e, applications cannot display
the real name of the certificate, but instead something like
xn--oxaaa.xn--kxawhkp. One could use the ToUnicode operation of TR#46
but still that is guessing expected by the implementers.

regards,
Nikos

---

From: Viktor Dukhovni <[email-masked]>
Date: Fri, 25 Nov 2016 04:37:46 -0500

> On Nov 25, 2016, at 3:10 AM, Nikos Mavrogiannopoulos <[email-masked]> wrote:
> 

Firefox has no problem displaying that site in the address bar as:

   http://γγγ.νίκος/

while the reverse mapping may not be exactly the Greek that was
originally entered to create xn--oxaaa.xn--kxawhkp, it is an
unambiguous form of that input, no guessing, which is stable under
further repeated decoding/encoding.  So applications can, as needed,
just decode the punycode back to the relevant UTF-8 code-points.

Displaying A-labels in some suitable encoding is not something that
a library like GnuTLS should be doing.  That falls squarely into the
application space, where the application may need to map from UTF-8
to some other local encoding, and perhaps only conver labels whose
alphabets the user can read.  While I can generally distinguish code
points in Latin and Cyrillic, showing me certificate names in Chinese,
Arabic or Tamil is rather pointless, the for me, the punycode is much
more usable.

GnuTLS just needs to be able to compare reference identifiers with
certificate content.  Mapping certificate content back to A-labels
to UTF-8 is not IMHO something that GnuTLS should be doing.

-- 
-- 
	Viktor.

---

From: Nikos Mavrogiannopoulos <[email-masked]>
Date: Fri, 25 Nov 2016 11:25:53 +0100

> Firefox has no problem displaying that site in the address bar as:

That's good, though I guess that other browsers wouldn't.

> to UTF-8 is not IMHO something that GnuTLS should be doing.

You raise different points with the above, though I disagree with
them. Expecting all applications to handle this gracefully is a recipe
for disaster (you can actually verify that by checking today how many
applications except firefox do that). It is the job of the components
that handle (store,generate,use) these structures to convert to and
from the needed formats.

regards,
Nikos

---

From: Viktor Dukhovni <[email-masked]>
Date: Fri, 25 Nov 2016 12:20:07 -0500

> On Nov 25, 2016, at 5:25 AM, Nikos Mavrogiannopoulos <[email-masked]> wrote:
> 

Applications that never expected multi-byte encodings such as UTF-8 are
likely to mishandle UTF-8 strings.  You can certainly provide a new
function that returns UTF-8 decoded peer names from the certificate
(you can't decode all the subjectAltNames in all cases, some will
hold custom OIDs whose meaning you don't know), but FWIW my advice
is to return the DNS names as verbatim A-labels from the certificate.

For example, an application that obtained the peername from MX or
SRV records starts out with A-labels, and giving at UTF-8 names
back is likely counterproductive and may result in incorrect
behaviour.

Displaying incorrect names from certificates is very UI-specific,
and I strongly recommend (as a long-timea TLS pplication developer,
not recently new OpenSSL team member) that you avoid taking on
that responsibility.  Applications are better off seeing the actual
certificate content.  If they are UTF-8 aware, they can call the
relevant UTS#46 (or other library) calls.

-- 
	Viktor.

---

From: Andrew Sullivan <[email-masked]>
Date: Fri, 25 Nov 2016 12:30:39 -0500

> would have conformed to IDNA2008 where it differs from UTS#46... ]

The reason I'm following up is that this has direct relevance to
things that saag discusses.

The _reasons_ that IDNA2008 didn't conform with the running code of
IDNA2003 in all cases are outlined in RFC 4690, but at least three
issues were significant in the backwards-incompatibility corner cases.
The first was the built-in mapping of IDNA2003, which made certain
things impossible and therefore ruled certain widely-used strings in
certain scripts impossible to write under IDNA.  The second was that
IDNA2003 used the wrong kind of normalization, which meant that it was
round-trip lossy.  The third was that internationalizing common label
practices in the DNS required internationalizing LDH, not permitting
every possible character in Unicode (this is why emoji "don't work"
with IDNA -- they're not letters or digits.  Of course, some people
have tried ot use them anyway).

The idea in the idnabis WG was that, if we were going to have to break
something, we should break it as early as possible so that the
installed base is smaller.

The position of some others in the industry is that something once
released onto the Internet must be made stable forever. So remaining
backward compatible for all time becomes a hard requirement.

I think the IETF traditions militate in favour of the first of these
views, and I note that the security practice has certainly tended that
way.  The second option means that any mistake will haunt us forever,
making the requirement for perfection in even Proposed Standards even
higher than it is today.  I think we're going to need to confront this
problem more and more often over time.

Best regards,

A

-- 
Andrew Sullivan
[email-masked]

---

From: Andrew Sullivan <[email-masked]>
Date: Fri, 25 Nov 2016 12:34:58 -0500

> but still that is guessing expected by the implementers.

This is not correct.   Any valid A-label under IDNA2008 corresponds to
_exactly_ one U-label under IDNA2008, and conversely.  Indeed, this is
one problem that IDNA2008 was designed to fix, because in IDNA2003 it
isn't true (which caused trouble).

There should be no guessing of any kind.

Best regards,

A

-- 
Andrew Sullivan
[email-masked]

---

From: Watson Ladd <[email-masked]>
Date: Fri, 25 Nov 2016 12:57:14 -0500

> There should be no guessing of any kind.

There are two parsers involved: the parser the machine has for unicode
strings, and the parser my eyes have for unicode strings. These are
different, even after normalization. I can't tell the difference
between SMALL CYRILLIC O and SMALL LATIN O, or SMALL GREEK OMICRON,
let alone the various Georgian curly fries, or the smiling animals of
Luwian.

A user interface for security critical manual comparison of strings
needs to restrict the output character set significantly and in a
culturally aware way.

>
> Best regards,
>
> A
>
> --
> Andrew Sullivan
> [email-masked]
>
> _______________________________________________
> saag mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/saag

-- 
"Man is born free, but everywhere he is in chains".
--Rousseau.

---

From: Viktor Dukhovni <[email-masked]>
Date: Fri, 25 Nov 2016 16:36:07 -0500

> On Nov 25, 2016, at 12:57 PM, Watson Ladd <[email-masked]> wrote:
> 

And firefox seems to make some effort to get this right.  While the all-Greek
domain is displayed in Greek letters, a fake gооglе.com (the two 'o' glyphs
and the 'e' glyph are Cyrillic) displays as undecoded A-labels:

	xn--ggl-tdd6ba.com

ditto with Chrome and Safari.  And also for all three with www.xn--h-7sb7bav.com
(уаhоо.com).  By contrast, "гоогле.com" displays in Cyrillic.  Which IMHO supports
the view that the underlying crypto toolkit should avoid decoding and leave that
decision to the UI layer.

Of course we're still left with "мас.com" and some other potentially confusing
glyphs, but dealing with that is still a UI issue.

-- 
	Viktor.