Subject: [dnssd] A concrete example of the Punycode fallback variants for the <Domain> portion

From: Ilya Kulakov <[email-masked]>
Date: Sun, 15 Dec 2013 14:15:15 +0200

I’m trying to implement the algorithm suggested by the RFC #6763:
In cases where the
DNS server returns a negative response for the name in question,
client software MAY choose to retry the query using the "Punycode"
algorithm [RFC3492] to convert the UTF-8 name to an IDNA "A-label"
[RFC5890], beginning with the top-level label, then issuing the query
repeatedly, with successively more labels translated to IDNA A-labels
each time, and giving up if it has converted all labels to IDNA
A-labels and the query still fails.

But unfortunately there is no concrete example.

If I have domain is ‘my.домен.рф.’ what are the fallback variants? Should I try every combination:
‘my.домен.рф.’
‘my.домен.xn--p1ai.’
‘my.xn--d1acufc.рф.’
''my.xn--d1acufc.xn--p1ai.’
Or I should avoid “gaps”:
‘my.домен.рф.’
‘my.домен.xn--p1ai.’
''my.xn--d1acufc.xn--p1ai.’
?

---

From: Andrew Sullivan <[email-masked]>
Date: Sun, 15 Dec 2013 10:53:48 -0500

> ''my.xn--d1acufc.xn--p1ai.’

This is the only correct answer.  You have to cover all possible
combinations.  However, the text doesn't read that way.  It reads as

> ''my.xn--d1acufc.xn--p1ai.’

I suspect this is because the authors figured there'd be no way anyone
would have an A-label inside a pure UTF-8 label.  I think this is a
serious underestimation of the inconsistencies actually found on the
Internet.

If I were doing it, I'd use your strategy 2 first on the grounds of
optimism, but I'd fall back to 1 if I got Name Error for everything.

Best,

A

-- 
Andrew Sullivan
[email-masked]

---

From: manning bill <[email-masked]>
Date: Mon, 16 Dec 2013 08:49:31 -0800

ah the naive authors….   not only have mixed labels been considered and tested, there was an initial exploration of mixed encoding within a label…
testing -every- combination has to be bound in some way - otherwise this,in practice creates a condition that we are  unable to observe in a finite time.
/bill
Neca eos omnes.  Deus suos agnoscet.

On 15December2013Sunday, at 7:53, Andrew Sullivan <[email-masked]> wrote:

> _______________________________________________
> dnssd mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dnssd

---

From: Andrew Sullivan <[email-masked]>
Date: Mon, 16 Dec 2013 12:03:28 -0500

> testing -every- combination has to be bound in some way - otherwise this,in practice creates a condition that we are  unable to observe in a finite time.

Yes, of course.  The problem, however, is the distributed nature of
the DNS.  A parent cannot control what a child's administration policy
will be once the namespace is delegated.  Similarly, a child cannot
control the parent's administrative policies.  So in fact names could
end up in either state.

The basic problem here is that we have top-level and second-level
domains with a policy that still requires LDH in the DNS.  The
rationale for not using IDNA is made clear in 6763, but it's an
obvious fact that, if DNS-SD is going to be generally useful on the
wider Internet, it's going to have to accommodate itself to the way
internationalization is handled in the DNS.  One can make lots of
arguments why IDNA sucks.  But it's the way this has been handled, and
so there's no hope that we're going to get away from mixed labels.

The only alternative, of course, is to duplicate the IDNA U-labels
with the plain UTF-8 data.  This seems at least error prone.  It also
has the problem of potentially inconsistent caches when a change is
introduces and one of the label "forms" has previously been queried
but not the other.

Best regards,

A

-- 
Andrew Sullivan
[email-masked]

---

From: Douglas Otis <[email-masked]>
Date: Mon, 16 Dec 2013 09:07:31 -0800

Dear Andrew,

This might not be as bad as you might think.

1) Hostnames should be subjected to the same canonicalization to establish valid UTF-8 strings prior to conversion.  Bonjour already supports a means to override naming conflicts often seen as a prefixed number value (n).  Once proper canonicalization is applied, users may be required  to adjust the hostname if unhappy with the results. 

2) There are two processes normally involved.  One is discovering services after selecting from a list of domains, and resolving services after selecting from a list of hostnames.  In each case, both of these presentations can be done using UTF-8 (provided proper  canonicalization is applied.  Whether the label gets converted to an A-Label will depend on the selection of the domain.

Regards,
Douglas Otis

---

From: Andrew Sullivan <[email-masked]>
Date: Mon, 16 Dec 2013 12:45:33 -0500

Hi,

I'm not sure I entirely understand your message.  But. . .

On Mon, Dec 16, 2013 at 09:07:31AM -0800, Douglas Otis wrote:
> 
> 1) Hostnames should be subjected to the same canonicalization to
> establish valid UTF-8 strings prior to conversion.  Bonjour already
> supports a means to override naming conflicts often seen as a
> prefixed number value (n).  Once proper canonicalization is applied,
> users may be required to adjust the hostname if unhappy with the
> results. 

When you say "the same canonicalization", what do you mean?  The DNS
owner names that result from RFC 6763 are in Net-Unicode (RFC 5198);
this entails NFC, but not much more.  The U-label portions of any
owner name arising from IDNA2008 have more restrictions -- most
punctuation is disallowed, upper case characters are not permitted
because they're not stable under case folding and NFKC, spaces are
disallowed, and so on.  This difference is why I suggested a common
interoperable profile (see draft-sullivan-dnssd-label-miprofile-00.
Stuart had some quibbles about the recommendations in there, but in
any case the recommendations are outside our charter.  I'm supposed to
pull out the problem statement part, which I guess I could be doing
rather than writing this mail).

> 2) There are two processes normally involved.  One is discovering
> services after selecting from a list of domains, and resolving
> services after selecting from a list of hostnames.  In each case,
> both of these presentations can be done using UTF-8 (provided proper
> canonicalization is applied.  Whether the label gets converted to an
> A-Label will depend on the selection of the domain.  

If I understand the above, you're suggesting that the user will know
how to cope with the issue.  Frankly, I think that is just naïve.
Users do not have a theory of operation of IDNA vs. plain UTF-8 labels
in the DNS, and subtle inconsistencies between them are going to be at
least mystifying and at worst opportunities for bad behaviour.

Best,

A

-- 
Andrew Sullivan
[email-masked]

---

From: Douglas Otis <[email-masked]>
Date: Mon, 16 Dec 2013 12:00:45 -0800

Dear Andrew,

I hope to create a write-up better describing this situation.  Bonjour should not be considered equivalent to DNS, although DNS plays a role. Once resolving services beyond link local are in scope, a type of canonicalization which includes RFC5198 normalization is required in checks for hostname conflicts.  Bonjour hostnames are not constrained to what is legal for registered domains, but this freedom is for a single left-most label when resolving the hostname.  Labels that do not represent hostnames should fall within the constraints imposed by DNS.  

> least mystifying and at worst opportunities for bad behaviour.

Users should only see UTF-8 strings, not A-Labels.  A resolution process must apply rules needed to satisfy Bonjour and DNS separately as a two or three step process,  Domain/Service/Hostname or Service/Hostname.  Of course DNS will require registered domains expressed as A-labels, but hidden from users.  Don't expect users to recognize A-label domains. 

Combining domain selection with hostname selection will not scale well when scope goes beyond link local.  Use of search domains must be handled carefully and may even require confirmation by users.  

Regards,
Douglas Otis

---

From: Andrew Sullivan <[email-masked]>
Date: Mon, 16 Dec 2013 15:40:56 -0500

On Mon, Dec 16, 2013 at 12:00:45PM -0800, Douglas Otis wrote:
> 
> I hope to create a write-up better describing this situation.
> Bonjour should not be considered equivalent to DNS, although DNS
> plays a role.

I don't think of Bonjour as equivalent to the DNS.  We're talking
about DNS-SD, however, not Bonjour (which uses DNS-SD but is more than
that).

> normalization is required in checks for hostname conflicts.  Bonjour
> hostnames are not constrained to what is legal for registered
> domains, but this freedom is for a single left-most label when
> resolving the hostname.

What exactly do you mean by "legal for registered domains"?  If what
you're saying is that Bonjour names are not constrained to LDH, then I
agree with you.  The point is -- and DNS-SD relies on this -- that
nameservers themselves are in fact not constrained to LDH, and can
happily serve labels that are any combination of octets, including
labels that are valid UTF-8.

> Labels that do not represent hostnames
> should fall within the constraints imposed by DNS.

The constraints on labels imposed by DNS as a protocol are minimal.
The problem is that there is a preferred syntax that led us to develop
IDNA.  And so now there are two ways to "spell" the same string in the
DNS.  DNS-SD assumes a label will be in the form "σόλος", and IDNA
assumes the label will be in the form "xn--wxaikc6b" (which has a
U-label "σόλος").  Once we have several levels of internationalized
label in the name, we have to expect that we'll get a mix of
U-label/A-label pairs and plain UTF-8 in the DNS.

Now, the problem that I've been trying to talk about is that DNS-SD
does not impose a number of restrictions on the labels that IDNA
does.  For example, under IDNA2008 Σόλος is not a valid U-label.
That's a perfectly acceptable label under DNS-SD, however; and indeed
RFC 6763 goes out of its way to recommend mixed case, spaces, and so
on.  There's nothing wrong with that, but if we're going to expect
users to be able to cope with this mechanism without needing to know
when they're looking at an IDNA label and when they're not, then the
rules are going to have to be consistent enough not to be surprising.  

> Users should only see UTF-8 strings, not A-Labels.

But what I am trying to say is that there are two classes of UTF-8
string here: the ones that DNS-SD permits and the subset of that which
constitutes protocol-valid labels under IDNA.  These two classes have
different properties, and I think you may be conflating them.

> Use of search domains must be handled carefully and may even require
> confirmation by users.

Again, I want to emphasise that if our answer is "confirmation or
selection by users" for any parts of the machinery, we will have
failed completely.  The whole point of this work is that people have
asked for a do-what-I-mean solution.  The answer can't be "get
confirmation from the user".

Best regards,

A

-- 
Andrew Sullivan
[email-masked]