Subject: Re: using AA in queries

From: Gregory Neil Shapiro <[email-masked]>
Date: Thu, 27 Jul 2000 09:24:02 -0700 (PDT)

Note that I am not part of the namedroppers mailing list.  A member of the
list forwarded this message to me for comment.  Please include me in any
replies.

>> Here's the scenario that prompted this: When sendmail tries to locate
>> the server for a mail domain, it issues a query for ANY in an effort to
>> get the A and MX records for the mail domain in a single response.

Actually, this *only* happens when sendmail checks to make sure the host
exists (dns_getcanonname()).  At delivery time, it specifically asks for MX
records, then AAAA and A records (getmxrr()).  The use of T_ANY for looking
up delivery information is a misconception.

pk> Yes, and other MTAs act in a similar fashion, which sometimes leads to
pk> problems if the result to an "ANY" query is larger than fits into a DNS
pk> UDP packet.

dns_getcanonname() deals with this situation (and variations).  From the
top of sendmail/domain.c:

/*
**  The standard udp packet size PACKETSZ (512) is not sufficient for some
**  nameserver answers containing very many resource records. The resolver
**  may switch to tcp and retry if it detects udp packet overflow.
**  Also note that the resolver routines res_query and res_search return
**  the size of the *un*truncated answer in case the supplied answer buffer
**  it not big enough to accommodate the entire answer.
*/

As well as later in the code:

				/*
				**  If the ANY query is larger than the
				**  UDP packet size, the resolver will
				**  fall back to TCP.  However, some
				**  misconfigured firewalls block 53/TCP
				**  so the ANY lookup fails whereas an MX
				**  or A record might work.  Therefore,
				**  don't fail on ANY queries.
				**
				**  The ANY query is really meant to prime
				**  the cache so this isn't dangerous.
				*/

>> this is NS data for the domain, provided by the servers for the parent
>> domain). In the latter case, sendmail has to deal with the incomplete
>> response by generating multiple unique queries that explicit request the
>> A and MX records for the domain.

pk> So that optimization turns out not to be one?

If the res_querydomain(..., T_ANY, ...) request returns -1 and 
(h_errno != TRY_AGAIN || h_errno != HOST_NOT_FOUND) then we try for
individual MX, AAAA, or A records.  However, if HOST_NOT_FOUND is returned,
we do not go for the specific records.  Likewise, if TRY_AGAIN is returned,
we return a temporary error.

to unsubscribe send a message to [email-masked] with
the word 'unsubscribe' in a single line as the message text body.

---

From: Gregory Neil Shapiro <[email-masked]>
Date: Thu, 27 Jul 2000 09:47:23 -0700 (PDT)

>> **  the cache so this isn't dangerous.

ehall> Interesting. Is this a complete and true statement by itself? If so, my
ehall> point example is false to begin with.

Partially.  In 99% of the cases, sendmail will canonify a hostname before
delivery time.  If that canonification is able to get the necessary
information in cache (MX, AAAA, A records) into the cache, then it is
successful in this goal since at delivery time, the specific MX and AAAA/A
queries can be answered out of the cache.  The comment is also meant to
convey the fact that even if the T_ANY request fails, it doesn't break
anything.

---

From: "Eric A. Hall" <[email-masked]>
Date: Thu, 27 Jul 2000 09:38:16 -0700

>                           **  the cache so this isn't dangerous.

Interesting. Is this a complete and true statement by itself? If so, my
point example is false to begin with.

-- 
Eric A. Hall                                      http://www.ehsco.com/
Internet Core Protocols        http://www.oreilly.com/catalog/coreprot/

---

From: "Eric A. Hall" <[email-masked]>
Date: Thu, 27 Jul 2000 10:06:47 -0700

> Partially.  In 99% of the cases, sendmail will canonify a hostname
> before delivery time.

Ok, this is what we're talking about. The process above occurs with ANY,
followed by A and MX when ANY fails. We're not talking about delivery
yet, we're only talking about finding the servers for the mail domain
(canonifying the target as you say).

My point was that ANY fails on most external lookups because the servers
for the destination domain's parent often answer a query of ANY with the
NS records from the destination domain's delegation. For example,
a.root-servers.net. responds to ANY query for ehsco.com. with NS records
for ehsco.com. since those records satisfies the query for "all of the
known RRs associated with ehsco.com."

Thus, in those cases where mail is for a remote domain that has not been
recently resolved, the query for ANY is almost guaranteed to fail and
will be followed with two additional queries. However, since the
Authority Section of the (incomplete) answer points to the authoritative
servers for the mail domain, the A and MX queries go directly to the
authoritative servers and are able to get answered immediately and
completely.

It would save a lot of work if ANY went to those servers in the first
place is my thinking, and is an example of a proposed beneficary of an
Authoritative Answer Required flag for queries.

The other approach is not to use ANY at all, and just do A and MX in the
first place. However, in those cases where the data is cached (such as
would happen with recently or frequently resolved domains), using ANY is
more efficient since it is just one transaction.

Just trying to clearly identify what's going on.

> at delivery time, the specific MX and AAAA/A queries can be answered
> out of the cache.

Yeah, but that's a different part of the cycle. Getting ANY answered
correctly wouldn't affect that.

---

From: Robert Elz <[email-masked]>
Date: Fri, 28 Jul 2000 03:35:59 +1000

Date:        Thu, 27 Jul 2000 10:06:47 -0700
    From:        "Eric A. Hall" <[email-masked]>
    Message-ID:  <[email-masked]>

  | We're not talking about delivery
  | yet, we're only talking about finding the servers for the mail domain
  | (canonifying the target as you say).

No, canonifying the target is testing whether it is an alias, and
if so, substituting the RHS of the CNAME record.

Using ANY to look for CNAME is generally successful, as if there are
any RRs at all, then they must be the CNAME - otherwise no CNAME exists.
The side-effect of perhaps getting other records into the local cache
while determining that is just a mild efficiency hack (might as well
get the auth server to send any data that exists, rather than just
saying "no CNAME", if you end up talking to it).

  | My point was that ANY fails on most external lookups because the servers
  | for the destination domain's parent often answer a query of ANY with the
  | NS records from the destination domain's delegation. For example,
  | a.root-servers.net. responds to ANY query for ehsco.com. with NS records
  | for ehsco.com. since those records satisfies the query for "all of the
  | known RRs associated with ehsco.com."

No, that has nothing whatever to do with ANY queries - any query at all
sent to the root servers for that domain would return the same answer.
That's a referral.

  | Thus, in those cases where mail is for a remote domain that has not been
  | recently resolved, the query for ANY is almost guaranteed to fail

The local cache (resolver back end) should really be following any referral
even if it was looking for ANY - it should be sending the ANY query to the
auth servers.   What broken server are you using that isn't doing that?
(note: this assumes that there was no local cached data for the name
previously - simply returning what is in the local cache is a useful
optimisation, even if it does lead to some of the data not being available).

  | It would save a lot of work if ANY went to those servers in the first
  | place is my thinking, and is an example of a proposed beneficary of an
  | Authoritative Answer Required flag for queries.

That should happen without any AA flag - the only difference the proposed AA
would make would be if there was already data in the cache.  For sendmail
purposes that's irrelevant though, as that "other data" is all sendmail
needs to know to know there is no CNAME.   If the other data is not A or MX
then the specific query will go look for that later.

kre

---

From: "Eric A. Hall" <[email-masked]>
Date: Thu, 27 Jul 2000 11:06:55 -0700

> any RRs at all, then they must be the CNAME - otherwise no CNAME
> exists. The side-effect of perhaps getting other records into the
> local cache while determining that is just a mild efficiency hack
> (might as well get the auth server to send any data that exists,
> rather than just saying "no CNAME", if you end up talking to it).

This clarifies things somewhat.

> | My point was that ANY fails on most external lookups because the
> | servers for the destination domain's parent often answer a query
> | of ANY with the NS records from the destination domain's delegation.
> | For example, a.root-servers.net. responds to ANY query for
> | ehsco.com. with NS records for ehsco.com. since those records
> | satisfies the query for "all of the known RRs associated with
> | ehsco.com."
> 
> No, that has nothing whatever to do with ANY queries - any query at
> all sent to the root servers for that domain would return the same
> answer. That's a referral.

Queries for ANY return non-authoritative answer data, while queries for
explicit types return referrals.

[email-masked]>  dig @a.root-servers.net. ehsco.com. ANY

; <<>> DiG 8.3 <<>> @a.root-servers.net. ehsco.com. any 
; (1 server found)
;; res options: init recurs defnam dnsrch
;; got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 6
;; flags: qr rd; QUERY: 1, ANSWER: 5, AUTHORITY: 5, ADDITIONAL: 5
;; QUERY SECTION:
;;      ehsco.com, type = ANY, class = IN

;; ANSWER SECTION:
ehsco.com.              2D IN NS        ARACHNID.ehsco.com.
ehsco.com.              2D IN NS        NAMESERVER.CONCENTRIC.NET.
ehsco.com.              2D IN NS        NAMESERVER3.CONCENTRIC.NET.
ehsco.com.              2D IN NS        NAMESERVER2.CONCENTRIC.NET.
ehsco.com.              2D IN NS        NAMESERVER1.CONCENTRIC.NET.

;; AUTHORITY SECTION:
ehsco.com.              2D IN NS        ARACHNID.ehsco.com.
ehsco.com.              2D IN NS        NAMESERVER.CONCENTRIC.NET.
ehsco.com.              2D IN NS        NAMESERVER3.CONCENTRIC.NET.
ehsco.com.              2D IN NS        NAMESERVER2.CONCENTRIC.NET.
ehsco.com.              2D IN NS        NAMESERVER1.CONCENTRIC.NET.

;; ADDITIONAL SECTION:
ARACHNID.ehsco.com.     2D IN A         209.31.7.46
NAMESERVER.CONCENTRIC.NET.  2D IN A  207.155.183.72
NAMESERVER3.CONCENTRIC.NET.  2D IN A  206.173.119.72
NAMESERVER2.CONCENTRIC.NET.  2D IN A  207.155.184.72
NAMESERVER1.CONCENTRIC.NET.  2D IN A  207.155.183.73

;; Total query time: 364 msec
;; FROM: Goose.NTRG.com to SERVER: a.root-servers.net.  198.41.0.4
;; WHEN: Thu Jul 27 10:56:14 2000
;; MSG SIZE  sent: 27  rcvd: 317

[email-masked]>  dig @a.root-servers.net. ehsco.com. MX 

; <<>> DiG 8.3 <<>> @a.root-servers.net. ehsco.com. MX 
; (1 server found)
;; res options: init recurs defnam dnsrch
;; got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 6
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 5, ADDITIONAL: 5
;; QUERY SECTION:
;;      ehsco.com, type = MX, class = IN

;; AUTHORITY SECTION:
ehsco.com.              2D IN NS        ARACHNID.ehsco.com.
ehsco.com.              2D IN NS        NAMESERVER.CONCENTRIC.NET.
ehsco.com.              2D IN NS        NAMESERVER3.CONCENTRIC.NET.
ehsco.com.              2D IN NS        NAMESERVER2.CONCENTRIC.NET.
ehsco.com.              2D IN NS        NAMESERVER1.CONCENTRIC.NET.

;; ADDITIONAL SECTION:
ARACHNID.ehsco.com.     2D IN A         209.31.7.46
NAMESERVER.CONCENTRIC.NET.  2D IN A  207.155.183.72
NAMESERVER3.CONCENTRIC.NET.  2D IN A  206.173.119.72
NAMESERVER2.CONCENTRIC.NET.  2D IN A  207.155.184.72
NAMESERVER1.CONCENTRIC.NET.  2D IN A  207.155.183.73

;; Total query time: 363 msec
;; FROM: Goose.NTRG.com to SERVER: a.root-servers.net.  198.41.0.4
;; WHEN: Thu Jul 27 11:02:18 2000
;; MSG SIZE  sent: 27  rcvd: 247

Returns NS answers for ANY, referrals for explicits.

The ANY query is completely processed and returned to the calling
process with the answer data.

> | It would save a lot of work if ANY went to those servers in the
> | first place is my thinking, and is an example of a proposed
> | beneficary of an Authoritative Answer Required flag for queries.
> 
> That should happen without any AA flag

This seems to be a common refrain. I will submit a bug report for BIND.

---

From: Peter Koch <[email-masked]>
Date: Fri, 28 Jul 2000 17:06:46 +0200

> [email-masked]>  dig @a.root-servers.net. ehsco.com. ANY

a.root-servers.net (like "F" and "G") not only serves "." but also "COM",
so of course an ANY query returns the NS RRs (as would an explicit NS query),
while an MX query is answered by a referral (due to non-recursiveness).
The NS RR responses are not authoritative because they are the delegation
info from the parent (COM) zone.  No magic.

-Peter

---

From: "Eric A. Hall" <[email-masked]>
Date: Fri, 28 Jul 2000 08:31:14 -0700

[ please continue this over in dnsop -- randy ]

> a.root-servers.net (like "F" and "G") not only serves "." but also

Just because I sent the dig query to a.gtld-servers.net. doesn't mean
that it's the only server that's returning incomplete answers to queries
for ANY. This behavior manifests whenever a query for ANY encounters a
BIND server in the last-hop delegation, so the same thing happens with
*.gtld-servers.net., various places within the ccTLD delegation trees,
and also within private delegations (punch in "dig @ns.ibm.com.
chips.ibm.com. ANY" for example). It happens everwhere.

> so of course an ANY query returns the NS RRs (as would an explicit
> NS query), while an MX query is answered by a referral (due to
> non-recursiveness).

If you believe that "all records" is satisfied by "RRs I know about",
then yes "of course" it does.

> The NS RR responses are not authoritative because they are the
> delegation info from the parent (COM) zone.  No magic.

Yep.

---

From: Paul A Vixie <[email-masked]>
Date: Fri, 28 Jul 2000 17:43:59 -0700

> info from the parent (COM) zone.  No magic.

note that bind8 wrongly sends back an answer for ANY and NS queries while
bind9 correctly sends back a referral in both cases since the authoritative
answer has to come from the authority zone rather than the delegating zone.

---

From: "D. J. Bernstein" <[email-masked]>
Date: 29 Jul 2000 06:47:37 -0000

Paul A Vixie writes:
> note that bind8 wrongly sends back an answer for ANY and NS queries
> while bind9 correctly sends back a referral in both cases since the
> authoritative answer has to come from the authority zone rather than
> the delegating zone.

Caches around the net will still save the NS records from the parent and
give them to clients on subsequent requests. There's no guarantee that
the child NS records will show up to replace the parent NS records in
the cache before this happens.

If the child servers are unreachable for a few seconds, for example,
then the client's retry of its * query will be answered by the parent's
NS records, rather than timing out.

As for ``wrongly'': The BIND 8 behavior and the cache behavior are
entirely permissible under RFC 1034 and RFC 1035. Under those RFCs, the
way to stop the roots from spreading bad data for (say) munnari.oz.au is
to avoid giving them bad data in the first place.

---Dan

P.S. My server sends back a referral in this situation. I found this
much easier to implement than the BIND 8 behavior.

---

From: Robert Elz <[email-masked]>
Date: Sun, 30 Jul 2000 06:44:08 +1000

Date:        29 Jul 2000 06:47:37 -0000
    From:        "D. J. Bernstein" <[email-masked]>
    Message-ID:  <[email-masked]>

  | Caches around the net will still save the NS records from the parent and
  | give them to clients on subsequent requests.

That's true, and relatively harmless.

  | As for ``wrongly'': The BIND 8 behavior and the cache behavior are
  | entirely permissible under RFC 1034 and RFC 1035.

Perhaps.   And perhaps this is another area that could do with some
clarification.   I would have thought (and obviously did so in the
earlier message I sent) that a server sending referrals should send
just referrals (ie: if RA isn't set, and the server isn't authoritative).

On the other hand, if the server will attempt to return answers (RA is
set, and the request contained RD) then returning the delegation (glue)
NS records as answers is just fine (as well as returning them in the
auth section, though I always thought returning the same RRs twice in
an answer as bordering on stupidity).

  | Under those RFCs, the
  | way to stop the roots from spreading bad data for (say) munnari.oz.au is
  | to avoid giving them bad data in the first place.

If only it were easy to make them do that.   Unfortunately, the broken
and totally bogus database used (not BIND, the thing they use to create
the zone files for BIND) insists on returning only half of the RRset that
is munnari.oz.au's A records.

If you have any insights on how to convince them to fix that broken
behaviour, please advise...

  | P.S. My server sends back a referral in this situation. I found this
  | much easier to implement than the BIND 8 behavior.

And much more correct I believe.

kre