Subject: [Tools-team] tools.ietf.org omits to include w3c robot

From: Bob Briscoe <[email-masked]>
Date: Thu, 29 Oct 2009 12:18:28 +0000

Henrik,

While running the W3C hyperlink checker <http://validator.w3.org/> 
over one of my Web pages, it told me it couldn't check all my 
hyperlinks to tools.ietf.org due to robot exclusion rules.

You may want to add the W3C link checker to <http://tools.ietf.org/robots.txt>

Here's the suggested text from the W3C:
<http://validator.w3.org/docs/checklink.html#bot>

Cheers

Bob

________________________________________________________________
Bob Briscoe,                                BT Innovate & Design

---

From: Henrik Levkowetz <[email-masked]>
Date: Thu, 29 Oct 2009 13:55:57 +0100

Hi Bob,

> <http://validator.w3.org/docs/checklink.html#bot>

I've updated tools.ietf.org/robots.txt accordingly.  Thanks for the note :-)

Best,

	Henrik