Subject: Re: [Tools-team] tools.ietf.org omits to include w3c robot

From: Bob Briscoe <[email-masked]>
Date: Mon, 02 Nov 2009 14:53:23 +0000

Henrik,

Excellent. I've tested and the new rules allow in the W3C robot.

Cheers

Bob

At 12:55 29/10/2009, Henrik Levkowetz wrote:
>Hi Bob,
>
>On 2009-10-29 13:18 Bob Briscoe said the following:
> > Henrik,
> >
> > While running the W3C hyperlink checker <http://validator.w3.org/>
> > over one of my Web pages, it told me it couldn't check all my
> > hyperlinks to tools.ietf.org due to robot exclusion rules.
> >
> > You may want to add the W3C link checker to 
> <http://tools.ietf.org/robots.txt>
> >
> > Here's the suggested text from the W3C:
> > <http://validator.w3.org/docs/checklink.html#bot>
>
>I've updated tools.ietf.org/robots.txt accordingly.  Thanks for the note :-)
>
>
>Best,
>
>         Henrik

________________________________________________________________
Bob Briscoe,                                BT Innovate & Design