Subject: [Tools-discuss] iCal agenda.ics and Google calendar

From: "Eggert, Lars" <[email-masked]>
Date: Mon, 5 Mar 2012 11:25:00 +0000

Hi,

when I try to import this ical calendar

	https://datatracker.ietf.org/meeting/83/agenda.ics?cfrg,dtnrg,iccrg,mobopts,ncrg,samrg

into my Google calendar, I get this error

	"Could not fetch the url because robots.txt prevents us from crawling the url."

and nothing happens.

If I retry a second time, a calendar gets imported, but is empty.

Importing into iCal on the Mac works fine.

Lars

---

From: Henrik Levkowetz <[email-masked]>
Date: Mon, 05 Mar 2012 13:59:22 +0100

Hi Lars,

> Importing into iCal on the Mac works fine.

I think it's debatable if Google should care about robots.txt when you
point to a specific url.  However, quite apart from that, we do want
google to index the datatracker, and the block-all robots.txt only should
be present in the test and development deployments.  I've taken out the
generic block from the file now.

Thanks for the alert :-)

Best regards,

	Henrik

---

From: "Eggert, Lars" <[email-masked]>
Date: Mon, 5 Mar 2012 13:07:07 +0000

On Mar 5, 2012, at 13:59, Henrik Levkowetz wrote:
> point to a specific url.

Agreed.

>  However, quite apart from that, we do want

Hm. I still get the robots.txt error when I try to add the calendar at https://datatracker.ietf.org/meeting/83/agenda.ics?cfrg,dtnrg,iccrg,mobopts,ncrg,samrg

Lars

---

From: Henrik Levkowetz <[email-masked]>
Date: Mon, 05 Mar 2012 15:01:16 +0100

Hi Lars,

> Hm. I still get the robots.txt error when I try to add the calendar at https://datatracker.ietf.org/meeting/83/agenda.ics?cfrg,dtnrg,iccrg,mobopts,ncrg,samrg

They probably cache the robots.txt information.  I'd made the content empty,
but have now changed it to have content, but not disallow anything:

  https://datatracker.ietf.org/robots.txt

Best regards,

	Henrik