Subject: [Tools-discuss] Why does the Datatracker take so long to notice new RFCs?

From: Paul Hoffman <[email-masked]>
Date: Thu, 30 Aug 2012 07:09:21 -0700

RFC 6722 was posted last night. As of this moment, the following search fails:

https://datatracker.ietf.org/doc/search/?name=6722&rfcs=on&search_submit=

Why the delay?

--Paul Hoffman

---

From: Alice Russo <[email-masked]>
Date: Thu, 30 Aug 2012 12:15:24 -0400

Paul,

The datatracker relies on rfc-index.xml (from rfc-editor.org) to get this data. On the RFC Editor end, after publication yesterday, we had an unusual delay (12 hours) in updating that XML file due to internal upgrades. 

In general, when an RFC publication announcement goes out, the RFC is available from the RFC search page (http://www.rfc-editor.org/rfcsearch.html) and various RFC index files (including the XML file) have been updated. Typically, there is a short delay before it's visible in the datatracker.  (I understand rfc-index.xml is polled daily.)

Btw, I think one goal of the upcoming implementation of extensions for datatracker / IANA / RFC Editor interaction (specified in RFC 6359) is to eliminate the typical delay -- or come close.

Thank you.

Alice Russo
RFC Production Center

On Aug 30, 2012, at 10:09 AM, Paul Hoffman wrote:

> _______________________________________________
> Tools-discuss mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tools-discuss
>

---

From: Paul Hoffman <[email-masked]>
Date: Thu, 30 Aug 2012 09:34:17 -0700

On Aug 30, 2012, at 9:15 AM, Alice Russo <[email-masked]> wrote:

> The datatracker relies on rfc-index.xml (from rfc-editor.org) to get this data.

Thanks, that's useful information.

> On the RFC Editor end, after publication yesterday, we had an unusual delay (12 hours) in updating that XML file due to internal upgrades. 

Also useful.

> In general, when an RFC publication announcement goes out, the RFC is available from the RFC search page (http://www.rfc-editor.org/rfcsearch.html) and various RFC index files (including the XML fi…

"Daily" seems silly; hourly or even quarter-hourly seems much more sensible.

The reason I bring this up is that we have people who want to subscribe to the Atom feeds for RFCs in which they are interested. An RFC announcement comes out; a person says "that's an interesting RFC" and starts to read it; they think it is interesting enough to want to subscribe to the Atom feed so they know when the RFC has been updated or there is an errata.

It is a disservice to make them have to wait an indeterminate amount of time in order to be able to subscribe to the Atom feed.

> Btw, I think one goal of the upcoming implementation of extensions for datatracker / IANA / RFC Editor interaction (specified in RFC 6359) is to eliminate the typical delay -- or come close.

That work seems off in the future; a simple fix would be for the Datatracker to poll for rfc-index.xml much more often.

--Paul Hoffman

---

From: Henrik Levkowetz <[email-masked]>
Date: Thu, 30 Aug 2012 21:12:55 +0200

Hi,

Expanding on Alice's statement below regarding an upcoming enhancement to
the IETF datatracker:

On 2012-08-30 18:15 Alice Russo said the following:

> Btw, I think one goal of the upcoming implementation of extensions
> for datatracker / IANA / RFC Editor interaction (specified in RFC
> 6359) is to eliminate the typical delay -- or come close.

Yes.  There's a project which is nearing completion where we're working
towards having state updates from both the RFC-Editor and IANA to the IETF
datatracker, (and vice versa, eventually), work by providing a method which
lets the originator of a state change push a notification (by using a HTTP
POST to a well-known URL) as soon as there's a state change to fetch.

This will let us exchange timely state change updates, and will still let us
fall back to cron-based polling if the notifications fail.

I believe the project should be ready for deployment within a few weeks,
for notifications from the RFC-Editor and IANA to the IETF, with the other
direction coming somewhat later.  There may be some initial tuning needed,
but we should nevertheless be in a much better place fairly soon.

Best regards,

	Henrik