Subject: [Tools-discuss] RFC search engine

From: "David Harrington" <[email-masked]>
Date: Thu, 28 Apr 2011 12:40:35 -0400

Hi,

What is the best way to search for all BCPs in the datatracker?
The old RFC search allowed you to request BCP, STD, and FYI as well as
RFC.

Is there a way to display the keywords associated with search results?
Is there a way to update the keywords for a document? (e.g., add a
keyword)

David Harrington
Director, IETF Transport Area
[email-masked] (preferred for ietf)
[email-masked]
+1 603 828 1401 (cell)

---

From: Alice Hagens <[email-masked]>
Date: Thu, 28 Apr 2011 16:27:23 -0700

David,

Although you are asking about the datatracker, from http://www.rfc-editor.org/rfcsearch.html you can search as follows. On the right, select Search: BCP and Show Keywords: On.  Enter text in the search box, such as simply the character "." (as it does not allow empty search).  This yields a list of 159 RFCs and their keywords.  Additionally, lists of RFCs by status are available from http://www.rfc-editor.org/category.html.

The RFC Editor maintains the keywords for RFCs.  Updates should be sent to [email-masked].

Thank you.

Alice
for the RFC Production Center

On Apr 28, 2011, at 9:40 AM, David Harrington wrote:

> _______________________________________________
> Tools-discuss mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tools-discuss
>

---

From: Fred Baker <[email-masked]>
Date: Thu, 28 Apr 2011 13:53:35 -0700

> Is there a way to update the keywords for a document? (e.g., add a

I have built myself a few ad hoc tools that build on an internet draft or RFC mirror I keep on my laptop, but to be honest I'm not happy with them. I do a lot of old-fashioned grepping.

I have attached some files. You may find the data useful. Or not. :-)

I'm giving you two versions. I have built myself a few ad hoc tools that build on an internet draft or RFC mirror I keep on my laptop, but to be honest I'm not happy with them. I do a lot of old-fashioned grepping. I frequently use a hack tool I wrote for myself (which I call get-refs). It is essentially a limited version of grep that will look through a text file for paragraphs (not lines) that *do* contain one or more of a set of character strings and *do not* contain any of another set of character strings. The character strings may contain white space, which includes spanning lines. Hence, it can look not only for words but phrases. As with any unix filter, it can be used in tandem through a pipe for certain purposes. I find it useful, but if I were to commission a proper tool I wouldn't do it this way (I would probably want some form of database query). My point here is partly to give you some data that may be useful to your question, and to give some discussion of tools that may give you and Henrik some ideas on questions I find myself asking and therefore possible use cases for the tool. One thing that I would change if I could easily think of a way to do it would be to be able to look for this *and* that - "contains '(status: best current practice)' *and* 'Nominations Committee', but neither 'obsoleted' nor 'historic'".

The attached "best-current-practice.txt" file is the result of

	rfc '(status: best current practice)' 

which devolves to 

	get-refs ~/rfc/rfc-index.txt -obsoleted -historic '(status: best current practice)' 

In other words, it scans rfc-index.txt for paragraphs that contain '(status: best current practice)' and do not contain either 'historic' or 'obsoleted' and writes them to stdout. It also concocts a URL for the file, which I find useful primarily when I'm telling someone else about the result. 

I have a similar tool for internet drafts, with the exception that it scans a list of current internet draft versions for the text I give it and then gives me the corresponding text from 1id-index.txt along with a URL. If I ask it about 6to4, for example, I find that there are four current internet drafts that have the character string in their moniker:

http://datatracker.ietf.org/doc/draft-ietf-v6ops-6to4-advisory
http://tools.ietf.org/html/draft-ietf-v6ops-6to4-advisory
  "Advisory Guidelines for 6to4 Deployment", Brian Carpenter, 26-Apr-11,
  <draft-ietf-v6ops-6to4-advisory-01.txt>

http://datatracker.ietf.org/doc/draft-ietf-v6ops-6to4-to-historic
http://tools.ietf.org/html/draft-ietf-v6ops-6to4-to-historic
  "Request to move Connection of IPv6 Domains via IPv4 Clouds (6to4) to
  Historic status", Ole Troan, 27-Apr-11,
  <draft-ietf-v6ops-6to4-to-historic-01.txt>

http://datatracker.ietf.org/doc/draft-kuarsingh-v6ops-6to4-provider-managed-tunnel
http://tools.ietf.org/html/draft-kuarsingh-v6ops-6to4-provider-managed-tunnel
  "6to4 Provider Managed Tunnels", Victor Kuarsingh, Yiu Lee, Olivier
  Vautrin, 13-Mar-11,
  <draft-kuarsingh-v6ops-6to4-provider-managed-tunnel-02.txt>

http://datatracker.ietf.org/doc/draft-troan-v6ops-6to4-to-historic
http://tools.ietf.org/html/draft-troan-v6ops-6to4-to-historic
  "Request to move Connection of IPv6 Domains via IPv4 Clouds (6to4) to
  Historic status", Ole Troan, 9-Mar-11,
  <draft-troan-v6ops-6to4-to-historic-01.txt>

The "bcps in numeric order" is the result of a similar search, but this time in a command-line script. What I notices was that I had 162 documents listed as BCPs, but the highest BCP number was 

> foreach i (00 01 02 03 04 05 06 07 08 09 `count 10 99`)
> foreach? rfc bcp00$i >> bcps-in-numeric-order.txt
> foreach? end
> foreach i (`count 100 200`)
> foreach? rfc bcp0$i >> bcps-in-numeric-order.txt
> foreach? end

and gives you (surprise) the BCPs in numeric order. I got curious about the fact that we have more documents classified as BCPs than we have BCP numbers. It turns out that we have some documents that update BCPs, and they have the same BCP number as the things they update. We also have some BCP numbers that are unused, at least in the rfc-index.txt file - there is no BCP0001 or BCP0002, and a couple of others.

The other thing that I do is ask myself what there is to know about a particular topic, which might mean to search the IEN and RFC series for documents that mention "multihom" (eg, multihome, multihoming, multihomed), for example, and then go look at what they say. That is a place for fgrep, but "gee wouldn't it be nice" if we could somehow identify those files and pop up a web page something akin to the attached file "multihoming-search.txt". No, my hack tool doesn't generate this easily, but it was used in generating this example. The value of the web page is to help someone zero in quickly on the document s/he needs to read and give him/her a link to find it.

---

From: Paul Hoffman <[email-masked]>
Date: Fri, 29 Apr 2011 06:29:41 -0700

On Apr 28, 2011, at 1:53 PM, Fred Baker wrote:

> I have built myself a few ad hoc tools that build on an internet draft or RFC mirror I keep on my laptop, but to be honest I'm not happy with them.

Not related to David's original request, but relevant to that point: Russ Housley and I have also been developing such tools and are tentatively scheduled to bring them to the tools team in May. Our hope is that they can be released to those in the community who want such tools but don't want to write themselves. Look for more from us in a bit.

> I do a lot of old-fashioned grepping.

We don't do that for the content, but we do do it for the filenames. Thus, you can say "draft baker" and get all the drafts that have "baker" in the filename. I'd be happy to consider how we could put in some content searching as well.

Our intention was that these tools could be delivered from the Tools site but would not be considered actual Tools Team products.

--Paul Hoffman