Subject: [Rfcplusplus] Apocrypha and Other Herrings

From: "Adrian Farrel" <[email-masked]>
Date: Thu, 21 Jun 2018 03:24:13 +0100

Hi,

I wanted to pile in on the lengthening list of anecdotal stories relating to
"confusion" about what is and is not an "RFC of merit".

Companies and their marketing machines will portray documents in any way they
see fit. At the moment we label things as RFCs and the company is tempted to say
or imply that all RFCs are equal and carry full IETF blessing. Many companies
have done this at one time or another whether through disingenuity or confusion.
But changing labels will at best only have a partial beneficial impact on this -
if I can call black white, then you renaming black to red will not stop me
calling red white.

It is certainly true that all sorts of RFCs show up in procurement specs. I
recall the RFI that asked whether our software implemented RFC 2119 (I think we
responded that it SHOULD do). But actually, anything might turn up on an RFI: I
have seen RFCs of any maturity and level, plenty of Internet-Drafts, and
assorted other documentation. Also plenty of undocumented requirements. And do
what? Sometimes this is a good opportunity to engage with the customer; other
times it simply is a statement of what the customer wants implemented and the
maturity or documentation level is not relevant. There are no Internet Police in
this theatre and no one proposes changing the label of an Internet-Draft just
because someone is "confused" about its status.

We might observe further that individual contribution Internet-Drafts are
sometimes perceived as being IETF work. This confusion spreads through the whole
community although may be seen most among newcomers. We've had some sticky
moments in our relationships with other SDOs as a result of such drafts being
assumed to represent IETF opinion. But no one seems to be suggesting that we
change the filename or labels of such drafts.

Additionally, we should note that the IETF has published one or two poor ideas
as Standards Track RFCs over the years. No blame attached to anyone, but
sometimes the RFCs are not implementable, sometimes they lack sufficient detail
to achieve interoperable implementations, sometimes they have bugs (often well
known, but rarely written up), and from time to time they describe protocols
that will never be coded let alone deployed. The Internet Standard numbering
system does address this, but the reluctance that we seem to have to push work
to full standard has kept the number of such documents quite small. Furthermore,
as was noted elsewhere, this numbering system is neither well understood nor
widely used.

Now, in all this I want to suggest a little more rigour. Not only should we try
more carefully to articulate the problem being addressed, but we should try to
understand the experiment that is proposed. Any experiment must have measurable
outcomes to be in any way valid. It seems to me that this experiment as designed
can only determine whether it does measurable harm in any as-yet-unknown way: it
cannot measure any benefits. This can be reasoned as follows:

- There is no quantified statement of the problem. What harm can actually be
shown to be being done? How damaging is that harm to how the Internet works such
that this experiment can be examined to see whether it has successfully "made
the Internet work better"?

- There is no attempt to address the existing cannon. But in the 3 years the
experiment is supposed to run for we can expect around 230 new documents per
year (thus around 700 over the 3 years) in what is currently the whole RFC
series. That will represent 7.5% to 8% of the total documents (RFCs and RFC++
documents) ever published. Presumably, therefore, over 90% of whatever problem
is believed to exist will continue even at the end of the 3 year period. That
would make the proposed change very hard to measure. Only by changing the
labelling of existing RFCs could we expect to see a properly measurable change.

Thanks,
Adrian

PS. Please note that I am in a somewhat delicate position contributing to this
discussion. I hope it can be clear that as ISE I serve as appointed (or removed)
by the IAB and strive to deliver according the current definition of the
Independent Submissions stream and the documented duties of the ISE. If those
definitions change and I continue to serve as ISE I would strive to deliver
according to the new definitions. The opinions expressed in this email are my
own based on several years in the IETF and time developing and selling
standards-based networking products in the wider community.