Subject: [Rfcplusplus] If it ain't broke ...

From: "Andrew G. Malis" <[email-masked]>
Date: Wed, 6 Jun 2018 03:57:10 -0400

I'm the author of some 40-odd RFCs (some of them very odd), some of them
preceding the formation of the IETF, and over the years appearing in most
of the available categories - historic, informational, proposed standard,
draft standard, and full standard (I don't recall writing an experimental
or BCP RFC, but the point is that I've had some amount of experience with
RFCs).

I've worked for organizations that are primarily generators of RFCs (the
vendor community) and those that are primarily consumers of RFCs (the
operator community). Of course, to some extent most organizations in our
industry are both producers and consumers of RFCs, but again, you get the
point.

Over the years, including during my stint on the IAB, I've heard the
various concerns about RFC confusion. However, I really haven't witnessed
that confusion in practice. When I worked for an operator, and we needed to
evaluate the various features implemented in vendor equipment, everyone I
worked with understood the various categories of what was what, including
the risks associated working with our vendors on features that were at that
point in time only drafts in progress.

During my stints at vendors, I've again found our customers in the operator
community to be well informed regarding the status of ongoing and completed
work in the IETF, what they would like to see implemented in the equipment
they deploy, and why. They want to see the results of interoperability
testing (and often run the tests themselves, or sponsor the work in test
labs). They know how the protocols work, and why they would want to use
them in their networks. In some cases, they're willing to deploy bleeding
edge drafts, and understand the risks but have business reasons why it
makes sense for them. In other cases, they take a more conservative
approach to their networks. But in all cases, they're well informed
regarding the status of the contents of any particular RFC or draft.

I've not (to my memory) been contacted by anyone that had any
misunderstanding that needed clearing up regarding the particular status of
any of my RFCs.

I realize that this is just one data point, and anecdotal in nature, but
based on my experience, like Bob I'm skeptical of the changes being
proposed.

Cheers,
Andy

---

From: Barry Leiba <[email-masked]>
Date: Wed, 6 Jun 2018 05:56:58 -0400

The problem is that it *is* broke(n).

> Over the years, including during my stint on the IAB, I've heard the various
> concerns about RFC confusion. However, I really haven't witnessed that
> confusion in practice.

I have.  I have experience with (1) perceptions that Informational and
Experimental IETF-stream RFCs are standards, along with demands that
they be implemented, (2) perceptions that IAB-stream RFCs are BCPs
that must be followed, and, probably worst, (3) perceptions that
Independent-stream Informational documents that got little review and
no consensus are standards, along with demands that they be
implemented.

Maybe the situation is worse at the app layer than at the routing
layer, but it's been a clear problem at the app layer in my
experience.

(Now off to read Brian's draft...)

Barry

---

From: "Paul Hoffman" <[email-masked]>
Date: Wed, 06 Jun 2018 06:45:04 -0700

On 6 Jun 2018, at 2:56, Barry Leiba wrote:

> I have experience with (1) perceptions that Informational and

What was the outcome of those misperceptions? How much different was it 
than if those people had correctly the perceived that the documents they 
were looking at were not standards?

--Paul Hoffman

---

From: Barry Leiba <[email-masked]>
Date: Wed, 6 Jun 2018 19:33:59 -0400

> What was the outcome of those misperceptions? How much different was it than
> if those people had correctly the perceived that the documents they were
> looking at were not standards?

A fair pair of questions, with a common answer.

One situation for the first -- one I was closely involved in and can
remember the details of -- was that the customers said something to
marketing sorts, who brought back to technical marketing support that
"the customer wants to know why we don't have the LMNOP standard
implemented."  The tech support folks looked around and also didn't
realize the problem and it bubbled its way around until it eventually
got to the product management, who thought we'd missed something basic
and took it to the developers.

We finally explained to them that it was an experimental protocol, not
a standard, and wasn't meant to be widely implemented in products...
and why did the customer want it in the first place.  No one seemed to
know the answer to that last question, and it took us a while to get
the mid-level managers to understand why we shouldn't put it into the
product.  It was similarly annoying to get the explanation to the
marketing folks, and then eventually back to the customer reps, who
told the customer.

Apparently, someone on the customer side had read about LMNOP and saw
that it was RFC 9876, and expected a company as savvy as IBM to have
implemented this latest stuff, so a bit more hand-holding and
re-explanation was needed before the flak stopped.

In the end, it all worked out.  But a good bit of time was wasted that
would not have been wasted if the customer hadn't seen "RFC" and
thought that it was an IETF standard.

Just one case study.  The other situations I remember were less of a
big deal, but were generated from the same misunderstanding.
Explanations were quicker and more brief, and responded to with, more
or less, "Ah, OK."

Barry

---

From: Adam Roach <[email-masked]>
Date: Wed, 6 Jun 2018 18:59:44 -0500

If we're collecting anecdotes, I want to point out that my experience 
lines up with Barry's. I've worked on multiple RFPs that included 
spreadsheets from the customer asking, line by line, for product 
compliance with long lists of RFCs. These lists included informational 
documents, IETF process documents, and experimental documents. Although 
I didn't run into quite the level of difficulty as the first situation 
Barry describes, I've had plenty of situations congruent with what he 
describes in his final paragraph.

I also have a somewhat cloudy recollection of finding a mailing list 
thread involving an argument among Linux kernel developers regarding 
implementation of an experimental RFC that was not intended for 
widespread public use. One side argued that the kernel implementation 
was incomplete without it ("you're not RFC compliant") while the other 
tried to explain that you don't have to implement all RFCs to comply 
with a protocol, and that some are, in fact, potentially harmful to 
deploy widely. I wish I could recall the exact topic so as to find the 
thread again, but it's been a relatively long time since I came across this.

/a

On 6/6/18 6:33 PM, Barry Leiba wrote:
> _______________________________________________
> Rfcplusplus mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rfcplusplus

---

From: "Paul Hoffman" <[email-masked]>
Date: Wed, 06 Jun 2018 17:09:32 -0700

On 6 Jun 2018, at 16:59, Adam Roach wrote:

> documents, IETF process documents, and experimental documents. 
> Although I didn't run into quite the level of difficulty as the first 
> situation Barry describes, I've had plenty of situations congruent 
> with what he describes in his final paragraph.
>

So, for the first question, the outcome in these cases was "we had to 
explain the situation to the customers and they eventually (and possibly 
grumpily) understood". I am not sure what the answer to the second 
question would be, particularly for that first story. They were 
obviously pulling at "IETF documents", not standards, so if they existed 
in different series, it sounds like the result might have been the same.

--Paul Hoffman

---

From: Adam Roach <[email-masked]>
Date: Wed, 6 Jun 2018 19:22:38 -0500

On 6/6/18 7:09 PM, Paul Hoffman wrote:
> ...it sounds like the result might have been the same. 

One runs experiments to test hypotheses. Your gut-feel "might" is only 
very vaguely aligned with my gut feel "probably would not" on this 
point, in that they both leave room for the other possibility to be true.

/a

---

From: "Paul Hoffman" <[email-masked]>
Date: Wed, 06 Jun 2018 17:51:30 -0700

> point, in that they both leave room for the other possibility to be 

It seems unlikely that we can run experiments on this. Given that, we 
should look at the data that we find by looking around in the 
environment. As pattern-seeking humans, we'll try to fit that data into 
useful conclusions; we'll also probably sit in the same room as others 
who come to different conclusions for the same data.

--Paul Hoffman

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 6 Jun 2018 22:18:55 -0400

I have seen the exact sequence you describe (and worse) where the 
document in quesiton was an Internet Draft.   I have seen this more than 
once.

Thus, based on the evidence I have seen, changing names is unlikely to help.

Yours,
Joel

PS: And changing names as an experiment is a recipe for more serious 
problems.

---

From: "Andrew G. Malis" <[email-masked]>
Date: Thu, 7 Jun 2018 02:21:05 -0400

I was just about to make Joel's last point in response to an earlier email.
If we deem the "experiment" a failure, how do we go back? Do we reissue all
of the non-RFCs as RFCs?

I would be much happier with us working on a very clear set of text to put
into the first page of each RFC, so that if anyone were to take the trouble
to read it, they would see a statement like:

THIS DOCUMENT DEFINES AN EXPERIMENT THAT IS NOT INTENDED FOR WIDESPREAD USE
IN OPERATIONAL NETWORKS.

or

THIS DOCUMENT IS FOR INFORMATION ONLY, AND DOES NOT SPECIFY AN
IETF-APPROVED PROTOCOL.

or

THIS DOCUMENT REPRESENTS THE CONSENSUS OF THE IETF AND HAS BEEN PROPOSED TO
BECOME AN IETF STANDARD.

or

THIS DOCUMENT IS AN IETF STANDARD.

Whatever we agree on for the wording, it should be shouted at the reader in
all caps, and in bold as well once we support styled text, with a box
around it ... whatever we need to ensure that you can't miss it.

Cheers,
Andy

On Wed, Jun 6, 2018 at 10:18 PM, Joel M. Halpern <[email-masked]>
wrote:

> I have seen the exact sequence you describe (and worse) where the document
> in quesiton was an Internet Draft.   I have seen this more than once.
>
>>> than

---

From: Adam Roach <[email-masked]>
Date: Thu, 7 Jun 2018 01:28:42 -0500

On 6/7/18 1:21 AM, Andrew G. Malis wrote:
> I was just about to make Joel's last point in response to an earlier 
> email. If we deem the "experiment" a failure, how do we go back? Do we 
> reissue all of the non-RFCs as RFCs?

The BOF description currently says: "At the end of the experiment, the 
community would discuss whether these stream identifiers were 
sufficiently useful in technical references and discussions to maintain 
the technical dialog among the relevant technical communities. If they 
are not, RFC numbers would be allocated for each document published 
under the experiment..."

/a

---

From: Barry Leiba <[email-masked]>
Date: Thu, 7 Jun 2018 02:57:44 -0400

> community would discuss whether these stream identifiers were sufficiently
> useful in technical references and discussions to maintain the technical
> dialog among the relevant technical communities. If they are not, RFC
> numbers would be allocated for each document published under the
> experiment..."

Apart from that, I would suggest that we continue the consecutive
numbering as though we were issuing RFCs, and then change the prefix.
So, RFC9112, IAB9113, BCP9114, RFC9115, EXP9116, etc.  That will make
it trivial to switch them all back to "RFC" if we decide to.

Barry

---

From: <[email-masked]>
Date: Thu, 7 Jun 2018 08:32:29 +0000

Hi Adam, all, 

It is completely fine to have informational or experimental RFCs be listed in RFPs. This can be on purpose by the RFP issuers.

For example, RFPs can include RFC 6830 (experimental), RFC 6824 (experimental), RFC 6877 (info), RFC 6234 (info),...

IMHO, introducing a new serie does not prevent that informational or experimental specs to be included in RFPs. 

Cheers,
Med

> -----Message d'origine-----
> De : Rfcplusplus [mailto:[email-masked]] De la part de Adam
> Roach
> Envoyé : jeudi 7 juin 2018 02:00
> À : Barry Leiba; Paul Hoffman
> Cc : [email-masked]
> Objet : Re: [Rfcplusplus] If it ain't broke ...
>

---

From: Barry Leiba <[email-masked]>
Date: Thu, 7 Jun 2018 10:55:01 +0200

> once.

Indeed, you’re right.  It’s clear that “RFC” is not the only thing that
confuses people, and there’s certainly evidence that some consider anything
posted to the IETF to be a standard.  At best, what some of us are
proposing will only solve part of the problem.

> Thus, based on the evidence I have seen, changing names is unlikely to
help.

And you might well be right about that too.  My sense is that it will help,
but not fix everything.  I certainly could be wrong about that.

Barry

---

From: "Heather Flanagan (RFC Series Editor)" <[email-masked]>
Date: Thu, 7 Jun 2018 10:43:56 +0100

> On 6/7/18 9:55 AM, Barry Leiba wrote:
> Indeed, you’re right.  It’s clear that “RFC” is not the only thing that confuses people, and there’s certainly evidence that some consider anything posted to the IETF to be a standard.  At best, what some of us are proposing will only solve part of the problem.
> 
> And you might well be right about that too.  My sense is that it will help, but not fix everything.  I certainly could be wrong about that.
And this is probably the heart of my concerns with the experiment as proposed. I think we have quite a bit of anecdotal information about the problem (or set of problems), but we have no measurements to     baseline and prove the existence of those problems. We also have a number of ideas on how to fix whatever we're defining the problem to be today, but no way to measure whether the fix worked (in part because we have insufficient data regarding the scope of the problem(s) today). 

I'm not opposed to making changes, but I want to start from a proper evaluation and a review of options (similar to what I did in RFC 6949 for the format work) before we jump in with change for change's sake.

-Heather

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 7 Jun 2018 08:36:57 -0400

There are several problems with the notion of experiment.

The first is that I can not understand how we would judge the success or 
failure of the experiment after three years.  The stated goal is to 
reduce confusion.  Since such confusion is extremely hard to measure, I 
wonder how we will know whether and how much reduction we would achieve. 
  (IN most other experiment proposals the IESG has asserted, for good 
reason, that this is an important property of the experiment.)

The second problem is the drawbacks of declaring failure of the 
experiment.  It is not just that we need RFC numbers for all the 
documents.  (Yes, I saw that in the proposal.)  It is that we will still 
have a commitment to maintain the immutable documents of whatever name 
we have produced.  Principally because we will hav eno way of knowing 
who is making use of those names in discussions or citations.

Yours,
Joel

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 9 Jun 2018 10:22:11 +1200

> it trivial to switch them all back to "RFC" if we decide to.

There's a gotcha there, which is that flipping labels in the metadata of
a document is easy, but flipping them in the archival text isn't just difficult,
it's always been forbidden in the RFC series. The only obvious implementation
technique is that if we experimentally issue a document as, say, RFC-BCP9114
(to use that format as an example), and later decided the experiment had failed,
would be then to retroactively issue the text as RFC9114 and insert it in
the archive. Copies of RFC-BCP9114 would then float around for ever, as
IENs do today.

Also, this doesn't do what BCP and STD numbers do, which is bind together
the various RFCs that constitute a single BCP or STD. And it certainly
doesn't do what ISDs were proposed for, which is to bind together all
the documents of any status that are relevant to a single purpose.

I'm still not sure that the cure is better than the alleged disease.
And I think that ISDs would cure the alleged disease and two others
at the same time. They involve real work, however,not just labels.

The base reference for ISDs: https://tools.ietf.org/html/draft-klensin-isdbis-00

   Brian