Subject: [RTG-DIR] Routing area directorate early (QA) review of draft-ietf-babel-applicability

From: Alexander Vainshtein <[email-masked]>
Date: Tue, 31 Jan 2017 16:26:36 +0000

Hello,
I have been appointed as an early  (QA) RTG-DIR reviewer of draft-ietf-babel-applicability.

The review has been requested by one of the BABEL WG chairs. The purpose of an early review is to get an outside opinion on the work and to try to spot any major issues or objections early on in its lifecycle.

Document: draft-ietf-babel-applicability-01
Reviewer: Sasha Vainshtein
Review Date: 31-Jan-17
IETF LC End Date: N/A
Intended Status: Informational

Before going into the review proper, I would like to make a couple of introductory statements.

1.       From my POV this review falls somewhere in between the RtgDir QA review and the "full' RtgDir review. This explains deviations from the full RtgDir review template.

2.       I have looked up for suitable precedents to understand the expectations for a stand-alone Applicability Statement document since it was not clear to me what to expect.

a.       The draft under review is just 5 pages, which, from my POV, is clearly a blessing for an "outside" reviewer. So I have decided, rather arbitrarily, to exclude several very long Applicability Statement RFCs for routing and signaling protocols (e.g., RFC 7733<https://tools.ietf.org/html/rfc7733> - 38 pages, or RFC 6571<https://tools.ietf.org/html/rfc6571> - 35 pages) from my list of potential valid reference points.

b.      I have considered the following published documents as valid reference points for the purpose of this review:

                                                               i.      RFC 2081<https://www.rfc-editor.org/rfc/rfc2081.txt>

                                                             ii.      RFC 3037<https://tools.ietf.org/html/rfc3037>

                                                            iii.      RFC 3210<https://tools.ietf.org/html/rfc3037>

c.       I have noticed that these documents more or less follow a common scheme:

                                                               i.      A brief technical overview of the protocol (at some level or detail)

                                                             ii.      Objectives and actual results that can be achieved with the protocol

                                                            iii.      Specific limitations (e.g., scalability), possibly with information about the critical scale parameters

                                                           iv.      Security considerations

3.       I expected to see some level of compliance with the above-mentioned common scheme in the draft under review. At the same time I understand that my selection of reference points and the resulting conclusion about the common scheme are somewhat arbitrary and may be biased. Hence I ask to take the WG Chairs, the ADs and the RTG-DIR team to take the conclusions in my review with a grain of salt.

Summary:
I have some concerns about the document that can be summarized as "not enough information in the document to be published as an Informational RFC". From my POV, these concerns should be resolved before the document is progressed.
These concerns may be due to difference of expectations from the kind of document I has been assigned to review.

I have discussed my concerns in a private exchange with the author, but we have not resolved our differences at the time this review has been posted.

Comments:
The document is very short  and hence easy to read. It is divided into three main parts:

1.       Networks where use of BABEL has been successfully deployed

2.       Networks where use of BABEL is not recommended

3.       Security considerations

The IANA Considerations section is also present but it does not contain any information (as expected).

Issues:

The draft differs from the above-mentioned common scheme and does not provide sufficient level of detail. As a consequence, It does not meet my expectations for a stand-alone Applicability Statement document.

1.       The technical overview of the protocol in the draft is limited to a single sentence that includes a reference to the base BABEL spec - RFC 6126.

a.       I have found that Sections 1.1 "Features" and 1.2 "Limitations" of RFC 6126 contain quite a reasonable description of BABEL capabilities and weaknesses

b.      This would probably  suffice if we were dealing with an Applicability Statement as a separate section in the spec. But from my POV a separate Applicability document should be more or less self-contained

2.       The draft lists four classes of networks where BABEL has been successfully deployed:

a.       Medium-sized hybrid networks that combine "a wired, aggregated backbone with meshy wireless bits at the edges".

                                                               i.      Success of BABEL in these networks is attributed to its ability to "to deal with both classical, prefix-based ("Internet-style") routing and flat ("mesh-style") routing over non-transitive link technologies"

                                                             ii.      No specific parameters about the networks in this class are provided. From my POV "medium-sized" and "meshy bits" can mean very different things to different people

b.      Large overlay networks "built out of thousands of tunnels spanning continents" and "used with a metric computed from links' latencies".

                                                               i.      Success of BABEL in these networks is attributed to the ability of BABEL to "remain stable and efficient in the presence of  unstable metrics, even in the presence of a feedback loop"

                                                             ii.      No specific parameters (e.g., frequency and the range of metric changes) are provided

                                                            iii.      Wireless pure mesh networks. No explanation is given for the claimed BABEL ability "be competitive with dedicated    routing protocols for wireless mesh networks"

c.       Small unmanaged networks (three to five routers). BABEL is claimed to be a replacement for RIP in these networks

3.       By and of itself, reference to actual deployments could be a great advantage for an applicability document - if sufficient level of detail about these deployments is provided. While "sufficient level of detail' clearly is in the eye of the reader, from my POV:

a.       The only class of networks where, the level of detail provided in the draft is quite sufficient, are "small, unmanaged networks (three to five routers)"

b.      In the remaining cases the descriptions are somewhat vague. In particular, there is no information about:

                                                               i.      The number of nodes and links per node encountered in specific deployments

                                                             ii.      Specific information about non-transitive behavior of links in these deployments (e.g. the numbers of transitive vs. non-transitive links per node)

                                                            iii.      Frequency of changes in the metric and observed range of variations (where applicable) etc.

4.       I have raised these points in a personal exchange with the author.

a.       It seems that in some cases the information is publicly available (and so could be included in the draft

b.      In some other cases the operators who have deployed BABEL object to publication of any specific information about their networks.

5.       One more detail that IMHO is lacking is usage -or non-usage - of any extensions to the base BABEL spec in successful deployments and their impact (or lack thereof) on the success.  This kind of detail could be important - or not

6.       The part of the draft that discusses the scenarios in which use of BABEL is not recommended is more in line with the common scheme. However, I have noticed that:

a.       Section 1.2 of RFC 6126 explicitly mentions two potential weaknesses of BABEL:

                                                               i.      Periodic distribution of routing info.

                                                             ii.      Potential transient black-holing when an aggregated route is replaced by more specific ones

b.      Only the first of these issues is mentioned in the draft as the root cause preventing effective use of BABEL in specific networks

                                                               i.      This may be due to the fact that the second kind of problem has never been encountered in actual deployments - or not.

                                                             ii.      In any case, I would expect correlation between the limitations of BABEL as presented in the base spec and the impact of these limitations on actual deployments in the applicability document.

c.       I also think that non-applicability of BABEL as a routing protocol in "large, stable, well-administrated networks" is not only due to the fact that it uses periodic distribution of routing info - especially since the protocols recommended for this purpose (OSPF and IS-IS) also do that (due to aging of their Link State databases even if, presumably, with much longer periods)

7.       IMHO and FWIW, security concerns associated with BABEL, and the mechanisms for their mitigation  are  adequately presented in the draft.

I believe that the issues I've raised can be resolved to my satisfaction (at the price of making the draft somewhat longer to read) by providing:

-          A reasonably detailed technical overview of BABEL in the draft based on a similar overview in the base spec

-          Some level of detail about the networks where BABEL has been successfully deployed - where information is available and can be published without causing unnecessary conflicts

-          Information about actual impact of de-aggregation of routes on black-holing in existing BABEL deployments. If such impact has not been observed, a mere statement of the fact would suffice, of course.

NITS:

I did not check the draft for nits.

Regards,
Sasha

Office: +972-39266302
Cell:      +972-549266302
Email:   [email-masked]

From: Jonathan Hardwick [mailto:[email-masked]]
Sent: Monday, January 09, 2017 1:42 PM
To: Alexander Vainshtein <[email-masked]>
Cc: Jon Hudson ([email-masked]) <[email-masked]>; Yemin (Amy) <[email-masked]>; Donald Eastlake <[email-masked]>; 'Zhangxian (Xian)' <[email-masked]>
Subject: Routing area directorate early (QA) review of draft-ietf-babel-applicability

Hi Sasha

Happy new year to you!  I wonder if you would be able to do an early directorate review of this short draft?  The review has been requested by the babel chairs and is due by 31 Jan.
https://datatracker.ietf.org/doc/draft-ietf-babel-applicability/?include_text=1

Just to remind you, the purpose of an early review is to get an outside opinion on the work and to try to spot any major issues or objections early on in its lifecycle.  Please send comments to the babel mailing list, rtg-dir mailing list and babel chairs.

You will hopefully have also received an email from our new review tracking system!

Please let me know if you can accept this assignment.

Best regards and all the best for 2017!
Jon

---

From: Alia Atlas <[email-masked]>
Date: Tue, 31 Jan 2017 11:37:06 -0500

Sasha,

Thank you very much for your review and detailed suggestions.
I am certain the authors and WG Chairs will be proactive in improving the
document.

Regards,
Alia

On Tue, Jan 31, 2017 at 11:26 AM, Alexander Vainshtein <
[email-masked]> wrote:

> The review has been requested by one of the BABEL WG chairs. The purpose
> of an early review is to get an outside opinion on the work and to try to
> spot any major issues or objections early on in its lifecycle.
>
>
>
> 1.       From my POV this review falls somewhere in between the RtgDir QA
> review and the “full’ RtgDir review. This explains deviations from the full
> RtgDir review template.
>
> 2.       I have looked up for suitable precedents to understand the
> expectations for a stand-alone Applicability Statement document since it
> was not clear to me what to expect.
>
> a.       The draft under review is just 5 pages, which, from my POV, is
> clearly a blessing for an “outside” reviewer. So I have decided, rather
> arbitrarily, to exclude several very long Applicability Statement RFCs for
> routing and signaling protocols (e.g., RFC 7733
> <https://tools.ietf.org/html/rfc7733> – 38 pages, or RFC 6571
> <https://tools.ietf.org/html/rfc6571> – 35 pages) from my list of
> potential valid reference points.
>
> b.      I have considered the following published documents as valid
> reference points for the purpose of this review:
>
>                                                                i.      RFC
> 2081 <https://www.rfc-editor.org/rfc/rfc2081.txt>
>
>                                                              ii.      RFC
> 3037 <https://tools.ietf.org/html/rfc3037>
>
>                                                             iii.      RFC
> 3210 <https://tools.ietf.org/html/rfc3037>
>
>                                                                i.      A
> brief technical overview of the protocol (at some level or detail)
>
>                                                              ii.      Objectives
> and actual results that can be achieved with the protocol
>
>                                                             iii.      Specific
> limitations (e.g., scalability), possibly with information about the
> critical scale parameters
>
>                                                            iv.      Security
> considerations
>
> 3.       I expected to see some level of compliance with the
> above-mentioned common scheme in the draft under review. At the same time I
> understand that my selection of reference points and the resulting
> conclusion about the common scheme are somewhat arbitrary and may be
> biased. Hence I ask to take the WG Chairs, the ADs and the RTG-DIR team to
> take the conclusions in my review with a grain of salt.
>
>
>
> *Summary*:
>
> I have some concerns about the document that can be summarized as “not
> enough information in the document to be published as an Informational
> RFC”. From my POV, these concerns should be resolved before the document is
> progressed.
>
> I have discussed my concerns in a private exchange with the author, but we
> have not resolved our differences at the time this review has been posted.
>
>
>
> *Comments*:
>
> *Issues*:
> The draft differs from the above-mentioned common scheme and does not
> provide sufficient level of detail. As a consequence, It does not meet my
> expectations for a stand-alone Applicability Statement document.
>
> 1.       The technical overview of the protocol in the draft is limited
> to a single sentence that includes a reference to the base BABEL spec – RFC
> 6126.
>
> a.       I have found that Sections 1.1 “Features” and 1.2 “Limitations”
> of RFC 6126 contain quite a reasonable description of BABEL capabilities
> and weaknesses
>
> b.      This would probably  suffice if we were dealing with an
> Applicability Statement as a separate section in the spec. But from my POV
> a separate Applicability document should be more or less self-contained
>
> a.       Medium-sized hybrid networks that combine “a wired, aggregated
> backbone with meshy wireless bits at the edges”.
>
>                                                                i.      Success
> of BABEL in these networks is attributed to its ability to “to deal with
> both classical, prefix-based ("Internet-style") routing and flat
> ("mesh-style") routing over non-transitive link technologies”
>
>                                                              ii.      No
> specific parameters about the networks in this class are provided. From my
> POV “medium-sized” and “meshy bits” can mean very different things to
> different people
>
> b.      Large overlay networks “built out of thousands of tunnels
> spanning continents” and “used with a metric computed from links'
> latencies”.
>
>                                                                i.      Success
> of BABEL in these networks is attributed to the ability of BABEL to “remain
> stable and efficient in the presence of  unstable metrics, even in the
> presence of a feedback loop”
>
>                                                              ii.      No
> specific parameters (e.g., frequency and the range of metric changes) are
> provided
>
>                                                             iii.      Wireless
> pure mesh networks. No explanation is given for the claimed BABEL ability
> “be competitive with dedicated    routing protocols for wireless mesh
> networks”
>
> c.       Small unmanaged networks (three to five routers). BABEL is
> claimed to be a replacement for RIP in these networks
>
> 3.       By and of itself, reference to actual deployments could be a
> great advantage for an applicability document – if sufficient level of
> detail about these deployments is provided. While “sufficient level of
> detail’ clearly is in the eye of the reader, from my POV:
>
> a.       The only class of networks where, the level of detail provided
> in the draft is quite sufficient, are “small, unmanaged networks (three to
> five routers)”
>
> b.      In the remaining cases the descriptions are somewhat vague. In
> particular, there is no information about:
>
>                                                                i.      The
> number of nodes and links per node encountered in specific deployments
>
>                                                              ii.      Specific
> information about non-transitive behavior of links in these deployments
> (e.g. the numbers of transitive vs. non-transitive links per node)
>
>                                                             iii.      Frequency
> of changes in the metric and observed range of variations (where
> applicable) etc.
>
> a.       It seems that in some cases the information is publicly
> available (and so could be included in the draft
>
> b.      In some other cases the operators who have deployed BABEL object
> to publication of any specific information about their networks.
>
> 5.       One more detail that IMHO is lacking is usage –or non-usage - of
> any extensions to the base BABEL spec in successful deployments and their
> impact (or lack thereof) on the success.  This kind of detail could be
> important – or not
>
> 6.       The part of the draft that discusses the scenarios in which use
> of BABEL is not recommended is more in line with the common scheme.
> However, I have noticed that:
>
>                                                              ii.      Potential
> transient black-holing when an aggregated route is replaced by more
> specific ones
>
> b.      Only the first of these issues is mentioned in the draft as the
> root cause preventing effective use of BABEL in specific networks
>
>                                                                i.      This
> may be due to the fact that the second kind of problem has never been
> encountered in actual deployments – or not.
>
>                                                              ii.      In
> any case, I would expect correlation between the limitations of BABEL as
> presented in the base spec and the impact of these limitations on actual
> deployments in the applicability document.
>
> c.       I also think that non-applicability of BABEL as a routing
> protocol in “large, stable, well-administrated networks” is not only due to
> the fact that it uses periodic distribution of routing info – especially
> since the protocols recommended for this purpose (OSPF and IS-IS) also do
> that (due to aging of their Link State databases even if, presumably, with
> much longer periods)
>
> 7.       IMHO and FWIW, security concerns associated with BABEL, and the
> mechanisms for their mitigation  are  adequately presented in the draft.
>
>
>
> I believe that the issues I’ve raised can be resolved to my satisfaction
> (at the price of making the draft somewhat longer to read) by providing:
>
> -          A reasonably detailed technical overview of BABEL in the draft
> based on a similar overview in the base spec
>
> -          Some level of detail about the networks where BABEL has been
> successfully deployed – where information is available and can be published
> without causing unnecessary conflicts
>
> -          Information about actual impact of de-aggregation of routes on
> black-holing in existing BABEL deployments. If such impact has not been
> observed, a mere statement of the fact would suffice, of course.
>
>
>
> *NITS*:
>
>
>
> Office: +972-39266302 <+972%203-926-6302>
>
> Cell:      +972-549266302 <+972%2054-926-6302>
>
> *From:* Jonathan Hardwick [mailto:[email-masked]]
> *Sent:* Monday, January 09, 2017 1:42 PM
> *To:* Alexander Vainshtein <[email-masked]>
> *Cc:* Jon Hudson ([email-masked]) <[email-masked]>; Yemin
> (Amy) <[email-masked]>; Donald Eastlake <[email-masked]>;
> 'Zhangxian (Xian)' <[email-masked]>
> *Subject:* Routing area directorate early (QA) review of
> draft-ietf-babel-applicability
>
>
>
> Hi Sasha
>
>
>
> Happy new year to you!  I wonder if you would be able to do an early
> directorate review of this short draft?  The review has been requested by
> the babel chairs and is due by 31 Jan.
>
> Just to remind you, the purpose of an early review is to get an outside
> opinion on the work and to try to spot any major issues or objections early
> on in its lifecycle.  Please send comments to the babel mailing list,
> rtg-dir mailing list and babel chairs.
>
>
>

---

From: Juliusz Chroboczek <[email-masked]>
Date: Tue, 31 Jan 2017 20:36:55 +0100

Dear Sasha,

Thank you very much for your review.  These comments are very helpful, and
I wish I had had your guidance when I first drafted this document.

While there are some minor points I wish to discuss with you (I'll CC the
list when I do), I'll jump to the conclusions right now.  You suggest the

> similar overview in the base spec

This sounds like a good idea, assuming that the technical introduction
does not need to be removed from 6126bis.  I like the technical
introduction in 6126bis.

> - Some level of detail about the networks where BABEL has been successfully
> deployed – where information is available and can be published without causing
> unnecessary conflicts

I'm not going to reply to this before I've at least attempted to gather
suitable data, but for now I'll just mention that:

  - in some cases, the data is easily available, or can be obtained
    without too much effort, and could be added to the document;
  - in some cases, the data is available in the peer-reviewed papers cited
    in the draft, and I'm not sure of the propriety of copy-pasting the
    results;
  - in other cases, the data is not easily available, or is available but
    cannot be published due to people having funny ideas about intellectual
    property (I'd name some names, but I'm not sure I'm allowed to);
  - in yet other cases, the data is simply not known, for many reasons, the
    main of which being that we haven't built an event-driven simulator
    for Babel yet.  (Let him who enjoys ns3 cast me the first stone.)

> observed, a mere statement of the fact would suffice, of course.

As you justly guessed, we know that we don't know how to do automatic
aggregation/deaggregation in Babel, so we don't do it.  I'm planning to
work on that, since I believe it would be useful to at least three of the
major users of Babel.

Thanks again for your review,

-- Juliusz