Subject: [lmap] FW: New Version Notification for draft-starkcarey-lmap-protocol-criteria-00.txt

From: "STARK, BARBARA H" <[email-masked]>
Date: Thu, 15 Jan 2015 14:57:24 +0000

This is the draft that Tim Carey and I put together for LMAP protocol selection criteria. It is taken from the slides we presented on the December teleconference, with a few changes based on the discussion during that call.
Barbara

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: Thursday, January 15, 2015 8:46 AM
To: STARK, BARBARA H; STARK, BARBARA H
Subject: New Version Notification for draft-starkcarey-lmap-protocol-criteria-00.txt

A new version of I-D, draft-starkcarey-lmap-protocol-criteria-00.txt
has been successfully submitted by Barbara Stark and posted to the IETF repository.

Name:		draft-starkcarey-lmap-protocol-criteria
Revision:	00
Title:		LMAP Protocol Selection Criteria
Document date:	2015-01-15
Group:		Individual Submission
Pages:		6
URL:            http://www.ietf.org/internet-drafts/draft-starkcarey-lmap-protocol-criteria-00.txt
Status:         https://datatracker.ietf.org/doc/draft-starkcarey-lmap-protocol-criteria/
Htmlized:       http://tools.ietf.org/html/draft-starkcarey-lmap-protocol-criteria-00

Abstract:
   This draft identifies criteria to be used in evaluating and selecting
   Control and Reporting Protocols described by
   [I-D.ietf-lmap-framework].

Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org.

The IETF Secretariat

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Thu, 15 Jan 2015 16:13:04 +0000

Thanks, Barbara and Tim!

All WG participants - please read and comment. Remember that this document aims to be a reference and help tool in the protocols selection process. 

Regards,

Dan

> From: lmap [mailto:[email-masked]] On Behalf Of STARK, BARBARA H
> Sent: Thursday, January 15, 2015 4:57 PM
> To: [email-masked]
> Subject: [lmap] FW: New Version Notification for draft-starkcarey-lmap-
> protocol-criteria-00.txt
> 
> This is the draft that Tim Carey and I put together for LMAP protocol selection
> criteria. It is taken from the slides we presented on the December
> teleconference, with a few changes based on the discussion during that call.
> Barbara
> 
> URL:            https://urldefense.proofpoint.com/v2/url?u=http-
> 3A__www.ietf.org_internet-2Ddrafts_draft-2Dstarkcarey-2Dlmap-
> 2Dprotocol-2Dcriteria-
> 2D00.txt&d=AwICAg&c=BFpWQw8bsuKpl1SgiZH64Q&r=I4dzGxR31OcNXCJfQ
> zvlsiLQfucBXRucPvdrphpBsFA&m=XIN9C1RI85JcLlc_HK4JmbtSpeNxLXsPfKInZ
> 3TM5DE&s=iICyiTfARFWNLQ0nYcKm8ZPu566aGDwcFFnLgS_IG64&e=
> Status:         https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__datatracker.ietf.org_doc_draft-2Dstarkcarey-2Dlmap-2Dprotocol-
> 2Dcriteria_&d=AwICAg&c=BFpWQw8bsuKpl1SgiZH64Q&r=I4dzGxR31OcNXCJ
> fQzvlsiLQfucBXRucPvdrphpBsFA&m=XIN9C1RI85JcLlc_HK4JmbtSpeNxLXsPfKI
> nZ3TM5DE&s=2gt4tnmgmwjx4-w3530xDA0goc1qR8bCfyBfR2e8lbw&e=
> Htmlized:       https://urldefense.proofpoint.com/v2/url?u=http-
> 3A__tools.ietf.org_html_draft-2Dstarkcarey-2Dlmap-2Dprotocol-2Dcriteria-
> 2D00&d=AwICAg&c=BFpWQw8bsuKpl1SgiZH64Q&r=I4dzGxR31OcNXCJfQzvls
> iLQfucBXRucPvdrphpBsFA&m=XIN9C1RI85JcLlc_HK4JmbtSpeNxLXsPfKInZ3T
> M5DE&s=V31dBlNm5qEW5oYktaORSpaT_-hXOvAf8uyD4S1KJLU&e=
> 
> 
> Abstract:
> Please note that it may take a couple of minutes from the time of submission
> until the htmlized version and diff are available at tools.ietf.org.
> 
> The IETF Secretariat
> 
> _______________________________________________
> lmap mailing list
> [email-masked]
> https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__www.ietf.org_mailman_listinfo_lmap&d=AwICAg&c=BFpWQw8bsuKpl
> 1SgiZH64Q&r=I4dzGxR31OcNXCJfQzvlsiLQfucBXRucPvdrphpBsFA&m=XIN9C1
> RI85JcLlc_HK4JmbtSpeNxLXsPfKInZ3TM5DE&s=EIGzvXolZ7jKxm1NoOMSbML
> AFc0FaDbViOExWfdxwNU&e=

---

From: Charles Cook <[email-masked]>
Date: Thu, 22 Jan 2015 10:24:15 -0700

Barbara
Tim,

I generally like what is conveyed in the draft.

Question:  Is there a need to indicate that the protocol(s) have the
ability to support encryption?  Or would encryption, if used, be
transparent to the protocol(s)?

Charles

On 1/15/2015 7:57 AM, STARK, BARBARA H wrote:
> https://www.ietf.org/mailman/listinfo/lmap

-- 

Charles Cook 
Principal Architect
Network
5325 Zuni Street; Suite 224
Denver, CO  80221
Tel:  303.992.8952  Fax:  925.281.0662
[email-masked]

---

From: "Joan Luciani" <[email-masked]>
Date: Sun, 25 Jan 2015 12:57:25 -0500

Hello,

Thank you for putting this document together.  Definitely gets one thinking
about these issues and think this will be a very insightful and helpful
document.  Some questions and comments below.

Thanks,
 -Joan

Specific Comments:

Section 2.0
Would it be possible to include a list of protocols that currently meet
these criteria?   Not necessarily an inclusive list, but something like:
Although this is not an complete list, examples of protocols that meet these
criteria are:...then list a few examples?

Section 2.1
The sentence which starts:  "Note that although..." is confusing to me.  It
seems to be saying, follow these criteria, but if it is not in 
the protocol, that's okay.   Is this what is meant?

CP-MUST-1 and CP-MUST-2, suggest saying "securely established", instead of
just "established".  The word "must" is used, and so was wondering if this
is "MUST" as outlined in section 1.1 (using key words)?

Section 2.2  Same comment as above with regard to the sentence that starts
"Note that although..."

Some of these CP-DIFF list items seem to be information that the protocol
needs to make available to the operator  (CP-DIFF-1, CP-DIFF-2, CP-DIFF-5).
Maybe these items (data) should be a sub-category?  (Would some
consideration about rate limit be appropriate?)

CP-DIFF-8 If possible, could examples be given?

CP-DIFF-11  Would like to see this mandatory.  To ask a related question is
there a benefit from not having version number?  If so, what is that
benefit?

CP-DIFF-12 is a subquestion of CP-DIFF-11.  Could the be combined with
CP-DIFF-11?  My understanding is that these protocols are standardized (or
on the standards track), doesn't the standards process provide a requirement
for versioning? Is that adequate for protocol numbering or is something else
wanted by LMAP?

Section 3.1
RP-MUST-1 and RP-MUST-2  Is this "must" or "MUST" as outlined in the key
words section?

RP-DIFF-4 Personally, would like to see compression mandatory depending on
the amount of data.  

RP-DIFF-5 Is this a MUST? ( as in MUST specify the bytes of overhead
required...)

RP-DIFF-6 What determines "widely used"?  I am not certain that I see this
as a criterion. While maybe a focus initially, do not think this needs to be
a criterion long-term (just my opinion) but yes, I think that LMAP would
like to focus on protocols that are widely used initially.

RP-DIFF-7 Who determines interoperability? Are folks envisioning independent
interoperability testing?

RP-DIFF-10 and RP-DIFF-11  same comments as for the CP-DIFF-11 and
CP-DIFF-12 above.

--

> From: lmap [mailto:[email-masked]] On Behalf Of Romascanu, Dan
> (Dan)
> Sent: Thursday, January 15, 2015 11:13 AM
> To: STARK, BARBARA H; [email-masked]
> Subject: Re: [lmap] New Version Notification for draft-starkcarey-lmap-
> protocol-criteria-00.txt
> 
> Thanks, Barbara and Tim!
> 
> All WG participants - please read and comment. Remember that this
> document aims to be a reference and help tool in the protocols selection
> process.
> 
> Regards,
> 
> Dan
> 
> 
> > Subject: [lmap] FW: New Version Notification for
> > draft-starkcarey-lmap- protocol-criteria-00.txt
> >
> > This is the draft that Tim Carey and I put together for LMAP protocol
> > selection criteria. It is taken from the slides we presented on the
> > December teleconference, with a few changes based on the discussion
> during that call.
> > Barbara
> >
> > Please note that it may take a couple of minutes from the time of
> > submission until the htmlized version and diff are available at
tools.ietf.org.

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Tue, 27 Jan 2015 15:45:36 +0000

Thanks for the comments. Replies in-line.
Barbara

> criteria are:...then list a few examples?

I believe the idea is that other people in the WG will propose protocols that they believe meet the mandatory criteria and that do a good job with the comparative criteria. The WG would then focus itself on assessing just those protocols. I believe it would be inappropriate for the two of us who initially created this list of criteria to try to provide examples of protocols, unless we are actually proposing those protocols for LMAP consideration. There has been a call for protocol proposals into LMAP. This draft is intended purely as a list of criteria for evaluating whatever protocols are proposed. Effort spent discussing and evaluating whether and how well some protocol satisfies the criteria would be a waste of effort (IMO) if that protocol is not being proposed by someone for consideration.

> the protocol, that's okay.   Is this what is meant?
...
> Section 2.2  Same comment as above with regard to the sentence that starts

No. The protocol (that LMAP WG selects) MUST define how the criterion is achieved in the context of that protocol. It is not required that the protocol mandate that all implementations do it. It would be possible for LMAP (if LMAP selected the protocol) to mandate that LMAP implementations do it (or do it in certain cases).

> CP-MUST-1 and CP-MUST-2, suggest saying "securely established", instead
> of just "established".  The word "must" is used, and so was wondering if this

Yes, I need to capitalize all such normative language in the next revision. Thx.
 
> Maybe these items (data) should be a sub-category?  

This list of criteria is not intended for operator use. It is intended for use by the LMAP WG in assessing and comparing protocols submitted to LMAP WG for consideration as the LMAP WG selected and recommended protocol. Some operators may choose not to make use of the protocol recommended by LMAP WG. If that is the case, then those operators will have their own set of evaluation criteria and methodology for evaluating. I do not believe that helping operators do their own evaluation is in scope of LMAP WG. I do not believe there is any intention of this draft ever achieving RFC status. It is for internal WG use only. If there are criteria listed that are not appropriate for WG use in protocol assessment, then these criteria need to be removed.

> (Would some consideration about rate limit be appropriate?)

It may indeed be appropriate. What do others think? Would anyone care to propose a criterion around rate limiting?
 
> CP-DIFF-8 If possible, could examples be given?

Yes. If others think it is useful, I could include a list of examples, such as "(e.g., test tools, plugfests, certification programs, reference implementations)".
 
> there a benefit from not having version number?  If so, what is that benefit?
>
> CP-DIFF-12 is a subquestion of CP-DIFF-11.  Could the be combined with CP-
> DIFF-11?  My understanding is that these protocols are standardized (or on
> the standards track), doesn't the standards process provide a requirement

What do others think? I have no strong opinion.

> RP-MUST-1 and RP-MUST-2  Is this "must" or "MUST" as outlined in the key

Yes, I need to capitalize all such normative language in the next revision. Thx.
 
> RP-DIFF-4 Personally, would like to see compression mandatory depending
> on the amount of data.

What do others think? I have no strong opinion.
 
> RP-DIFF-6 What determines "widely used"?  I am not certain that I see this as
> a criterion. While maybe a focus initially, do not think this needs to be a
> criterion long-term (just my opinion) but yes, I think that LMAP would like to
> focus on protocols that are widely used initially.

LMAP WG needs to determine "widely used" when it compares different protocol proposals in the very near future. How much weight LMAP wishes to give this criterion is up to LMAP. These criteria are not expected to be long-term, as they are not expected to be maintained once LMAP has made its near-future protocol selection.
 
> RP-DIFF-7 Who determines interoperability? Are folks envisioning
> independent interoperability testing?

I can provide the same list of examples as I suggested for the Controller protocol: "(e.g., test tools, plugfests, certification programs, reference implementations)". LMAP WG would need to decide whether it is likely that implementations of a protocol will be interoperable, based on the available mechanisms (and any known history regarding interoperability). Since these criteria are strictly for use by LMAP WG in its near-future protocol selection exercise, LMAP WG will make all determinations for proposed protocols. All are welcome to participate in LMAP WG to help make these determinations.
 
> RP-DIFF-10 and RP-DIFF-11  same comments as for the CP-DIFF-11 and

What do others think? I have no strong opinion.

---

From: "Carey, Timothy (Timothy)" <[email-masked]>
Date: Wed, 28 Jan 2015 18:58:03 +0000

Joan and Barbara,

My comments to Barbara's inline <TAC>

BR,
Tim

-----Original Message-----
From: STARK, BARBARA H [mailto:[email-masked]] 
Sent: Tuesday, January 27, 2015 9:46 AM
To: Joan Luciani; 'Romascanu, Dan (Dan)'; [email-masked]
Subject: Re: [lmap] New Version Notification for draft-starkcarey-lmap-protocol-criteria-00.txt

> Although this is not an complete list, examples of protocols that meet 
> these criteria are:...then list a few examples?

> Section 2.1
> The sentence which starts:  "Note that although..." is confusing to 
> me.  It seems to be saying, follow these criteria, but if it is not in
...
> Section 2.2  Same comment as above with regard to the sentence that 
> starts "Note that although..."

> CP-MUST-1 and CP-MUST-2, suggest saying "securely established", 
> instead of just "established".  The word "must" is used, and so was 
> wondering if this is "MUST" as outlined in section 1.1 (using key words)?

Yes, I need to capitalize all such normative language in the next revision. Thx.
<TAC> Actually we saw CP-MUST-3 as providing the "secure" session. What we want to make sure of is that a secure session capability is possible but again we would let LMAP WG determine if a implementation is mandated to use a secure session through some normative text in the Framework, IM or Protocols drafts.
 
> Some of these CP-DIFF list items seem to be information that the 
> protocol needs to make available to the operator  (CP-DIFF-1, CP-DIFF-2, CP-DIFF-5).

It may indeed be appropriate. What do others think? Would anyone care to propose a criterion around rate limiting?
<TAC> Yes I can see asking if there are inherent rate limiting (throttling) capabilities in a protocol for comparative purposes. The actual requirements for what is means to rate limit would have to be developed in the protocol draft. My guess is that this feature would be something that needs added-onto any specific base protocol but you never know.

Yes. If others think it is useful, I could include a list of examples, such as "(e.g., test tools, plugfests, certification programs, reference implementations)".
<TAC> I would think examples are useful - the examples seem to clarify by what we meant by interoperable.
 
> CP-DIFF-11  Would like to see this mandatory.  To ask a related 
> question is there a benefit from not having version number?  If so, what is that benefit?
>
> CP- DIFF-11?  My understanding is that these protocols are 
> standardized (or on the standards track), doesn't the standards 
> process provide a requirement for versioning? Is that adequate for 
> protocol numbering or is something else wanted by LMAP?

What do others think? I have no strong opinion.
<TAC> Hmmm - if we are saying a protocol will not be considered for selection if it does not handle elements with different versions - then yes we need to make this mandatory. Frankly I would elevate this to a requirement.

<TAC> On CP-DIFF-12 - Versioning and Extensibility are 2 separate things. I might be able to live with a protocol that doesn't provide a mechanism for vendor extension but I couldn't live with a protocol that didn't allow for different versions.

> Section 3.1
> RP-MUST-1 and RP-MUST-2  Is this "must" or "MUST" as outlined in the 
> key words section?

> RP-DIFF-4 Personally, would like to see compression mandatory 
> depending on the amount of data.

What do others think? I have no strong opinion.
<TAC> Again this is more of a requirement that would need to be placed on a protocol; like versioning. We aren't creating requirements for a protocol but for comparing and selecting protocols. So again if we are saying we will not select a protocol that doesn't provide a compression capability then it should be a requirement not a comparative criteria. I don't think compression elevates to that "status". It is important though.
 
> RP-DIFF-6 What determines "widely used"?  I am not certain that I see 
> this as a criterion. While maybe a focus initially, do not think this 
> needs to be a criterion long-term (just my opinion) but yes, I think 
> that LMAP would like to focus on protocols that are widely used initially.

 

What do others think? I have no strong opinion.
<TAC> I would think versioning is a requirement; extensibility is comparative.