Subject: [lmap] lmap charter - draft 2

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Wed, 29 May 2013 12:34:48 +0000

Hi, 

Please see below another try at the lmap wg charter. I have included the edits discussed until now on the list which we seem to have reached agreement. I have also aligned the milestones of the Control and Report protocols and data models and mentioned the option of issuing these as one or separate documents for each (to be decided later by the WG). 

Other comments or clarifications? 

Is this text good enough for IESG review? 

Regards,

Dan

------------------------------

Name: Large-Scale Measurement of Broadband Performance (LMAP)

AREA: Operations and Management

Chairs:
TBD
TBD

OPS Area Directors:
     
Benoit Claise <bclaise at cisco.com>
Joel Jaeggli <joelja at bogus.com>

OPS Area Advisor:
     Benoit Claise <bclaise at cisco.com>

Mailing Lists:
     General Discussion: lmap at ietf.org
     To Subscribe:       http://www.ietf.org/mailman/listinfo/lmap
     Archive:            http://www.ietf.org/mail-archive/web/lmap

Description of Working Group

Measuring portions of the Internet on a large scale is vital for 
accurate characterizations of performance over time and geography, for 
network diagnostic investigations by providers and their users, and for 
collecting information to support public policy development. The 
ultimate goal is to have the same measurements for a large number of 
points on the Internet, conducted according to the same 
characterization plan, and to have the results collected and stored in 
the same form. 

Many measurement systems that exist today use proprietary, 
custom-designed mechanisms to coordinate their Measurement Agents (MAs) 
deployed across networks, to communicate between MAs and measurement 
Controllers, and to transfer results to measurement Collectors. 
Standardizing these mechanisms (Control protocol, Report protocol, 
and corresponding data models) will make it possible to incorporate 
interoperable measurement communications into home and enterprise 
edge routers, personal computers, mobile devices and other networking 
devices, whether wired or wireless. Standards will help these management 
capabilities become more pervasive and facilitate measurement results 
that are directly comparable.

The Large-Scale Measurement of Broadband Performance (LMAP) working 
group is chartered to determine the form of data models and 
select/extend one or more protocols for the communications between the 
Measurement Agents' (MAs) function and their Controller function and 
Collector function. These three functions comprise the LMAP measurement 
system.

The standardized LMAP mechanisms will allow a Controller to instruct 
MAs what performance metrics to measure, when to measure them,  
how/when to report the measurement results to a Collector, and then for 
the MA to report the results to the Collector. The primary 
products of the working group are the data models that define the 
measurement instructions and reports, and protocols for securely 
delivering these data models to/from the MAs. Data models should be 
extensible for new and additional measurements.

A key assumption constraining the initial work is that the measurement 
system is under the control of a single organization (for example, an 
Internet Service Provider or a regulator). However, the components of 
an LMAP measurement system can be deployed in administrative domains 
that are not owned by the measuring organization. Thus, the system of 
functions deployed by a single organization constitutes a single LMAP 
domain which may span ownership or other administrative boundaries. 

The LMAP architecture will allow for measurements that utilize either 
IPv4 or IPv6, or possibly both. Devices containing MAs may have several 
interfaces using different link technologies. Multiple address families 
and interfaces must be considered in the Control and Report 
protocols.

It is assumed that different organization's LMAP measurement domains 
can overlap, and that active measurement packets appear along with 
normal user traffic when crossing another organization's network. In 
the initial chartering phase, there is no requirement to specify a 
mechanism for coordination between the LMAP measurements in overlapping 
domains (for instance a home network with MAs on the home hub, set top 
box and laptop). In principle, there are no restrictions on the type of 
device in which the MA function resides. 

Both active and passive measurements are in scope, although there may 
be differences in their applicability to specific use cases, or in the 
security measures needed according to the threats specific to each 
measurement category. At a high level, LMAP systems are agnostic to the 
measurements and results, and extensible to incorporate evolution in 
the measurement area, but the details such as the data models must be 
standardized to match the measurements.

Inter-organization communication of results is not addressed in the 
LMAP system and is deferred to bi-lateral agreements.

Many measurement aspects are already within the charter of IPPM. These 
include standardized definitions of performance metrics, MA-to-MA 
measurement protocols, and a registry of frequently-used metrics and 
parameter settings so they can be identified in an efficient and 
consistent fashion. Neither the definition of the new metrics and 
methods of measurement, nor the post-processing and analysis of results 
falls within the remit of LMAP. 

The case where an end user can independently perform network diagnostic 
measurements (beyond their private network) is not directly in scope, 
recognizing that users have many opportunities to do this today. 
However, end users can obtain an MA to run measurement tasks if desired 
and report their results to whomever they want, most likely the 
supplier of the MA. This provides for user-initiated on-demand 
measurement, which is an important component of the ISP use case.

Another area that is out of scope is the management protocol to 
bootstrap the MAs in measurement devices, although a bootstrapping 
process may be described and conducted in many ways, such as 
configuration during manufacture or with a local USB interface. 
Discovering the service parameters on the MAs or sharing the service 
parameters between MAs are out of the scope of the LMAP charter. 
However, if the service parameters are available to the MAs, they could 
be combined with the measurement results in the Report Protocol.

Service parameters, such as product category, can be useful to decide 
which measurements to run and how to interpret the results. These 
parameters are already gathered and  stored by existing operations 
systems. Deciding the set of measurements to run is a business decision 
and is out of scope.
 
Exhaustive protection against all possibilities of gaming the 
measurements - where gaming is defined as intentionally (and perhaps 
maliciously) inserting inaccuracy into the overall system or 
measurement process - is beyond the scope of work. Some protections are 
lawyer problems, not engineering problems. However, the working group 
may include protections that do not add significant complexity, as 
determined by working group consensus.

The LMAP working group will coordinate with other standards bodies 
working in this area (e.g., BBF, IEEE 802.16, ETSI), and other IETF 
working groups in the areas of data models, protocols, multiple 
interface management, and measurement of performance metrics.

LMAP will consider re-use of existing protocols and data model 
languages in its efforts to produce the following work items:

- The LMAP Framework - provides common terminology and justifies the 
simplifying constraints 
- The LMAP Use Cases - provides the motivating use cases as a basis for 
the work 
- Information Model, the abstract definition of the information carried
from the Controller to the MA and the information carried from the MA
to the Collector. It includes
  - the metric(s) to be measured and
  values for its parameters such as the Peer MA participating in the
  measurement and the desired environmental conditions (for example, only
  conduct the measurement when there is no user traffic observed) 
  - the schedule: when the measurement should be run and how
  the results should be reported (when and to which Collector)
  - the report: the metric(s) measured and when, the actual
  result, and supporting metadata such as location. Result reports may 
  be organized in batches or may be reported immediately, such as for an
  on-demand measurement.
- The Control protocol and the associated data model: The definition of
  how instructions are delivered from a Controller to a MA; this
  includes a Data Model consistent with the Information Model plus a
  transport protocol.  This is a simple instruction - response
  protocol, and LMAP will specify how it operates over an existing
  protocol (to be selected, perhaps REST-style HTTP(s) or Netconf).
- The Report protocol and the associated data model: The definition of
  how the Report is delivered from a MA to a Collector; this includes
  a Data Model consistent with the Information Model plus a transport
  protocol (to be selected, perhaps REST-style HTTP(s) or IPFIX).

The WG will decide later whether protocols and data models (for Control, 
respectively Report) will be defined in one or separated documents. 
 
    Goals and Milestones

Sep 2013: Initial WG I-D for the LMAP Framework including terminology 
Sep 2013: Initial WG I-D for the LMAP Use cases
Dec 2013: Submit the LMAP Framework I-D to the IESG for consideration 
as Informational RFC
Dec 2013: Submit I-D on the LMAP Use cases to the IESG for 
consideration as Informational RFC
Jan 2014: Initial WG I-D for LMAP Information models
Apr 2014: Initial WG I-D for the Control protocol 
Apr 2014: Initial WG I-D for the Report protocol 
Apr 2014: Initial WG I-D for a Data model for LMAP control information
Apr 2014: Initial WG I-D for a Data model for LMAP report information
July 2014: Submit the LMAP Information models I-D to the IESG for 
consideration as Standards track RFC
Dec 2014: Submit the Control protocol I-D to the IESG for 
consideration as Standards track RFC
Dec 2014: Submit the Report protocol I-D to the IESG for consideration as 
Standards track RFC
Dec 2014: Submit the Data model for LMAP control I-D to the IESG for 
consideration as Standards track RFC
Dec 2014: Submit the Data model for LMAP report I-D to the IESG for 
consideration as Standards track RFC

---

From: <[email-masked]>
Date: Wed, 29 May 2013 16:38:46 +0100

No more comments, definitely seems good enough for iesg review

Thanks!
phil

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Romascanu, Dan (Dan)
> Sent: 29 May 2013 13:35
> To: [email-masked]
> Subject: [lmap] lmap charter - draft 2
> 
> Hi,
> 
> Please see below another try at the lmap wg charter. I have included
> the edits discussed until now on the list which we seem to have reached
> agreement. I have also aligned the milestones of the Control and Report
> protocols and data models and mentioned the option of issuing these as
> one or separate documents for each (to be decided later by the WG).
> 
> Many measurement systems that exist today use proprietary, custom-
> designed mechanisms to coordinate their Measurement Agents (MAs)
> Standardizing these mechanisms (Control protocol, Report protocol, and
> corresponding data models) will make it possible to incorporate
> interoperable measurement communications into home and enterprise edge
> routers, personal computers, mobile devices and other networking
> devices, whether wired or wireless. Standards will help these
> management capabilities become more pervasive and facilitate
> measurement results that are directly comparable.
> 
> MAs what performance metrics to measure, when to measure them, how/when
> to report the measurement results to a Collector, and then for the MA
> to report the results to the Collector. The primary products of the
> working group are the data models that define the measurement
> instructions and reports, and protocols for securely delivering these
> data models to/from the MAs. Data models should be extensible for new
> and additional measurements.
> 
> and interfaces must be considered in the Control and Report protocols.
> 
> The WG will decide later whether protocols and data models (for
> Control, respectively Report) will be defined in one or separated
> documents.
> 
>     Goals and Milestones
> 
> Sep 2013: Initial WG I-D for the LMAP Use cases Dec 2013: Submit the
> LMAP Framework I-D to the IESG for consideration as Informational RFC
> consideration as Informational RFC Jan 2014: Initial WG I-D for LMAP
> Information models Apr 2014: Initial WG I-D for the Control protocol
> Apr 2014: Initial WG I-D for the Report protocol Apr 2014: Initial WG
> I-D for a Data model for LMAP control information Apr 2014: Initial WG
> I-D for a Data model for LMAP report information July 2014: Submit the
> LMAP Information models I-D to the IESG for consideration as Standards
> track RFC Dec 2014: Submit the Control protocol I-D to the IESG for
> consideration as Standards track RFC Dec 2014: Submit the Report
> protocol I-D to the IESG for consideration as Standards track RFC Dec
> 2014: Submit the Data model for LMAP control I-D to the IESG for
> consideration as Standards track RFC Dec 2014: Submit the Data model
> for LMAP report I-D to the IESG for consideration as Standards track
> RFC
> 
> 
> 
> 
> _______________________________________________
> lmap mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/lmap

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Wed, 29 May 2013 19:30:24 +0000

Sorry I didn't manage to write my comments last week. I kept running out of time, because I didn't want to just throw my comments over the wall without suggesting language.

I have a big concern about language in the charter that suggests lmap will standardize The Protocols to Be Used, and suggestions that this would be particularly useful. I think that attempting to pick The Protocols to Be Used would result in a futile religious war. It's futile because we've agreed that the MA, Controller and Collector are all under a single administrative domain -- so whoever owns that domain will select the protocols they will use. I would prefer if the charter were to identify existing IETF-defined protocols that *can* be used, describe any nuances to their use for lmap purposes, define extensions as necessary, and define any data models that would be needed to use them for lmap purposes. All without any hint or suggestion to imply they *must* be used. And before doing that, requirements for what constitutes a good protocol would also be useful. Such requirements can not only be used to identify the gaps in existing IETF protocols, but would also be useful to any "domain administrators" in developing criteria for what they need out of a protocol that is used in systems they procure (that wasn't invented here).

In addition, I've had people tell me privately that they think my idea of defining a standard DNS-SD Service Instance for an xml page on a router that can provide whatever info that router may want to share about WAN link utilization, its LAN link metrics, service attributes it may know, etc. would be a good idea. But for some reason, they don't support the idea publicly on this list. So I'm going to try one more time to put this in scope. If no-one speaks up, I'll consider the idea dead here, and will pursue it in BBF. I think it will be less useful as a BBF-only defined DNS-SD Service Instance, because I think it would be also useful if off-the-shelf consumer CE routers (e.g., D-Link, Belkin) could be encouraged to use this to provide standard-format information about WAN link utilization and LAN link metrics to home networked devices. But so be it.

> routers, personal computers, mobile devices and other networking devices,
> whether wired or wireless. Standards will help these management
> capabilities become more pervasive and facilitate measurement results that
> are directly comparable.

Recommend re-wording to make this focus on the need for requirements of protocols and not on a specific set of standardized protocols:

Standardizing the requirements for mechanisms (Control protocol, Report protocol, and corresponding data models) will ensure that all protocols that are used for these purposes are capable of controlling the same set of measurements and reporting the measurement data in a standardized manner. This will allow for consistency in protocol capabilities and behavior, without constraining the deploying entity unnecessarily in protocol selection. It will make it possible to incorporate interoperable measurement communications into home and enterprise edge routers, personal computers, mobile devices and other networking devices, whether wired or wireless. Standardized requirements will help these management capabilities become more pervasive and facilitate measurement results that are directly comparable.

> group is chartered to determine the form of data models and select/extend
> one or more protocols for the communications between the Measurement
> Agents' (MAs) function and their Controller function and Collector function.
> These three functions comprise the LMAP measurement system.

Recommend adding the protocol requirements as a deliverable, and limiting the scope of protocol extension to IETF-specified protocols:

The Large-Scale Measurement of Broadband Performance (LMAP) working group is chartered to determine protocol requirements, determine the form of data models and identify/extend one or more IETF-specified protocols for the communications between the Measurement Agents' (MAs) function and their Controller function and Collector function. These three functions comprise the LMAP measurement system.

> The standardized LMAP mechanisms will allow a Controller to instruct MAs
> what performance metrics to measure, when to measure them, how/when
> to report the measurement results to a Collector, and then for the MA to
> report the results to the Collector. The primary products of the working
> group are the data models that define the measurement instructions and
> reports, and protocols for securely delivering these data models to/from the
> MAs. Data models should be extensible for new and additional
> measurements.

I recommend: 
The requirements for LMAP mechanisms will allow a Controller to instruct MAs what performance metrics to measure, when to measure them, how/when to report the measurement results to a Collector, and then for the MA to report the results to the Collector. The primary products of the working group are the protocol requirements, the data models that define the measurement instructions and reports, and identification/extension of IETF-specified protocols for securely delivering these data models to/from the MAs. Data models should be extensible for new and additional measurements.

> and report their results to whomever they want, most likely the supplier of
> the MA. This provides for user-initiated on-demand measurement, which is
> an important component of the ISP use case.

I would prefer to delete the phrase ", most likely the supplier of the MA". I actually think the most likely choice here will be for the end user to report the results to the end user. But I don't think this point is particular worthy of discussion.
 
> However, if the service parameters are available to the MAs, they could be
> combined with the measurement results in the Report Protocol.

Recommend:
Mechanisms for exchanging service parameters external to an end user's network is out of scope of the LMAP charter. However, the data model of these service attributes is in scope, where they could be combined with the measurement results in the Report Protocol. A mechanism that would allow a CE Router inside the end user's network to provide other devices inside the end user network with information is in scope.

> Service parameters, such as product category, can be useful to decide which
> measurements to run and how to interpret the results. These parameters
> are already gathered and  stored by existing operations systems. Deciding
> the set of measurements to run is a business decision and is out of scope.

The last sentence seems out of place, since I don't see how it relates to the first 2 sentences. I think the last sentence could be added at the end of a different paragraph, like the one that starts "It is assumed that different organization's LMAP measurement domains can overlap..." or the one that starts "Both active and passive measurements are in scope..."

> LMAP will consider re-use of existing protocols and data model languages in
> its efforts to produce the following work items:
> - The LMAP Framework - provides common terminology and justifies the simplifying constraints
<snip>
>   protocol (to be selected, perhaps REST-style HTTP(s) or IPFIX).

As it relates to deliverables, I think the protocol requirements should be included in the framework, and don't need to be a separate deliverable.
BTW, the term "transport protocol" kind of threw me in the description of the protocols. I was wondering if "application protocol" might be better.
Recommend:
- The LMAP Framework - provides common terminology and justifies the simplifying constraints. The framework includes requirements for the Control and Report protocols.

- The Control protocol and the associated data model: The definition of
  how instructions are delivered from a Controller to a MA; this
  includes a Data Model consistent with the Information Model plus identification/extension of one or more IETF-specified
  application protocols.  This is a simple instruction - response
  protocol, and LMAP will specify how it operates over one or more existing
  protocols (perhaps REST-style HTTP(s) or Netconf).
- The Report protocol and the associated data model: The definition of
  how the Report is delivered from a MA to a Collector; this includes
  a Data Model consistent with the Information Model plus identification/extension of one or more IETF-specified
  application protocols (perhaps REST-style HTTP(s) or IPFIX).

And add:
- Mechanism to allow a CE Router to provide devices inside the end user network with various information that could be useful when analyzing measurement results.

> Apr 2014: Initial WG I-D for the Control protocol 
Recommend: Apr 2014: Initial WG I-D for the Control protocol(s)

> Apr 2014: Initial WG I-D for the Report protocol 
Recommend: Apr 2014: Initial WG I-D for Report protocol(s)

Milestone around this same time for the information delivery mechanism.

Barbara

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Wed, 29 May 2013 19:52:10 +0000

Hi Barbara, 

Even if the scope of the initial phase of LMAP is designed on the assumption that the measurement system is under the control of a single organization, its components (controllers, MAs, collectors) may originate from different sources (vendors of equipment and applications). How will we ensure interoperability between these components if we do not define/select at least one mandatory-to-implement set of protocols and associated data models? 

Thanks and Regards,

Dan

> From: STARK, BARBARA H [mailto:[email-masked]]
> Sent: Wednesday, May 29, 2013 10:30 PM
> To: Romascanu, Dan (Dan); [email-masked]
> Subject: RE: lmap charter - draft 2
> 
> Sorry I didn't manage to write my comments last week. I kept running out
> of time, because I didn't want to just throw my comments over the wall
> without suggesting language.
> 
> I have a big concern about language in the charter that suggests lmap
> will standardize The Protocols to Be Used, and suggestions that this
> would be particularly useful. I think that attempting to pick The
> Protocols to Be Used would result in a futile religious war. It's futile
> because we've agreed that the MA, Controller and Collector are all under
> a single administrative domain -- so whoever owns that domain will
> select the protocols they will use. I would prefer if the charter were
> to identify existing IETF-defined protocols that *can* be used, describe
> any nuances to their use for lmap purposes, define extensions as
> necessary, and define any data models that would be needed to use them
> for lmap purposes. All without any hint or suggestion to imply they
> *must* be used. And before doing that, requirements for what constitutes
> a good protocol would also be useful. Such requirements can not only be
> used to identify the gaps in existing IETF protocols, but would also be
> useful to any "domain administrators" in developing criteria for what
> they need out of a protocol that is used in systems they procure (that
> wasn't invented here).
> 
> In addition, I've had people tell me privately that they think my idea
> of defining a standard DNS-SD Service Instance for an xml page on a
> router that can provide whatever info that router may want to share
> about WAN link utilization, its LAN link metrics, service attributes it
> may know, etc. would be a good idea. But for some reason, they don't
> support the idea publicly on this list. So I'm going to try one more
> time to put this in scope. If no-one speaks up, I'll consider the idea
> dead here, and will pursue it in BBF. I think it will be less useful as
> a BBF-only defined DNS-SD Service Instance, because I think it would be
> also useful if off-the-shelf consumer CE routers (e.g., D-Link, Belkin)
> could be encouraged to use this to provide standard-format information
> about WAN link utilization and LAN link metrics to home networked
> devices. But so be it.
> 
> Recommend re-wording to make this focus on the need for requirements of
> protocols and not on a specific set of standardized protocols:
> 
> Standardizing the requirements for mechanisms (Control protocol, Report
> protocol, and corresponding data models) will ensure that all protocols
> that are used for these purposes are capable of controlling the same set
> of measurements and reporting the measurement data in a standardized
> manner. This will allow for consistency in protocol capabilities and
> behavior, without constraining the deploying entity unnecessarily in
> protocol selection. It will make it possible to incorporate
> devices, whether wired or wireless. Standardized requirements will help
> these management capabilities become more pervasive and facilitate
> Collector function.
> Recommend adding the protocol requirements as a deliverable, and
> limiting the scope of protocol extension to IETF-specified protocols:
> 
> group is chartered to determine protocol requirements, determine the
> form of data models and identify/extend one or more IETF-specified
> protocols for the communications between the Measurement Agents' (MAs)
> function and their Controller function and Collector function. These
> three functions comprise the LMAP measurement system.
> 
> > how/when to report the measurement results to a Collector, and then
> > for the MA to report the results to the Collector. The primary
> The requirements for LMAP mechanisms will allow a Controller to instruct
> group are the protocol requirements, the data models that define the
> measurement instructions and reports, and identification/extension of
> IETF-specified protocols for securely delivering these data models
> to/from the MAs. Data models should be extensible for new and additional
> measurements.
> 
> > The case where an end user can independently perform network
> > diagnostic measurements (beyond their private network) is not directly
> > in scope, recognizing that users have many opportunities to do this
> today.
> > However, end users can obtain an MA to run measurement tasks if
> > desired and report their results to whomever they want, most likely
> > the supplier of the MA. This provides for user-initiated on-demand
> I would prefer to delete the phrase ", most likely the supplier of the
> MA". I actually think the most likely choice here will be for the end
> user to report the results to the end user. But I don't think this point
> is particular worthy of discussion.
> 
> > However, if the service parameters are available to the MAs, they
> > could be combined with the measurement results in the Report Protocol.
> 
> Recommend:
> Mechanisms for exchanging service parameters external to an end user's
> network is out of scope of the LMAP charter. However, the data model of
> these service attributes is in scope, where they could be combined with
> the measurement results in the Report Protocol. A mechanism that would
> allow a CE Router inside the end user's network to provide other devices
> inside the end user network with information is in scope.
> 
> > systems. Deciding the set of measurements to run is a business
> decision and is out of scope.
> 
> The last sentence seems out of place, since I don't see how it relates
> to the first 2 sentences. I think the last sentence could be added at
> the end of a different paragraph, like the one that starts "It is
> assumed that different organization's LMAP measurement domains can
> overlap..." or the one that starts "Both active and passive measurements
> are in scope..."
> 
> As it relates to deliverables, I think the protocol requirements should
> be included in the framework, and don't need to be a separate
> deliverable.
> BTW, the term "transport protocol" kind of threw me in the description
> of the protocols. I was wondering if "application protocol" might be
> better.
> Recommend:
> simplifying constraints. The framework includes requirements for the
> Control and Report protocols.
> 
>   includes a Data Model consistent with the Information Model plus
> identification/extension of one or more IETF-specified
>   a Data Model consistent with the Information Model plus
> identification/extension of one or more IETF-specified
> - Mechanism to allow a CE Router to provide devices inside the end user
> network with various information that could be useful when analyzing
> measurement results.
> 
> >     Goals and Milestones

---

From: "Bugenhagen, Michael K" <[email-masked]>
Date: Wed, 29 May 2013 23:54:49 +0000

Dan,  

    I believe  we need to put an interface for the customer as part of the scope, I agree which protocol that interface uses comes later - However the question of creation of an interface should be part of the scope conversation.  Personally I would like to see the IETF get accolades for creating that, So I think it should be part of the project.

Thanks
Mike

Sent from my iPhone

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Thu, 30 May 2013 08:09:26 +0000

Hi Mike, 

Can you be more specific what you mean by 'interface for the customer'? More exact:
- who is the customer you have in mind? 
- what kind of interface needs to be standardized? 

Thanks and Regards,

Dan

> From: Bugenhagen, Michael K
> [mailto:[email-masked]]
> Sent: Thursday, May 30, 2013 2:55 AM
> To: Romascanu, Dan (Dan)
> Cc: STARK, BARBARA H; [email-masked]
> Subject: Re: [lmap] lmap charter - draft 2
> 
> Dan,
> 
>     I believe  we need to put an interface for the customer as part of
> the scope, I agree which protocol that interface uses comes later -
> However the question of creation of an interface should be part of the
> scope conversation.  Personally I would like to see the IETF get
> accolades for creating that, So I think it should be part of the
> project.
> 
> Thanks
> Mike
> 
> Sent from my iPhone
> 
> > Even if the scope of the initial phase of LMAP is designed on the
> assumption that the measurement system is under the control of a single
> organization, its components (controllers, MAs, collectors) may
> originate from different sources (vendors of equipment and
> applications). How will we ensure interoperability between these
> components if we do not define/select at least one mandatory-to-
> implement set of protocols and associated data models?
> >
> > Thanks and Regards,
> >
> > Dan
> >
> >
> >
> >
> >> Sorry I didn't manage to write my comments last week. I kept running
> >> out of time, because I didn't want to just throw my comments over the
> >> wall without suggesting language.
> >>
> >> Protocols to Be Used would result in a futile religious war. It's
> >> futile because we've agreed that the MA, Controller and Collector are
> >> all under a single administrative domain -- so whoever owns that
> >> domain will select the protocols they will use. I would prefer if the
> >> charter were to identify existing IETF-defined protocols that *can*
> >> be used, describe any nuances to their use for lmap purposes, define
> >> extensions as necessary, and define any data models that would be
> >> needed to use them for lmap purposes. All without any hint or
> >> suggestion to imply they
> >> *must* be used. And before doing that, requirements for what
> >> constitutes a good protocol would also be useful. Such requirements
> >> can not only be used to identify the gaps in existing IETF protocols,
> >> but would also be useful to any "domain administrators" in developing
> >> criteria for what they need out of a protocol that is used in systems
> >> they procure (that wasn't invented here).
> >>
> >> In addition, I've had people tell me privately that they think my
> >> idea of defining a standard DNS-SD Service Instance for an xml page
> >> on a router that can provide whatever info that router may want to
> >> share about WAN link utilization, its LAN link metrics, service
> >> attributes it may know, etc. would be a good idea. But for some
> >> reason, they don't support the idea publicly on this list. So I'm
> >> going to try one more time to put this in scope. If no-one speaks up,
> >> I'll consider the idea dead here, and will pursue it in BBF. I think
> >> it will be less useful as a BBF-only defined DNS-SD Service Instance,
> >> because I think it would be also useful if off-the-shelf consumer CE
> >> routers (e.g., D-Link, Belkin) could be encouraged to use this to
> >> provide standard-format information about WAN link utilization and
> >> LAN link metrics to home networked devices. But so be it.
> >>
> >>> edge routers, personal computers, mobile devices and other
> >>> networking devices, whether wired or wireless. Standards will help
> >> Recommend re-wording to make this focus on the need for requirements
> >> of protocols and not on a specific set of standardized protocols:
> >>
> >> Standardizing the requirements for mechanisms (Control protocol,
> >> Report protocol, and corresponding data models) will ensure that all
> >> protocols that are used for these purposes are capable of controlling
> >> the same set of measurements and reporting the measurement data in a
> >> standardized manner. This will allow for consistency in protocol
> >> capabilities and behavior, without constraining the deploying entity
> >> unnecessarily in protocol selection. It will make it possible to
> >> incorporate interoperable measurement communications into home and
> >> enterprise edge routers, personal computers, mobile devices and other
> >> networking devices, whether wired or wireless. Standardized
> >> requirements will help these management capabilities become more
> >> pervasive and facilitate measurement results that are directly
> comparable.
> >>
> >>> select/extend one or more protocols for the communications between
> >>> the Measurement Agents' (MAs) function and their Controller function
> >>> and
> >> Collector function.
> >> protocols for the communications between the Measurement Agents'
> >> (MAs) function and their Controller function and Collector function.
> >> The requirements for LMAP mechanisms will allow a Controller to
> >> instruct MAs what performance metrics to measure, when to measure
> >> them, how/when to report the measurement results to a Collector, and
> >> then for the MA to report the results to the Collector. The primary
> >> products of the working group are the protocol requirements, the data
> >> models that define the measurement instructions and reports, and
> >> identification/extension of IETF-specified protocols for securely
> >>> diagnostic measurements (beyond their private network) is not
> >>> directly in scope, recognizing that users have many opportunities to
> >>> do this
> >> today.
> >> I would prefer to delete the phrase ", most likely the supplier of
> >> the MA". I actually think the most likely choice here will be for the
> >> end user to report the results to the end user. But I don't think
> >> this point is particular worthy of discussion.
> >>
> >> Mechanisms for exchanging service parameters external to an end
> >> user's network is out of scope of the LMAP charter. However, the data
> >> model of these service attributes is in scope, where they could be
> >> combined with the measurement results in the Report Protocol. A
> >> mechanism that would allow a CE Router inside the end user's network
> >> to provide other devices inside the end user network with information
> is in scope.
> >>
> >>> Service parameters, such as product category, can be useful to
> >>> decide which measurements to run and how to interpret the results.
> >>> These parameters are already gathered and  stored by existing
> >>> operations systems. Deciding the set of measurements to run is a
> >>> business
> >> The last sentence seems out of place, since I don't see how it
> >> relates to the first 2 sentences. I think the last sentence could be
> >> added at the end of a different paragraph, like the one that starts
> >> "It is assumed that different organization's LMAP measurement domains
> >> can overlap..." or the one that starts "Both active and passive
> >> measurements are in scope..."
> >>
> >>> - The Report protocol and the associated data model: The definition
> >>> of  how the Report is delivered from a MA to a Collector; this
> >>> transport  protocol (to be selected, perhaps REST-style HTTP(s) or
> IPFIX).
> >>
> >> As it relates to deliverables, I think the protocol requirements
> >> should be included in the framework, and don't need to be a separate
> >> deliverable.
> >> BTW, the term "transport protocol" kind of threw me in the
> >> description of the protocols. I was wondering if "application
> >> protocol" might be better.
> >> Recommend:
> >> - The Control protocol and the associated data model: The definition
> >> of  how instructions are delivered from a Controller to a MA; this
> >> identification/extension of one or more IETF-specified  application
> >> protocols.  This is a simple instruction - response  protocol, and
> >> LMAP will specify how it operates over one or more existing
> >> - The Report protocol and the associated data model: The definition
> >> of  how the Report is delivered from a MA to a Collector; this
> >> identification/extension of one or more IETF-specified  application
> >> protocols (perhaps REST-style HTTP(s) or IPFIX).
> >>
> >> And add:
> >> - Mechanism to allow a CE Router to provide devices inside the end
> >> user network with various information that could be useful when
> >> analyzing measurement results.
> >>
> >>>    Goals and Milestones

---

From: "Bugenhagen, Michael K" <[email-masked]>
Date: Thu, 30 May 2013 20:23:30 +0000

Sent from my iPhone

> - who is the customer you have in mind? 
    Consumers of the Internet is what I meant here 
> - what kind of interface needs to be standardized? 
      A consumer should be able to access measurement data which equals an interface of some sort

Thanks
Mike
>>>> of

---

From: Henning Schulzrinne <[email-masked]>
Date: Thu, 30 May 2013 20:55:52 +0000

A good example are DOCSIS cable modems - consumers can and do buy those at retail, and they are portable across ISPs. The same is true for DSL modems in Europe, which are routinely bought from third parties (FritzBox being one example).

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Romascanu, Dan (Dan)
Sent: Wednesday, May 29, 2013 3:52 PM
To: STARK, BARBARA H; [email-masked]
Subject: Re: [lmap] lmap charter - draft 2

Hi Barbara, 

Thanks and Regards,

Dan

> charter were to identify existing IETF-defined protocols that *can* be 
> used, describe any nuances to their use for lmap purposes, define 
> about WAN link utilization, its LAN link metrics, service attributes 
> it may know, etc. would be a good idea. But for some reason, they 
> don't support the idea publicly on this list. So I'm going to try one 
> more time to put this in scope. If no-one speaks up, I'll consider the 
> idea dead here, and will pursue it in BBF. I think it will be less 
> useful as a BBF-only defined DNS-SD Service Instance, because I think 
> it would be also useful if off-the-shelf consumer CE routers (e.g., 
> D-Link, Belkin) could be encouraged to use this to provide 
> standard-format information about WAN link utilization and LAN link 
> metrics to home networked devices. But so be it.
> 
> pervasive and facilitate measurement results that are directly comparable.
> 
> user to report the results to the end user. But I don't think this 
> point is particular worthy of discussion.
> 
> network is out of scope of the LMAP charter. However, the data model 
> of these service attributes is in scope, where they could be combined 
> with the measurement results in the Report Protocol. A mechanism that 
> would allow a CE Router inside the end user's network to provide other 
> devices inside the end user network with information is in scope.
> 
> overlap..." or the one that starts "Both active and passive

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Fri, 31 May 2013 13:16:22 +0000

> A good example are DOCSIS cable modems - consumers can and do buy
> those at retail, and they are portable across ISPs. The same is true for DSL
> modems in Europe, which are routinely bought from third parties (FritzBox
> being one example).

And is the same set of protocols the right answer for both of these?

I suggested to Shane that an "architecture profile" or "device profile" (as done in v6ops) might be appropriate, that identified the set of protocols for a particular environment, rather than suggesting that the same protocol was "right" for all situations.

Personally, I would also think that a DOCSIS modem device profile would be better defined by CableLabs, who has a critical mass of people who understand cable environments, and who could also really and truly ensure interoperability through their testing program. Similarly, I think a European DSL modem device profile would be better defined by HGI, who has a critical mass of experts understanding their environment, and who has a certification program that can truly ensure interoperability. If the providers who operate in these environments want IETF to define device profiles for them, I have no objection to that. But if they want to do their profiles in CableLabs or HGI, I don't think IETF needs to insist on inserting itself into that process. 
Barbara