Subject: [lmap] charter discussion status

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 14 May 2013 10:41:40 +0200

Hi,

I am wondering where we are with the charter discussions. I have not
seen any revised charter proposal since the BOF in Orlando. I assume
that some of the questions asked on this list recently are related to
activities revising the draft charter.

I am wondering when the draft charter will come back to this list for
general review and comments. Perhaps some questions are even easier to
answer with a concrete draft charter text available.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Benoit Claise <[email-masked]>
Date: Tue, 14 May 2013 12:44:20 +0200

Hi Jüergen,

Timely question.
As far as I'm concerned, the couple of clarifying questions I asked on 
the mailing list are now addressed.
Al, Dan, and I are working on the charter proposal.
You can expect some charter proposal very soon.

Regards, Benoit

---

From: "MORTON JR., ALFRED C (AL)" <[email-masked]>
Date: Wed, 22 May 2013 11:39:45 -0400

> -----Original Message-----
> From: Benoit Claise [mailto:[email-masked]]
> Sent: Tuesday, May 14, 2013 6:44 AM
> To: MORTON JR., ALFRED C (AL); Dan Romascanu; [email-masked]
> Subject: Re: charter discussion status
> 
> Hi Jüergen,
> 
> Timely question.

The "first" draft charter text is appended below.

It would be ideal if readers could hold-off on nit-level comments
and post the most significant issues first (but we won't try to achieve
consensus on significance). You get points for suggesting editorial
improvements, but please send them to Dan and Al, not the list.
And last, let us suggest that commenters take the extra time necessary to 
write a brief message when posting to the list. All subscribers will 
appreciate your efforts.

regards,
Dan and Al

-=-=-=-=-=-=-=-=-=-

Name: Large-Scale Measurement of Broadband Performance (LMAP)

AREA: Operations and Management

Chairs:
TBD
TBD

OPS Area Directors:
     
Benoit Claise <bclaise at cisco.com>
Joel Jaeggli <joelja at bogus.com>

OPS Area Advisor:
     Benoit Claise <bclaise at cisco.com>

Mailing Lists:
     General Discussion: lmap at ietf.org
     To Subscribe:       http://www.ietf.org/mailman/listinfo/lmap
     Archive:            http://www.ietf.org/mail-archive/web/lmap

Description of Working Group

Measuring portions of the Internet on a large scale is vital for 
accurate characterizations of performance over time and geography, for 
network diagnostic investigations by providers and their users, and for 
collecting information to support public policy development. The 
ultimate goal is to have the same measurements for a large number of 
points on the Internet, conducted according to the same 
characterization plan, and to have the results collected and stored in 
the same form. 

Many measurement systems that exist today use proprietary, 
custom-designed mechanisms to coordinate their Measurement Agents (MAs) 
deployed across networks, to communicate between MAs and measurement 
Controllers, and to transfer results to measurement Collectors. 
Standardizing these mechanisms (Control protocol, Report protocol, 
and corresponding data models) will make it possible to incorporate 
interoperable measurement communications into home and enterprise 
edge routers, personal computers, mobile devices and other networking 
devices, whether wired or wireless. Standards will help these management 
capabilities become more pervasive and facilitate measurement results 
that are directly comparable.

The Large-Scale Measurement of Broadband Performance (LMAP) working 
group is chartered to determine the form of data models and 
select/extend one or more protocols for the communications between the 
Measurement Agents' (MAs) function and their Controller function and 
Collector function. These three functions comprise the LMAP measurement 
system.

The standardized LMAP mechanisms will allow a Controller to instruct 
MAs what performance metrics to measure, when to measure them, and 
how/when to report the measurement results to a Collector.  The primary 
products of the working group are the data models that define the 
measurement instructions and reports, and protocols for securely 
delivering these data models to/from the MAs. Data models should be 
extensible for new and additional measurements.

A key assumption constraining the initial work is that the measurement 
system is under the control of a single organization (for example, an 
Internet Service Provider or a regulator). However, the components of 
an LMAP measurement system can be deployed in administrative domains 
that are not owned by the measuring organization. Thus, the system of 
functions deployed by a single organization constitutes a single LMAP 
domain which may span ownership or other administrative boundaries. 

The LMAP architecture will allow for measurements that utilize either 
IPv4 or IPv6, or possibly both. Devices containing MAs may have several 
interfaces using different link technologies. Multiple address families 
and interfaces must be considered in the Control and Reporting 
protocols.

It is assumed that different organization's LMAP measurement domains 
can overlap, and that active measurement packets appear along with 
normal user traffic when crossing another organization's network. In 
the initial chartering phase, there is no requirement to specify a 
mechanism for coordination between the LMAP measurements in overlapping 
domains (for instance a home network with MAs on the home hub, set top 
box and laptop). In principle, there are no restrictions on the type of 
device in which the MA function resides. 

Both active and passive measurements are in scope, although there may 
be differences in their applicability to specific use cases, or in the 
security measures needed according to the threats specific to each 
measurement category. At a high level, LMAP systems are agnostic to the 
measurements and results, and extensible to incorporate evolution in 
the measurement area, but the details such as the data models must be 
standardized to match the measurements.

Inter-organization communication of results is a consideration not 
addressed in the LMAP system and deferred to bi-lateral agreements.

Many measurement aspects are already within the charter of IPPM. These 
include standardized definitions of performance metrics, MA-to-MA 
measurement protocols, and a registry of frequently-used metrics and 
parameter settings so they can be identified in an efficient and 
consistent fashion. Neither the definition of the new metrics and 
methods of measurement, nor the post-processing and analysis of results 
falls within the remit of LMAP. 

The case where an end user can independently perform network diagnostic 
measurements (beyond their private network) is not directly in scope, 
recognizing that users have many opportunities to do this today. 
However, end users can obtain an MA to run measurement tasks if desired 
and report their results to whomever they want, most likely the 
supplier of the MA. This provides for user-initiated on-demand 
measurement, which is an important component of the ISP use case.

Another area that is out of scope is the management protocol to 
bootstrap the MAs in measurement devices, although a bootstrapping 
process may be described and conducted in many ways, such as 
configuration during manufacture or with a local USB interface. 
Discovering the service parameters on the MAs or sharing the service 
parameters between MAs are out of the scope of the LMAP charter. 
However, if the service parameters are available to the MAs, they could 
be combined with the measurement results in the Report Protocol.

Service parameters can be used to decide which measurements to run, or 
to interpret the results (for example, when assigning the correct 
product categorization). These parameters are already gathered and 
stored by existing operations systems. Deciding the set of measurements 
to run is a business decision and is out of scope.
 
Exhaustive protection against all possibilities of gaming the 
measurements - where gaming is defined as intentionally (and perhaps 
maliciously) inserting inaccuracy into the overall system or 
measurement process - is beyond the scope of work. Some protections are 
lawyer problems, not engineering problems. However, the working group 
may include protections that do not add significant complexity, as 
determined by working group consensus.

The LMAP working group will coordinate with other standards bodies 
working in this area (e.g., BBF, IEEE 802.16, ETSI), and other IETF 
working groups in the areas of data models, protocols, multiple 
interface management, and measurement of performance metrics.

LMAP will consider re-use of existing protocols and data model 
languages in its efforts to produce the following work items:

- The LMAP Framework - provides common terminology and justifies the 
simplifying constraints 
- The LMAP Use Cases - provides the motivating use cases as a basis for 
the work 
- Information Models, the abstract definition of the measurement tasks 
and the Report; the semantics of the fields and their arrangement (the 
order they appear in and any hierarchy).
- A Data model that defines how the Controller tells an MA to execute a 
measurement instruction. It includes the metric(s) to be measured and 
values for its parameters such as the Peer MA participating in the 
measurement and the desired environmental conditions (for example, only 
conduct the measurement when there is no user traffic observed). It 
also includes the schedule (when the measurement should be run) and how 
the results should be reported (when and to which Collector). 
- A Data model that defines the report that the MA sends to the 
Collector. It includes the metric(s) measured and when, the actual 
result, and supporting metadata such as location. Results reports may 
be organized in batches or may be reported immediately, such as for an 
on-demand measurement. 
- The Control protocol: The definition of how instructions are 
delivered from a Controller to a MA; a Data Model plus a transport 
protocol. This is a simple instruction - response protocol, and LMAP 
will specify how it operates over an existing protocol (to be selected, 
perhaps REST-style HTTP(s) or Netconf). 
- The Report protocol: The definition of how the Report is delivered 
from a MA to a Collector; a Data Model plus a transport protocol (to be 
selected, possibly IPFIX).

    Goals and Milestones

Sep 2013: Initial WG I-D for the LMAP Framework including terminology 
Sep 2013: Initial WG I-D for the LMAP Use cases
Dec 2013: Submit the LMAP Framework I-D to the IESG for consideration 
as Informational RFC
Dec 2013: Submit I-D on the LMAP Use cases to the IESG for 
consideration as Informational RFC
Jan 2014: Initial WG I-D for LMAP Information models
Jan 2014: Initial WG I-D for the Control protocol 
Jan 2014: Initial WG I-D for the Report protocol 
Apr 2014: Initial WG I-D for a Data model for LMAP control information
Apr 2014: Initial WG I-D for a Data model for LMAP report information
July 2014: Submit the LMAP Information models I-D to the IESG for 
consideration as Standards track RFC
July 2014: Submit the Control protocol I-D to the IESG for 
consideration as Standards track RFC
July 2014: Submit Report protocol I-D to the IESG for consideration as 
Standards track RFC
Dec 2014: Submit the Data model for LMAP control I-D to the IESG for 
consideration as Standards track RFC
Dec 2014: Submit the Data model for LMAP report I-D to the IESG for 
consideration as Standards track RFC

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 22 May 2013 22:11:41 +0200

On Wed, May 22, 2013 at 11:39:45AM -0400, MORTON JR., ALFRED C (AL) wrote:
 
> consensus on significance).

[...]

> selected, possibly IPFIX).

The phrase 'fields and their arrangement (the order they appear in and
any hierarchy)' should be something like 'fields and their
relationships'. It would be wrong to prescribe certain hierarchical
nestings since protocols may have different constraints there.

The first item is an information model. This is fine. The second and
third items are data models. I would argue that the details what
should be cover should be moved to the information model - the data
models simply have to consistent with the information model. The
fourth and fifth items talk about protocols and once again data
models. This means confusing overlap.

To sum up, here is a significantly simplified version:

- Information Model, the abstract definition of the information carried
  from the Controller to the MA and the information carried from the MA
  to the Collector. It includes the metric(s) to be measured and
  values for its parameters such as the Peer MA participating in the
  measurement and the desired environmental conditions (for example, only
  conduct the measurement when there is no user traffic observed). It
  also includes the schedule (when the measurement should be run) and how
  the results should be reported (when and to which Collector). It
  further includes the metric(s) measured and when, the actual
  result, and supporting metadata such as location. Result reports may
  be organized in batches or may be reported immediately, such as for an
  on-demand measurement.
- The Control protocol and the associated data model: The definition of
  how instructions are delivered from a Controller to a MA; this
  includes a Data Model consistent with the Information Model plus a
  transport protocol.  This is a simple instruction - response
  protocol, and LMAP will specify how it operates over an existing
  protocol (to be selected, perhaps REST-style HTTP(s) or Netconf).
- The Report protocol and the associated data model: The definition of
  how the Report is delivered from a MA to a Collector; this includes
  a Data Model consistent with the Information Model plus a transport
  protocol (to be selected, perhaps REST-style HTTP(s) or IPFIX).

Sorry for the length of this message.

/js

---

From: "MORTON JR., ALFRED C (AL)" <[email-masked]>
Date: Wed, 22 May 2013 16:50:41 -0400

Thanks for your suggestion of revised text, Juergen.

When you go to the trouble of clarifying sections by re-writing them,
there's no limit on message length, and excerpted text doesn't count. ;-)

Taking it a step further, does your proposal to emphasize the Information
Model obviate the separate memos on the Control and Reporting Data Models,
and thus we should remove them from the milestones, or is the non-abstract
expression of each Data Model useful to keep separate from the Protocol memos?

Al

> From: Juergen Schoenwaelder [mailto:[email-masked]]
> Sent: Wednesday, May 22, 2013 4:12 PM
...

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Thu, 23 May 2013 07:13:10 +0000

This looks good to me. 

May I suggest a slight reorganization of the Information Model bullet to avoid the repetition of 'it includes': 

- Information Model, the abstract definition of the information carried
  from the Controller to the MA and the information carried from the MA
  to the Collector. It includes 
	- the metric(s) to be measured and
  values for its parameters such as the Peer MA participating in the
  measurement and the desired environmental conditions (for example, only
  conduct the measurement when there is no user traffic observed) 
       - the schedule (when the measurement should be run) and how
  the results should be reported (when and to which Collector)
       - the metric(s) measured and when, the actual
  result, and supporting metadata such as location. Result reports may 
  be organized in batches or may be reported immediately, such as for an
  on-demand measurement.

Regards,

Dan

> Sent: Wednesday, May 22, 2013 11:12 PM
> To: MORTON JR., ALFRED C (AL)
> Cc: Benoit Claise; Romascanu, Dan (Dan); [email-masked]
> Subject: Re: [lmap] charter discussion status
> 
> > It would be ideal if readers could hold-off on nit-level comments and
> > post the most significant issues first (but we won't try to achieve
> > - A Data model that defines how the Controller tells an MA to execute
> > a measurement instruction. It includes the metric(s) to be measured
> > and values for its parameters such as the Peer MA participating in the
> > measurement and the desired environmental conditions (for example,
> > only conduct the measurement when there is no user traffic observed).
> > It also includes the schedule (when the measurement should be run) and
> > how the results should be reported (when and to which Collector).
> > will specify how it operates over an existing protocol (to be
> > selected, perhaps REST-style HTTP(s) or Netconf).
> > from a MA to a Collector; a Data Model plus a transport protocol (to
> > be selected, possibly IPFIX).
> 
> third items are data models. I would argue that the details what should
> be cover should be moved to the information model - the data models
> simply have to consistent with the information model. The fourth and
> fifth items talk about protocols and once again data models. This means
> confusing overlap.
>

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Thu, 23 May 2013 07:39:16 +0000

My preference is for separate documents for the Information Model (possibly only one), Data Models (two), and Protocols (two). 

Dan

> From: MORTON JR., ALFRED C (AL) [mailto:[email-masked]]
> Sent: Wednesday, May 22, 2013 11:51 PM
> To: Juergen Schoenwaelder
> Subject: RE: [lmap] charter discussion status
> 
> Taking it a step further, does your proposal to emphasize the
> Information Model obviate the separate memos on the Control and
> Reporting Data Models, and thus we should remove them from the
> milestones, or is the non-abstract expression of each Data Model useful
> to keep separate from the Protocol memos?
> 
> Al
> 
> > From: Juergen Schoenwaelder
> > [mailto:[email-masked]]
> only

---

From: Benoit Claise <[email-masked]>
Date: Thu, 23 May 2013 11:21:31 +0200

> consensus on significance).
And you have no significant issues, please let us know.

Regards, Benoit
> You get points for suggesting editorial

---

From: Benoit Claise <[email-masked]>
Date: Thu, 23 May 2013 11:22:03 +0200

> My preference is for separate documents for the Information Model (possibly only one), Data Models (two), and Protocols (two).
That makes sense to me.

Regards, Benoit

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 23 May 2013 11:32:28 +0200

On Wed, May 22, 2013 at 04:50:41PM -0400, MORTON JR., ALFRED C (AL) wrote:
 
> milestones, or is the non-abstract expression of each Data Model
> useful to keep separate from the Protocol memos?

For some possible protocols, the data model is clearly separated from
the protocol. For other possible protocols, the data model is part of
the resource model you work with, e.g. you define what the REST
resources are and how they react to GET/PUT/POST/DELETE and what they
expects as input and return as output. In this case, separating the
data model description from the protocol makes things likely difficult
to read and understand.  (See the ALTO specification as an example of
a REST approach where things are bundled together.)

Hence I decided to mention protocol and data model under one item,
assuming that the WG will take a decision later when the protocols are
selected whether these will end up in different documents or not.

/js

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Thu, 23 May 2013 09:55:02 +0000

Good point. 

Dan

> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Juergen Schoenwaelder
> Sent: Thursday, May 23, 2013 12:32 PM
> expects as input and return as output. In this case, separating the data
> model description from the protocol makes things likely difficult to
> read and understand.  (See the ALTO specification as an example of a
> REST approach where things are bundled together.)
> 
> _______________________________________________
> lmap mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/lmap

---

From: "Bugenhagen, Michael K" <[email-masked]>
Date: Thu, 23 May 2013 12:56:31 +0000

Benoit -

  I see two specific clauses that I feel need to be added, and of course I'll make contributions to such areas in the future provided the group adds these to the charter....

1)  Access Service Attribute Validation - ( I would put this under the section where it describes a measurement domain putting MA's in ISP domain)

	LMAP should define a standard interface, and protocol to obtain near-real time access service provisioning attributes to ensure the test results are being conducted and compared to actual ISP terms of service.

2)  Testing Control group mechanisms  ( I recommend changing the "gaming" paragraph to reflect this)

	LMAP may recommend creation of "control group tests" which are specifically designed to validate and investigate results, thereby lowering the risk of gaming the test results by special treatment of test traffic.

Thanks,
Mike   

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Benoit Claise
Sent: Thursday, May 23, 2013 4:22 AM
To: MORTON JR., ALFRED C (AL)
Cc: Dan Romascanu; [email-masked]
Subject: Re: [lmap] charter discussion status

On 22/05/2013 17:39, MORTON JR., ALFRED C (AL) wrote:
>> As far as I'm concerned, the couple of clarifying questions I asked 
>> on the mailing list are now addressed.

Regards, Benoit
> You get points for suggesting editorial improvements, but please send 
> them to Dan and Al, not the list.
> And last, let us suggest that commenters take the extra time necessary 
> to write a brief message when posting to the list. All subscribers 
> will appreciate your efforts.
>
> regards,
> Dan and Al
>
> -=-=-=-=-=-=-=-=-=-
>
> network diagnostic investigations by providers and their users, and 
> for collecting information to support public policy development. The 
> custom-designed mechanisms to coordinate their Measurement Agents 
> (MAs) deployed across networks, to communicate between MAs and 
> measurement Controllers, and to transfer results to measurement Collectors.
> Standardizing these mechanisms (Control protocol, Report protocol, and 
> corresponding data models) will make it possible to incorporate 
> interoperable measurement communications into home and enterprise edge 
> routers, personal computers, mobile devices and other networking 
> devices, whether wired or wireless. Standards will help these 
> management capabilities become more pervasive and facilitate 
> measurement results that are directly comparable.
>
> Collector function. These three functions comprise the LMAP 
> measurement system.
>
> how/when to report the measurement results to a Collector.  The 
> primary products of the working group are the data models that define 
> the measurement instructions and reports, and protocols for securely 
> IPv4 or IPv6, or possibly both. Devices containing MAs may have 
> several interfaces using different link technologies. Multiple address 
> families and interfaces must be considered in the Control and 
> Reporting protocols.
>
> mechanism for coordination between the LMAP measurements in 
> overlapping domains (for instance a home network with MAs on the home 
> hub, set top box and laptop). In principle, there are no restrictions 
> on the type of device in which the MA function resides.
>
> measurement category. At a high level, LMAP systems are agnostic to 
> the measurements and results, and extensible to incorporate evolution 
> in the measurement area, but the details such as the data models must 
> be standardized to match the measurements.
>
> methods of measurement, nor the post-processing and analysis of 
> results falls within the remit of LMAP.
>
> The case where an end user can independently perform network 
> diagnostic measurements (beyond their private network) is not directly 
> in scope, recognizing that users have many opportunities to do this today.
> However, end users can obtain an MA to run measurement tasks if 
> desired and report their results to whomever they want, most likely 
> the supplier of the MA. This provides for user-initiated on-demand 
> However, if the service parameters are available to the MAs, they 
> could be combined with the measurement results in the Report Protocol.
>
> stored by existing operations systems. Deciding the set of 
> measurements to run is a business decision and is out of scope.
>   
> measurement process - is beyond the scope of work. Some protections 
> are lawyer problems, not engineering problems. However, the working 
> group may include protections that do not add significant complexity, 
> as determined by working group consensus.
>
> - The LMAP Use Cases - provides the motivating use cases as a basis 
> for the work
> Sep 2013: Initial WG I-D for the LMAP Use cases Dec 2013: Submit the 
> LMAP Framework I-D to the IESG for consideration as Informational RFC 
> consideration as Informational RFC Jan 2014: Initial WG I-D for LMAP 
> Information models Jan 2014: Initial WG I-D for the Control protocol 
> Jan 2014: Initial WG I-D for the Report protocol Apr 2014: Initial WG 
> I-D for a Data model for LMAP control information Apr 2014: Initial WG 
> I-D for a Data model for LMAP report information July 2014: Submit the 
> LMAP Information models I-D to the IESG for consideration as Standards 
> track RFC July 2014: Submit the Control protocol I-D to the IESG for 
> consideration as Standards track RFC July 2014: Submit Report protocol 
> I-D to the IESG for consideration as Standards track RFC Dec 2014: 
> Submit the Data model for LMAP control I-D to the IESG for 
> consideration as Standards track RFC Dec 2014: Submit the Data model 
> for LMAP report I-D to the IESG for consideration as Standards track 
> RFC
>
>
>
>

---

From: "Jiankang Yao" <[email-masked]>
Date: Mon, 27 May 2013 10:43:21 +0800

whether the data models and protocols are separated or not, it is suggested that 
Framework docuemnt should clear about it and say the relationship between the data models documents and protocol documents if they are separated.

Jiankang Yao

From: Juergen Schoenwaelder
Date: 2013-05-23 17:32
To: MORTON JR., ALFRED C (AL)
CC: Benoit Claise; Dan Romascanu; [email-masked]
Subject: Re: [lmap] charter discussion status

/js

---

From: <[email-masked]>
Date: Wed, 29 May 2013 11:57:06 +0100

Thanks for the work on this, it looks good to me. 

I like Juergen's changes and agree with deciding later whether each data model & protocol is bundled together. Also agree with doing a single Information Model document, because the report is likely to echo several parts of the Instruction.

> configuration during manufacture or with a local USB interface.

At one point we discussed detailing the bootstrap process but not the bootstrap protocol. from the text, I'm not sure whether we'll work on the bootstrap process. To the extent it could be covered in the framework doc, I'm ok to include it (but don't object if we don't)

...
> Dec 2013: Submit the LMAP Framework I-D to the IESG for consideration as Informational RFC

I think an important milestone is for us to reach consensus on which protocol(s) we will re-use or extend for the Control and Report protocol (I assume we won't define a new protocol)
I think the framework doc would be a good place to record this consensus and that the timescale is realistic (ie the WG makes a decision in the autumn)

...
> Jan 2014: Initial WG I-D for LMAP Information models 

I think this is very pessimistic (for your info, some of us are planning to submit an individual i-d for Berlin)

Juergen's changes mean the other milestones need adjusting, I suggest:-

Jan 2014: Initial WG I-D for the Control protocol and the associated data model 
Apr 2014: Initial WG I-D for the Report protocol and the associated data model
July 2014: Submit WG I-D for the Control protocol and the associated data model to the IESG for consideration as Standards track RFC
July 2014: Submit WG I-D for the Report protocol and the associated data model to the IESG for consideration as Standards track RFC

I suggested staggering the initial I-D deadlines. Seems more natural to do Control first. We should finish them together, as there may be dependencies.

Thanks
phil

---

From: <[email-masked]>
Date: Wed, 29 May 2013 12:04:03 +0100

Mike

I think both of the topics are interesting

On 1, I think this is something the framework doc could consider (at a high level) but I don't favour including it in the first phase of lmap work - think it's better to concentrate on the Controller-MA and MA-Collector interfaces.  

On 2, this isn't a topic I know about, do you have references on ways this can be done? would like to be educated. is it known how to do this or is it an area where research is needed? 
Formally, I guess it would fit better in IPPM, as it is a test.

Best wishes
phil

> Bugenhagen, Michael K
> Sent: 23 May 2013 13:57
> To: 'Benoit Claise'; MORTON JR., ALFRED C (AL)
>   I see two specific clauses that I feel need to be added, and of
> course I'll make contributions to such areas in the future provided the
> group adds these to the charter....
> 
> 
> 
> 
> 1)  Access Service Attribute Validation - ( I would put this under the
> section where it describes a measurement domain putting MA's in ISP
> domain)
> 
> 	LMAP should define a standard interface, and protocol to obtain
> near-real time access service provisioning attributes to ensure the
> test results are being conducted and compared to actual ISP terms of
> service.
> 
> 
> 
> 
> 	LMAP may recommend creation of "control group tests" which are
> specifically designed to validate and investigate results, thereby
> lowering the risk of gaming the test results by special treatment of
> test traffic.
> 
> 
> 
> 
> Thanks,
> Mike
> 
> 
> 
> 
> 
> 
> 
> 
> Benoit Claise
> and
> edge