Subject: [ippm] Fwd: [lmap] Proposed LMAP Charter

From: Brian Trammell <[email-masked]>
Date: Wed, 23 Jan 2013 13:31:21 +0100

Greetings, all,

For those of you interested in LMAP (large-scale measurement control) topics who aren't already on the [email-masked] mailing list, there's a discussion for a potential charter starting. This is of specific interest to IPPM, as the proposal being discussed explicitly mentions restricting the scope of metrics used in LMAP to those defined by IPPM. 

(This will be my only cross-forwarding of this thread; please head over to [email-masked] to discuss.)

Best regards,

Brian

Begin forwarded message:

> From: <[email-masked]>
> Subject: Re: [lmap] Proposed LMAP Charter
> Date: 23 January 2013 10:54:39 GMT+01:00
> To: <[email-masked]>, <[email-masked]>, <[email-masked]>
> 
> Thanks Marc. We’re very interested in discussing a potential charter.
> One thing we think is worth trying to do is to simplify the proposed work as much as possible, as the potential scope is big. Of course this could be done entirely as part of the WG’s work, but doing this beforehand would enable a more rapid start. Below is our attempt to suggest some simplifying assumptions.
> Each of the simplifying assumptions  has some penalty, so it would be great to hear whether people think they’re reasonable, or over-restrict the use case that you have in mind. If it’s helpful we could write up the architectural assumptions in an I-D – co-authors very welcome.
> So a key question is whether the prospective WG should start with a phase of reaching consensus on architecture and choosing starting protocols, or whether we can reach enough consensus before the next IETF to move straight to standards work. I’m not sure that we can decide this right now, as it depends on how the discussion goes over the next month or so. There are some work items below that might apply in the latter case.
> --
> Measuring broadband service on a large scale is important for network diagnostics by providers and users, as well for public policy.  {Comment: operators may want measurement to help network planning and network mgt like fault identification; not sure whether this needs explicitly noting.}  To conduct service measurements, Measurement Agents (MAs) on user networks gather data, either on their own initiative or instructed by a Controller, and then upload the measurement results to a designated Collector. Currently measurement platforms contain up to a few thousand MAs. However, the vision is that MAs could potentially be embedded in every home hub, cable modem, enterprise edge router, set top box, smartphone and so on. Standards will help this capability be more pervasive, manageable and directly comparable.
> RFCs from the IPPM WG define how to measure metrics – requests to define new tests should be handled by IPPM. These tests effectively define the communications between a MA and Test Server. {Comment: there are likely new metrics that need to be defined (eg latency under load) or tests under new conditions (eg current test for speed may not scale well to high line speeds).} . In a few cases the IPPM definition may need to be extended to move results data from the Test Server to the MA for later reporting.
> The LMAP WG makes the following architectural assumptions – in order to make rapid progress with an acceptable loss of generality:-
> ·         We assume that Measurement Agent, tests Controller and results Collector are all under control of same organisation (eg Samknows, FCC, BT). Reason: Inter-organisation interactions (for example to share results) is better handled by business-level negotiation than a control protocol.
> ·         We assume the MA initiates a measurement. Reasons: having the test schedule on MA avoids it having to check frequently with the Controller; it is easier if the MA is behind a NAT; and the MA knows whether the user is active and therefore whether to run test or delay it.
> ·         We assume measurement results are sent from MA, and not from test server. Reason-1: It is easier to secure the MA to Collector communication. One implication is that for an upload test measured by the test server, the result is reported back to the MA which reports it in turn. Reason-2: Test servers may be controlled by a third party (‘load time for IETF home page’).
> ·         We assume no negotiation is needed between Controller and MA. The Controller already knows the characteristics of the MA, for instance via TR069 for Broadband Forum devices. The Controller simply instructs (securely) the MA “Measure this”; the MA replies “OK /error”.
> ·         Similarly we assume no negotiation is needed between the MA and Collector. The protocol simply delivers (securely) the measurement results from the MA to the Collector, and returns an ack or error code.
> ·         We assume each MA is controlled by only one Controller. Reason: to avoid different Controllers giving the MA conflicting schedules. Note: a measurement platform may have several Controllers, with each Controller controlling a subset of the MAs, for example one controlling wireline MAs and one mobile MAs.
> ·         We assume the raw measurement results are reported; filtering /averaging of measuremnts by the MA is not in the initial scope, except as defined by IPPM as part of the metric definition.
> ·         We assume that uploading a new test capability to the MA is out of scope. This can be done as a firmware upgrade for home hub, or new app for PC, etc.
> ·         We assume as out of scope a network parameter server (which stores nominal values such as the contractual data rate). The functional role of the Network Paraemeter Server may be fulfilled by existing OSS (such as network or home gateway management systems). Interfaces to the NPS are pre-measurement (used by the Controller to decide which tests to schedule) or post-measurement (used to analyse the measurement data).
> ·         We assume data filtering is out of the initial scope. Filtering includes: removal of outlier measurements, archiving (curating) data for scientific research, and so on. There are certainly some aspects where best practice would be useful but data filtering is not a first priority.
> ·         We assume that data privacy issues are out of scope although security is considered. The MA, Controller and Collector are all under control of the same organisation. IPPM considers the implications of any passive measurement tests that it defines.
> The LMAP WG standardises the following items:
> 1.      The data model that defines the test schedule, which includes:
> a.      The IPPM metric to be tested and values for its parameters. {Question: is the restriction to IPPM OK? Ie non-IPPM tests are out of scope}
> b.      Test schedule: when the test should be done and how often repeated (‘once an hour at xx.03’). A test may also be a one-off, done immediately on demand.
> c.       Output parameters: some IPPM metrics may define a choice of outputs
> d.      Test Server: which server or servers should be used for the test
> e.      Environmental conditions: such as ‘no user traffic’ which means a particular instance of the test is delayed or abandoned.
> f.        (perhaps) Metadata: such as a mobile’s location when the test should be made, the line’s nominal rate etc. There may be privacy implications for the scenario where the measurement plaform is not controlled by the operator; the initial scope will exclude this scenario.
> g.      (perhaps) Metadat to report: for example, the MA may be able to measure Layer 1 o 2 charcteristics such as DSL sync speed and interleaving; the actual test is out of scope. As for the previous bullet, there may be privacy implications for the general scenario, which we exclude from the initial scope.
> h.      (perhaps) Priority: the Controller decides which tests are done when. However a test might get delayed until it overlaps with another scheduled test – which should be done (or done first)? Or should it simply be abandoned?
> 2.       The data model that defines how an MA should report test results. This may be combined with (1) above or specified as a separate reporting schedule. This includes:
> a.       Where to report results to (one or more Collectors)
> b.      How often to report results for the different tests
> c.       What to do when Collector is not available (fail-over, try again)
> 3.      The data model that defines the report, which includes:
> a.      The metric that was measured
> b.      The time of the test
> c.       Whether the test is scheduled, one-off triggered by the Controller, or user-initiated
> d.      The measurement result – usually several results are batched together; the result of a one-off test may be wanted immediately
> e.      (perhaps) Metadata, such as the mobile’s location
> f.        Comment: it would make sense to use the same language as for 1, for instance XML, YANG…
> 4.      The application level protocol for the (secure) delivery of the test and reporting schedule (#1, #2)
> a.      A simple instruction – response protocol, with error codes defined
> b.      LMAP could specify how it operates over one existing protocol (REST style HTTP(s), Netconf…)
> 5.      The application level protocol for the (secure) delivery of the report (#3)
> a.      Same comments as for #4
> 6.      A document explaining the simplifying architectural assumptions in more detail. This document could also define some terminology and explain the use cases.
> Looking forward to the discussion,
> Best wishes,
> Trevor and Phil
> From: [email-masked] [mailto:[email-masked]] On Behalf Of Marc Linsner
> Sent: 18 January 2013 20:48
> To: [email-masked]
> Subject: [lmap] Proposed LMAP Charter
>  
> Let's discuss possible charter language for the proposed LMAP work.  At this point, IMO, we should propose a charter for the work that could either be added to an existing WG's charter, or standalone as a WG charter.  That's an AD decision.  A follow-on discussion to this would obviously be the output documents and milestones.
>  
> Here is a strawman to start with - 
>  
> "
> Measuring broadband service on a large scale is important for network diagnostics by providers and users, as well for public policy.  To conduct service measurements, user networks gather data, either on their own initiative or instructed by a measurement controller, and then upload the measurement results to a designated measurement server. There is a need to standardize a logical architecture and describe key requirements for protocols to connect the components.  A large-scale measurement system needs to support residential and small-enterprise networks, using either wired or wireless networks. The architecture needs to consider the management of the active and passive measurements.
>  
> The LMAP work will include describing various use cases considered to be Large Scale Performance Measurement systems, define the requirements of the management of these systems, and finally define the management communication architecture and protocols.
>  
> This work will also include discussions about the privacy and security concerns and will address them within its documents.
> "
>  
> Fire away!
>  
> -Marc Linsner-
> _______________________________________________
> lmap mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/lmap