Subject: Re: [lmap] LMAP: situation: Q1-Use Cases

From: "STARK, BARBARA H" <[email-masked]>
Date: Thu, 11 Apr 2013 20:22:55 +0000

> Question 1. Use cases
> Do we agree that we want to focus on the Internet Service Provider and the End User Network Diagnostics use cases for now? As a phase 2, we could focus on the 3rd party use case: multi-provider, regulator, over the top
> Question: in the End User Network Diagnostics use case, I'm unclear if the user initiated measurement is included?

Here is my opinion on what I would suggest considering as in scope:
1.a) The Access Provider / ISP use case relative to collecting metrics for ensuring the health of their own network. (or what is recommended for an ISP in the context of their internal measurement collection needs)
1.b) The Access Provider / ISP use case relative to meeting potential regulatory requests. (or what is recommended for an ISP who wants to support measurement collection for regulatory consumption). Note that this may involve recommendations for how an ISP provides service attributes to an end-user-network-located MA not under the ISP's control, or how an ISP provides other entities with access to the ISP's test endpoints, or how the ISP's end-user-network-located MAs may interact with test endpoints provided by others. I see some limited set of inter-domain discussion as being relevant to this usage. For this usage, I'd like to discuss the full set of scenarios for supporting regulatory requests where *something* is requested of the ISP. Not just the case where the ISP supplies the MA and tests against endpoints in the ISP network.
1.c) The Access Provider / ISP  use case relative to supporting the end user's diagnostics needs (or what is recommended for an ISP who wants to provide an end user with tools for helping with end user network diagnostics). As with 1.b), I see this as not just being about the case where the ISP supplies the MA and test endpoints, but also about supporting inter-domain measurements between end-user- or 3rd-party-managed and ISP-managed devices, as well as potentially supplying the end user with service attribute information. I think this usage should include user-initiated measurements.

Note that in the inter-domain interactions I mention above, I'm only suggesting inter-domain running of measurements and unidirectional providing of service attributes to a device inside the end user's network. These inter-domain interactions do not involve inter-domain management, control, or configuration -- which I would prefer to avoid.

It sounds like there may be additional interest in providing recommendations for end-user-controlled MAs that have the ability to send collected data to non-end-user-controlled Data Collectors. It may or may not be possible for the end user to specify the Data Collector? [Or perhaps the recommendations would only apply to an MA that does allow the end user to specify the Data Collector?] I'm a bit fuzzy on this one. It may instead be useful to just create a generic BCP for an MA that sends data to a Data Collector under someone else's control. Or even more generically, a BCP about which data elements may be considered end-user-identifiable/private info for consideration when one entity (e.g., the end user, or the ISP) supplies collected data to another entity (inter-domain data transfer). It might be useful if such a BCP included discussion as to how the release of that information to others could be used to invade an end user's privacy, to better understand why the information is recommended to be private.

While I have seen statements proclaiming the value of 3rd party measurement systems (and, BTW, I agree that such systems are valuable) I don't recall having seen requests for helping to architect such systems. People who have put such systems together seem to be quite happy with their systems. However, they may benefit from ippm recommendations and efforts. They may also benefit from mappings of ippm registry elements into certain IETF protocol models (such as SNMP MIBs, netconf data model, or ipfix data model). They may benefit from something that allows the ISP to provide Servcie Attributes to MAs inside the end user network. And they may benefit from privacy recommendations.

Given the above scope, I could envision potentially having the following deliverables:
1. ISP lmap Architecture that includes (1) intra-domain interfaces between an MA and other MAs (test endpoints) and operational support systems that manage/control/configure those MAs and the physical devices/elements that contain the MA, and (2) inter-domain interfaces for (a) requesting resources on ISP test endpoints, (b) ISP MA requesting resources on others' test endpoints, and (c) ISP providing service attribute info to an MA inside an end user's network.
2. Mechanism for intra- and inter-domain requesting of test endpoint resources
3. Mechanism for providing service attribute info
4. BCP on end-user identifiable data elements with discussion of threats to privacy and recommendations for maintaining privacy
5. Data models / MIBs for various IETF protocols, based on the ippm registry.

Personally, I would be willing to participate in and contribute to 1-4 above. I have no objection to 5, and think it would probably be useful to those who use such protocols.
Barbara

---

From: Shane Amante <[email-masked]>
Date: Fri, 12 Apr 2013 09:49:44 -0600

Barbara,

Thanks for your note.  I have some questions for clarification, below.

>> Question: in the End User Network Diagnostics use case, I'm unclear if the user initiated measurement is included?

[--snip--]

> 2. Mechanism for intra- and inter-domain requesting of test endpoint resources

WRT #1 and #2, above, I am wondering if you could clarify your use of the word "inter-domain"?  More specifically, I think it would be helpful if you could characterize inter-domain in the context of:
a)  "Control/Signaling Protocols", i.e.: those protocols that are used to communication from, say, a Measurement Controller to a Measurement Agent to start/stop one or more tests, Measurement Agent to Measurement Data Collector to communicate results of tests, etc.
b)  "Measurement Protocols" and the parts of the network over which they will operate, i.e.: those (IPPM) protocols that are run over-the-wire from Measurement Agent to Measurement Server to measure things like: bandwidth, latency, packet loss, jitter, etc.

FWIW, I think that (b) is more about what part, or parts, of the network are intended to be measured, whereas (a) is more about how the various 'entities' (MA, MS, MC, etc.) within the LMAP Architecture communicate with one another to accomplish the tasks to requesting a test, starting a test, uploading of measurement results, etc.

Thanks,

-shane

> _______________________________________________
> lmap mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/lmap
>

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Fri, 12 Apr 2013 18:46:36 +0000

> > 1. ISP lmap Architecture that includes (1) intra-domain interfaces between
> an MA and other MAs (test endpoints) and operational support systems that
> manage/control/configure those MAs and the physical devices/elements
> that contain the MA, and (2) inter-domain interfaces for (a) requesting
> resources on ISP test endpoints, (b) ISP MA requesting resources on others'
> test endpoints, and (c) ISP providing service attribute info to an MA inside an
> end user's network.
> WRT #1 and #2, above, I am wondering if you could clarify your use of the
> word "inter-domain"?  More specifically, I think it would be helpful if you
> could characterize inter-domain in the context of:
> a)  "Control/Signaling Protocols", i.e.: those protocols that are used to
> communication from, say, a Measurement Controller to a Measurement
> Agent to start/stop one or more tests, Measurement Agent to
> Measurement Data Collector to communicate results of tests, etc.
> b)  "Measurement Protocols" and the parts of the network over which they
> will operate, i.e.: those (IPPM) protocols that are run over-the-wire from
> Measurement Agent to Measurement Server to measure things like:
> bandwidth, latency, packet loss, jitter, etc.
> 
> 
> FWIW, I think that (b) is more about what part, or parts, of the network are
> intended to be measured, whereas (a) is more about how the various
> 'entities' (MA, MS, MC, etc.) within the LMAP Architecture communicate
> with one another to accomplish the tasks to requesting a test, starting a test,
> uploading of measurement results, etc.

Hi Shane,
By "intra-domain", I mean that the same entity has control/management over both ends of the interface. "Inter-domain" means that different entities control/manage the interface endpoints.

I have no interest in scoping lmap "phase 1" to try to deal with protocols for supporting "inter-domain" communication outside of "ask permission to run a test just before running it" and providing service attributes from a device inside the home network to another device inside the same home network. For Measurement Controller to MA to start/stop/schedule tests, I'm only interested in scoping lmap (phase 1) to discuss intra-domain. Same for MA to Data Collector. The same goes for communication between a device or network element to a Management Server. Which isn't to say that such inter-domain interfaces won't or can't or shouldn't exist (I fully support their right to exist) -- I'm just suggesting that they not be in lmap "phase 1" scope.

I think there is a big need for measurement protocols to be run inter-domain. Since there seems to be a desire by the owners of some test endpoints to require "ask permission first", this needs to be in lmap "phase 1" scope: for intra-domain (test endpoints under "the ISP"s control), and inter-domain (one end of the test under "the ISP"s control and the other end under someone else's control). Also, if the MA can be under someone else's control and needs to associate service attributes with the measurement data, then we need some way to supply those attributes from "the ISP" to the MA. [I've put "the ISP" in quotes because it's been suggested to be a central figure in the phase 1 scope.]

There's also a bunch of intra-domain interfaces I'm not interested in seeing in "phase 1". Basically, interfaces with the MA (or its physical device/element) as a communication endpoint I would like to see in scope. All other interfaces I would prefer not to have in scope for lmap "phase 1". For example, if or how a Management Server talks directly a Measurement Controller, or a Data Collector, I really don't care. 
Barbara

---

From: Shane Amante <[email-masked]>
Date: Fri, 12 Apr 2013 13:53:03 -0600

Hi Barbara,

On Apr 12, 2013, at 12:46 PM, "STARK, BARBARA H" <[email-masked]> wrote:
> I have no interest in scoping lmap "phase 1" to try to deal with protocols for supporting "inter-domain" communication outside of "ask permission to run a test just before running it"

I would agree that LMAP "Phase 1" must support a Inter-Domain capability to 'ask permission to run a test just before running it'.  (See below for why).

> and providing service attributes from a device inside the home network to another device inside the same home network.

I could agree with this, as well.  

By service attributes, I assume you mean something along the lines of the ISP "service package" the customer has subscribed to, e.g.: 100 Mbps down/50 Mbps up; DSL or DOCSIS access technology; etc. correct?

However, one addition to this would be that I think it's important to share 'service attributes' from a so-called Measurement Parameter Server to (a) a Measurement Collector (directly) -or- (b) from a so-called Measurement Parameter Server to a Measurement Agent (e.g.: in the home) and have the MA share the 'service attributes' with the Measurement Collector, (perhaps bundled in as part of the measurement results of a just performed test).  IMO, measurement results are only going to be meaningful if they also contain the so-called 'service attributes' so that it's easy for someone performing post-analysis of measurement results to know that they are looking at 'valid' data.  FWIW, I don't have a strong preference on (a) or (b) and would be open to discussing other operational models to facilitate this communication as well as whether (a) and (b) could "co-exist" in the overall LMAP architecture and it will be up to ISP's (or, others) to choose whichever one they prefer.

> For Measurement Controller to MA to start/stop/schedule tests, I'm only interested in scoping lmap (phase 1) to discuss intra-domain. Same for MA to Data Collector. The same goes for communication between a device or network element to a Management Server.

Just so I'm clear, in the above, do you equate a "MA" to exist on, say, only a CPE router/modem in a end-user's house?  And, the "Measurement Server" side does not contain a "MA"?  (I believe that is the appropriate frame of reference, given the various drafts in LMAP, so far).

This could fit the model you've proposed for LMAP "Phase 1" where there is Intra-Domain communication from a Measurement Controller to a MA (e.g.: to start/stop tests).  At the same time, there exists a third-party (Inter-Domain) "Measurement Server" to which the MA will conduct network performance tests against.  In this particular instance, as I understand it, the 3rd-party Measurement Server would have no "heads-up" (or, as you phrased it "be asked permission just before running it") ... at least, via an Inter-Domain communication method directly from a "Measurement Controller".  That could be one operational model, but it may lead to concerns with respect to a (unintended) DoS attack on the Measurement Server side of things or, worse, we obtain 'tainted'/suboptimal performance results, because the Measurement Server side is over-subscribed (e.g.: BW or CPU saturated).

IMO, it might be useful to think of allowing Inter-Domain communication from the "Measurement Controller" to the "Measurement Server" and from the "MA" to the "Measurement Server" strictly for the capability to perform some type of "Call Admission Control" (CAC) function.  The benefit of allowing the Measurement Controller to perform CAC to the Measurement Server is, presumably, it could aggregate a whole series of tests it is *about* to schedule on MA's to a Measurement Server and say: "Can you handle tests that may generate 10 Gbps of traffic in the next N minutes?"  The Measurement Server may simply respond: "No" or "No, but I can handle 5 Gbps".  OTOH, we likely need a similar CAC function on the "MA" to Measurement Server so that in the cases of various network and/or server failures, the "MA" can dynamically and autonomously adapt on its own to those conditions w/out having to be explicitly told to "knock it off" by a Measurement Controller.  The latter is likely critical in the case where the Measurement Controller is offline (failed) or otherwise not reachable from the MA.

I would agree that, in general, anything beyond a relatively straightforward Intra- and Inter-Domain CAC function from MC and MA toward the Measurement Server should be out-of-scope for a LMAP "Phase 1".

Does the above sound reasonable?

> Which isn't to say that such inter-domain interfaces won't or can't or shouldn't exist (I fully support their right to exist) -- I'm just suggesting that they not be in lmap "phase 1" scope.
> 
> I think there is a big need for measurement protocols to be run inter-domain. Since there seems to be a desire by the owners of some test endpoints to require "ask permission first", this needs to be in lmap "phase 1" scope: for intra-domain (test endpoints under "the ISP"s control), and inter-domain (one end of the test under "the ISP"s control and the other end under someone else's control).

I also think a CAC function would be useful (required?) from MA to MS and from MC to MS both in the intra- and inter-domain contexts.

> Also, if the MA can be under someone else's control and needs to associate service attributes with the measurement data, then we need some way to supply those attributes from "the ISP" to the MA. [I've put "the ISP" in quotes because it's been suggested to be a central figure in the phase 1 scope.]

Agreed.  As I said above, I'm not sure how it is best to accomplish that and, for that matter, whether we need to choose only one ... assuming that having more than one does not create complexity.

> There's also a bunch of intra-domain interfaces I'm not interested in seeing in "phase 1". Basically, interfaces with the MA (or its physical device/element) as a communication endpoint I would lik…

By "Management Server", do you mean something like a NMS?  (Network Management System?)

-shane

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Sat, 13 Apr 2013 00:28:27 +0000

> By service attributes, I assume you mean something along the lines of the ISP
> "service package" the customer has subscribed to, e.g.: 100 Mbps down/50
> Mbps up; DSL or DOCSIS access technology; etc. correct?

Yes
 
> However, one addition to this would be that I think it's important to share
> 'service attributes' from a so-called Measurement Parameter Server to (a) a
> Measurement Collector (directly)

If the ISP who knows the attributes also manages the Data Collector, then that's intra-domain, and the ISP should be able to figure out how to do that within there OSS network without help from IETF. Therefore, I don't see a need for the intra-domain interface to be in lmap phase 1 scope. If the ISP does not control the Data Collector, then I think this is a much more difficult problem to solve, because it involves the ISP knowing that the Data Collector controlling entity has the end user's permission to have access to this data. Because of the need for permission with all of its associated need to authenticate the Data Collector and "know" that the end user has authorized that Data Collector (and allow the end user to revoke that authorization, change that authorization, etc.), I consider this a much more complex problem to solve and would prefer for it not to be in lmap phase 1 scope.

> -or- (b) from a so-called Measurement
> Parameter Server to a Measurement Agent (e.g.: in the home) and have the
> MA share the 'service attributes' with the Measurement Collector, (perhaps
> bundled in as part of the measurement results of a just performed test).

Yes. If an MA is already inside the home network of the end user whose service it is, then providing information to that MA about the end user's broadband service should be ok. If the "MA" getting the info is a Trojan or spyware (in other words, inside the home network without the end user's permission or knowledge), then I think it having access to the end user's broadband service attributes may be the least of the end user's worries.

> IMO, measurement results are only going to be meaningful if they also
> contain the so-called 'service attributes' so that it's easy for someone
> performing post-analysis of measurement results to know that they are
> looking at 'valid' data.  FWIW, I don't have a strong preference on (a) or (b)
> and would be open to discussing other operational models to facilitate this
> communication as well as whether (a) and (b) could "co-exist" in the overall
> LMAP architecture and it will be up to ISP's (or, others) to choose whichever
> one they prefer.

Yes, this is an important topic. As I mentioned above, I do have a preference for which to address in the context of standardization (lmap phase 1). But even if IETF doesn't define all of the 3-way authorization/authentication and data exchange mechanisms for a fully standardized (a) inter-domain solution, it would still be possible for an ISP to do this on their own, or together with a regulator-identified 3rd party. So the ISP (or others) will still be free to choose that approach.
 
> > For Measurement Controller to MA to start/stop/schedule tests, I'm only
> interested in scoping lmap (phase 1) to discuss intra-domain. Same for MA to
> Data Collector. The same goes for communication between a device or
> network element to a Management Server.
> 
> Just so I'm clear, in the above, do you equate a "MA" to exist on, say, only a
> CPE router/modem in a end-user's house?  And, the "Measurement Server"
> side does not contain a "MA"?  (I believe that is the appropriate frame of
> reference, given the various drafts in LMAP, so far).

No. In my vision of the term, an MA (as a generic Measurement Agent) can exist on anything that has L1 connectivity or above. In the context of IETF, I would suggest that IETF can be helpful with solutions for MAs that have IP connectivity. I really think we should get away from Measurement Server. My terminology vision is that active tests can run between MAs. One MA is generally responsible for initiating a particular active test. In that case, I tried (I think) to refer to the one as the initiating MA and the other end of the test as the test endpoint. I think the lmap phase 1 scope is also supposed to support ISP tests that may be run between endpoints (MAs) that are all within the ISP's network. I also think that some ISPs may choose to run diagnostic tests (usually while on the phone with a customer, to troubleshoot a problem) that are initiated by the ISP network element, with the MA in the home network as the test endpoint. This is not uncommon. Because MAs can conduct passive measurement collection, initiate active tests or act as active test endpoints, I find the term "Measurement Server" to be very confusing. Is an MA that can initiate and respond to a ping test an MA when it initiates the ping but a Measurement Server when it responds to the ping? Or is something that participates in measurements in an ISP network element always a Measurement Server, no matter whether it initiates or responds to tests? Al Morton did a great job at describing this terminology problem in one of the early lmap threads. He convinced me. 
 
> This could fit the model you've proposed for LMAP "Phase 1" where there is
> Intra-Domain communication from a Measurement Controller to a MA (e.g.:
> to start/stop tests).  At the same time, there exists a third-party (Inter-
> Domain) "Measurement Server" to which the MA will conduct network
> performance tests against.  In this particular instance, as I understand it, the
> 3rd-party Measurement Server would have no "heads-up" (or, as you
> phrased it "be asked permission just before running it") ... at least, via an
> Inter-Domain communication method directly from a "Measurement
> Controller".  That could be one operational model, but it may lead to
> concerns with respect to a (unintended) DoS attack on the Measurement
> Server side of things or, worse, we obtain 'tainted'/suboptimal performance
> results, because the Measurement Server side is over-subscribed (e.g.: BW
> or CPU saturated).
> 
> IMO, it might be useful to think of allowing Inter-Domain communication
> from the "Measurement Controller" to the "Measurement Server" and from
> the "MA" to the "Measurement Server" strictly for the capability to perform
> some type of "Call Admission Control" (CAC) function.  The benefit of
> allowing the Measurement Controller to perform CAC to the Measurement
> Server is, presumably, it could aggregate a whole series of tests it is *about*
> to schedule on MA's to a Measurement Server and say: "Can you handle
> tests that may generate 10 Gbps of traffic in the next N minutes?"  The
> Measurement Server may simply respond: "No" or "No, but I can handle 5
> Gbps".  OTOH, we likely need a similar CAC function on the "MA" to
> Measurement Server so that in the cases of various network and/or server
> failures, the "MA" can dynamically and autonomously adapt on its own to
> those conditions w/out having to be explicitly told to "knock it off" by a
> Measurement Controller.  The latter is likely critical in the case where the
> Measurement Controller is offline (failed) or otherwise not reachable from
> the MA.
> 
> I would agree that, in general, anything beyond a relatively straightforward
> Intra- and Inter-Domain CAC function from MC and MA toward the
> Measurement Server should be out-of-scope for a LMAP "Phase 1".
> 
> > Which isn't to say that such inter-domain interfaces won't or can't or
> shouldn't exist (I fully support their right to exist) -- I'm just suggesting that
> they not be in lmap "phase 1" scope.
> >
> > I think there is a big need for measurement protocols to be run inter-
> domain. Since there seems to be a desire by the owners of some test
> endpoints to require "ask permission first", this needs to be in lmap "phase
> 1" scope: for intra-domain (test endpoints under "the ISP"s control), and
> inter-domain (one end of the test under "the ISP"s control and the other end
> under someone else's control).
> 
> I also think a CAC function would be useful (required?) from MA to MS and
> from MC to MS both in the intra- and inter-domain contexts.
 
If you're suggesting perhaps "reserving" resources ahead of time, for tests that are scheduled on masses of MAs, I think I could agree with the Measurement Controller doing the asking. Most usage scenarios I've heard of involve the Measurement Controller scheduling MAs to do mass testing, and only using start/stop control for specific MA on-demand tests. I would question the desirability of having huge numbers of MAs all start testing at the same time, even if the test endpoint can handle it. There may be a bottleneck somewhere in between that isn't scaled to expect all of the end users to max out their data connections at the same time. And think of what that would do all of the innocent end users who aren't involved in the test but suddenly have their service degraded. There's also the idea that if the end user is actively using their access connection, then the MA is supposed to wait and not test. Even in the on-demand start/stop case, it may be desired for the MA to say "No, I won't run the test unless you force me to" if there is significant traffic on the access link. And if there are masses of MAs that were asked to run at the same time, is the Controller ready to deal with a response from each all at the same time, where many may be saying "No not now"? I really think it's much cleaner to schedule tests and get the Measurement Controller out of the way, and for the MA to ask permission when it's ready. Note that if the Measurement Controller wants to ask on behalf of just one MA, then this could actually be the same CAC mechanism as MA to test endpoint. But I think I can agree to the need for us to consider the mass request for inter-domain mechanism to allow for a Measurement Controller to reserve scheduled resources.

> > There's also a bunch of intra-domain interfaces I'm not interested in seeing
> in "phase 1". Basically, interfaces with the MA (or its physical
> device/element) as a communication endpoint I would like to see in scope.
> All other interfaces I would prefer not to have in scope for lmap "phase 1".
> For example, if or how a Management Server talks directly a Measurement
> Controller, or a Data Collector, I really don't care.
> 

Yes. Or a TR-069 server. Or an SNMP server used to manage cable modems. There is an idea that it may be desirable to manage the MA as an application separate from the device. Therefore, device management is identified as a distinct function from MA management. This does not mean that the MA has to be managed separately from the device. It just allows us to talk about that and consider the implications. It could also allow for managed MAs to reside on unmanaged devices, like a desktop PC. Or for the user to download an MA app onto their smart phone, where the smart phone itself may be managed via OMA-DM by the wireless ISP.
Barbara

---

From: Shane Amante <[email-masked]>
Date: Sun, 14 Apr 2013 21:19:56 -0600

Barbara,

Snipping those parts where I think we're in agreement.  Please see below.

> If the ISP who knows the attributes also manages the Data Collector, then that's intra-domain, and the ISP should be able to figure out how to do that within there OSS network without help from IET…

At a minimum, the party (e.g.: regulatory entity) who are receiving the "service attributes" from several dozen ISPs would want to receive them in a standards-based data model, (e.g.: XML schema), to make processing of the data contained therein more straightforward.  So, for the defined regulatory use case, this should be in scope of "LMAP Phase 1", right?

[--snip--]

> If you're suggesting perhaps "reserving" resources ahead of time, for tests that are scheduled on masses of MAs, I think I could agree with the Measurement Controller doing the asking. Most usage scenarios I've heard of involve the Measurement Controller scheduling MAs to do mass testing, and only using start/stop control for specific MA on-demand tests.

Agreed.

> I would question the desirability of having huge numbers of MAs all start testing at the same time, even if the test endpoint can handle it. There may be a bottleneck somewhere in between that isn't scaled to expect all of the end users to max out their data connections at the same time.

FWIW, I'm not suggesting any such thing.  Rather, consider the scenario where an entire neighborhood loses commercial power and thousands (or, 10s of 1,000's) of the CPE devices power-on at the same time.  We definitely do not want all of those CPE's (and, more specifically the MA's on them) to all re-initiate their tests and cause catastrophic problems in the network.  Hence, we definitely will need a CAC mechanism at the MA's to ensure this doesn't happen.

> And think of what that would do all of the innocent end users who aren't involved in the test but suddenly have their service degraded. There's also the idea that if the end user is actively using their access connection, then the MA is supposed to wait and not test. Even in the on-demand start/stop case, it may be desired for the MA to say "
> No, I won't run the test unless you force me to" if there is significant traffic on the access link. And if there are masses of MAs that were asked to run at the same time, is the Controller ready to deal with a response from each all at the same time, where many may be saying "No not now"? I really think it's much cleaner to schedule tests and get the Measurement Controller out of the way, and for the MA to ask permission when it's ready. Note that if the Measurement Controller wants to ask on behalf of just one MA, then this could actually be the same CAC mechanism as MA to test endpoint. But I think I can agree to the need for us to consider the mass request for inter-domain mechanism to allow for a Measurement Controller to reserve scheduled resources.

OK.  We agree that a CAC mechanism is needed in the MA -> Measurement Server direction both for Intra- and Inter-Domain cases.

Thanks,

-shane

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Mon, 15 Apr 2013 13:20:54 +0000

Further snipping...

> >> However, one addition to this would be that I think it's important to
> >> share 'service attributes' from a so-called Measurement Parameter
> >> Server to (a) a Measurement Collector (directly)
> >
> > If the ISP who knows the attributes also manages the Data Collector, then
> that's intra-domain, and the ISP should be able to figure out how to do that
> within there OSS network without help from IETF. Therefore, I don't see a
> need for the intra-domain interface to be in lmap phase 1 scope. If the ISP
> does not control the Data Collector, then I think this is a much more difficult
> problem to solve, because it involves the ISP knowing that the Data Collector
> controlling entity has the end user's permission to have access to this data.
> Because of the need for permission with all of its associated need to
> authenticate the Data Collector and "know" that the end user has authorized
> that Data Collector (and allow the end user to revoke that authorization,
> change that authorization, etc.), I consider this a much more complex
> problem to solve and would prefer for it not to be in lmap phase 1 scope.
> 
> At a minimum, the party (e.g.: regulatory entity) who are receiving the
> "service attributes" from several dozen ISPs would want to receive them in a
> standards-based data model, (e.g.: XML schema), to make processing of the
> data contained therein more straightforward.  So, for the defined regulatory
> use case, this should be in scope of "LMAP Phase 1", right?

Would the xml representation of service attributes provided to a "Data Collector" be different than the xml representation provided to a MA? That is, MAs may also need to have a very precise understanding of the service attributes being provided to them. Therefore, I saw the creation of an xml representation for service attributes to an MA to be a very important part of that interface. I guess, though, that a Data Collector probably would need additional metadata, so they could associate a specific set of collected data with a specific set of service attributes. If you think that the service attributes to Data Collector would drive additional data elements, then I think it would be reasonable to put that interface "in scope", but only for the xml, and not for the protocol or authentication/authorization/trust modeling. I would also prefer to have just a single set of service attribute xml (sufficient for both interfaces), and not separate xml for service attributes to MA vs. service attributes to Data Collector.
Barbara

---

From: <[email-masked]>
Date: Mon, 15 Apr 2013 14:46:20 +0100

> 
> FWIW, I'm not suggesting any such thing.  Rather, consider the scenario
> where an entire neighborhood loses commercial power and thousands (or,
> 10s of 1,000's) of the CPE devices power-on at the same time.  We
> definitely do not want all of those CPE's (and, more specifically the
> MA's on them) to all re-initiate their tests and cause catastrophic
> problems in the network.  Hence, we definitely will need a CAC
> mechanism at the MA's to ensure this doesn't happen.
> 

I agree we don't want this to happen but I don't see why we need admission control to solve it.

After a power outage a number of things could happen:
- the MA still has its Measurement Schedule. It just re-starts running tests (Measurement Tasks)
- the MA has lost its Measurement Schedule, or it has expired. It simply waits to hear from the Controller with a new one
- the MA has forgotten who its Controller is. The initialisation process needs to run again (it needs to hear from the 'Management Server' about who its controller is and security details ('Management Server' = term from the Broadband forum liaison). This is probably a BBF defined process (at least for ISP-controlled MA). In any case, it's only this (re-)initialisation or re-homing process that needs some kind of load control (and which could be handled by a back-off algorithm rather than CAC)

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 15 Apr 2013 16:04:15 +0200

On Mon, Apr 15, 2013 at 02:46:20PM +0100, [email-masked] wrote:

> - the MA has forgotten who its Controller is. The initialisation process needs to run again (it needs to hear from the 'Management Server' about who its controller is and security details ('Management Server' = term from the Broadband forum liaison). This is probably a BBF defined process (at least for ISP-controlled MA).

Question: Do ISPs using (TV) cable network technology deploy BBF
technology or do they have their own technology to control the cable
modems? My question is whether ISP => BBF technology in the access
network is generally true.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Mon, 15 Apr 2013 14:41:52 +0000

No. Not all ISPs use BBF-defined architectures or technology.
In my experience, the main protocols used by service providers to manage devices are BBF TR-069 (aka CWMP), SNMP, and OMA-DM (for mobile devices).
But my own read on Phil's comment is to interpret it more generically as: This initialization process is probably not something that lmap needs to deal with in Phase 1, because different entities will want to do things different ways, and lmap may not have the operational experience during Phase 1 to recommend one way over another. Which is a sentiment I would agree with.
Barbara

> -----Original Message-----
> From: Juergen Schoenwaelder [mailto:j.schoenwaelder@jacobs-
> university.de]
> Sent: Monday, April 15, 2013 10:04 AM
> To: [email-masked]
> Cc: [email-masked]; STARK, BARBARA H; [email-masked]
> Subject: Re: [lmap] LMAP: situation: Q1-Use Cases
> 
> > - the MA has forgotten who its Controller is. The initialisation process needs
> to run again (it needs to hear from the 'Management Server' about who its
> controller is and security details ('Management Server' = term from the
> Broadband forum liaison). This is probably a BBF defined process (at least for
> ISP-controlled MA).
>

---

From: Shane Amante <[email-masked]>
Date: Mon, 15 Apr 2013 11:01:33 -0600

Phil,

> After a power outage a number of things could happen:

A number of things could happen, but the question is more likely: what do we (LMAP, if/when a WG is formed) want to *recommend should/must happen*?  Presumably, as part of a "Deployment BCP" (???) (or, an LMAP Architecture Doc) we'd want to recommend what is considered a Best Practice at the MA (and, other components of the overall Architecture).

> - the MA has forgotten who its Controller is. The initialisation process needs to run again (it needs to hear from the 'Management Server' about who its controller is and security details ('Managem…

Those are certainly valid suggestions.  However, we would still want to have a (relatively) straightforward CAC mechanism between the MA (CPE device) and Measurement Server?  At the end of the day, we don't want to be in a situation where a bunch of MA's (due to an unfortunate sequence of events), that are running tests autonomously, all end up over-subscribing a single Measurement Server, thus making any measurement results captured during that time to be considered suspect or, worse, invalid.

-shane

---

From: "MORTON JR., ALFRED C (AL)" <[email-masked]>
Date: Mon, 15 Apr 2013 13:32:08 -0400

Hi Shane,
you wrote:
> Those are certainly valid suggestions.  However, we would still want to
> have a (relatively) straightforward CAC mechanism between the MA (CPE
> device) and Measurement Server?  At the end of the day, we don't want to
> be in a situation where a bunch of MA's (due to an unfortunate sequence of
> events), that are running tests autonomously, all end up over-subscribing
> a single Measurement Server, thus making any measurement results captured
> during that time to be considered suspect or, worse, invalid.
> 

The CAC function (between MAs, one is your Server device above) 
could be accomplished by a dedicated testing  protocol, if I understand
the requirement here, using the pre-test control exchanges. A MA can reject
testing session requests when it has no remaining resources. 

This seems to be a critical part of LMAP systems, and certainly needs to be
mentioned in the framework draft, but it may simply end-up as a requirement 
for the active test protocol chosen and for the CAC outcome communicated by the
Complete MA to the Controller/Collector in the appropriate protocol.

hope this helps,
Al 

> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Shane Amante
> Sent: Monday, April 15, 2013 1:02 PM
> Cc: STARK, BARBARA H; [email-masked]
> A number of things could happen, but the question is more likely: what do
> we (LMAP, if/when a WG is formed) want to *recommend should/must happen*?
> Presumably, as part of a "Deployment BCP" (???) (or, an LMAP Architecture
> Doc) we'd want to recommend what is considered a Best Practice at the MA
> (and, other components of the overall Architecture).
> 
> 
> > - the MA has lost its Measurement Schedule, or it has expired. It simply
> waits to hear from the Controller with a new one
> > - the MA has forgotten who its Controller is. The initialisation process
> needs to run again (it needs to hear from the 'Management Server' about
> who its controller is and security details ('Management Server' = term
> from the Broadband forum liaison). This is probably a BBF defined process
> (at least for ISP-controlled MA). In any case, it's only this (re-
> )initialisation or re-homing process that needs some kind of load control
> (and which could be handled by a back-off algorithm rather than CAC)
> 
> Those are certainly valid suggestions.  However, we would still want to
> have a (relatively) straightforward CAC mechanism between the MA (CPE
> device) and Measurement Server?  At the end of the day, we don't want to
> be in a situation where a bunch of MA's (due to an unfortunate sequence of
> events), that are running tests autonomously, all end up over-subscribing
> a single Measurement Server, thus making any measurement results captured
> during that time to be considered suspect or, worse, invalid.
> 
> -shane

---

From: <[email-masked]>
Date: Mon, 15 Apr 2013 18:33:33 +0100

Hi Shane,
I hadn't thought about the power outage example before. So I'm interested whether you need admission control to cope with that problem. So far I don't think you do.

On the wider problem
> to be in a situation where a bunch of MA's (due to an unfortunate
> sequence of events), that are running tests autonomously, all end up
> over-subscribing a single Measurement Server, thus making any
> measurement results captured during that time to be considered suspect
> or, worse, invalid.

Agree in principle there could be a problem of overloading a measurement server (Remote-MA if you like terminology in http://datatracker.ietf.org/doc/draft-eardley-lmap-terminology/)

Alternatives to admission control include
- Measurement Method includes a back-off algorithm (I got no response so I'll stop sending big file)
- Remote-MA uses standard load balancing techniques
- Remote-MA ignores the other MA (doesn't respond to request to send big file)
- Remote-MA's site temporarily blocks 'misbehaving' domain (ie domain initiating too many Measurement Tasks at the moment)
- MA runs a sanity check with Remote-MA first time performs a particular Measurement Method with it (if you like, a one-off admission control)
- operational advice and good practice (eg start individual Measurement Tasks at a slightly randomised time; keep measurement traffic to <x%)
- a Measurement Method that creates a lot of load must include an initial msg from MA that Remote-MA can respond with an 'overloaded' error code [this is effectively admission control, but devolving the problem to the Measurement Method]

So there seem several alternatives to alleviate the risk of overload. 
BTW, I'm not sure what the right answer is - I suspect several techniques should be used in combination.
Not yet convinced either for or against devising a special admission control protocol 

Also admission protocol
- only protects the Remote-MA (not its network); 
- itself loads the Remote-MA - mustn't be limiting factor on Remote-MA's capacity
- is not applicable if the Remote-MA isn't lmap-enabled (say it's a DNS server)

phil

> From: Shane Amante [mailto:[email-masked]]
> Sent: 15 April 2013 18:02
> To: Eardley,PL,Philip,TUB8 R
> Cc: [email-masked]; [email-masked]
> >> FWIW, I'm not suggesting any such thing.  Rather, consider the
> >> scenario where an entire neighborhood loses commercial power and
> >> thousands (or, 10s of 1,000's) of the CPE devices power-on at the
> >> same time.  We definitely do not want all of those CPE's (and, more
> >> specifically the MA's on them) to all re-initiate their tests and
> >> cause catastrophic problems in the network.  Hence, we definitely
> >> will need a CAC mechanism at the MA's to ensure this doesn't happen.
> >>
> >
> A number of things could happen, but the question is more likely: what
> do we (LMAP, if/when a WG is formed) want to *recommend should/must
> happen*?  Presumably, as part of a "Deployment BCP" (???) (or, an LMAP
> Architecture Doc) we'd want to recommend what is considered a Best
> Practice at the MA (and, other components of the overall Architecture).
> 
> 
> > - the MA still has its Measurement Schedule. It just re-starts
> running
> > tests (Measurement Tasks)
> > - the MA has lost its Measurement Schedule, or it has expired. It
> > simply waits to hear from the Controller with a new one
> > - the MA has forgotten who its Controller is. The initialisation
> > process needs to run again (it needs to hear from the 'Management
> > Server' about who its controller is and security details ('Management
> > Server' = term from the Broadband forum liaison). This is probably a
> > BBF defined process (at least for ISP-controlled MA). In any case,
> > it's only this (re-)initialisation or re-homing process that needs
> > some kind of load control (and which could be handled by a back-off
> > algorithm rather than CAC)
> 
> device) and Measurement Server?  At the end of the day, we don't want
> to be in a situation where a bunch of MA's (due to an unfortunate
> sequence of events), that are running tests autonomously, all end up
> over-subscribing a single Measurement Server, thus making any
> measurement results captured during that time to be considered suspect
> or, worse, invalid.
> 
> -shane

---

From: Shane Amante <[email-masked]>
Date: Mon, 15 Apr 2013 12:01:34 -0600

Al,

> testing session requests when it has no remaining resources.

I completely agree.  

Agreed.

-shane

---

From: Shane Amante <[email-masked]>
Date: Mon, 15 Apr 2013 12:29:16 -0600

Hi Phil,

> - is not applicable if the Remote-MA isn't lmap-enabled (say it's a DNS server)

I'd agree with most of your points above.  At this point, I'm not trying to suggest any specific technical solution(s).  Instead, I'm attempting to clarify & suggest requirements for a (hopefully) forthcoming LMAP WG and it's short-term deliverables/scope, (e.g.: "LMAP Phase 1" as we've been calling it):
a)  An "admission control" method between two MA's that communicate directly with each other is REQUIRED, (e.g.: Measurement Agent on a CPE to a Measurement Agent on a 'server');
b)  The aforementioned "admission control" method MUST be the same for Intra-Domain and Inter-Domain MA's.

Note, at this point, I don't have an opinion on whether a "special/new" protocol is required to achieve the above -or- if the solution becomes, as Al states, an aspect of an (existing?) "pre-test protocol".  I believe the latter would certainly be preferred, but for now I'm trying to phrase the above requirements as I did to avoid getting into solutions prematurely.

Is there a particular draft you, Al or Barbara can suggest we should consider incorporating these into, for now?

-shane

---

From: Shane Amante <[email-masked]>
Date: Mon, 15 Apr 2013 12:47:09 -0600

Barbara,

Agreed.

-shane

---

From: "Bugenhagen, Michael K" <[email-masked]>
Date: Mon, 15 Apr 2013 21:06:42 +0000

Shane, Barbara,

 On the architecture including some sort of ISP attribute information --- 
   
     We could assume the tests are run separately and merged with ISP information, or we could assume the probe gathers service environment information and includes it with every test result (does a real-time encapsulation).
Using the post processing merge framework has significant challenges, some of which require local network environment validation regardless.

Examples -
     
1) Matching the test probe results to the ISP service attributes -
	a) Increases the Challenges associated with creating anonymous data 
	b) Doesn't reflect when a customer moves a box from one service to another (it happens)
	** Nothing I know reflects b) other than the probe validating it's where it should be (what's the environment) so this is required....

2)  Service Environment changes
	- When a service environment changes (enforcement of a bandwidth cap - mobile, provisioning speed change, ....)
	    In order to apply this to a specific test result from a ISP service record (multiple ones that is) one must.
		1) Create a timeline per service of it's state
		2) Map that state to each test result

	- My thought here is that post processing information from two different sources drives the solution in the wrong direction by making the anonymity more difficult, and creating expanded post processing requirements.  

    So having the probe collect and encapsulate the environment with each test result seems a no brainer to me.

Just my 2.5 cents...

Mike 

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Shane Amante
Sent: Monday, April 15, 2013 1:47 PM
To: STARK, BARBARA H
Cc: [email-masked]
Subject: Re: [lmap] LMAP: situation: Q1-Use Cases

Barbara,

On Apr 15, 2013, at 7:20 AM, "STARK, BARBARA H" <[email-masked]> wrote:
> Further snipping...
> 
>>>> However, one addition to this would be that I think it's important 
>>>> to share 'service attributes' from a so-called Measurement 
>>>> Parameter Server to (a) a Measurement Collector (directly)
>>> 
>> that's intra-domain, and the ISP should be able to figure out how to 
>> do that within there OSS network without help from IETF. Therefore, I 
>> don't see a need for the intra-domain interface to be in lmap phase 1 
>> scope. If the ISP does not control the Data Collector, then I think 
>> this is a much more difficult problem to solve, because it involves 
>> the ISP knowing that the Data Collector controlling entity has the end user's permission to have access to this data.
>> authenticate the Data Collector and "know" that the end user has 
>> authorized that Data Collector (and allow the end user to revoke that 
>> authorization, change that authorization, etc.), I consider this a 
>> much more complex problem to solve and would prefer for it not to be in lmap phase 1 scope.
>> 
>> At a minimum, the party (e.g.: regulatory entity) who are receiving 
>> the "service attributes" from several dozen ISPs would want to 
>> receive them in a standards-based data model, (e.g.: XML schema), to 
>> make processing of the data contained therein more straightforward.  
>> So, for the defined regulatory use case, this should be in scope of "LMAP Phase 1", right?
> 
> Would the xml representation of service attributes provided to a "Data 
> Collector" be different than the xml representation provided to a MA? 
> That is, MAs may also need to have a very precise understanding of the 
> service attributes being provided to them. Therefore, I saw the 
> creation of an xml representation for service attributes to an MA to 
> be a very important part of that interface. I guess, though, that a 
> Data Collector probably would need additional metadata, so they could 
> associate a specific set of collected data with a specific set of 
> service attributes. If you think that the service attributes to Data 
> Collector would drive additional data elements, then I think it would 
> be reasonable to put that interface "in scope", but only for the xml, 
> and not for the protocol or authentication/authorization/trust 
> modeling. I would also prefer to have just a single set of service 
> attribute xml (sufficient for both interfaces), and not separate xml 
> for service attributes to MA vs
 . 
> service attributes to Data Collector.

Agreed.

-shane

---

From: Shane Amante <[email-masked]>
Date: Mon, 15 Apr 2013 20:21:22 -0600

Michael,

[--snip--]

> 	- My thought here is that post processing information from two different sources drives the solution in the wrong direction by making the anonymity more difficult, and creating expanded post proce…

I agree the above may be true.  However, it could be a bit premature to take post-processing correlation off the table.  Specifically, we need to strive to keep the MA function(s) that are shipped in embedded devices, (CPE modem/routers, WiFi AP's, etc.), as "simple as possible, but no simpler" and, also, we know that SW on those devices may not be upgraded "often" ... then, that CPE-based MA could become a "bottleneck" in the architecture 

For example, if <something> packages 'service attributes' in XML, then ships them to the MA ... the question then becomes: will the MA need to "understand" the service attributes encapsulated in that XML before repackaging them (again) to upload them to the Measurement Collector (alongside the network performance measurement results)?  

So long as the MA can treat the service attributes it receives from <something> as "opaque", before shipping it to the Measurement Collector, then I would definitely agree with you above.

We shall see.

-shane

---

From: <[email-masked]>
Date: Tue, 16 Apr 2013 09:18:39 +0100

Making sure that the probe is on the line we expect is certainly a challenging problem, but one that seems to face us regardless of whether the line/subscriber information is being given to the MA or the Collector. In both cases the network can check the IP address used by the MA. It would be nice if the MA or any other home device were able to get some broadband identifier for its line but that isn't really an LMAP specific capability.

I would certainly support a simple way of the MA checking the line identity. What I don't think it needs to do is retreive further line/subscriber information. It seems in this case we're simply pushing information up to the MA only to collect it back again. Do we want to do this for every possible parameter that we are going to use for correlation in the data analysis? If we're not going to do it with all parameters, why do it for a few? - we would need to add the extra ones in after result collection in that case. Adding 20 parameters to 5 columns of test results is also a serious reporting overhead.

There are two other concerns with the MA approach. Firstly we would need very timely information from the various network OSS providing the parameters. Doing the splicing after collection removes any tight timing requirements for the availability of this data (although we do obviously need good timestamps). Having the MA check these parameters before every test is also a big overhead.

Secondly I worry about security (including both privacy and business confidentialility). Instead of managing an interface with a handful of Collectors, we need to manage access control for millions of MAs to make sure they can only get their own information. Also just because an MA is on a specific broadband line, what information should it really be able to get? Do we need to then have access policies on the MA to control which of this data is then reported to different collectors?

Just trying to paint the other side to the story. Enhancing the data after collection is not trivial. However, I think it is the better option than having the MA do it.

Trevor.

>Bugenhagen, Michael K
>Sent: 15 April 2013 22:07
>To: 'Shane Amante'; STARK, BARBARA H
>Cc: [email-masked]
>     We could assume the tests are run separately and merged with ISP
>information, or we could assume the probe gathers service environment
>information and includes it with every test result (does a real-time
>encapsulation).
>Using the post processing merge framework has significant challenges, some
>of which require local network environment validation regardless.
>
>Examples -
>
>	** Nothing I know reflects b) other than the probe validating it's
>where it should be (what's the environment) so this is required....
>
>
>	- When a service environment changes (enforcement of a bandwidth
>cap - mobile, provisioning speed change, ....)
>	- My thought here is that post processing information from two
>different sources drives the solution in the wrong direction by making the
>anonymity more difficult, and creating expanded post processing
>requirements.
>
>>> the ISP knowing that the Data Collector controlling entity has the end
>user's permission to have access to this data.

---

From: <[email-masked]>
Date: Tue, 16 Apr 2013 10:50:31 +0100

> Sent: 15 April 2013 19:29
> > I hadn't thought about the power outage example before. So I'm
> interested whether you need admission control to cope with that
> problem. So far I don't think you do.
> >
> > On the wider problem
> >> measurement results captured during that time to be considered
> >> suspect or, worse, invalid.
> >
> > Agree in principle there could be a problem of overloading a
> > measurement server (Remote-MA if you like terminology in
> > http://datatracker.ietf.org/doc/draft-eardley-lmap-terminology/)
> >
> >
> > - Remote-MA's site temporarily blocks 'misbehaving' domain (ie domain
> > initiating too many Measurement Tasks at the moment)
> > - MA runs a sanity check with Remote-MA first time performs a
> > particular Measurement Method with it (if you like, a one-off
> > admission control)
> > - operational advice and good practice (eg start individual
> > Measurement Tasks at a slightly randomised time; keep measurement
> > traffic to <x%)
> > - a Measurement Method that creates a lot of load must include an
> > initial msg from MA that Remote-MA can respond with an 'overloaded'
> > error code [this is effectively admission control, but devolving the
> > problem to the Measurement Method]
> >
> > BTW, I'm not sure what the right answer is - I suspect several
> techniques should be used in combination.
> I'd agree with most of your points above.  At this point, I'm not
> trying to suggest any specific technical solution(s).  Instead, I'm
> attempting to clarify & suggest requirements for a (hopefully)
> forthcoming LMAP WG and it's short-term deliverables/scope, (e.g.:
> "LMAP Phase 1" as we've been calling it):
> a)  An "admission control" method between two MA's that communicate
> directly with each other is REQUIRED, (e.g.: Measurement Agent on a CPE
> to a Measurement Agent on a 'server');
> Note, at this point, I don't have an opinion on whether a "special/new"
> protocol is required to achieve the above -or- if the solution becomes,
> as Al states, an aspect of an (existing?) "pre-test protocol".  I
> believe the latter would certainly be preferred, but for now I'm trying
> to phrase the above requirements as I did to avoid getting into
> solutions prematurely.
> 
> Is there a particular draft you, Al or Barbara can suggest we should
> consider incorporating these into, for now?
> 
> -shane

Where the IPPM protocol includes a pre-test that's great (such as OWAMP & TWAMP)
Where it doesn't, I don't see "admission control between the two MAs" as a MUST. I think that can be left to the test and the MA (on the 'server') to decide what to do amongst the many possibilities
phil

> >> A number of things could happen, but the question is more likely:
> >> what do we (LMAP, if/when a WG is formed) want to *recommend
> >> should/must happen*?  Presumably, as part of a "Deployment BCP"
> (???)
> >> (or, an LMAP Architecture Doc) we'd want to recommend what is
> >> considered a Best Practice at the MA (and, other components of the
> overall Architecture).
> >>
> >>
> >>> Server' about who its controller is and security details
> >>> ('Management Server' = term from the Broadband forum liaison). This
> >>> is probably a BBF defined process (at least for ISP-controlled MA).
> >>> In any case, it's only this (re-)initialisation or re-homing
> process
> >>> that needs some kind of load control (and which could be handled by
> >>> a back-off algorithm rather than CAC)
> >>
> >> Those are certainly valid suggestions.  However, we would still want
> >> to have a (relatively) straightforward CAC mechanism between the MA
> >> (CPE
> >> measurement results captured during that time to be considered
> >> suspect or, worse, invalid.
> >>
> >> -shane
> >
>

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Tue, 16 Apr 2013 12:29:43 +0000

Trying to focus on what is in scope and not in scope for lmap phase 1...

I thought Shane and I had sort of agreed that "in scope" would include:
1. Identifying service attributes and creating an extensible xml model of these attributes. This would consider the service attributes that would be needed if they were provided to an MA, inter-domain (from the access provider who has them to an authorized 3rd party who is collecting the data), and intra-domain (where the access provider has the service attributes and is doing data collection, so standard xml naming is really needed more for service attributes that will be included with collected data ultimately provided to others for analysis). I thought we had agreed that there was no need for a particular service attribute to have a different xml "name" or representation, depending on how it was delivered.
2. Mechanisms for providing service attributes directly to an MA (primarily targeted at the case where the MA is not controlled by the access provider, and does not provide its data to the access provider).

Just because something is "in scope" doesn't mean we necessarily end up managing to create a solution for it. But it does mean we discuss it and see what we think we can come up with.

"Out of scope" are mechanisms/protocols for intra-domain and inter-domain transfer of service attributes.

So, Mike and Trevor, are you disagreeing with this "in scope" and "out of scope" characterization? Specifically...

Mike, do you object to trying to come up with a holistic service attribute xml model, that considers the type of data needed by all of the various transfer usages?

Trevor, do you object to lmap phase 1 consideration of potential mechanisms for supplying service attributes to MAs that do not send data to the access provider? I absolutely agree that sending service attributes to a system that also gets the collected data is a great way to go, and how I would recommend an access provider do things, where the access provider collects the data and has the service attributes. But I also have no desire to ask the IETF to recommend mechanisms for getting that data from one internal access provider IT system to another. Are you wanting IETF to define a mechanism for intra-domain transfer of service attributes? I'm also not keen on asking IETF to try to solve the inter-domain transfer problem in lmap phase 1, because I think it involves 3-way trust relationships, which can be difficult to manage. To be sufficiently robust and flexible, the end user must be able to grant and revoke authority for a 3rd party to receive that end user's service attributes. The access provider has to trust the end user is the end user, that the end user has provided that authorization, and that the 3rd party is indeed the party that the end user has authorized. Somebody has to manage the trust relationship. Different regulators might want different ways to manage the trust relationship. It's complex.  Are you wanting to put mechanisms for inter-domain service attribute transfer in lmap phase 1?

Barbara

---

From: <[email-masked]>
Date: Tue, 16 Apr 2013 13:46:10 +0100

>From: STARK, BARBARA H [mailto:[email-masked]]
>Sent: 16 April 2013 13:30
>To: Burbridge,T,Trevor,TUB8 R; [email-masked];
>[email-masked]
>Cc: [email-masked]
>Subject: RE: [lmap] LMAP: situation: Q1-Use Cases
[snip]
>Trevor, do you object to lmap phase 1 consideration of potential
>mechanisms for supplying service attributes to MAs that do not send data to
>the access provider?

As you suggest below, the correct question from a requirements point-of-view is whether we include inter-domain service attribute transfer in phase 1. I certainly wouldn't push for it.

For intra-domain, do I think the correct implementation is via the MA - no. Would this change if inter-domain was in scope? Not convinced.

>I absolutely agree that sending service attributes to a
>system that also gets the collected data is a great way to go, and how I
>would recommend an access provider do things, where the access provider
>collects the data and has the service attributes. But I also have no desire to
>ask the IETF to recommend mechanisms for getting that data from one
>internal access provider IT system to another. Are you wanting IETF to define
>a mechanism for intra-domain transfer of service attributes? I'm also not
>keen on asking IETF to try to solve the inter-domain transfer problem in
>lmap phase 1, because I think it involves 3-way trust relationships, which can
>be difficult to manage. To be sufficiently robust and flexible, the end user
>must be able to grant and revoke authority for a 3rd party to receive that
>end user's service attributes. The access provider has to trust the end user is
>the end user, that the end user has provided that authorization, and that the
>3rd party is indeed the party that the end user has authorized. Somebody
>has to manage the trust relationship. Different regulators might want
>different ways to manage the trust relationship. It's complex.  Are you
>wanting to put mechanisms for inter-domain service attribute transfer in
>lmap phase 1?

No.

Trevor.

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Tue, 16 Apr 2013 13:32:26 +0000

> >mechanisms for supplying service attributes to MAs that do not send
> >data to the access provider?
> 
> As you suggest below, the correct question from a requirements point-of-
> view is whether we include inter-domain service attribute transfer in phase
> 1. I certainly wouldn't push for it.
> 
> For intra-domain, do I think the correct implementation is via the MA - no.
> Would this change if inter-domain was in scope? Not convinced.

Hi Trevor,
OK. I hear you, but I disagree that "service attribute to MA" should be "out of scope". I would like for some sort of solution to exist where a 3rd party can manage the MA and collect the MA's data, and have the MA in that case be able to get service attributes. This doesn't mean that I want there to be a BCP recommending this method. It doesn't mean it should preclude architectures where the access provider does data collection. It does mean that I don't think there is a "one size fits all" way to architect measurement collection that will be "the" right way for every provider and every government in the world. It means I want the mechanism to exist and be defined, and be available to whoever wants to implement it.

I also think that providing an MA with service attributes is useful in the case where the end user just wants to run his/her own tests and not share the data with anybody. End users should be able to have access to their service attributes.
Barbara

---

From: <[email-masked]>
Date: Tue, 16 Apr 2013 14:42:29 +0100

>Sent: 16 April 2013 14:32
>To: Burbridge,T,Trevor,TUB8 R
>Cc: [email-masked]
>> As you suggest below, the correct question from a requirements
>> point-of- view is whether we include inter-domain service attribute
>> transfer in phase 1. I certainly wouldn't push for it.
>>
>OK. I hear you, but I disagree that "service attribute to MA" should be "out
>of scope". I would like for some sort of solution to exist where a 3rd party
>can manage the MA and collect the MA's data, and have the MA in that case
>be able to get service attributes. This doesn't mean that I want there to be a
>BCP recommending this method. It doesn't mean it should preclude
>architectures where the access provider does data collection. It does mean
>that I don't think there is a "one size fits all" way to architect measurement
>collection that will be "the" right way for every provider and every
>government in the world. It means I want the mechanism to exist and be
>defined, and be available to whoever wants to implement it.
>
>I also think that providing an MA with service attributes is useful in the case
>where the end user just wants to run his/her own tests and not share the
>data with anybody. End users should be able to have access to their service
>attributes.
>Barbara

Not sure we're disagreeing.

I'm not precluding the MA reporting stuff. The report information model may contain context information. Certainly any individual test result is free to report whatever measured or local parameters it wants to.

On getting service attributes to the MA, the only question is whether there is a single standard way of doing that in LMAP. I'm not saying there shouldn't be ways of doing it. Some ISPs might push this info to the home gateway. Others might provide it on the customer web portal.

Trevor.

---

From: Shane Amante <[email-masked]>
Date: Tue, 16 Apr 2013 07:50:02 -0600

Trevor, Barbara,

> As you suggest below, the correct question from a requirements point-of-view is whether we include inter-domain service attribute transfer in phase 1. I certainly wouldn't push for it.

I think we need to be careful to separate out two aspects of the service attributes discussion:
a)  Representation of service attributes inside an XML schema.  IMO, it seems this is the right thing to do regardless of whether this is "intra-" or "inter-"domain, since everyone who is going to programmatically interpret measurement results is going to want a standardized representation of "service attributes" to make it straightforward to parse them.
b)  Transfer of the aforementioned "service attribute XML schema" from one host to another host and/or from one party to another party.  For this I would ask the question of: why wouldn't we just use HTTP or HTTPS? In doing so, it would allow LMAP to leverage a variety of existing, already-specified & implemented authentication mechanisms for HTTP/HTTPS.  And, for that matter, it could be up to ISPs to decide what they wanted to use, e.g.: simple username/password and/or SSL certificates.  At that point, it's up to a bi-lateral negotiation between Party A to Party B, (in the context of inter-domain), at the time they stand up the capability to 'download' (or, upload) the XML service attribute schema. 

Also, please note that I said: "bi-lateral negotiation" above.  IOW, two parties are dealing directly with each other, thus they should be able to negotiate amongst themselves (out-of-band, of course) what HTTP/HTTPS mechanism(s) are required to "safely" xfer the XML service attribute schema, etc.

-shane

---

From: "Bugenhagen, Michael K" <[email-masked]>
Date: Tue, 16 Apr 2013 13:58:57 +0000

Barbara,

   Short answer - YES on Service attributes XML model, otherwise we can't have true validated results with tests that compare the 
"throughput" to "sold speed" - Aka a non-qualifiable result if we don't know what the actual "sold speed" was, which of course is a service attribute.
This is part of the greater scope of the project, and is really not a "SHOULD" - it's more of a "MUST" requirement.

------------------------
Long answer (they why's examined) - and a hitch to more succinctly defining Inter-domain {look at 1. b) under Inter-domain classes}

Ok - for my phase 1 in/out preferences & reasoning, I'll have to answer those multi-part questions one by one

1. a) Identifying service attributes and creating an extensible xml model of these attributes This would consider the service attributes that would be needed if they were provided to an MA.

		Yes - Definitely should be part of the Scope for Phase 1, and should be in a standard XML format... 
			I'll extend this to say that the test results should also be in a standard format as well, and should include the Service attributes
			That I've somewhat coined the term "test environment" (looking for a more suitable nomenclature but better term has been ID'd yet)				
			Logic behind this (test results including the service attributes in place during the test) -  
			Data analysis after the fact become increasingly difficult to conduct accurately without the environmental data of the test 					environment.  The last thing anyone needs is to complete an analysis not understanding the environment and coming up with incorrect 			conclusions due to missing data.  That itself would discredit the methodology and framework itself.

1. b) inter-domain (from the access provider who has them to an authorized 3rd party who is collecting the data), and intra-domain (where the access provider has the service attributes and is doing data collection, so standard xml naming is really needed more for service attributes that will be included with collected data ultimately provided to others for analysis). I thought we had agreed that there was no need for a particular service attribute to have a different xml "name" or representation, depending on how it was delivered.

	I) I believe knowing the test environment is required to provide validated test results regardless of who's testing.
		- Which aligns with classifying a validated or qualified vs. non-validated/qualified test result 
			(either you know the service attributes or didn't)

	II) Inter-Domain testing classes -
			This could be multiple things - we need to define it.
				a)  MA, Controller, and Collector in one domain, but a second test MA is outside (no or partial results shared) - IN SCOPE
				b)  MA, Controller, and 3rd party Collector - OUT OF SCOPE
				c)  3rd party MA, 3rd party Controller, and 3rd party Collector - IN SCOPE (supports customer conducting testing)

2.) XML model for service attributes - Definitely in scope to support the a & c models and create a validated / Qualified test

Best Regards,
Mike
-------------------------------------------------------------------------------------------------------------------------

-----Original Message-----
From: STARK, BARBARA H [mailto:[email-masked]] 
Sent: Tuesday, April 16, 2013 7:30 AM
To: [email-masked]; Bugenhagen, Michael K; [email-masked]
Cc: [email-masked]
Subject: RE: [lmap] LMAP: situation: Q1-Use Cases

2. - I

Barbara

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 16 Apr 2013 16:00:05 +0200

> a)  Representation of service attributes inside an XML schema.  IMO, it seems this is the right thing to do regardless of whether this is "intra-" or "inter-"domain, since everyone who is going to…

How realistic is it that we can define a standard data model for
service attributes that matches a significant portion of the products
out there in the wild?

> b)  Transfer of the aforementioned "service attribute XML schema" from one host to another host and/or from one party to another party.  For this I would ask the question of: why wouldn't we just u…

Securing the protocol is not the problem, it is key management that is
usually the problem. Do we expect an MA to have credentials that allow
it to fetch service attributes?  How are those credentials managed
during the lifecycle of the MA? It may be much harder to manage the
necessary credentials on thousands of MAs compared to a solution where
a few data collector or a data analysis function fetch service
attributes. (Of course, this still requires proper authorization but
the key management problem seems to be at a very different order of
scale - at least if we can assume #controller << #MAs or 
#data-analyzer << #MAs.)

/js

---

From: "Bugenhagen, Michael K" <[email-masked]>
Date: Tue, 16 Apr 2013 14:13:06 +0000

Trevor -

  A few thoughts on your points --- 

You first statement mentioned -
There are two other concerns with the MA approach. Firstly we would need very timely information from the various network OSS providing the parameters. Doing the splicing after collection removes any tight timing requirements for the availability of this data (although we do obviously need good timestamps). Having the MA check these parameters before every test is also a big overhead.

   -- Your statement assumes a methodology given it's considered "big overhead,"  -  I think the approach used here is key, and there may be ways of 	making this easy vs. difficult.  I would agree that using today's constructs this is difficult, however we collectively have the ability to create 	a new methodology to accomplish making the SA (service attributes) obtainable, and low overhead.   

	- I would also like to point out that there really only a small amount of "provisionable" service attributes, but a large amount of different types 
	   	Of performance that can be tracked.   I don't think it's appropriate to ask a ISP to track every performance aspect of every line, that is 		more intuitive of a business service vs. a Internet Service, so the subset of "SET by the ISP" attributes are really the ones we're 			interested in.

Your second statement -

	-- My thoughts here are that we have 3 distinct events occurring in the framework (Assuming we ignore the MA getting the service attributes)
		1) Some controller programs a MA to run tests (BOTs work a lot like this so high security is required)
		2) MA to MA testing occurs - and a "may I run this test" like control protocol is required, however high security (not so much)
		3) The MA send test results to the collector (seems light security would be ok here)

	   For #1 - Given all currently placed elements have some sort of provisioning security ... logic would say to use that
		#2 & #3 are much lighter security issues vs. #1.

Cheers,
Mike
----------------------

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: Tuesday, April 16, 2013 3:19 AM
To: Bugenhagen, Michael K; [email-masked]; [email-masked]
Cc: [email-masked]
Subject: RE: [lmap] LMAP: situation: Q1-Use Cases

Trevor.

>Using the post processing merge framework has significant challenges, 
>some of which require local network environment validation regardless.
>
>Examples -
>
>	    In order to apply this to a specific test result from a ISP 
>service record (multiple ones that is) one must.
>different sources drives the solution in the wrong direction by making 
>the anonymity more difficult, and creating expanded post processing 
>requirements.
>
>    So having the probe collect and encapsulate the environment with 
>each test result seems a no brainer to me.
>
>Just my 2.5 cents...
>
>Mike
>
>
>
>
>
>
>
>>> do that within there OSS network without help from IETF. Therefore, 
>>> I don't see a need for the intra-domain interface to be in lmap 
>>> phase 1 scope. If the ISP does not control the Data Collector, then 
>>> I think this is a much more difficult problem to solve, because it 
>>> involves the ISP knowing that the Data Collector controlling entity 
>>> has the end
>>> Because of the need for permission with all of its associated need 
>>> to authenticate the Data Collector and "know" that the end user has 
>>> authorized that Data Collector (and allow the end user to revoke 
>>> that authorization, change that authorization, etc.), I consider 
>>> this a much more complex problem to solve and would prefer for it 
>>> not to be in
>lmap phase 1 scope.
>>>
>>> So, for the defined regulatory use case, this should be in scope of 
>>> "LMAP
>Phase 1", right?
>>
>> Would the xml representation of service attributes provided to a 
>> "Data Collector" be different than the xml representation provided to a MA?
>> That is, MAs may also need to have a very precise understanding of 
>> the service attributes being provided to them. Therefore, I saw the

---

From: Shane Amante <[email-masked]>
Date: Tue, 16 Apr 2013 08:15:58 -0600

> service attributes that matches a significant portion of the products

I'm pretty confident it can be done, assuming we confine ourselves to representing the most common of service attributes:
- Upload speed, (in Kbps)
- Download speed, (in Kbps)
- Service package 'name', (i.e.: product/service-tier as sold)
- ISP name
- Access technology: DSL, DOCSIS, FTTH, WiFi, Wireless/Cellular, Other?
- Customer or Line ID <-- note, this would have to be some pseudo-random identifier that associates this particular service to a particular broadband "line", but doesn't reveal anything else about the customer to ensure privacy/anonymity of the customer.

> scale - at least if we can assume #controller << #MAs or 

+1.  Great points.  Yet another reason to consider xfer'ing of XML service attribute schema on a "Network Parameter Server" on ISP A to, for example, Data Collector owned/operated by Entity B, combined with post-correlation of customer/line ID in the XML service attribute schema to measurement performance results.

-shane

---

From: "Bugenhagen, Michael K" <[email-masked]>
Date: Tue, 16 Apr 2013 14:17:45 +0000

Juergen -

     Part of the solution to securing the Service attributes is what part of the network you have to be attached to in order to obtain it.

IMO that should be on the Customer LAN, if the customer lets you attach to their LAN then at least part of the problem goes away.

Regards,
Mike

-----Original Message-----
From: Juergen Schoenwaelder [mailto:[email-masked]] 
Sent: Tuesday, April 16, 2013 9:00 AM
To: Shane Amante
Cc: [email-masked]; [email-masked]; [email-masked]; Bugenhagen, Michael K
Subject: Re: [lmap] LMAP: situation: Q1-Use Cases

> a)  Representation of service attributes inside an XML schema.  IMO, it seems this is the right thing to do regardless of whether this is "intra-" or "inter-"domain, since everyone who is going to…

How realistic is it that we can define a standard data model for service attributes that matches a significant portion of the products out there in the wild?

> b)  Transfer of the aforementioned "service attribute XML schema" from one host to another host and/or from one party to another party.  For this I would ask the question of: why wouldn't we just u…

Securing the protocol is not the problem, it is key management that is usually the problem. Do we expect an MA to have credentials that allow it to fetch service attributes?  How are those credentials managed during the lifecycle of the MA? It may be much harder to manage the necessary credentials on thousands of MAs compared to a solution where a few data collector or a data analysis function fetch service attributes. (Of course, this still requires proper authorization but the key management problem seems to be at a very different order of scale - at least if we can assume #controller << #MAs or #data-analyzer << #MAs.)

/js

---

From: "Bugenhagen, Michael K" <[email-masked]>
Date: Tue, 16 Apr 2013 14:28:18 +0000

Shane - Nice start.. I tweaked changed, and added some

  This is the (Service attributes from the ISP to the MA context)..
   I think we'll need the MA to store more information about the Device it's in / or on in it's MIB itself...

Suggested Tweaks -
 
  I split line rate and User information rate..., added PON, and tweaked Product name, and Unique ID (removed customer - we're identifying a access service link, not a person)... 

Service Attributes - 
- IP upload speed, (in Kbps)     - aka User information rate up
- IP download speed, (in Kbps)   - aka User information rate down
- Link speed upload, (in Kbps)
- Link speed download, (in Kbps)
- ISP Product name, (i.e.: product/service-tier as sold)
- ISP name
- Access technology: PON, DSL, DOCSIS, FTTH, WiFi, Wireless/Cellular, Other?
- Unique Line ID <-- note, this would have to be some pseudo-random identifier that associates this particular service to a particular broadband "line", but doesn't reveal anything else about the customer to ensure privacy/anonymity of the customer.

Regards,
Mike

---------------------------------------

-----Original Message-----
From: Shane Amante [mailto:[email-masked]] 
Sent: Tuesday, April 16, 2013 9:16 AM
To: Juergen Schoenwaelder
Cc: [email-masked]; [email-masked]; [email-masked]; Bugenhagen, Michael K
Subject: Re: [lmap] LMAP: situation: Q1-Use Cases

-shane

---

From: <[email-masked]>
Date: Tue, 16 Apr 2013 15:30:06 +0100

> 		2) Map that state to each test result

The bandwidth cap is interesting.
For the device that enforces (or the one that decides about) a bandwidth cap - does anyone know what kind of mgt interface it has?
Is the info available? Is it time stamped?

---

From: Shane Amante <[email-masked]>
Date: Tue, 16 Apr 2013 08:42:41 -0600

Mike,

>   I think we'll need the MA to store more information about the Device it's in / or on in it's MIB itself...

That could be useful.

>  I split line rate and User information rate..., added PON, and tweaked Product name, and Unique ID (removed customer - we're identifying a access service link, not a person)... 

Can you clarify why you think splitting "line rate" and "user information rate" is important?  For example, are you trying to distinguish between throughput with and without packet/framing overhead, (e.g.: PPPoA/PPPoE overhead?)  I was thinking and hoping to avoid that, because when we're talking about _Internet_ network performance, we're talking about IP throughput.  Plus, IP is thee common denominator across every access network technology.

> - Link speed download, (in Kbps)

Not sure I understand/agree with the above, yet, but I'm hoping you can clarify.

> - Unique Line ID <-- note, this would have to be some pseudo-random identifier that associates this particular service to a particular broadband "line", but doesn't reveal anything else about the c…

I like the above modifications.

Thanks!

-shane

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Tue, 16 Apr 2013 14:51:35 +0000

> b)  Transfer of the aforementioned "service attribute XML schema" from one
> host to another host and/or from one party to another party.  For this I
> would ask the question of: why wouldn't we just use HTTP or HTTPS? In
> doing so, it would allow LMAP to leverage a variety of existing, already-
> specified & implemented authentication mechanisms for HTTP/HTTPS.  And,
> for that matter, it could be up to ISPs to decide what they wanted to use,
> e.g.: simple username/password and/or SSL certificates.  At that point, it's up
> to a bi-lateral negotiation between Party A to Party B, (in the context of
> inter-domain), at the time they stand up the capability to 'download' (or,
> upload) the XML service attribute schema.
> 
> Also, please note that I said: "bi-lateral negotiation" above.  IOW, two parties
> are dealing directly with each other, thus they should be able to negotiate
> amongst themselves (out-of-band, of course) what HTTP/HTTPS
> mechanism(s) are required to "safely" xfer the XML service attribute
> schema, etc.

I was hoping to consider the possibility of service discovery mechanisms to see if "service attribute" data existed somewhere in the home.
That is, might the process of figuring out where to direct the MA to find the info be made easier if the access provider CE router (if there is one) uses something like DNS-SD and/or mDNS to make it easy to find the xml page on the CE router that can provide the info.
BTW, I'm also interested in the MA being able to learn whether the WAN link is currently in use, so I would expect this to be part of what could be supplied to the MA. Even an off-the-shelf CE router might be able to provide that. But only if the MA had an easy way to find the info and the info were provided using standard nomenclature.

I'm not convinced that every single piece of useful data that the CE router might provide to other devices inside the home network requires top notch security credentials to access. I'm not convinced that informing devices inside the home network about some aspects of the service package is necessarily a violation of privacy (where views of privacy vary from country to country). After all, these in-home devices are already capable of collecting all sorts of passive measurements, discovering topology, seeing all the services being advertised in the home (UPnP and Bonjour), running performance tests to see what bandwidth is available to them, etc. I do agree that bi-lateral negotiation of credentials could be a possibility. It may be that the end user is instructed to put their email address and login (if these are from the access provider -- which in many cases they are) into the MA. There's a wide range of possibilities. I'm not interested in figuring out how to exchange credentials, but I think it's possible to do without too much trouble, and so it shouldn't be a reason for refusing to allow discussion of the "info to MA" interface.

Anyway, I thought there were some interesting intricacies around mechanisms for providing ancillary data (including service attributes) to MAs that at least were worthy of discussion. I'm getting the sense that others would prefer for such discussion to be out of scope. If that's the consensus, then I'll be happy to accept it.
Barbara

---

From: <[email-masked]>
Date: Tue, 16 Apr 2013 17:08:19 +0100

I think that lmap phase 1 should make the assumption that the measurement system is under the control of a single organisation (the MA, controller and collector)

- the measurement peer may be under another org's control (eg DNS server)
- an MA can run tests if it wants and report results to whoever it wants (cf. ookla speed tests). Lmap doesn't try to stop this happening (it's up to customer)

I think this assumption simplifies the security and policy considerations a lot

phil

> 				a)  MA, Controller, and Collector in one domain,
> but a second test MA is outside (no or partial results shared) - IN
> SCOPE
> 				c)  3rd party MA, 3rd party Controller, and 3rd
> party Collector - IN SCOPE (supports customer conducting testing)
> 
...
> Trying to focus on what is in scope and not in scope for lmap phase
...
> So, Mike and Trevor, are you disagreeing with this "in scope" and "out
> of scope" characterization? Specifically...
> 
> Mike, do you object to trying to come up with a holistic service
> attribute xml model, that considers the type of data needed by all of
> the various transfer usages?
> 
> data to the access provider? I absolutely agree that sending service
> attributes to a system that also gets the collected data is a great way
> to go, and how I would recommend an access provider do things, where
> the access provider collects the data and has the service attributes.
> But I also have no desire to ask the IETF to recommend mechanisms for
> getting that data from one internal access provider IT system to
> another. Are you wanting IETF to define a mechanism for intra-domain
> transfer of service attributes? I'm also not keen on asking IETF to try
> to solve the inter-domain transfer problem in lmap phase 1, because I
> think it involves 3-way trust relationships, which can be difficult to
> manage. To be sufficiently robust and flexible, the end user must be
> able to grant and revoke authority for a 3rd party to receive that end
> user's service attri  butes. The access provider has to trust the end
> user is the end user, that the end user has provided that
> authorization, and that the 3rd party is indeed the party that the end
> user has authorized. Somebody has to manage the trust relationship.
> Different regulators might want different ways to manage the trust
> relationship. It's complex.  Are you wanting to put mechanisms for
> inter-domain service attribute transfer in lmap phase 1?
> 
> Barbara
>

---

From: "Bugenhagen, Michael K" <[email-masked]>
Date: Tue, 16 Apr 2013 17:05:44 +0000

Phil,

   I concur, I like yours it's simpler and cleaner...

--M

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: Tuesday, April 16, 2013 11:08 AM
To: Bugenhagen, Michael K; [email-masked]; [email-masked]; [email-masked]
Cc: [email-masked]
Subject: Re: [lmap] LMAP: situation: Q1-Use Cases

phil

> 				a)  MA, Controller, and Collector in one domain, but a second test 
> MA is outside (no or partial results shared) - IN SCOPE
> 				c)  3rd party MA, 3rd party Controller, and 3rd party Collector - 
> IN SCOPE (supports customer conducting testing)
> 
...
...
> attributes to a system that also gets the collected data is a great 
> way to go, and how I would recommend an access provider do things, 
> where the access provider collects the data and has the service attributes.
> transfer of service attributes? I'm also not keen on asking IETF to 
> try to solve the inter-domain transfer problem in lmap phase 1, 
> because I think it involves 3-way trust relationships, which can be 
> difficult to manage. To be sufficiently robust and flexible, the end 
> user must be able to grant and revoke authority for a 3rd party to 
> receive that end user's service attri  butes. The access provider has 
> to trust the end user is the end user, that the end user has provided 
> that authorization, and that the 3rd party is indeed the party that 
> the end user has authorized. Somebody has to manage the trust relationship.

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 16 Apr 2013 21:37:33 +0200

> - Unique Line ID <-- note, this would have to be some pseudo-random identifier that associates this particular service to a particular broadband "line", but doesn't reveal anything else about the c…

I have heard about service contracts where there are limits like a
certain amount of data (with varying definitions of 'data') per month
(or some other time frame) and traffic shaping by the provider is
explicitely allowed either if you reach the data limit or if you
inject traffic that is for some reason disliked by the ISP (e.g.  they
reserve the right to throttle p2p traffic). If we consider cellular
networks, things can get even more interesting.

The devil really is in the details. What does Kbps mean? What does
'amount of data' mean? What does throttling mean? How do you report
whether the line is being throttled due to the fact that the traffic
crossed some threshold or carries stuff that somehow is disliked by
the ISP (see some of the Glasnost results)?

/js

---

From: "Weil, Jason" <[email-masked]>
Date: Tue, 16 Apr 2013 15:41:19 -0400

[Snip]
"...On getting service attributes to the MA, the only question is whether
there is a single standard way of doing that in LMAP. I'm not saying there
shouldn't be ways of doing it. Some ISPs might push this info to the home
gateway. Others might provide it on the customer web portal."

A pull model may be useful as well where the MA requests service attribute
information from an service attribute service. The service offering that
information could offer varying level of information detail based on the
trust relationship with the MA.

Jason

On 4/16/13 9:42 AM, "[email-masked]" <[email-masked]>
wrote:

>I'm not precluding the MA reporting stuff. The report information model
>may contain context information. Certainly any individual test result is
>free to report whatever measured or local parameters it wants to.
>
>On getting service attributes to the MA, the only question is whether
>there is a single standard way of doing that in LMAP. I'm not saying
>there shouldn't be ways of doing it. Some ISPs might push this info to
>the home gateway. Others might provide it on the customer web portal.
>
>Trevor.

This E-mail and any of its attachments may contain Time Warner Cable proprietary information, which is privileged, confidential, or subject to copyright belonging to Time Warner Cable. This E-mail is intended solely for the use of the individual or entity to which it is addressed. If you are not the intended recipient of this E-mail, you are hereby notified that any dissemination, distribution, copying, or action taken in relation to the contents of and attachments to this E-mail is strictly prohibited and may be unlawful. If you have received this E-mail in error, please notify the sender immediately and permanently delete the original and any copy of this E-mail and any printout.

---

From: "Weil, Jason" <[email-masked]>
Date: Tue, 16 Apr 2013 15:50:17 -0400

I'm confused on why a distinction is needed between provisioned speed
(user information rate) and link speed as well.How would bonded links be
described by this link attribute?

Jason

On 4/16/13 10:42 AM, "Shane Amante" <[email-masked]> wrote:

>Mike,
>
>On Apr 16, 2013, at 8:28 AM, "Bugenhagen, Michael K"
><[email-masked]> wrote:
>>  I split line rate and User information rate..., added PON, and tweaked
>>Product name, and Unique ID (removed customer - we're identifying a
>>access service link, not a person)...
>
>Can you clarify why you think splitting "line rate" and "user information
>rate" is important?  For example, are you trying to distinguish between
>throughput with and without packet/framing overhead, (e.g.: PPPoA/PPPoE
>overhead?)  I was thinking and hoping to avoid that, because when we're
>talking about _Internet_ network performance, we're talking about IP
>throughput.  Plus, IP is thee common denominator across every access
>network technology.
>
>
>> Service Attributes -
>> - Unique Line ID <-- note, this would have to be some pseudo-random
>>identifier that associates this particular service to a particular
>>broadband "line", but doesn't reveal anything else about the customer to
>>ensure privacy/anonymity of the customer.
>
>> On Apr 16, 2013, at 8:00 AM, Juergen Schoenwaelder
>><[email-masked]> wrote:
>>>>>point-of-view is whether we include inter-domain service attribute
>>>> a)  Representation of service attributes inside an XML schema.  IMO,
>>>>it seems this is the right thing to do regardless of whether this is
>>>>"intra-" or "inter-"domain, since everyone who is going to
>>>>programmatically interpret measurement results is going to want a
>>>>standardized representation of "service attributes" to make it
>>>>straightforward to parse them.
>>>
>> I'm pretty confident it can be done, assuming we confine ourselves to
>>representing the most common of service attributes:
>> - Customer or Line ID <-- note, this would have to be some
>>pseudo-random identifier that associates this particular service to a
>>particular broadband "line", but doesn't reveal anything else about the
>>customer to ensure privacy/anonymity of the customer.
>>
>>
>>
>>>> b)  Transfer of the aforementioned "service attribute XML schema"
>>>>from one host to another host and/or from one party to another party.
>>>>For this I would ask the question of: why wouldn't we just use HTTP or
>>>>HTTPS? In doing so, it would allow LMAP to leverage a variety of
>>>>existing, already-specified & implemented authentication mechanisms
>>>>for HTTP/HTTPS.  And, for that matter, it could be up to ISPs to
>>>>decide what they wanted to use, e.g.: simple username/password and/or
>>>>SSL certificates.  At that point, it's up to a bi-lateral negotiation
>>>>between Party A to Party B, (in the context of inter-domain), at the
>>>>time they stand up the capability to 'download' (or, upload) the XML
>>>>service attribute schema.
>>>
>> +1.  Great points.  Yet another reason to consider xfer'ing of XML
>>service attribute schema on a "Network Parameter Server" on ISP A to,
>>for example, Data Collector owned/operated by Entity B, combined with
>>post-correlation of customer/line ID in the XML service attribute schema
>>to measurement performance results.
>>
>> -shane
>>
>>
>>> /js
>>>
>>> --

---

From: "Weil, Jason" <[email-masked]>
Date: Tue, 16 Apr 2013 15:51:35 -0400

Phil,

I agree, phase 1 scope should be limited to a single organization under
control of the MA, Controller and Collector with the caveat that the MA in
this set should be MA that resides within the end-user's environment.

Jason

As an aside, I see the argument for MA being a general term to cover both
endpoints,

On 4/16/13 12:08 PM, "[email-masked]" <[email-masked]> wrote:

>I think that lmap phase 1 should make the assumption that the measurement
>system is under the control of a single organisation (the MA, controller
>and collector)
>
>- an MA can run tests if it wants and report results to whoever it wants
>(cf. ookla speed tests). Lmap doesn't try to stop this happening (it's up
>to customer)
>

---

From: "Bugenhagen, Michael K" <[email-masked]>
Date: Tue, 16 Apr 2013 20:04:09 +0000

Jason,

Short answer to why link rate & User info rate (IP throughput capacity).

	Some ISP's sell service at the link rate, others sell service as the IP throughput rate (only universal standards).
	So for a customer to identify what they bought and it's IP throughput capacity (user info rate) they need both in order to relate that to the ISP service terms.   Plus, IP throughput capacity is a big deal for the customer.

Why did selling link rate happen (IMO) -
   Historically before IP was the universal protocol for User information rate the "access" link rate was standard and then what ever protocol or channel partitioning impacted the actual user information rate.  So when DSL was created they followed the status Quo (not everyone but some) and sell "link rate" or "train rate" - however given the IP was over ATM they had overhead, and the actual user information rate (IP rate) is lower than the link rate.

Mike

-----Original Message-----
From: Weil, Jason [mailto:[email-masked]] 
Sent: Tuesday, April 16, 2013 2:50 PM
To: Shane Amante; Bugenhagen, Michael K
Cc: [email-masked]; Juergen Schoenwaelder; [email-masked]; [email-masked]
Subject: Re: [lmap] LMAP: situation: Q1-Use Cases

I'm confused on why a distinction is needed between provisioned speed (user information rate) and link speed as well.How would bonded links be described by this link attribute?

Jason

On 4/16/13 10:42 AM, "Shane Amante" <[email-masked]> wrote:

>>   I think we'll need the MA to store more information about the 
>>Device it's in / or on in it's MIB itself...
>
>That could be useful.
>
>
>> Suggested Tweaks -
>>
>>  I split line rate and User information rate..., added PON, and 
>>tweaked Product name, and Unique ID (removed customer - we're 
>>identifying a access service link, not a person)...
>
>Can you clarify why you think splitting "line rate" and "user 
>information rate" is important?  For example, are you trying to 
>distinguish between throughput with and without packet/framing 
>overhead, (e.g.: PPPoA/PPPoE
>>broadband "line", but doesn't reveal anything else about the customer 
>>to ensure privacy/anonymity of the customer.
>
>>>>>> mechanisms for supplying service attributes to MAs that do not 
>>>>>> send data to the access provider?
>>>>>
>>>> a)  Representation of service attributes inside an XML schema.  
>>>>IMO, it seems this is the right thing to do regardless of whether 
>>>>this is "intra-" or "inter-"domain, since everyone who is going to 
>>> service attributes that matches a significant portion of the 
>>> products out there in the wild?
>>
>>particular broadband "line", but doesn't reveal anything else about 
>>the customer to ensure privacy/anonymity of the customer.
>>
>>
>>
>>>>For this I would ask the question of: why wouldn't we just use HTTP 
>>>>or HTTPS? In doing so, it would allow LMAP to leverage a variety of 
>>>>decide what they wanted to use, e.g.: simple username/password 
>>>>and/or SSL certificates.  At that point, it's up to a bi-lateral 
>>>>negotiation between Party A to Party B, (in the context of 
>>>>inter-domain), at the time they stand up the capability to 
>>>>'download' (or, upload) the XML service attribute schema.
>>>
>>> Securing the protocol is not the problem, it is key management that 
>>> is usually the problem. Do we expect an MA to have credentials that 
>>> allow it to fetch service attributes?  How are those credentials 
>>> managed during the lifecycle of the MA? It may be much harder to 
>>> manage the necessary credentials on thousands of MAs compared to a 
>>> solution where a few data collector or a data analysis function 
>>> fetch service attributes. (Of course, this still requires proper 
>>> authorization but the key management problem seems to be at a very 
>>> different order of scale - at least if we can assume #controller << 
>>> #MAs or #data-analyzer << #MAs.)
>>
>>post-correlation of customer/line ID in the XML service attribute 
>>schema to measurement performance results.
>>
>> -shane
>>
>>
>>> /js
>>>
>>> --

---

From: <[email-masked]>
Date: Fri, 19 Apr 2013 11:29:40 +0100

Since Benoit probably won't read all 44 emails in this thread, I tried to draft an answer to his original question.
Is it a fair summary of the consensus?

> Question 1. Use cases
> Do we agree that we want to focus on the Internet Service Provider and
> the End User Network Diagnostics use cases for now? As a phase 2, we
> could focus on the 3rd party use case: multi-provider, regulator, over
> the top
> Question: in the End User Network Diagnostics use case, I'm unclear if
> the user initiated measurement is included?

Here's a proposed answer:-

In order to simplify the security and policy considerations, we propose that LMAP phase 1 should make the assumption that the measurement system is under the control of a single organisation. 'Measurement system' refers to the Measurement Agent, Controller and Collector. There is likely to be interest in LMAP phase 2 in scenarios where the measurement system is not under the control of a single organisation.

Note that the Measurement Peer may be under another organisation's control (for example, the Measurement Peer might be in example.com, with the Measurement Task to download example.com's homepage or to send a request to its DNS server). However, this doesn't affect LMAP's 'single organisation' assumption since the job of defining Measurement Tasks is left to IPPM.

So in terms of Benoit's questions:
- The Internet Service Provider use case is in scope, assuming that the ISP controls the functions for the Measurement Agent, Controller and Collector
- The regulator use case is in scope, assuming that the regulator controls the functions for the Measurement Agent, Controller and Collector
- The End User Network Diagnostics use case is not directly in scope. However, the end user can get its MA to run Measurement Tasks if it wants and report results to whoever it wants; LMAP doesn't try to stop this happening.

On separate topics (but should be mentioned somewhere in reply to Benoit)

Discussion has revealed one new capability that may be required. Before a Measurement Task actually runs, the Measurement Agent may need to check that the Peer can carry out the Task. If one considers this as an initial stage of the Task, then it would be something for IPPM to define.

We have also discussed about the bootstrapping of a Measurement Agent (for example, about what Controller to contact). Here the consensus is to define the process but not a protocol. Because bootstrapping could be done in different ways, depending on the device and the measurement system, for instance: loaded at manufacture, updated locally via USB port, or orchestrated via a protocol.

---

From: "Bugenhagen, Michael K" <[email-masked]>
Date: Fri, 19 Apr 2013 12:36:06 +0000

Phil, 

   Thanks! That's a nice summation IMO.

The added points of "MA registration," and MA to MA capability resolution is also critical.

Regards,
Mike

---

From: Brian Trammell <[email-masked]>
Date: Mon, 22 Apr 2013 09:32:38 +1200

Phil,

Thanks for the summation! Summary reads good to me (as an interested lurker in this thread)...

Cheers,

Brian