Subject: [lmap] framework comments

From: "STARK, BARBARA H" <[email-masked]>
Date: Sun, 20 Apr 2014 12:59:45 +0000

First let me apologize for not having gotten my comments in before the deadline. I left for vacation shortly after the start of WGLC and have had very, very limited email and Internet access, or time for non-vacation activities.
Barbara

General comments
1. I'm concerned about descriptions of "the scope of LMAP" vs. "the scope of LMAP WG" vs. "the scope of this framework for LMAP" vs. "the scope of the LMAP WG charter". I think that what we chartered the LMAP WG to do limits the scope of this framework for large-scale measurement platforms (LMAP) at this time. It does not limit the scope of all possible LMAPs (future WG charters or all large-scale measurement platforms) for all time. Where there is distinction described between "LMAP" and "IPPM" scope (so that LMAP is really the WG more than the term), I agree with the description of "out-of-scope" for LMAP (WG) being permanent. Where we could conceive of the LMAP WG possibly being re-chartered in the future to take on additional tasks, or other organizations describing large-scale measurement platforms that encompass more, I think we should describe the restriction as being out-of-scope of this framework for LMAP, or out-of-scope of the current LMAP WG charter, rather than out-of-scope of (all) large-scale measurement platforms (LMAP) or the LMAP WG (forever). Part of the problem for me does seem to be that LMAP is both the WG and a term, and it's sometimes hard to tell which is being referred to. 
I suggest doing a search for "scope" and making the object of scoping "LMAP WG", "the current LMAP WG charter", or "this framework for LMAP", as appropriate.

2. Location of MA: This framework seems to assume the MA is always within a customer premises network. Is that necessarily true? Use-cases doesn't seem to indicate this. For example, in the case where measurements are done in an ONT (fiber termination point, which may be dedicated to a single subscriber or serve many in a MDU), the MA would be fully within the ISP network. Or it could be in a DSLAM and not specific to a single subscriber. If we consider the non-regulatory, ISP-controlled, trouble-shooting use case (which I thought was included in scope and is described in use-cases) then it doesn't seem like an MA *always* is associated with a single subscriber. But this case does not seem to be covered by this framework, because there are many statements that assume MA-to-subscriber association is part of the basic framework. Note that BBF use of MA does not restrict it to being internal to or at edge of subscriber network, or to having a necessary subscriber relationship. Following are some statements where I would recommend expanding the description to include elements outside the customer premises:
1. Introduction: "These devices could be software based agents on PCs, embedded agents in consumer devices (e.g. blu-ray players), service provider controlled devices such as set-top players and home gateways, or simply dedicated probes."
6.2.  Measurement Agent [this section should have examples of MAs located elsewhere in ISP networks and such.]

-------------------------------------------------------------------------------
Specific Comments:
1. Introduction
   o  Large-scale - [I-D.ietf-lmap-use-cases] envisages Measurement
      Agents in every home gateway and edge device such as set-top-boxes
      and tablet computers.  It is expected that a measurement system
      could easily encompass a few hundred thousand  Measurement Agents.
      Existing systems have up to a few thousand MAs (without judging
      how much further they could scale).
Comment: Why only a few hundred thousand? For the incredibly bandwidth-heavy regulatory-type use case, I think this is to many. For ISP use cases, I think we're looking at millions.
--------------------------------------------------------------------
2.  Outline of an LMAP-based measurement system
   The main work of the LMAP working group is to define the Control
   Protocol between the Controller and MA, and the Report Protocol
   between the MA and Collector.   
Comment: I realize Greg already commented on this and Phil responded that he agreed. But it wasn't clear to me exactly what would be done about it. My preference would be to use the charter phrase: " The LMAP working group is chartered to specify an information model, the associated data models, and select/extend one or more protocols for the secure communication:" I would prefer not to debate which of these items are "main" (implying the others are lesser deliverables). 
--------
A device with a
   Measurement Agent may have multiple interfaces  (Wi-Fi, Ethernet, DSL,
   fibre, etc.) and the Measurement Tasks may specify any one of these.
Comment: Only PHY interfaces are listed. I would recommend listing other interfaces as well (that carry IP) to make it clear the term is not limited to PHY interfaces. Such as PPPoE, L2TP, 6rd, DS-Lite, IPsec.
---------
   Measurement Tasks may be Active (the MA generates Active Measurement
   Traffic and measures some metric associated with its transfer),
   Passive (the MA observes user traffic), or some hybrid form of the
   two.
Comment: I recommend deleting "user" from the phrase, unless we really are restricting MAs to only be associated with specific subscribers.
---------
   The data analysis tools receive the results from the Collector or via
   the Results repository.  They might visualise the data or identify
   which component or link is likely to be the cause of a fault or
   degradation.
Comment: What does it mean to visualize data? I'm unfamiliar with this word in this context.
----------------------------------------------
3.  Terminology
   Subscriber: An entity (associated with one or more users) that is
   engaged in a subscription with a service provider.  The Subscriber is
   allowed to subscribe and un-subscribe services, and to register a
   user or a list of users authorized to enjoy these services.  [Q1741].
   Both the Subscriber and service provider are allowed to set the
   limits relative to the use that associated users make of subscribed
   services.
Comment: The reference is to a 3GPP document and the last 2 sentences are very 3GPP-specific and not relevant to the concept of a wireline subscriber. I recommend deleting the last 2 sentences from this definition.
-----------------------------------------------
4.1.  Measurement system is under the direction of a single organisation

   In the LMAP framework, the measurement system is under the direction
   of a single organisation that is responsible for any impact that
   measurements have on a user's quality of experience and privacy.
Comment: Recommend "... that is responsible for any impact that its measurements have..." They cannot be responsible for the impact of others' measurements.
------------------------------------------------
5.3.1.  Starting and Stopping Measurement Tasks
...
   o  the MA checking that there is no cross-traffic.  In other words, a
      check that the end-user isn't already sending traffic;
Comment: It may not be the end user directly engaged in sending the traffic. It may be some other autonomous or scheduled application. Recommend "...that the end-user or other application isn't..."
,,,
   o  the first part of the Measurement Task  consisting of traffic that
      probes the path to make sure it isn't overloaded;
   o  the first part of the Measurement Task  checking that the device
      with the MA has enough resources to execute the Measurement Task
      reliably.  Note that the designer of the measurement system should
      ensure that the device's capabilities are normally sufficient to
      comfortably operate the Measurement Tasks.
   It is possible that similar checks continue during the Measurement
   Task , ...
Comment: This describes a very unusual and inefficient way of designing "Measurement Tasks". It suggests that a Measurement Method can be something very complex with multiple, disparate test protocols. Personally, I would use a Measurement Schedule to identify a sequence of atomic Measurement Tasks, rather than define mega Measurement Methods that combine lots of stuff together so that a single Measurement Task is really a whole sequence of different tests. I don't object to people doing it this way if that's what they want, I just want to make absolutely sure that it's not going to end up being mandated or assumed that mega Measurement Methods must be defined, instead of allowing measurement system operators to define a sequence of individual Tasks within a Measurement Schedule.
.......
To avoid the MA generating
   traffic forever after a Controller has permanently failed , it is
   suggested that the Measurement Schedule includes a time limit...
Comment: I thought we were also going to allow for the MA to just stop doing measurements if it didn't hear from the Controller in a while.
.......
   It is likely to  be important to include in the Measurement Report the
   fact that the Measurement Task overlapped with another.
Comment: I think "It may be important..." rather than "likely to be important". What is important is knowing whether or not (and how much) cross-traffic is present. The precise nature of the cross traffic (that it was another Measurement Task) isn't that important.
............................................................................
5.5.  Operation of LMAP over the underlying transport protocol
 And subsequent uses of "underlying transport protocol" in this section.
Comment: The description of "transport protocol" in this section is unlike any use of "transport protocol" I've ever seen. Especially the part about there being Push, Pull, and multicast transport protocols. Multicast only happens at IP or Ethernet, to the best of my knowledge. I've never heard of a higher layer protocol capable of multicast. I usually associate "transport protocol" with protocols that operate at the transport layer, like TCP and UDP, or with protocols like RTP (that has Transport Protocol in its name). But RTP, TCP, UDP and such have no Push, Pull, or multicast characteristics. I'm not sure what to recommend about this section -- parts seem usable, but the way it's described doesn't relate to my understanding of the nature of various protocols.
............................................................................
5.6.1.  End-user-controlled measurement system
...
       Note that a user can't directly initiate a Measurement
       Task on an ISP- (or regulator-) controlled MA.
Comment: Why not? Why isn't it allowed to include this ability in a MA design (such as one that is just for the ISP use case and has nothing to do with regulatory use case)?
-------------------------------------------------------------
6.1.  Controller and the measurement system
...
; if it will, then the
   measurement system may need to compensate the Subscriber, for
   instance .
Comment: This statement needs to be deleted. This document is no place for speculation regarding compensation models.
-------------------------------------------------------------
6.2.2.  Measurement Agent embedded in site gateway
...
   However, if the site gateway is owned and operated by the service
   provider, the Measurement Agent will generally not be directly
   available for over the top providers, the regulator, end users or
   enterprises.
Comment: True. But it's also true that where the gateway is owned by the end user, it's even less likely that it will be available for service providers, ott providers or regulators. End-user owned gateway is a common model, so I would suggest mentioning this fact as well.
--------------------------------------------------------------
6.2.3.  Measurement Agent embedded behind site NAT /Firewall

   The Measurement Agent could also be embedded behind a NAT, a
   firewall, or both.  In this case the Controller may not be able to
   unilaterally contact the Measurement Agent unless either static port
   forwarding configuration or firewall pin holing is configured.  For
   the former, protocols such as PCP [RFC6887], TR-069 [TR-069]or UPnP
   [UPnP]could be used.  For the latter, the Measurement Agent could
   send keepalives towards the Controller to prop open the firewall (and
   perhaps use these also as a network reachability test).
Comment: Regarding the last sentence, I don't understand what keepalives has to do with UPnP. Why not just mention STUN as an option for keeping NAT table entries from timing out?
-------------------------------------------------------------
7.  Security considerations
   Reporting by the MA must also be secured to maintain confidentiality.
   The results must be encrypted such that only the authorised Collector
   can decrypt the results to prevent the leakage of confidential or
   private information.   
Comment: There are other ways of securing communications. For example, where the MA is in a fiber ONT or other ISP network element (generally for ISP troubleshooting and not the regulatory use case), the communication may occur over the ISP's dedicated management network. In this case, encryption is not needed. Therefore I disagree that the results must be encrypted. The second sentence could be changed to start "Where encryption is used, the results must be..."
...
   A malicious party could "game the system".  For example, where a
   regulator is running a measurement system in order to benchmark
   operators, an operator could try to identify the broadband lines that
   the regulator was measuring and prioritise that traffic.  This
   potential issue is currently handled by a code of conduct.   It is
   outside the scope of the LMAP WG to consider the issue.
Comment: The existence of a code of conduct is not universally true. I suggest deleting this sentence.
--------------------------------
8.3.  Key Distinction Between Active and Passive Measurement Tasks

   Passive and Active Measurement Tasks raise different privacy issues.

   Passive Measurement Tasks are conducted on a user's traffic ,...
Comment: This is not necessarily true if it is possible for the MA not to be located in a place where it is associated with a single end user or subscriber. I recommend "   Passive Measurement Tasks can be conducted on a user's traffic ,..."
-------------------------------------
8.4.1.  MA Bootstrapping
...
   During the Bootstrap process, the MA receives its MA-ID which is a
   persistent pseudonym for the Subscriber ...
Comment: Only true if the MA is associated with a specific subscriber. 
--------------------------------------
8.5.  Threats
Comment: I recommend mentioning some of the threats described in the Security section here, especially the possibility of DoS attacks. I think DoS attacks pose a huge threat.
--------------------------------------
8.5.4.  Secondary Use and Disclosure
...
   Passive Measurement Tasks are a form of Secondary Use, and the
   Subscribers' permission and the measured ISP's permission should be
   obtained beforehand .  
Comment: This "should" statement as it applies to subscribers is dependent on national laws and regulations. How it applies to ISPs, I don't understand. In what case is someone supposed to get the ISP's permission to do passive measurements (presumably from a location outside the ISP network?)? I recommend deleting this sentence, or at least the 2nd part of the sentence (", and the Subscribers'...beforehand").
------------------------------------------
8.6.1.  Data Minimisation
...
   There are two levels of information needed for LMAP results to be
   useful for a specific task : troubleshooting and general results
   reporting.
Comment: I don't understand "levels of information". I recommend: "There are two main uses of LMAP results:" 
-------------------------------------------
8.6.4.  Other Mitigations
...
   The informed consent of the Subscriber (and, if
   different, the end user) is needed ,...
Comment: This depends on local laws and regulations. I suggest it "may be needed".
...
   It is good practice to time limit the storage of
   personal information.
Comment: I suggest: "Where privacy is important, it is good practice ..."

------------------------------------------------------------------------------------------------
Nits
 - Inconsistent use of "set-top players" vs "set-top-boxes". My preference would be "set-top boxes".
 - Instruction vs. Instructions: If the term is really supposed to be singular, consider searching for "Instructions" and replacing either with "Instruction" or "Instruction messages" or "Instruction-carrying messages" or such.
 - "for example:  "Report results once a day in a batch at 4am + Unif[0,180] seconds; if the end user is active then delay the report 5 minutes ".  " Comment: Interesting that the "delay" example is for delayed reporting instead of delayed measurement. I'm not familiar with a lot of need to delay reporting.

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Mon, 21 Apr 2014 08:07:33 +0000

Hi,

One observation about general comment #1. I understand the comment and I agree to a large extent with the intent. We need to be clear about the places when we are talking about the broader scope of LMAP (like when tracing the delineation line between what LMAP does and what IPPM or other groups or organization do) and the places where we refer to the current charter. This document may contain both (broader and narrower) scope items, as a framework may draw lines for work beyond the current charter. We just need to be clear. 

Having said this, I do not believe that we should talk explicitly about the LMAP WG charter  in this document, and this is for two reasons. One is that WGs come and go, while RFCs stay, so we would like to write the document in a language that is meaningful even after the WG has completed work. Second is that the LMAP charter may and probably will change in time, so talking about "the current LMAP WG charter" may point to something different in the future. I would suggest to use a more neutral term, like "the first phase of the LMAP work".

Regards,

Dan

> -----Original Message-----
> From: lmap [mailto:[email-masked]] On Behalf Of STARK, BARBARA H
> Sent: Sunday, April 20, 2014 4:00 PM
> To: [email-masked]
> Subject: [lmap] framework comments
> 
> First let me apologize for not having gotten my comments in before the
> deadline. I left for vacation shortly after the start of WGLC and have had
> very, very limited email and Internet access, or time for non-vacation
> activities.
> Barbara
> 
> General comments
> 1. I'm concerned about descriptions of "the scope of LMAP" vs. "the scope of
> LMAP WG" vs. "the scope of this framework for LMAP" vs. "the scope of the
> LMAP WG charter". I think that what we chartered the LMAP WG to do limits
> the scope of this framework for large-scale measurement platforms (LMAP)
> at this time. It does not limit the scope of all possible LMAPs (future WG
> charters or all large-scale measurement platforms) for all time. Where there
> is distinction described between "LMAP" and "IPPM" scope (so that LMAP is
> really the WG more than the term), I agree with the description of "out-of-
> scope" for LMAP (WG) being permanent. Where we could conceive of the
> LMAP WG possibly being re-chartered in the future to take on additional
> tasks, or other organizations describing large-scale measurement platforms
> that encompass more, I think we should describe the restriction as being out-
> of-scope of this framework for LMAP, or out-of-scope of the current LMAP
> WG charter, rather than out-of-sc  ope of (all) large-scale measurement
> platforms (LMAP) or the LMAP WG (forever). Part of the problem for me
> does seem to be that LMAP is both the WG and a term, and it's sometimes
> hard to tell which is being referred to.
> I suggest doing a search for "scope" and making the object of scoping "LMAP
> WG", "the current LMAP WG charter", or "this framework for LMAP", as
> appropriate.
> 
> 2. Location of MA: This framework seems to assume the MA is always within
> a customer premises network. Is that necessarily true? Use-cases doesn't
> seem to indicate this. For example, in the case where measurements are
> done in an ONT (fiber termination point, which may be dedicated to a single
> subscriber or serve many in a MDU), the MA would be fully within the ISP
> network. Or it could be in a DSLAM and not specific to a single subscriber. If
> we consider the non-regulatory, ISP-controlled, trouble-shooting use case
> (which I thought was included in scope and is described in use-cases) then it
> doesn't seem like an MA *always* is associated with a single subscriber. But
> this case does not seem to be covered by this framework, because there are
> many statements that assume MA-to-subscriber association is part of the
> basic framework. Note that BBF use of MA does not restrict it to being
> internal to or at edge of subscriber network, or to having a necessary
> subscriber relationship. Following
>   are some statements where I would recommend expanding the description
> to include elements outside the customer premises:
> 1. Introduction: "These devices could be software based agents on PCs,
> embedded agents in consumer devices (e.g. blu-ray players), service
> provider controlled devices such as set-top players and home gateways, or
> simply dedicated probes."
> Comment: Why only a few hundred thousand? For the incredibly bandwidth-
> heavy regulatory-type use case, I think this is to many. For ISP use cases, I
> think we're looking at millions.
> Comment: I realize Greg already commented on this and Phil responded that
> he agreed. But it wasn't clear to me exactly what would be done about it. My
> preference would be to use the charter phrase: " The LMAP working group is
> chartered to specify an information model, the associated data models, and
> select/extend one or more protocols for the secure communication:" I would
> prefer not to debate which of these items are "main" (implying the others
> are lesser deliverables).
> --------
> A device with a
> Comment: Only PHY interfaces are listed. I would recommend listing other
> interfaces as well (that carry IP) to make it clear the term is not limited to PHY
> interfaces. Such as PPPoE, L2TP, 6rd, DS-Lite, IPsec.
> ---------
> Comment: I recommend deleting "user" from the phrase, unless we really
> are restricting MAs to only be associated with specific subscribers.
> ---------
> Comment: The reference is to a 3GPP document and the last 2 sentences are
> very 3GPP-specific and not relevant to the concept of a wireline subscriber. I
> recommend deleting the last 2 sentences from this definition.
> Comment: Recommend "... that is responsible for any impact that its
> measurements have..." They cannot be responsible for the impact of others'
> measurements.
> 5.3.1.  Starting and Stopping Measurement Tasks ...
> Comment: It may not be the end user directly engaged in sending the traffic.
> It may be some other autonomous or scheduled application. Recommend
> "...that the end-user or other application isn't..."
> ,,,
> Comment: This describes a very unusual and inefficient way of designing
> "Measurement Tasks". It suggests that a Measurement Method can be
> something very complex with multiple, disparate test protocols. Personally, I
> would use a Measurement Schedule to identify a sequence of atomic
> Measurement Tasks, rather than define mega Measurement Methods that
> combine lots of stuff together so that a single Measurement Task is really a
> whole sequence of different tests. I don't object to people doing it this way if
> that's what they want, I just want to make absolutely sure that it's not going
> to end up being mandated or assumed that mega Measurement Methods
> must be defined, instead of allowing measurement system operators to
> define a sequence of individual Tasks within a Measurement Schedule.
> .......
> Comment: I thought we were also going to allow for the MA to just stop
> doing measurements if it didn't hear from the Controller in a while.
> .......
> Comment: I think "It may be important..." rather than "likely to be
> important". What is important is knowing whether or not (and how much)
> cross-traffic is present. The precise nature of the cross traffic (that it was
> another Measurement Task) isn't that important.
> 5.5.  Operation of LMAP over the underlying transport protocol  And
> subsequent uses of "underlying transport protocol" in this section.
> Comment: The description of "transport protocol" in this section is unlike any
> use of "transport protocol" I've ever seen. Especially the part about there
> being Push, Pull, and multicast transport protocols. Multicast only happens at
> IP or Ethernet, to the best of my knowledge. I've never heard of a higher
> layer protocol capable of multicast. I usually associate "transport protocol"
> with protocols that operate at the transport layer, like TCP and UDP, or with
> protocols like RTP (that has Transport Protocol in its name). But RTP, TCP,
> UDP and such have no Push, Pull, or multicast characteristics. I'm not sure
> what to recommend about this section -- parts seem usable, but the way it's
> described doesn't relate to my understanding of the nature of various
> protocols.
> 5.6.1.  End-user-controlled measurement system ...
> Comment: Why not? Why isn't it allowed to include this ability in a MA design
> (such as one that is just for the ISP use case and has nothing to do with
> regulatory use case)?
> 6.1.  Controller and the measurement system ...
> ; if it will, then the
> Comment: This statement needs to be deleted. This document is no place for
> speculation regarding compensation models.
> 6.2.2.  Measurement Agent embedded in site gateway ...
> Comment: True. But it's also true that where the gateway is owned by the
> end user, it's even less likely that it will be available for service providers, ott
> providers or regulators. End-user owned gateway is a common model, so I
> would suggest mentioning this fact as well.
> Comment: Regarding the last sentence, I don't understand what keepalives
> has to do with UPnP. Why not just mention STUN as an option for keeping
> NAT table entries from timing out?
> Comment: There are other ways of securing communications. For example,
> where the MA is in a fiber ONT or other ISP network element (generally for
> ISP troubleshooting and not the regulatory use case), the communication
> may occur over the ISP's dedicated management network. In this case,
> encryption is not needed. Therefore I disagree that the results must be
> encrypted. The second sentence could be changed to start "Where
> encryption is used, the results must be..."
> ...
> Comment: This is not necessarily true if it is possible for the MA not to be
> located in a place where it is associated with a single end user or subscriber. I
> recommend "   Passive Measurement Tasks can be conducted on a user's
> traffic ,..."
> Comment: I recommend mentioning some of the threats described in the
> Security section here, especially the possibility of DoS attacks. I think DoS
> attacks pose a huge threat.
> Comment: This "should" statement as it applies to subscribers is dependent
> on national laws and regulations. How it applies to ISPs, I don't understand. In
> what case is someone supposed to get the ISP's permission to do passive
> measurements (presumably from a location outside the ISP network?)? I
> recommend deleting this sentence, or at least the 2nd part of the sentence
> (", and the Subscribers'...beforehand").
>  - Instruction vs. Instructions: If the term is really supposed to be singular,
> consider searching for "Instructions" and replacing either with "Instruction"
> or "Instruction messages" or "Instruction-carrying messages" or such.
>  - "for example:  "Report results once a day in a batch at 4am + Unif[0,180]
> seconds; if the end user is active then delay the report 5 minutes ".  "
> Comment: Interesting that the "delay" example is for delayed reporting
> instead of delayed measurement. I'm not familiar with a lot of need to delay
> reporting.
> 
> _______________________________________________
> lmap mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/lmap

---

From: <[email-masked]>
Date: Tue, 22 Apr 2014 17:33:30 +0100

Barbara,

Thanks very much for your detailed comments!

Some follow-ups in-line

phil

> Sent: 20 April 2014 14:00
> To: [email-masked]
> deadline. I left for vacation shortly after the start of WGLC and have
> had very, very limited email and Internet access, or time for non-
> vacation activities.
> Barbara
>
> General comments
> 1. I'm concerned about descriptions of "the scope of LMAP" vs. "the
> scope of LMAP WG" vs. "the scope of this framework for LMAP" vs. "the
> scope of the LMAP WG charter". I think that what we chartered the LMAP
> WG to do limits the scope of this framework for large-scale measurement
> platforms (LMAP) at this time. It does not limit the scope of all
> possible LMAPs (future WG charters or all large-scale measurement
> platforms) for all time. Where there is distinction described between
> "LMAP" and "IPPM" scope (so that LMAP is really the WG more than the
> term), I agree with the description of "out-of-scope" for LMAP (WG)
> being permanent. Where we could conceive of the LMAP WG possibly being
> re-chartered in the future to take on additional tasks, or other
> organizations describing large-scale measurement platforms that
> encompass more, I think we should describe the restriction as being
> out-of-scope of this framework for LMAP, or out-of-scope of the current
> LMAP WG charter, rather than out-of-sc  ope of (all) large-scale
> measurement platforms (LMAP) or the LMAP WG (forever). Part of the
> problem for me does seem to be that LMAP is both the WG and a term, and
> it's sometimes hard to tell which is being referred to.
> I suggest doing a search for "scope" and making the object of scoping
> "LMAP WG", "the current LMAP WG charter", or "this framework for LMAP",
> as appropriate.

[phil] will try and follow Dan's guidance

>
> 2. Location of MA: This framework seems to assume the MA is always
> within a customer premises network. Is that necessarily true? Use-cases
> doesn't seem to indicate this. For example, in the case where
> measurements are done in an ONT (fiber termination point, which may be
> dedicated to a single subscriber or serve many in a MDU), the MA would
> be fully within the ISP network. Or it could be in a DSLAM and not
> specific to a single subscriber. If we consider the non-regulatory,
> ISP-controlled, trouble-shooting use case (which I thought was included
> in scope and is described in use-cases) then it doesn't seem like an MA
> *always* is associated with a single subscriber. But this case does not
> seem to be covered by this framework, because there are many statements
> that assume MA-to-subscriber association is part of the basic
> framework. Note that BBF use of MA does not restrict it to being
> provider controlled devices such as set-top players and home gateways,
> or simply dedicated probes."
> 6.2.  Measurement Agent [this section should have examples of MAs
> located elsewhere in ISP networks and such.]

[phil] I agree with you, the MA does not have to be in the customer's premises.
S6 - if anyone can contribute text of MAs located elsewhere, that would be very helpful.

> heavy regulatory-type use case, I think this is to many. For ISP use
> cases, I think we're looking at millions.

[phil] how about:
could easily encompass a few hundred thousand or even millions of Measurement Agents.

...

> Comment: What does it mean to visualize data? I'm unfamiliar with this

http://en.wikipedia.org/wiki/Data_visualization

> Comment: The reference is to a 3GPP document and the last 2 sentences
> are very 3GPP-specific and not relevant to the concept of a wireline
> subscriber. I recommend deleting the last 2 sentences from this
> definition.

[phil] I think the definition is copied from Q.1741. personally am ok with deleting the last 2 sentences.

...

> something very complex with multiple, disparate test protocols.
> Personally, I would use a Measurement Schedule to identify a sequence
> of atomic Measurement Tasks, rather than define mega Measurement
> Methods that combine lots of stuff together so that a single
> Measurement Task is really a whole sequence of different tests. I don't
> object to people doing it this way if that's what they want, I just
> want to make absolutely sure that it's not going to end up being
> mandated or assumed that mega Measurement Methods must be defined,
> instead of allowing measurement system operators to define a sequence
> of individual Tasks within a Measurement Schedule.

[phil] yes, that's ok. whatever - the issue is that the later task or sub-task runs (or not) depending on the result of the earlier (sub)task.

> doing measurements if it didn't hear from the Controller in a while.

[phil] seems to me that these are equivalent, though I don't mind adding it.

...

> Comment: The description of "transport protocol" in this section is
> unlike any use of "transport protocol" I've ever seen. Especially the
> part about there being Push, Pull, and multicast transport protocols.
> Multicast only happens at IP or Ethernet, to the best of my knowledge.
> I've never heard of a higher layer protocol capable of multicast. I
> usually associate "transport protocol" with protocols that operate at
> the transport layer, like TCP and UDP, or with protocols like RTP (that
> has Transport Protocol in its name). But RTP, TCP, UDP and such have no
> Push, Pull, or multicast characteristics. I'm not sure what to
> recommend about this section -- parts seem usable, but the way it's

Very open to suggested phrasings.
Key word is "underlying".
Simply mean that (say) the LMAP Control Protocol is Controller to MA, but for instance if it operates over http, then at this "underlying" layer the MA does a request (pull) to the Controller and its reply includes the LMAP Control Protocol info. Or the "underlying" layer may be push or even multicast.

"transport" didn't mean layer 4 but something transporting the LMAP info.

Would "underlying packet transfer mechanism" or "model" be clearer?

> Comment: Why not? Why isn't it allowed to include this ability in a MA
> design (such as one that is just for the ISP use case and has nothing
> to do with regulatory use case)?

[phil] then there would be two ways of controlling the MA, which is something we want to avoid.

> Comment: This statement needs to be deleted. This document is no place
> for speculation regarding compensation models.

[phil] I'm ok to delete this phrase

> Comment: Regarding the last sentence, I don't understand what
> keepalives has to do with UPnP. Why not just mention STUN as an option
> for keeping NAT table entries from timing out?

[phil] "for the former" refers to "static port forwarding configuration" and "for the latter" to "firewall pin holing"

> where the MA is in a fiber ONT or other ISP network element (generally
> for ISP troubleshooting and not the regulatory use case), the
> communication may occur over the ISP's dedicated management network. In
> this case, encryption is not needed. Therefore I disagree that the
> results must be encrypted. The second sentence could be changed to
> start "Where encryption is used, the results must be..."

[phil] this may need some more discussion. I'm not sure we should mention this. Anyway, here's a suggestion:-

(new penultimate paragraph for the section)
The security mechanisms described above may not be strictly necessary if the network's design ensures the LMAP components and their communications are already secured, for example potentially if they are all part of an ISP's dedicated management network.

(replacement Reporting para)
Reporting by the MA must be encrypted to maintain confidentiality, to prevent the leakage of confidential or private information. Reporting must also be authenticated (to ensure that it comes from a trusted MA) and not vulnerable to tampering (which can be ensured through integrity and replay checks). It must not be possible to fool a MA into injecting falsified data and the results must also be held and processed securely after collection and analysis (see section 8.5.2 ...).

I also think we should mention that the Control messages also need to be confidential.
"The Instruction messages also need to be encrypted to maintain confidentiality, as the information might be useful to an attacker."

> suggest deleting this sentence.

[phil] think we should add "normally" to the sentence but keep it. The topic has come up several times, so seems to be something that readers often wonder about.

> Comment: This is not necessarily true if it is possible for the MA not
> to be located in a place where it is associated with a single end user
> or subscriber. I recommend "   Passive Measurement Tasks can be
> conducted on a user's traffic ,..."

[phil] how about " Passive Measurement Tasks are conducted on user traffic"?
(I think "can be conducted" misleadingly hints "but may not be")

[phil] ok, this needs "may" or "for a MA located at an end-user"

> Security section here, especially the possibility of DoS attacks. I
> think DoS attacks pose a huge threat.

[phil] agree DOS attacks are the most worrying ones/.

> Comment: This "should" statement as it applies to subscribers is
> dependent on national laws and regulations. How it applies to ISPs, I
> don't understand. In what case is someone supposed to get the ISP's
> permission to do passive measurements (presumably from a location
> outside the ISP network?)? I recommend deleting this sentence, or at
> least the 2nd part of the sentence (", and the
> Subscribers'...beforehand").

[phil] how about shortening the second para:-
Secondary use could be made of the Results from both Active and Passive Measurement Tasks, for example to inform an unauthorised marketing campaign. Secondary use may break national laws and regulations, and may violate people's expectations or desires.

-- At the moment the section says that all Passive measurements are intrinsically secondary use - is this really true?

---

From: <[email-masked]>
Date: Thu, 24 Apr 2014 15:32:31 +0100

Looking at the i-d, I think all of the currently ambiguous text about "scope" falls into the category of "out of scope of LMAP WG" (rather than "out of scope of the first phase of the LMAP work")

Am currently updating the i-d, based on the WBLC comments and discussions

phil

> From: lmap [mailto:[email-masked]] On Behalf Of Romascanu, Dan
> (Dan)
> Sent: 21 April 2014 09:08
> To: STARK, BARBARA H; [email-masked]
> Subject: Re: [lmap] framework comments
> 
> Hi,
> 
> One observation about general comment #1. I understand the comment and
> I agree to a large extent with the intent. We need to be clear about
> the places when we are talking about the broader scope of LMAP (like
> when tracing the delineation line between what LMAP does and what IPPM
> or other groups or organization do) and the places where we refer to
> the current charter. This document may contain both (broader and
> narrower) scope items, as a framework may draw lines for work beyond
> the current charter. We just need to be clear.
> 
> Having said this, I do not believe that we should talk explicitly about
> the LMAP WG charter  in this document, and this is for two reasons. One
> is that WGs come and go, while RFCs stay, so we would like to write the
> document in a language that is meaningful even after the WG has
> completed work. Second is that the LMAP charter may and probably will
> change in time, so talking about "the current LMAP WG charter" may
> point to something different in the future. I would suggest to use a
> more neutral term, like "the first phase of the LMAP work".
> 
> Regards,
> 
> Dan
> 
> 
> > had very, very limited email and Internet access, or time for
> > non-vacation activities.
> > Barbara
> >
> > General comments
> > WG to do limits the scope of this framework for large-scale
> > measurement platforms (LMAP) at this time. It does not limit the
> scope
> > of all possible LMAPs (future WG charters or all large-scale
> > measurement platforms) for all time. Where there is distinction
> > described between "LMAP" and "IPPM" scope (so that LMAP is really the
> > WG more than the term), I agree with the description of "out-of-
> > LMAP WG possibly being re-chartered in the future to take on
> > additional tasks, or other organizations describing large-scale
> > measurement platforms that encompass more, I think we should describe
> > the restriction as being out- of-scope of this framework for LMAP, or
> > out-of-scope of the current LMAP WG charter, rather than out-of-sc
> > ope of (all) large-scale measurement platforms (LMAP) or the LMAP WG
> > (forever). Part of the problem for me does seem to be that LMAP is
> both the WG and a term, and it's sometimes hard to tell which is being
> referred to.
> > "LMAP WG", "the current LMAP WG charter", or "this framework for
> > LMAP", as appropriate.
> >

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Tue, 29 Apr 2014 08:13:03 +0000

Hi Phil.
Snipping out parts where I have nothing else to say and I agree with what you propose.
Barbara

> > within a customer premises network. Is that necessarily true?
> > Use-cases doesn't seem to indicate this. For example, in the case
> > where measurements are done in an ONT (fiber termination point, which
> > may be dedicated to a single subscriber or serve many in a MDU), the
> > MA would be fully within the ISP network. Or it could be in a DSLAM
> > and not specific to a single subscriber. If we consider the
> > non-regulatory, ISP-controlled, trouble-shooting use case (which I
> > thought was included in scope and is described in use-cases) then it
> > doesn't seem like an MA
> > *always* is associated with a single subscriber. But this case does
> > not seem to be covered by this framework, because there are many
> > statements that assume MA-to-subscriber association is part of the
> >   are some statements where I would recommend expanding the
> > description to include elements outside the customer premises:

<bhs> Yes, I'll propose some additional text. I'll do that in a separate email later this week.

> deleting the last 2 sentences.

<bhs> Right, I see that Q.1741 is not a 3GPP document -- it's an ITU-T document entitled "GSM-evolved UMTS core network". I would still prefer to delete the last 2 sentences, but if others want to propose modifying them to indicate that they *may* be true, that could be ok, too.  
 
> [phil] seems to me that these are equivalent, though I don't mind adding it.
<bhs> Given the extensive discussion that was had about this, I'm curious as to what others think.

> > the transport layer, like TCP and UDP, or with protocols like RTP
> > (that has Transport Protocol in its name). But RTP, TCP, UDP and such
> > have no Push, Pull, or multicast characteristics. I'm not sure what to
> Simply mean that (say) the LMAP Control Protocol is Controller to MA, but for
> instance if it operates over http, then at this "underlying" layer the MA does
> a request (pull) to the Controller and its reply includes the LMAP Control
> Protocol info. Or the "underlying" layer may be push or even multicast.
> 

<bhs> So what I read you saying is that there are some characteristics that the Control or Report Protocol can "inherit" from underlying protocols (effectively defined as any protocol at any layer that is below the Control or Report Protocol). This much I agree with, and I think it would be good to rephrase along this line. 
However, http does not necessarily imply "Pull". Just because 2 endpoints are in a client/server relationship for a Control or Report Protocol, doesn't mean that they maintain that same client/server relationship as it relates to http. That is, it's possible for the Controller to be the http client that initiates a http session to the MA (which is the http server). It can be the higher layer Control or Report protocol that dictates which endpoint can or must be the http client/server. For example, TR-069 does allow the TR-069 config server to initiate http with the TR-069 client for the purpose of sending a "connect to your config server ASAP" message. But only the TR-069 client can initiate a http session for purpose of receiving configuration info (in which case the TR-069 client is the http client and the TR-069 config server is the http server). And so I consider TR-069 to be a "Pull" protocol. But it doesn't inherit this from http. Because either side can initiate http to the other at different times. It is TR-069 itself that defines the RPCs (remote procedure calls) and when different RPCs can be used.
So I still have a problem with the idea that Push and Pull are necessarily inherited from underlying protocols, or that http implies "Pull". 
I agree that multicast is inherited.

> something we want to avoid.

<bhs> Multiple methods of remote configuration are indeed complicated, which was why I very much agreed with the "one Controller" restriction. But we have a lot of experience in coordinating between person-directed local configuration and remote automated configuration on various devices. Which is why I don't see it necessary to forbid this.

> [phil] "for the former" refers to "static port forwarding configuration" and
> "for the latter" to "firewall pin holing"

<bhs> Hmm. I think we have terminology-understanding differences at multiple levels here. In my use of the words, a port forwarding rule or mapping is a pinhole. They can be statically (manually, TR-069, SNMP) or dynamically (UPnP IGD, PCP) configured, or dynamically generated as a result of outbound TCP and UDP packets.  Because pinholes configured through PCP and UPnP IGD do expire after a while, and are created on an "as needed" basis, they tend to be considered dynamic and not static. Pinholes dynamically generated by the NAT or firewall processes as a result of outbound TCP or UDP packets don't tend to be considered "configured". I would suggest rewording this to:
  The Measurement Agent could also be embedded behind a NAT, a
  stateful-packet inspection firewall, or both.  In this case the Controller may not be able to
 unilaterally contact the Measurement Agent unless configured or dynamically-generated pinholes exist.  
 Protocols such as PCP [RFC6887], CWMP [TR-069] or UPnP IGD
 [UPnP IGD] can be used to configure pinholes.  For pinholes that are dynamically-generated as a result of outbound TCP or UDP packets, 
  the Measurement Agent could
 send keepalives, such as those of the STUN protocol [STUN], towards the Controller to keep open the pinholes
(and perhaps use these also as a network reachability test).

I recommend referencing UPnP IGD, and not just UPnP. And calling the protocol described in TR-069 "CWMP".
 
> (I think "can be conducted" misleadingly hints "but may not be")

<bhs> As has been discussed on this list, Passive Measurements can also be conducted on Active Measurement traffic. I don't consider MA- or MP-generated Active Measurement traffic to be user traffic. For MAs located outside the customer premises, it is also possible for an ISP to measure management traffic (between its network elements and systems).

---

From: <[email-masked]>
Date: Tue, 29 Apr 2014 11:31:23 +0100

Thanks!

> separate email later this week.

Al wrote this text, what do you reckon?

6.2.5.  Measurement Agent embedded in ISP Networks

   A MA may be embedded on a device that is part of an ISP's network,
   such as a router or switch.  Usually the network devices with an
   embedded MA will be stategically located, such as a Carrier Grade NAT
   or ISP Gateway.  [I-D.ietf-ippm-lmap-path] gives many examples where
   a MA might be located within a network to provide an intermediate
   measurement point on the end-to-end path.  Other examples include a
   network device whose primary role is to host MA functions and the
   necessary measurement protocol.

> > > Comment: I thought we were also going to allow for the MA to just
> > > stop doing measurements if it didn't hear from the Controller in a
> while.
> >

[phil] ok. I have changed to say:
To avoid the MA generating
   traffic forever after a Controller has permanently failed, the MA can
   be configured with a time limit; if the MA doesn't hear from the
   Controller for this length of time, thenl it stops operating
   Measurement Tasks.

> > > unlike any use of "transport protocol" I've ever seen. Especially
> > > the part about there being Push, Pull, and multicast transport
> protocols.
> > > usually associate "transport protocol" with protocols that operate
> > > at the transport layer, like TCP and UDP, or with protocols like
> RTP
> > > (that has Transport Protocol in its name). But RTP, TCP, UDP and
> > > such have no Push, Pull, or multicast characteristics. I'm not sure
> > > what to recommend about this section -- parts seem usable, but the
> > > way it's described doesn't relate to my understanding of the nature
> > > of various protocols.
> >
> > Simply mean that (say) the LMAP Control Protocol is Controller to MA,
> > but for instance if it operates over http, then at this "underlying"
> > layer the MA does a request (pull) to the Controller and its reply
> > includes the LMAP Control Protocol info. Or the "underlying" layer
> may be push or even multicast.
> >
> <bhs> So what I read you saying is that there are some characteristics
> that the Control or Report Protocol can "inherit" from underlying
> protocols (effectively defined as any protocol at any layer that is
> below the Control or Report Protocol). This much I agree with, and I
> think it would be good to rephrase along this line.
> However, http does not necessarily imply "Pull". Just because 2
> endpoints are in a client/server relationship for a Control or Report
> Protocol, doesn't mean that they maintain that same client/server
> relationship as it relates to http. That is, it's possible for the
> Controller to be the http client that initiates a http session to the
> MA (which is the http server). It can be the higher layer Control or
> Report protocol that dictates which endpoint can or must be the http
> client/server. For example, TR-069 does allow the TR-069 config server
> to initiate http with the TR-069 client for the purpose of sending a
> "connect to your config server ASAP" message. But only the TR-069
> client can initiate a http session for purpose of receiving
> configuration info (in which case the TR-069 client is the http client
> and the TR-069 config server is the http server). And so I consider TR-
> 069 to be a "Pull" protocol. But it doesn't inherit this from http.
> Because either side can initiate http to the other at different times.
> It is TR-069 itself that defines the RPCs (remote procedure calls) and
> when different RPCs can be used.
> So I still have a problem with the idea that Push and Pull are
> necessarily inherited from underlying protocols, or that http implies
> "Pull".

In terms of how to change the text, the first 2 paras seem ok, but replace the last sentence with:
How this is done depends on the design of the Control and Report Protocols, the underlying packet transfer mechanism and how it is configured (for example, which endpoint is the HTTP client and which the server).

And then change to:
For the Control Protocol, the underlying packet transfer mechanism could be used in a:

   o  a 'push' mode (that is, from the Controller to the MA)

and similar change for the other bullets.

Does this work?

> > > Comment: Why not? Why isn't it allowed to include this ability in a
> > > MA design (such as one that is just for the ISP use case and has
> > > nothing to do with regulatory use case)?
> >
> <bhs> Multiple methods of remote configuration are indeed complicated,
> which was why I very much agreed with the "one Controller" restriction.
> But we have a lot of experience in coordinating between person-directed
> local configuration and remote automated configuration on various
> devices. Which is why I don't see it necessary to forbid this.

Would be interested to read more about this, do you have a ref please.

> > > keepalives has to do with UPnP. Why not just mention STUN as an
> > > option for keeping NAT table entries from timing out?
> >
> > [phil] "for the former" refers to "static port forwarding
> > configuration" and "for the latter" to "firewall pin holing"
> 
> <bhs> Hmm. I think we have terminology-understanding differences at
> multiple levels here. In my use of the words, a port forwarding rule or
> mapping is a pinhole. They can be statically (manually, TR-069, SNMP)
> or dynamically (UPnP IGD, PCP) configured, or dynamically generated as
> a result of outbound TCP and UDP packets.  Because pinholes configured
> through PCP and UPnP IGD do expire after a while, and are created on an
> "as needed" basis, they tend to be considered dynamic and not static.
> Pinholes dynamically generated by the NAT or firewall processes as a
> result of outbound TCP or UDP packets don't tend to be considered
> "configured". I would suggest rewording this to:
>   stateful-packet inspection firewall, or both.  In this case the
> Controller may not be able to  unilaterally contact the Measurement
> Agent unless configured or dynamically-generated pinholes exist.
>  Protocols such as PCP [RFC6887], CWMP [TR-069] or UPnP IGD  [UPnP IGD]
> can be used to configure pinholes.  For pinholes that are dynamically-
> generated as a result of outbound TCP or UDP packets,
>  send keepalives, such as those of the STUN protocol [STUN], towards
> the Controller to keep open the pinholes (and perhaps use these also as
> a network reachability test).
> 

Wfm. Thanks for the text.

> > > Comment: This is not necessarily true if it is possible for the MA
> > > not to be located in a place where it is associated with a single
> end user
> <bhs> As has been discussed on this list, Passive Measurements can also
> be conducted on Active Measurement traffic. I don't consider MA- or MP-
> generated Active Measurement traffic to be user traffic. For MAs
> located outside the customer premises, it is also possible for an ISP
> to measure management traffic (between its network elements and
> systems).

Good point.

Thanks
Phil.

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Tue, 29 Apr 2014 08:11:26 -0400

comments on the last point:

> From: lmap [mailto:[email-masked]] On Behalf Of
> [email-masked]
> Sent: Tuesday, April 29, 2014 6:31 AM
...
> > to measure management traffic (between its network elements and
When active measurement traffic is only observed (passively), we call that a
*hybrid* measurement, a combination of active and passive. In the IPPM
working group discussions (the registry, new metrics, etc.) we keep 
active and passive measurements at opposite ends of the continuum
and the various combinations which are between the ends are hybrid measurements.

We're talking about pure passive measurements here, and we've currently
clarified the text like this:

Passive Measurement Tasks are conducted on one or more user's traffic, 
such that sensitive information is present and stored in the 
measurement system (however briefly this storage may be).

hope this helps (but we may need to mention hybrid measurements here now,
like we added recently in the core registry draft).

Al

---

From: Charles Cook <[email-masked]>
Date: Tue, 29 Apr 2014 09:55:12 -0600

Phil,

Regarding prevention of the MA from continuously executing Measurement
Tasks when contact with the Controller has been lost, I believe there
needs to be a default time.  See inline below.

Charles

>    Controller for this length of time, thenl it stops operating
<cc> I don't think this solves the problem unless there is a default
time limit that is used if a time interval is not specifically configured.

Perhaps something like,

"To avoid the MA generating traffic forever after a Controller has permanently failed, the MA will stop operating Measurement Tasks if the MA does not hear from the Controller for a default <tbd> length of time.  Optionally, the MA can be configured with a specific time limit; if the MA doesn't hear from the Controller for this length of time, then it stops operating Measurement Tasks."

-- 

Charles Cook 
Principal Architect
Network
5325 Zuni Street; Suite 224
Denver, CO  80221
Tel:  303.992.8952  Fax:  925.281.0662
[email-masked]