Subject: [lmap] Overlapping Measurement Tasks (WGLC comments on draft-ietf-lmap-framework-03)

From: <[email-masked]>
Date: Mon, 17 Feb 2014 11:48:31 +0000

There were a couple of comments about overlapping measurement tasks.

draft-ietf-lmap-framework-03 says

<<   {Comment: It is possible that the set of measurement schedules
   implies overlapping Measurement Tasks.  It is not clear the best
   thing to do.  Our current suggestion is to leave this to the protocol
   document.}

I think we should say something in the framework doc - and not just punt this to the protocol doc (and in retrospect I'm not sure that's a good place).

I still think it's not clear what the best thing to do is. As Charles says, one approach is to run the first Task to start, then run the next. But one can imagine tasks could get delayed - say a task was defined to "wait" if it detected cross-user traffic, and try again a minute later. A task might be written to re-try many times. Should you try and run the next task in the meantime? Should you allow multiple tasks to run at once? How many tasks do you allow all to be waiting for others to complete? Are the considerations different if the Task is defined so that Measurement Peer sends traffic towards the MA? These sorts of things seem quite complex and need accurate knowledge of the characteristics of the task.

Here's a proposal to say something meaningful but simple about overlapping measurement tasks.

--

In principle there are several reasons why a MA might have two Tasks that it are supposed to run at the same time:
*           The operator of the measurement system has unintentionally created a set of Schedules with overlapping Tasks
*           The completion of one Task is delayed beyond when another Task is due to start (for example the Task may be designed to wait for a lull in end-user traffic, or the Task may be of uncertain duration)

In the initial stage of LMAP, overlapping Tasks will not be considered due to their complexity.

The above issues are addressed as follows
*           The operator of the measurement system should design their set of Schedules with sufficient gaps between Tasks, so that one Task will comfortably complete before another begins
*           It is suggested that Tasks are never delayed - they either run at the scheduled time or they don't
*           In line with this, if a (second) Task is due to start when a (first) Task is still on-going, then the (second) Task is never started
*           A Task may delay itself internally (within the definition of the Task); this is invisible to the Schedule. However, caution is advised as this may prevent other Tasks from executing
*           Rather than running a Passive Measurement Task continuously in the background, instead it is run when there are no other Tasks operating
*           Rather than testing for the impact of cross-traffic by having two Tasks that run simultaneously, instead a single Task generates two types of traffic.

More complex possibilities could be explored in LMAP2.0.

As a general principle, if Tasks (especially Active ones) do overlap, this should be noted in the Report, so that the Result can be treated with caution.

---
what do you reckon?

Thanks
phil

From: Charles Cook [mailto:[email-masked]]
Sent: 05 February 2014 17:41
To: Eardley,PL,Philip,TUB8 R
Cc: [email-masked]
Subject: Re: [lmap] FW: New Version Notification for draft-ietf-lmap-framework-03.txt

Phil,

I like the improvements over the previous version.  I did not have many additional comments on this draft.  Attached is my marked up review.

The main comments I had are as follows In summary:

 *   Conflict Resolution:   If two or more Measurement Tasks overlap in their schedule, how should the MA behave?  If two or more Measurement Tasks are scheduled to start at the same time, the first one listed in the set of Measurement Schedules can be run first.  When complete, the conflicting Measurement Task can be run.  If the Measurement Schedules overlap, but are not scheduled to start at the same time, run the one scheduled to start first to completion, and then run the overlapping Measurement Schedule.  The Results Report should contain a start and stop time stamp that can be used in post processing.  Note:  Some Measurement Tasks may be able to run in parallel without impacting the Results of other Measurement Tasks.  In this case, scheduling conflicts are not an issue.  But these will need to be will defined ahead of time.
 *   MA Performing Pre-processing:  I don't think that the MA should perform pre-processing unless specifically instructed to do so in a Measurement Method.  The MA should provide enough detail in the Measurement Result to enable post processing.  Any Measurement Methods with pre-processing will need to be very carefully constructed to ensure that the results are repeatable/comparable.
 *   MP Registration and Suppression:  MPs also need to Register with the Controller so that the Controller knows they exist.  The Controller will also need to know the capabilities of the MP, and also have the ability to Suppress/Unsuppress the MP as needed.  I didn't see this in the draft.  We just need to add it.
 *   Underlying Transport Protocol:  Push and/or Pull:  The LMAP Framework identifies Push and Pull as possible underlying transport protocols, but does not specify a requirement.  A discussion is needed to determine if one or both of these are required for the Framework.  It sounds like Push is preferred in general, but Pull may be needed in order to perform a Measurement Task when the MA or MP is behind a NAT.  Additional details are needed.  For example should a Controller, in addition to Pushing out Instructions, provide a mechanism for an MA to Pull an Instruction if the MA is behind a NAT?  If a MA or MP is behind a NAT, should it execute a keep-alive so that the Controller knows where it is at?

Great work.

Charles

On 1/21/2014 8:00 AM, [email-masked]<mailto:[email-masked]> wrote:

We submitted a new version of the framework document.

http://tools.ietf.org/html/draft-ietf-lmap-framework

Here's a summary of the changes:

   o  alignment with the Information Model

      [I-D.burbridge-lmap-information-model] as this is agreed as a WG

      document

   o  One-off and periodic Measurement Schedules are kept separate, so

      that they can be updated independently

   o  Measurement Suppression in a separate sub-section.  Can now

      optionally include particular Measurement Tasks &/or Schedules to

      suppress, and start/stop time

   o  for clarity, concept of Channel split into Control, Report and MA-

      to-Controller Channels

   o  numerous editorial changes, mainly arising from a very detailed

      review by Charles Cook

best wishes,

phil, al, marcelo, trevor, paul, aamer

_______________________________________________

lmap mailing list

[email-masked]<mailto:[email-masked]>

https://www.ietf.org/mailman/listinfo/lmap

--

Charles Cook

Principal Architect

Network

5325 Zuni Street; Suite 224

Denver, CO  80221

Tel:  303.992.8952  Fax:  925.281.0662

[email-masked]<mailto:[email-masked]>

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Mon, 17 Feb 2014 13:57:25 +0000

*           Rather than running a Passive Measurement Task continuously in the background, instead it is run when there are no other Tasks operating
<bhs> I think this one item in the proposal is problematic and I don't see a need for it. Lots of devices are continually running passive measurements, and I don't want them to have to turn these off to start an active measurement. I'm also completely at a loss as to what the harm is. For example, in an RG, ADSL line statistics are always being captured, as are Ethernet frames in/out and IP packets in/out. The firewall is logging various messages it sees (and I really don't want to turn that off in order to run an active measurement). There has also been the suggestion that the "responding" end of an active measurement task can simultaneously do passive measurements on the traffic of that active measurement task. IMO, these are 2 separate tasks (though, given the next bullet, we may not have a common definition of Measurement Task, either). Anyway, unless the device is so incredibly processor- or memory-limited that it can't collect passive measurements while doing active measurements (a serious design flaw, IMO, which would make me question this device's ability to even accurately do a single active measurement task), I see no reason to disable any passive measurements while other measurements are occurring. Multiple passive measurements can also run simultaneously. Oh -- and measuring internal processor- and memory-utilization during the running of measurement tasks is also a passive measurement task that I wouldn't want to see disabled.

*           Rather than testing for the impact of cross-traffic by having two Tasks that run simultaneously, instead a single Task generates two types of traffic.
<bhs> This is the point that makes me wonder if we're using the same definition of "Task". Since Measurement Task is an instantiation of a Measurement Method, this would imply that the Measurement Method would have to be defined in a way that allows it to generate all of the needed types of traffic. But the Measurement Methods (and the Registry of Measurement Methods) are being defined external to lmap. This assumption would have the lmap framework imposing requirements on Measurement Methods and the associated Registry in order to run certain tests. That makes me very uneasy.
I'm also not understanding the proposed cross-traffic test, but maybe that's beside the point. I thought cross-traffic was something where we wanted to know whether or not there was any cross traffic present prior to starting certain Active Measurement Tasks, or during an Active Measurement Task (which BTW would require something to do some passive measuring while the Active Measurement Task was happening). Understanding what sort of impact one type of traffic has on the simultaneous running of another type of traffic, from the same MA, strikes me as a rather complex thing to measure, and I'm not sure I want to do much design around that, right now.

---

From: <[email-masked]>
Date: Mon, 17 Feb 2014 14:37:01 +0000

In-line
Thanks
phil

From: STARK, BARBARA H [mailto:[email-masked]]
Sent: 17 February 2014 13:57
To: Eardley,PL,Philip,TUB8 R; [email-masked]
Subject: RE: Overlapping Measurement Tasks (WGLC comments on draft-ietf-lmap-framework-03)

[phil] should suppression by default impact all Tasks or just Active ones? The argument for the former is that it's simpler - it avoids having to classify Tasks as active or passive - and for the MA having to remember this info. It avoids getting into discussion about whether the passive tasks are generating a lot of Reports (should they be suppressed), what to do if an active task has been mis-classified as a passive task, what happens if the operator wants to suppress tasks for reasons other than a network overload (perhaps some privacy attack), can we treat an active task that only generates 1 or 2 packets as (almost) passive.

I have sympathy with your viewpoint (in fact, it was my viewpoint earlier). At the moment, given suppression is only a rare thing, I've been persuaded it's better to go for simplicity and make suppression affect all Tasks by default. (of course, one can use the suppression option to list specific Tasks that are suppressed.)
*           Rather than testing for the impact of cross-traffic by having two Tasks that run simultaneously, instead a single Task generates two types of traffic.
<bhs> This is the point that makes me wonder if we're using the same definition of "Task". Since Measurement Task is an instantiation of a Measurement Method, this would imply that the Measurement Method would have to be defined in a way that allows it to generate all of the needed types of traffic. But the Measurement Methods (and the Registry of Measurement Methods) are being defined external to lmap. This assumption would have the lmap framework imposing requirements on Measurement Methods and the associated Registry in order to run certain tests. That makes me very uneasy.
I'm also not understanding the proposed cross-traffic test, but maybe that's beside the point. I thought cross-traffic was something where we wanted to know whether or not there was any cross traffic present prior to starting certain Active Measurement Tasks, or during an Active Measurement Task (which BTW would require something to do some passive measuring while the Active Measurement Task was happening). Understanding what sort of impact one type of traffic has on the simultaneous running of another type of traffic, from the same MA, strikes me as a rather complex thing to measure, and I'm not sure I want to do much design around that, right now.

[phil] I meant something on the lines of your last sentence: understanding the impact of one sort of traffic on another sort. I agree this is quite complex - some people have mentioned a test on these lines - so I was just pointing out an implication for the proposed way of handling overlapping Tasks.

---

From: <[email-masked]>
Date: Mon, 17 Feb 2014 14:41:03 +0000

Sorry please ignore, I will re-write shortly without the confusing reference to suppression!
(am currently preparing an email about suppression - too many overlapping tasks in my brain!!)

From: Eardley,PL,Philip,TUB8 R
Sent: 17 February 2014 14:37
To: 'STARK, BARBARA H'; [email-masked]
Subject: RE: Overlapping Measurement Tasks (WGLC comments on draft-ietf-lmap-framework-03)

In-line
Thanks
phil

From: STARK, BARBARA H [mailto:[email-masked]]
Sent: 17 February 2014 13:57
To: Eardley,PL,Philip,TUB8 R; [email-masked]<mailto:[email-masked]>
Subject: RE: Overlapping Measurement Tasks (WGLC comments on draft-ietf-lmap-framework-03)

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 17 Feb 2014 16:09:37 +0100

> I still think it's not clear what the best thing to do is. As Charles says, one approach is to run the first Task to start, then run the next. But one can imagine tasks could get delayed - say a ta…

I do not think that overlapping tasks are necessarily evil. It really
depends on the nature of the measurement tasks whether this is a
problem or not and it even depends on the purpose of the measurement
(e.g., I might run a bulk download measurement just to generate lots
of background traffic for something else I am interested to study
under load).  I think we should refrain from hardcoding assumptions
about the nature of measurement tasks into the LMAP framework or
information model or protocol. We should separate policy from the
mechanism.

If we want to tackle this problem, we should allow people to express
which sets of tasks may not overlap and the action to take if this
constrained is violated (delay starting, do not start at all, kill the
running task and start, start anyway but report the issue, ...). I
actually think it is not too hard to provide the mechanisms such that
suitable policies can be configured. But all this comes with a cost of
course (but the added robustness may be worth it - I have my own set
of experience of cron jobs going wild in certain situations).

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Charles Cook <[email-masked]>
Date: Mon, 17 Feb 2014 08:42:36 -0700

Phil,

See inline <cc>

Charles

On 2/17/2014 4:48 AM, [email-masked] wrote:
> I think we should say something in the framework doc -- and not just
> punt this to the protocol doc (and in retrospect I'm not sure that's a
> good place).
>
>  
>
> I still think it's not clear what the best thing to do is. As Charles
> says, one approach is to run the first Task to start, then run the
> next. But one can imagine tasks could get delayed -- say a task was
> defined to "wait" if it detected cross-user traffic, and try again a
> minute later. A task might be written to re-try many times. Should you
> try and run the next task in the meantime? Should you allow multiple
> tasks to run at once? How many tasks do you allow all to be waiting
> for others to complete? Are the considerations different if the Task
> is defined so that Measurement Peer sends traffic towards the MA?
> These sorts of things seem quite complex and need accurate knowledge
> of the characteristics of the task.
>
>  
>
> In principle there are several reasons why a MA might have two Tasks
> that it are supposed to run at the same time:
>
> .           The operator of the measurement system has unintentionally
> created a set of Schedules with overlapping Tasks
>
> .           The completion of one Task is delayed beyond when another
> Task is due to start (for example the Task may be designed to wait for
> a lull in end-user traffic, or the Task may be of uncertain duration)
>
>  
>
> .           The operator of the measurement system should design their
> set of Schedules with sufficient gaps between Tasks, so that one Task
> will comfortably complete before another begins
>
> .           It is suggested that Tasks are never delayed -- they
> either run at the scheduled time or they don't
>
> .           In line with this, if a (second) Task is due to start when
> a (first) Task is still on-going, then the (second) Task is never started
>
> .           A Task may delay itself internally (within the definition
> of the Task); this is invisible to the Schedule. However, caution is
> advised as this may prevent other Tasks from executing
>
> .           Rather than running a Passive Measurement Task
> continuously in the background, instead it is run when there are no
> other Tasks operating
>
<cc> I am comfortable with no overlapping active Measurement Tasks, and
if two active Measurement Tasks overlap, the second Measurement Task is
not performed.  I need to think more about whether active Measurement
Tasks should be handled differently from passive Measurement Tasks. 
>
> .           Rather than testing for the impact of cross-traffic by
> having two Tasks that run simultaneously, instead a single Task
> generates two types of traffic.
>
<cc>  Maybe this needs to be discussed separately.  It appears to impact
what the general format of what a Measurement Method will look like. 
Perhaps this should be deferred to a later version of the LMAP Framework.
> As a general principle, if Tasks (especially Active ones) do overlap,
> this should be noted in the Report, so that the Result can be treated
> with caution.
>
<cc> Agree.
>
>  
>
> ---
>
> what do you reckon?
>
>  
>
> Thanks
>
> phil
>
>  
>
> *From:*Charles Cook [mailto:[email-masked]]
> *Sent:* 05 February 2014 17:41
> *To:* Eardley,PL,Philip,TUB8 R
> *Cc:* [email-masked]
> *Subject:* Re: [lmap] FW: New Version Notification for
> draft-ietf-lmap-framework-03.txt
>
>  
>
> Phil,
>
> I like the improvements over the previous version.  I did not have
> many additional comments on this draft.  Attached is my marked up
> review. 
>
>   * *Conflict Resolution: *  If two or more Measurement Tasks overlap
>     in their schedule, how should the MA behave?  If two or more
>     Measurement Tasks are scheduled to start at the same time, the
>     first one listed in the set of Measurement Schedules can be run
>     first.  When complete, the conflicting Measurement Task can be
>     run.  If the Measurement Schedules overlap, but are not scheduled
>     to start at the same time, run the one scheduled to start first to
>     completion, and then run the overlapping Measurement Schedule. 
>     The Results Report should contain a start and stop time stamp that
>     can be used in post processing.  Note:  Some Measurement Tasks may
>     be able to run in parallel without impacting the Results of other
>     Measurement Tasks.  In this case, scheduling conflicts are not an
>     issue.  But these will need to be will defined ahead of time. 
>   * *MA Performing Pre-processing:*  I don't think that the MA should
>     perform pre-processing unless specifically instructed to do so in
>     a Measurement Method.  The MA should provide enough detail in the
>     Measurement Result to enable post processing.  Any Measurement
>     Methods with pre-processing will need to be very carefully
>     constructed to ensure that the results are repeatable/comparable.
>   * *MP Registration and Suppression:*  MPs also need to Register with
>     the Controller so that the Controller knows they exist.  The
>     Controller will also need to know the capabilities of the MP, and
>     also have the ability to Suppress/Unsuppress the MP as needed.  I
>     didn't see this in the draft.  We just need to add it.
>   * *Underlying Transport Protocol:*  Push and/or Pull:  The LMAP
>     Framework identifies Push and Pull as possible underlying
>     transport protocols, but does not specify a requirement.  A
>     discussion is needed to determine if one or both of these are
>     required for the Framework.  It sounds like Push is preferred in
>     general, but Pull may be needed in order to perform a Measurement
>     Task when the MA or MP is behind a NAT.  Additional details are
>     needed.  For example should a Controller, in addition to Pushing
>     out Instructions, provide a mechanism for an MA to Pull an
>     Instruction if the MA is behind a NAT?  If a MA or MP is behind a
>     NAT, should it execute a keep-alive so that the Controller knows
>     where it is at?
>
>
> Great work.
>
> Charles
>
>     [email-masked] <mailto:[email-masked]>
>
> [email-masked] <mailto:[email-masked]>
>
>
> [email-masked]

-- 

Charles Cook 
Principal Architect
Network
5325 Zuni Street; Suite 224
Denver, CO  80221
Tel:  303.992.8952  Fax:  925.281.0662
[email-masked]

---

From: <[email-masked]>
Date: Tue, 18 Feb 2014 08:48:01 +0000

Barbara,
Thanks for the comments.

With these two bullets I was pointing out the implications of the proposed way of not allowing overlapping Tasks (don't start, and never start, a (second) Task if a (first) Task is still on-going).

On your second comment below, I meant something on the lines of your last sentence: understanding the impact of one sort of traffic on another sort. I was just saying that such a test could still be achieved, but would have to be defined as a single test rather than two. I agree this is quite complex - but some people have mentioned a test on these lines

On your first bullet, I have some sympathy with this. It is a technical impact.
Counter arguments are that it's not necessarily clear cut what's a passive and what's an active Task, and that it adds some complexity to have to classify all tasks as active or passive and to tell the MA this. An active task misconfigured as passive is nasty.

Possible approaches are:

-       Leave as original proposal. No overlapping Tasks, therefore passive have to be run at a separate time to active tasks

-       Modify original proposal, so the MA has a list of Tasks and runs through them one a time. Doesn't affect the 2 bullets you mention

-       Allow MA to do what it wants, ie start overlapping Tasks (but record this in the Results). Solves your 2 bullets, but also can have multiple overlapping active tasks

-       Distinguish active and passive tasks. Keep original proposal for active tasks; passive tasks can be run simultaneously.

phil

---

From: "Sharam Hakimi" <[email-masked]>
Date: Tue, 18 Feb 2014 08:45:04 -0500

Phil,

One thing that is missing is whether "suppress" is stateful. I believe
that it should be. One would not want to power cycle a device
(intentional or power outage) and have the device return to executing
measurement tasks.

I suggest adding another bullet that :

*         A device in suppression must enter suppression mode after a
power cycle.  

Cheers,

Sharam

From: lmap [mailto:[email-masked]] On Behalf Of
[email-masked]
Sent: Tuesday, February 18, 2014 3:48 AM
To: [email-masked]; [email-masked]
Subject: Re: [lmap] Overlapping Measurement Tasks (WGLC comments on
draft-ietf-lmap-framework-03)

Barbara,

Thanks for the comments.

With these two bullets I was pointing out the implications of the
proposed way of not allowing overlapping Tasks (don't start, and never
start, a (second) Task if a (first) Task is still on-going). 

On your second comment below, I meant something on the lines of your
last sentence: understanding the impact of one sort of traffic on
another sort. I was just saying that such a test could still be
achieved, but would have to be defined as a single test rather than two.
I agree this is quite complex - but some people have mentioned a test on
these lines 

On your first bullet, I have some sympathy with this. It is a technical
impact.

Counter arguments are that it's not necessarily clear cut what's a
passive and what's an active Task, and that it adds some complexity to
have to classify all tasks as active or passive and to tell the MA this.
An active task misconfigured as passive is nasty.

Possible approaches are:

-       Leave as original proposal. No overlapping Tasks, therefore
passive have to be run at a separate time to active tasks

-       Modify original proposal, so the MA has a list of Tasks and runs
through them one a time. Doesn't affect the 2 bullets you mention

-       Allow MA to do what it wants, ie start overlapping Tasks (but
record this in the Results). Solves your 2 bullets, but also can have
multiple overlapping active tasks

-       Distinguish active and passive tasks. Keep original proposal for
active tasks; passive tasks can be run simultaneously.

phil

From: STARK, BARBARA H [mailto:[email-masked]] 
Sent: 17 February 2014 13:57
To: Eardley,PL,Philip,TUB8 R; [email-masked]
Subject: RE: Overlapping Measurement Tasks (WGLC comments on
draft-ietf-lmap-framework-03)

*           Rather than running a Passive Measurement Task continuously
in the background, instead it is run when there are no other Tasks
operating

<bhs> I think this one item in the proposal is problematic and I don't
see a need for it. Lots of devices are continually running passive
measurements, and I don't want them to have to turn these off to start
an active measurement. I'm also completely at a loss as to what the harm
is. For example, in an RG, ADSL line statistics are always being
captured, as are Ethernet frames in/out and IP packets in/out. The
firewall is logging various messages it sees (and I really don't want to
turn that off in order to run an active measurement). There has also
been the suggestion that the "responding" end of an active measurement
task can simultaneously do passive measurements on the traffic of that
active measurement task. IMO, these are 2 separate tasks (though, given
the next bullet, we may not have a common definition of Measurement
Task, either). Anyway, unless the device is so incredibly processor- or
memory-limited that it can't collect passive measurements while doing
active measurements (a serious design flaw, IMO, which would make me
question this device's ability to even accurately do a single active
measurement task), I see no reason to disable any passive measurements
while other measurements are occurring. Multiple passive measurements
can also run simultaneously. Oh -- and measuring internal processor- and
memory-utilization during the running of measurement tasks is also a
passive measurement task that I wouldn't want to see disabled.

*           Rather than testing for the impact of cross-traffic by
having two Tasks that run simultaneously, instead a single Task
generates two types of traffic.

<bhs> This is the point that makes me wonder if we're using the same
definition of "Task". Since Measurement Task is an instantiation of a
Measurement Method, this would imply that the Measurement Method would
have to be defined in a way that allows it to generate all of the needed
types of traffic. But the Measurement Methods (and the Registry of
Measurement Methods) are being defined external to lmap. This assumption
would have the lmap framework imposing requirements on Measurement
Methods and the associated Registry in order to run certain tests. That
makes me very uneasy.

I'm also not understanding the proposed cross-traffic test, but maybe
that's beside the point. I thought cross-traffic was something where we
wanted to know whether or not there was any cross traffic present prior
to starting certain Active Measurement Tasks, or during an Active
Measurement Task (which BTW would require something to do some passive
measuring while the Active Measurement Task was happening).
Understanding what sort of impact one type of traffic has on the
simultaneous running of another type of traffic, from the same MA,
strikes me as a rather complex thing to measure, and I'm not sure I want
to do much design around that, right now.

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Tue, 18 Feb 2014 14:03:10 +0000

Thanks Phil.
Personally, I prefer Option 3 (3rd dashed item) in your list:
" -           Allow MA to do what it wants, ie start overlapping Tasks (but record this in the Results). Solves your 2 bullets, but also can have multiple overlapping active tasks"

Not all MAs are for regulatory use cases, and different use cases may have different tolerances regarding accuracy and precision. And as I mentioned in a previous email, tasks that use minimal resources and are not attempting to actively measure total resource availability can provide reasonably accurate results during times when the device is not resource-constrained -- independent of what other tasks (or non-measurement related functions) may be using those same resources. It also makes no sense to restrict what a single MA can do simultaneously when other functions inside the device can be doing all sorts of things at the same time. Including utilization measures of various device resources with Results is to me far preferable.

We need to remember that this framework is not supposed to cost an arm and a leg to implement. It is not intended to provide perfect measurements. It is intended to provide "good enough" measurements. What is "good enough" will vary per use case, country/jurisdiction, entity procuring/designing the MAs, etc.
Barbara

From: [email-masked] [mailto:[email-masked]]
Sent: Tuesday, February 18, 2014 3:48 AM
To: STARK, BARBARA H; [email-masked]
Subject: RE: Overlapping Measurement Tasks (WGLC comments on draft-ietf-lmap-framework-03)

Barbara,
Thanks for the comments.

Possible approaches are:

phil

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Tue, 18 Feb 2014 15:44:49 +0000

This suggestion is attempting to impose requirements on the device where the MA is. I think that's problematic. But perhaps you meant MA instead of device?

If survivability of suppression is so important, I'm not sure this is the best way to ensure it. I could envision a MA that is factory-configured to run a set of basic measurement tasks. If reset to factory configuration, it will go back to doing those (and suppression will be lost) in the absence of updated configuration (Instructions) from the Controller. I could also envision a case where the suppression message was sent while a MA was powered down.

On the other hand, I would expect that one of the very first things a MA does on boot is contact its Controller. The MA should never assume that its configuration is current after a power cycle. We've said (I think) that if a MA can't talk to its Controller, it shouldn't run measurements. So the MA shouldn't be starting up measurement tasks before that's been successful. I would suggest that rather than rely on an MA's ability to remember suppression, a Controller would be expected to transmit any currently active suppression whenever an MA contacts it. If a Controller doesn't indicate active suppression to a MA at any time when it is communicating with the MA, the MA can assume it is not suppressed.
Barbara

From: lmap [mailto:[email-masked]] On Behalf Of Sharam Hakimi
Sent: Tuesday, February 18, 2014 8:45 AM
To: [email-masked]; STARK, BARBARA H; [email-masked]
Subject: Re: [lmap] Overlapping Measurement Tasks (WGLC comments on draft-ietf-lmap-framework-03)

Phil,
One thing that is missing is whether "suppress" is stateful. I believe that it should be. One would not want to power cycle a device (intentional or power outage) and have the device return to executing measurement tasks.

*         A device in suppression must enter suppression mode after a power cycle.

Cheers,
Sharam

From: lmap [mailto:[email-masked]] On Behalf Of [email-masked]<mailto:[email-masked]>
Sent: Tuesday, February 18, 2014 3:48 AM
To: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: Re: [lmap] Overlapping Measurement Tasks (WGLC comments on draft-ietf-lmap-framework-03)

Barbara,
Thanks for the comments.

Possible approaches are:

phil

---

From: "Sharam Hakimi" <[email-masked]>
Date: Tue, 18 Feb 2014 13:09:35 -0500

My intention was the MA inside a device, but a device is reset not he MA
so it is a catch 22. The reason I am suggesting to have suppression to
be stateful is for the that you mentioned. If a MA in a device is pre
configured and will power-up and starts running tests without having to
check with the MC, that would be a problem. If a device, pre configured
or not, must check with the MC before it starts running tests, then the
MC could either send a 'no task' to the MA or send a suppression message
to the MA during a suppression period.

Sharam 

From: STARK, BARBARA H [mailto:[email-masked]] 
Sent: Tuesday, February 18, 2014 10:45 AM
To: Sharam Hakimi; [email-masked]; [email-masked]
Subject: RE: [lmap] Overlapping Measurement Tasks (WGLC comments on
draft-ietf-lmap-framework-03)

This suggestion is attempting to impose requirements on the device where
the MA is. I think that's problematic. But perhaps you meant MA instead
of device?

If survivability of suppression is so important, I'm not sure this is
the best way to ensure it. I could envision a MA that is
factory-configured to run a set of basic measurement tasks. If reset to
factory configuration, it will go back to doing those (and suppression
will be lost) in the absence of updated configuration (Instructions)
from the Controller. I could also envision a case where the suppression
message was sent while a MA was powered down.

On the other hand, I would expect that one of the very first things a MA
does on boot is contact its Controller. The MA should never assume that
its configuration is current after a power cycle. We've said (I think)
that if a MA can't talk to its Controller, it shouldn't run
measurements. So the MA shouldn't be starting up measurement tasks
before that's been successful. I would suggest that rather than rely on
an MA's ability to remember suppression, a Controller would be expected
to transmit any currently active suppression whenever an MA contacts it.
If a Controller doesn't indicate active suppression to a MA at any time
when it is communicating with the MA, the MA can assume it is not
suppressed. 

Barbara

Phil,

Cheers,

Sharam

Barbara,

Thanks for the comments.

Possible approaches are:

phil

---

From: <[email-masked]>
Date: Tue, 18 Feb 2014 18:22:20 +0000

This sounds sensible to me ie after a re-boot ask Controller to re-send Instruction. I know there's a bit about re-boot in the framework, will check what it says and update

Thanks
phil

From: STARK, BARBARA H [mailto:[email-masked]]
Sent: 18 February 2014 15:45
To: Sharam Hakimi; Eardley,PL,Philip,TUB8 R; [email-masked]
Subject: RE: [lmap] Overlapping Measurement Tasks (WGLC comments on draft-ietf-lmap-framework-03)

From: lmap [mailto:[email-masked]] On Behalf Of Sharam Hakimi
Sent: Tuesday, February 18, 2014 8:45 AM
To: [email-masked]<mailto:[email-masked]>; STARK, BARBARA H; [email-masked]<mailto:[email-masked]>
Subject: Re: [lmap] Overlapping Measurement Tasks (WGLC comments on draft-ietf-lmap-framework-03)

Phil,

Cheers,
Sharam

Barbara,
Thanks for the comments.

Possible approaches are:

phil