Subject: [lmap] draft-ietf-lmap-framework-08: AD review - part 1

From: Benoit Claise <[email-masked]>
Date: Sat, 08 Nov 2014 21:43:02 -1000

Dear all,

Here is my AD review.
Note: I haven't reviewed yet the section 7 "security considerations" and 
section 8 "privacy considerations", but thought I should anyway send 
this before the meeting this week.
Note2: if some points were discussed already, don't hesitate to let me know.

A first impression is that there are some repetitions in the draft. It 
seems like some sections, written by different persons, were glued 
together. Reading this draft again with a fresh mind would highlight this.

One important issue, which I mentioned already during the interim 
meeting: the lack of passive monitoring aspects in the draft.

    Section 2

    The MA may
    generate Measurement Traffic and measure some metric associated with
    its transfer, or the MA may observe existing traffic, or there may be
    some kind of hybrid of these two possibilities.

So the document speaks a little bit about active and passive monitoring. 
Good, because this is mentioned in the charter:

    Both active and passive measurements are in scope, although there
    may be differences in their applicability to specific use cases, or
    in the security measures needed according to the threats specific to
    each measurement category. LMAP will not standardize performance
    metrics.

I recall the charter discussion. I specifically asked: "do you want to 
have passive monitoring in the charter? Do you understand all the 
consequences (for example, traffic filters, for observed packet/flow, 
which may imply RFC6728)?". Both answers were "yes".
So, the draft clearly misses the passive monitoring aspects, starting 
with the primary figure in this framework, figure 1, which only covers 
the active monitoring

Some other issues, simply listed in order
-

    These devices could be software based
    agents on PCs, embedded agents in consumer devices (e.g.  Blu-ray
    players),

Does a blu-ray player need a broadband test?

-
  o  Diversity - a measurement system should handle different types of
       Measurement Agents - for example Measurement Agents may come from
       different vendors, be in wired and wireless networks, be able to
       execute different sorts of Measurement Task and be on devices with
       IPv4 or IPv6 addresses.

Is "different types" the right term?
You have introduced the term "capabilities"

-

    It is also useful to define a registry
    for commonly-used Metrics [I-D.manyfolks-ippm-metric-registry  <http://tools.ietf.org/html/draft-ietf-lmap-framework-08#ref-I-D.manyfolks-ippm-metric-registry>] so
    that a Metric with its associated Measurement Method can be referred
    to simply by its identifier in the registry.

The new draft is http://tools.ietf.org/html/draft-ietf-ippm-metric-registry

-
OLD:

    Finally we introduce several components that are outside the scope of
    initial LMAP work and will be provided through existing protocols or
    applications.

NEW:

    Finally we introduce several components that are outside the scope of
    initial LMAP work (see in figure 1 "out of scope" and will be provided
    through existing protocols or applications.

Then I would list four bullet points for the next paragraph.
Otherwise it's not obvious which components you speak about

- figure 1
     * Since the above mentions the Results repository  (The data 
analysis tools receive the results from the Collector or via the Results 
repository. ), the figure should replace repository by Results repository
     * "IPPM scope" is wrong (see my first remark)
     * Mention the Control Protocol and Report Protocol, or Control 
Channel/Report Channel. Note: if you replace Instruction by Control 
Protocol, then the arrow must be bidirectional (according to the Control 
Protocol definition)

-

    Control Channel: A Channel between a Controller and a MA over which
    Instruction Messages and Capabilities, Failure and Logging
    Information are sent.

    Control Protocol: The protocol delivering Instruction(s) from a
    Controller to a Measurement Agent.  It also delivers Capabilities,
    Failure and Logging Information from the Measurement Agent to the
    Controller.  It can also be used to update the MA's Configuration.

Do you want to mention that the Control Protocol runs on the Control Channel?

-
    Measurement Method: The process for assessing the value of a Metric;
    the process of measuring some performance or reliability parameter
    associated with the transfer of traffic; where this process involves
    multiple MAs or MPs, each may perform different roles.

I don't understand what "where this process involves multiple MAs or MPs, each may perform different roles." adds to the def.

-
   Report Channel: A Channel between a Collector and a MA over which
    Report messages are sent.

Report messages -> Report Messages

-
4.2
  An operator may have several Controllers, perhaps with a Controller
    for different types of MA (home gateways, tablets) or location
    (Ipswich, Edinburgh).

BT specific, maybe? :-)

- Section 5.
Here is yet another example of repetition (see the last 3 bullets below. Why do you have to repeat what the Control Protocol does)

    An LMAP system goes through the following phases:

    o  a Bootstrapping process before the MA can take part in the other
       three phases

    o  a Control Protocol, which delivers Instruction Messages from a
       Controller to a MA, detailing what Measurement Tasks the MA should
       perform and when, and how it should report the Measurement
       Results.  It also delivers Capabilities, Failure and logging
       Information from a MA to its Controller.  Finally, it allows the
       Controller to update the MA's Configuration.

    o  the actual Measurement Tasks, which measure some performance or
       reliability parameter(s) associated with the transfer of packets.

       The LMAP work does not define Metrics and Measurement Methods,
       these are defined elsewhere (e.g.  IPPM).

    o  a Report Protocol, which delivers Reports from a MA to a
       Collector.  The Report contains the Measurement Results.

-
Would be nice to have "measurement system" in the terminology section.
It's mentioned so many times in the draft

-
5.2.  Control Protocol

    The primary purpose of the Control Protocol is to allow the
    Controller to configure a Measurement Agent with an Instruction about
    what Measurement Tasks to do, when to do them, and how to report the
    Measurement Results (Section 5.2.2).  The Measurement Agent then acts
    on the Instruction autonomously.  The Control Protocol also enables
    the MA to inform the Controller about its Capabilities and any
    Failure and Logging Information (Section 5.2.2).  Finally, the
    Control Protocol allows the Controller to update the MA's
    Configuration.

Another example of repetition.
 From the terminology:
    Control Protocol: The protocol delivering Instruction(s) from a
    Controller to a Measurement Agent.  It also delivers Capabilities,
    Failure and Logging Information from the Measurement Agent to the
    Controller.  It can also be used to update the MA's Configuration.

-
section 5.2.2
Instruction:                            ->
[(Measurement Task configuration(
    [Input Parameter],
    (interface),
    (Cycle-ID))),
  (Report Channel),
  (Schedule),
  (Suppression information)]
                                          <-          Response(details)

It seems from the above that "Measurement Task configuration" takes 3 arguments.
I'm surprised not to see the Metric (which is described in the section btw) above.
According to the Input Parameter def., I understand that the Metric is NOT part of the Input Parameter.

-
5.2.2.1.  Suppression

    The Instruction may include Suppression information.  The purpose of
    Suppression is to enable the Controller to instruct the MA not to
    perform Measurement Tasks.  It is used if the measurement system
    wants to eliminate inessential traffic, because there is some
    unexpected network issue for example.

Again, "eliminate inessential traffic". This is an active probing only view.
Same remark with
    o  a set of Measurement Schedules to suppress; the others are not
       suppressed.  For example, suppose the measurement system has
       defined two Schedules, one with the most critical Measurement
       Tasks and the other with less critical ones that create a lot of
       Measurement Traffic, then it may only want to suppress the second.

-
5.3.1.  Starting and Stopping Measurement Tasks

That's typical section completely dedicated to active probing:
    _Before sending Measurement Traffic_  the MA may run a pre-check.  (The
    pre-check could be defined as a separate, preceding Task or as the
    first part of a larger Task.)  Action could include:

    o_the MA checking that there is no cross-traffic_.  In other words, a
       check that the end-user isn't already sending traffic;

For passive monitoring, you typically want to check that there is traffic of interest...

-
5.4.1.  Reporting of Subscriber's service parameters

The charter says:
Service parameters, such as product category, can be useful to decide
which measurements to run and how to interpret the results. These
parameters are already gathered and stored by existing operations
systems.
Discovering the service parameters on the MAs or sharing
the service parameters between MAs are out of the scope. However, if the
  service parameters are available to the MAs, they could be reported
with the measurement results in the Report Protocol.

The key sentence is: "if the service parameters are available to the MAs"
I don't find back this concept in section 5.4.1

-
section 5.5

    It is also possible that a different choice is made for the
    Control and Report Protocols, for example NETCONF-YANG and IPFIX
    (Internet Protocol Flow Information Export) respectively.
  
Missing references

-
    For example, there
    could be different Controllers for different types of MA (home
    gateways, tablets) or locations (Ipswich, Edinburgh), for load
    balancing or to cope with failure of one Controller.

Another example of repetition.

-
section 6.1

    The measurement system also needs to consider carefully how to
    interpret missing Results; for example, if the missing Results are
    ignored and the lack of a Report is caused by its broadband being
    broken, then the estimate of overall performance, averaged across all
    MAs, would be too optimistic.  The correct interpretation may depend
    on the specifics of the Measurement Task and Measurement Schedule.

What do you mean by "missing Results"?
Missing from a measurement or reporting point of view?
And if it relates to measurement, it's completely different from a passive monitoring or active probing point of view.

- Appendix A:

One more example that there is some repetition, why do you need this:

    The LMAP framework defines two types of components involved in the
    actual measurement task, namely the Measurement Agent (MA) and the
    Measurement Peer (MP).  The fundamental difference conveyed in the
    definition of these terms is that the MA has a interface with the
    Controller/Collector while the MP does not.  The MP is broadly
    defined as a function that assists the MA in the Measurement Task but
    has no interface with the Controller/Collector.  There are many
    elements in the network that can fall into this broad definition of
    MP.  We believe that the MP terminology is useful to allow us to
    refer an element of the network that plays a role that is
    conceptually important to understand and describe the measurement
    task being performed.  We next illustrate these concepts by
    describing several deployment scenarios.

- Appendix Figure 2.

OLD:
       +----------------+    OWAMP     +----------------+    ^
       | OWAMP          |<--control--->| MP:            |    |
       | control-client |>test-traffic>| OWAMP server & |   IPPM

NEW:
       +----------------+    OWAMP     +----------------+    ^
       | OWAMP          |<--control--->| MP:            |    |
       | control-client |-test-traffic>| OWAMP server & |   IPPM

- Appendix: figure A4

    +-----+   +----------------+              +------+   ^
    | MP  |   |  MA: Monitor   |              | MP   | IPPM
    |     |<--|----------------|---traffic--->|      | Scope
    +-----+   |                |              +------+   |
       .......|................|.........................v...........

You mentioned in relation to this figure "Next, we consider Measurement Methods that measure user traffic."

So "IPPM scope" is wrong. This might be IPFIX/PSAMP

- Appendix: figure A4
I don't understand this sentence:

    if packets are not forwarded, the measurement task
    will not work.

- Also, related to the previous comment. What about a remote span to the 
probe? Is this in scope or not?

Regards, Benoit

---

From: Benoit Claise <[email-masked]>
Date: Sun, 09 Nov 2014 06:18:11 -1000

On 08/11/2014 21:43, Benoit Claise wrote:
> Dear all,
>
> Here is my AD review.
> Note: I haven't reviewed yet the section 7 "security considerations" 
> and section 8 "privacy considerations", but thought I should anyway 
> send this before the meeting this week.
For this part, I have requested an early review by the security directorate.

Regards, Benoit

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Sun, 9 Nov 2014 20:05:31 +0000

Thanks, Benoit!

Dear authors - please review and address this if possible  before the meeting on Thursday. At the meeting we should discuss the open issues.

Regards,

Dan

From: lmap [mailto:[email-masked]] On Behalf Of Benoit Claise
Sent: Sunday, November 09, 2014 9:43 AM
To: [email-masked]; [email-masked]
Cc: Paul Aitken; Aamer Akhter
Subject: [lmap] draft-ietf-lmap-framework-08: AD review - part 1

Dear all,

Here is my AD review.
Note: I haven't reviewed yet the section 7 "security considerations" and section 8 "privacy considerations", but thought I should anyway send this before the meeting this week.
Note2: if some points were discussed already, don't hesitate to let me know.

A first impression is that there are some repetitions in the draft. It seems like some sections, written by different persons, were glued together. Reading this draft again with a fresh mind would highlight this.

One important issue, which I mentioned already during the interim meeting: the lack of passive monitoring aspects in the draft.

   Section 2

   The MA may

So the document speaks a little bit about active and passive monitoring. Good, because this is mentioned in the charter:
Both active and passive measurements are in scope, although there may be differences in their applicability to specific use cases, or in the security measures needed according to the threats specific to each measurement category. LMAP will not standardize performance metrics.
I recall the charter discussion. I specifically asked: "do you want to have passive monitoring in the charter? Do you understand all the consequences (for example, traffic filters, for observed packet/flow, which may imply RFC6728)?". Both answers were "yes".
So, the draft clearly misses the passive monitoring aspects, starting with the primary figure in this framework, figure 1, which only covers the active monitoring

   players),

-

      IPv4 or IPv6 addresses.

-

   for commonly-used Metrics [I-D.manyfolks-ippm-metric-registry<http://tools.ietf.org/html/draft-ietf-lmap-framework-08#ref-I-D.manyfolks-ippm-metric-registry>] so

-
OLD:

   applications.
NEW:

- figure 1
    * Since the above mentions the Results repository  (The data analysis tools receive the results from the Collector or via the Results repository. ), the figure should replace repository by Results repository
    * "IPPM scope" is wrong (see my first remark)
    * Mention the Control Protocol and Report Protocol, or Control Channel/Report Channel. Note: if you replace Instruction by Control Protocol, then the arrow must be bidirectional (according to the Control Protocol definition)

-

   Information are sent.

-

-

-

4.2

   (Ipswich, Edinburgh).

BT specific, maybe? :-)

- Section 5.

      three phases

-

-

5.2.  Control Protocol

   Configuration.

-

section 5.2.2

Instruction:                            ->

   [Input Parameter],

   (interface),

   (Cycle-ID))),

 (Report Channel),

 (Schedule),

                                         <-          Response(details)

-

5.2.2.1.  Suppression

Same remark with

-

   Before sending Measurement Traffic the MA may run a pre-check.  (The

   o  the MA checking that there is no cross-traffic.  In other words, a

-

The charter says:

systems.

-

section 5.5

Missing references

-

   For example, there

-

section 6.1

- Appendix A:

- Appendix Figure 2.

OLD:

NEW:

- Appendix: figure A4

   +-----+   |                |              +------+   |

- Appendix: figure A4

   will not work.

- Also, related to the previous comment. What about a remote span to the probe? Is this in scope or not?

Regards, Benoit

---

From: <[email-masked]>
Date: Tue, 11 Nov 2014 16:51:10 +0000

Benoit,
Thanks very much for the review - some comments in line
Best wishes
phil

Sent: 09 November 2014 07:43

Dear all,

Here is my AD review.

[phil] ok will try to reduce. I think some redundancy is OK. for example, I think there should be some repetition between S1 & 2 (trying to provide an introduction, for people who want to read no further or haven't come across measurement stuff before); S3 (summarises terminology) and S5 (details the protocol model).

   Section 2

   The MA may

[phil] I'll mainly leave Paul & Aamer to handle this comment.
In terms of Figure 1, I can add a pointer to Appendix A4 which shows a passive monitoring case.
We can think about how to show passive monitoring in Fig 1 (not trivial - it doesn't show all active monitoring cases either, as doesn't show the scenario with a second MA instead of a M Peer).

   players),

[phil[ >tablets?

-

      IPv4 or IPv6 addresses.

[phil] don't mind. (The term capabilities hasn't yet been introduced)

-

-
OLD:

   applications.
NEW:

[phil] ok

- figure 1
    * Since the above mentions the Results repository  (The data analysis tools receive the results from the Collector or via the Results repository. ), the figure should replace repository by Results repository
    * "IPPM scope" is wrong (see my first remark)
[phil] sorry, I don't understand. Would you prefer something like "Scope of eg IPPM"?

    * Mention the Control Protocol and Report Protocol, or Control Channel/Report Channel. Note: if you replace Instruction by Control Protocol, then the arrow must be bidirectional (according to the Control Protocol definition)
[phil] Prefer to keep a mention of instruction and report.
Suggested change: to "Instruction (over Control Channel)" and "Report (over Report Channel)" and keep the arrows uni-directional.
-

   Information are sent.

-

[phil] ok.

-

-

4.2

   (Ipswich, Edinburgh).

BT specific, maybe? :-)

[phil] I'll add Diegem!

- Section 5.

Here is yet another example of repetition

[phil] ok, will shorten these (they're basically introducing the sub-sections of S5)

(see the last 3 bullets below. Why do you have to repeat what the Control Protocol does)

      three phases

-

[phil] with trepidation... since terminology always provokes excitement...

Measurement System: the set of LMAP-defined and related components that are operated by a single organisation, for the purpose of measuring performance aspects of the network.

-

5.2.  Control Protocol

   Configuration.

[phil] personally I think this repetition is ok. Section 5 is providing the protocol model, so needs to describe what the control protocol does.

-

section 5.2.2

Instruction:                            ->

   [Input Parameter],

   (interface),

   (Cycle-ID))),

 (Report Channel),

 (Schedule),

                                         <-          Response(details)

[phil] good spot, thanks.

-

5.2.2.1.  Suppression

[phil] does adding something like this solve the issue?

"or inessential processing by a networking component"

Same remark with

[phil] Paul, Aamer - please suggest some text - don't know what's appropriate for passive tasks

-

[phil] ok. will expand the list.

-

The charter says:

systems.

[phil] I think S5.4.1 does mention this:
The information could be transferred directly from the Subscriber
   parameter database to the data analysis tools.  It may also be
   possible to transfer the information via the MA.  How (and if) the MA
   knows such information is likely to depend on the device type.  The
   MA could either include the information in a Measurement Report or
   separately.

-

section 5.5

Missing references

-

   For example, there

[phil] ok, will delete this paragraph

-

section 6.1

[phil] How about:

   interpret missing Results. The correct interpretation depends

   on why the Results are missing, and potentially on the specifics of the Measurement Task and Measurement Schedule.

- Appendix A:

[phil] ok. will delete.

- Appendix Figure 2.

OLD:

NEW:

- Appendix: figure A4

   +-----+   |                |              +------+   |

You mentioned in relation to this figure "Next, we consider Measurement Methods that measure user traffic."
So "IPPM scope" is wrong. This might be IPFIX/PSAMP
[phil] ok, will replace with "IPFX/PSAMP/IPPM Scope"

- Appendix: figure A4

   will not work.

- Also, related to the previous comment. What about a remote span to the probe? Is this in scope or not?
[phil] I'd be ok to delete the whole of the last paragraph.

Regards, Benoit

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 11 Nov 2014 18:35:41 +0100

> Does a blu-ray player need a broadband test?

I suggest "TV" since TVs are consumer real devices that are morphing
into Internet video playing engines. And yes, since I mention it,
"gaming consoles" can perhaps be mentioned as well. Gamers really care
about latency. So my proposal would be to replace the parenthesis with
'(e.g., TVs or gaming consoles)'.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>