Subject: Re: [lmap] WGLC for http://www.ietf.org/id/draft-ietf-lmap-framework-03.txt

From: KEN KO <[email-masked]>
Date: Thu, 6 Feb 2014 21:36:02 +0000

Many thanks to the authors for this draft. It expresses many of the concepts for lmap clearly and concisely. Here are my comments for WGLC. They are lengthy because I am trying to include my reasoning, not because there are a lot of issues.

P.5, starting with 2nd para. Of Section 2.  Suggest that Measurement Method is described first so that we move from the general to the specific (Measurement Task). Suggested text:

   A Measurement Method defines how to measure a Metric of interest. It

   is very useful to standardise Measurement Methods, so that it is

   meaningful to compare measurements of the same Metric made at

   different times and places.  It is also useful to define a registry

   for commonly-used Metrics [I-D.bagnulo-ippm-new-registry-independent]

   so that a Measurement Method can be referred to simply by its

   identifier in the registry.  The Measurement Methods and registry

   will hopefully be referenced by other standards organisations.

   A Measurement Task is a specific instantiation of a Measurement Method

   which generates a Measurement Result.  An Active Measurement Task

   involves either a Measurement Agent (MA) injecting Active Measurement

   Traffic into the network destined for a Measurement Peer, and/or a

   Measurement Peer sending Active Measurement Traffic to a MA; one of

   them measures some parameter associated with the transfer of the

   packet(s).  A Passive Measurement Task involves only a MA, which

   simply observes existing traffic - for example, it could simply count

   bytes or it might calculate the average loss for a particular flow.

P.6, Collector: delete "It may also do some post-processing on the results, for instance to eliminate outliers, as they can severely impact the aggregated results."  And replace with "It then provides the results to a results repository (see below)."

Reasons for deleting this sentence:
-              Keep functional definitions well delineated. A Collector collects and stores data for use by other functions. A post-processing analysis function may modify that data.
-              Avoid implicit approval for any "post-processing" function that modifies the database so that raw results are no longer available for analysis. In particular, elimination of Outliers strikes me as a bad example - different analyses can create different useful algorithms to identify and remove outliers.
-              The fact that the Collector and the results repository are two logically separate functions reinforces the above point. There should be no post processing of the measurement results prior to storage in the repository.

P.9: Delete the definition of Composite Metric (isn't used anywhere, imprecise definition).

P.10, definition of Measurement Task: Delete "The act that yields a single Measurement Result." Later in the draft reference is made to Measurement Tasks that can yield multiple results.

P.14, bottom: Delete "Optionally any Suppression message can be sent over a different channel." There is no mention of this in the Suppression section, and Suppression is being handled as an element within the Instruction.

P.16: Change "Measurement Task" to "Measurement Task Configuration" in the first bullet. This configures parameters for a Method and potentially applies to many iterations of a Measurement Task.

P.17, 2nd full para.: change sentence to "For example, if the message includes  (only) a set of periodic Measurement Schedules, then that replaces the old set but does not alter the Measurement Task Configurations and Report Channels." The element may contain more than one Measurement Schedule.

Bottom of P.21/top of P.22: I think we need to encourage restraint in any preprocessing of Measurement Results before they get to the repository, since this precludes any chance of analyzing the raw results.

Overall:  I think the draft is missing a means for a Controller to provide explicit suppression to a Measurement Peer. In the current draft there is no mention of a controller interacting with a Peer and Figure 1 shows no connection between the two. For the most part, that is expected. MA and MP are two related but distinct logical functions - one initiates upon direction from a Controller, the other is limited to responding.

However, if suppression is required the Controller should be able to notify both ends, so that a MP can reject suppressed Tasks that are initiated either by MAs that have not yet received the suppression Instruction or by MAs controlled from another domain entirely. (I know, this is out of scope for lmap. But this seems to be an important omission, and I think we can justify it if need be out of a concern that even in the same domain, some MAs may not get timely notification).

This would not be necessary for an MP sitting in, say, a set-top box, because that MP will not interact with thousands of MAs. It is necessary for the large dedicated devices at network peering points, etc., and adding a controller interface for those devices doesn't measurably increase the complexity.  I expect them to also have resident MA functions anyway to support troubleshooting.

From: lmap [mailto:[email-masked]] On Behalf Of Romascanu, Dan (Dan)
Sent: Thursday, January 23, 2014 7:31 AM
To: [email-masked]
Subject: [lmap] WGLC for http://www.ietf.org/id/draft-ietf-lmap-framework-03.txt

This is a Working Group Last Call for the Internet-Draft 'A framework for large-scale measurement platforms (LMAP)' which can be found at
 http://www.ietf.org/id/draft-ietf-lmap-framework-03.txt. Please send your comments about this I-D to the WG mail list until 2/6/13.  If you have no comments but you believe that the document is ready to be submitted to the IESG for consideration as Informational RFC a short mail stating this is also welcome.

Thanks and Regards,

Dan

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Sun, 9 Feb 2014 12:51:23 +0000

Hi,

Please find below my comments – as a contributor – to draft-ietf-lmap-framework-03.txt

I focused my comments at this point on the content and mainly technical issues. I put aside the editorial comments as it seems to me it’s a little bit early for these.

1.       Juergen Schoenwaelder made in his review a comment (comment a) about the definition of channels between Controllers and MA which I fully support.

2.       The MAs are described as ‘pieces of code’ that can reside in a number of hosts, routers or devices deployed in the Internet. There is however no indication about how these ‘pieces of code’ can be upgraded. There is also no mention about the implications of deploying these ‘pieces of code’ on security and privacy. This may or may be not related to the components of the LMAP framework, but I feel this needs to be discussed and maybe addressed in the text.

3.       I do not know exactly what the following sentence in the Introduction section means: ‘The Measurement Tasks themselves may be on IPv4, IPv6, and on various services (DNS, HTTP, XMPP, FTP, VoIP, etc.).’ I suggest to delete.

4.       Also in the introduction – a standardized LMAP mechanism is justified by the fact that it ‘enables the operator of a measurement system to buy the various components from different vendors.’ I believe that what is meant really here is that standardized mechanisms may provide ‘criteria for evaluation of the different solutions that can be used for various purposes including buying decisions’.

5.       Several protocols which do not originate in the IETF are being mentioned in the document – TR-069, UPnP, maybe other. Please provide Informational References for these.

6.       I do not understand what the following sentence in Section 2 means: ‘The operator's OAM (Operations, Administration, and Maintenance) uses the results from the tools.’  I suggest to delete.

7.        I am confused by some of the terms in the Terminology section being labelled as (optional). I hope that it’s not the description of the term that is optional, and if what is meant is that the fields are optional in the LMAP protocols, this needs to be said someplace else.

8.       I do not believe that the term ‘metric’ really needs a definition – hopefully it has in LMAP the same meaning as used in other WGs In the IETF (IPPM, XRBLOCK, etc.) – especially as the description is problematic (what does ‘carefully specified’ mean?)

9.       In Section 5.2 I found the following:

   In LMAP, the MA can inform the

   Controller about its Capabilities.  This message could be sent in

   several circumstances: when the MA first communicates with a

   Controller; when the MA becomes capable of a new Measurement Method;

   when requested by the Controller (for example, if the Controller

   forgets what the MA can do or otherwise wants to resynchronize what

   it knows about the MA).  Note that Capabilities do not include

   dynamic information like the MA's currently unused CPU, memory or

   battery life.

My question is how does the MA inform the controller on asynchronous events? (for example the addition of capabilities). There is no hint about an asynchronous event passing from MAs to controllers in diagrams or further text.

10.   Also in Section 5.2:

     The MA can inform the Controller about a Failure.  There are two

   broad categories of failure: (1) the MA cannot action the Instruction

   (for example, it doesn't include a parameter that is mandatory for

   the requested Measurement Method; or it is missing details of the

   target Collector). (2) the MA cannot execute the Measurement Task or

   deliver the Report (for example, the MA unexpectedly has no spare CPU
   cycles; or the Collector is not responding).

How is the latest failure condition (collector not responding) detected by the MA? Do we assume that the Report Protocol incorporates either an ACK mechanism or some kind of keep-alive with each of the reporting MAs? If this is the case, we should mention this when speaking about the Report Protocol, I could not find anything on this respect.

11.   In Section 5.4:

    The measurement system may define what happens if a Collector

   unexpectedly does not hear from a MA, for example the Controller

   could send a fresh Report Schedule to the MA.

This implies an interaction between Collectors and Controllers which is not present in the basic architecture diagram and declared out of scope in Section 5.6 (point 2). This requires some clarification in the text at best – I am not sure that we need to define in the protocol option that relay on a channel that is explicitly out of the scope of the framework.

12.   In Section 6.2.2 - It is not clear why the first mechanism mentioned for pre-provisioning firewalls for traversal is a non-IETF protocol (TR-069), while PCP and another non-IETF protocol UPnP are later mentioned as alternatives. At least I would see them as part of the same enumeration.

13.   The security considerations section should say something about the threats related to the storage on the Collectors

14.   The security considerations section should say something about the threat of DoS attacks on collectors, for example by throttling them with dummy MA registrations and false reports.

Thanks and Regards,

Dan

From: lmap [mailto:[email-masked]] On Behalf Of Romascanu, Dan (Dan)
Sent: יום ה 23 ינואר 2014 15:31
To: [email-masked]
Subject: [lmap] WGLC for http://www.ietf.org/id/draft-ietf-lmap-framework-03.txt

This is a Working Group Last Call for the Internet-Draft ‘A framework for large-scale measurement platforms (LMAP)’ which can be found at
 http://www.ietf.org/id/draft-ietf-lmap-framework-03.txt. Please send your comments about this I-D to the WG mail list until 2/6/13.  If you have no comments but you believe that the document is ready to be submitted to the IESG for consideration as Informational RFC a short mail stating this is also welcome.

Thanks and Regards,

Dan