Subject: [lmap] Which performance metrics for LMAP?

From: Benoit Claise <[email-masked]>
Date: Thu, 07 Mar 2013 12:27:49 +0100

Dear all,

Another clarifying question: I'm not too sure which performance metric 
types LMAP targets.
draft-linsner-lmap-use-cases-00 mentions:

    It is assumed that measurement
    tests run by OTT providers would only include the metrics associated
    with layer 3 and up

The different drafts use different terms: service, end user experience, 
quality of experience.
Does Quality of Experience imply performance metrics per application, 
per 5 tuple (IP addresses, ports, protocol), per DCSP?
Or do we want to stay at layer 3 performance metrics, typically IPPM 
type of metrics?
If the latter, then LMAP is about large scale deployment of IPPM, right?

While I fully understand the business needs for "and up" in "include the 
metrics associated with layer 3 and up", let's not boil the ocean.
Performance Metrics at Other Layer (PMOL 
<http://datatracker.ietf.org/wg/pmol/charter/>), a concluded WG, tried 
and it proved to be difficult.

Some more discussions, on the mailing list or during the BoF, on this 
topic would be appreciated.

Regards, Benoit

---

From: "MORTON JR., ALFRED (AL)" <[email-masked]>
Date: Thu, 7 Mar 2013 10:53:28 -0500

Hi Benoit,

You've asked four good questions today, I'll take a stab at this one.

IMO, the metrics chosen need to serve two purposes, at least.

One set of metrics describe the packet transfer service (at the service layers).
Another (somewhat overlapping) set help subscribers understand
how well their applications will perform on the service
(inferred from measurements at the service layer).

These two points of view (POV) and their implications
on metric selection/parameter settings/reporting for IP packet transfer
were investigated in http://tools.ietf.org/html/rfc6703

IPPM's scope has always included transport layer, so your 5-tuple with
DSCP is covered there: these have all been part of the notion "packets of Type-P"
since the beginning, see http://tools.ietf.org/html/rfc2330#section-13
and these aspects are more important now than ever.

As a limit on "layer 3 and up", I would suggest that limited exceptions for
metrics at +transport layers would be useful, such as the DNS response time
metric constructed in
http://tools.ietf.org/html/draft-bagnulo-ippm-new-registry-independent-00#section-8

So IMO, LMAP infrastructure would support measurement at IP and transport layers
with a few notable exceptions, and look to others (IPPM WG) to select the best
metrics (and develop new ones, as necessary) for each POV that LMAP serves.

my 2cents,
Al

From: [email-masked] [mailto:[email-masked]] On Behalf Of Benoit Claise
Sent: Thursday, March 07, 2013 6:28 AM
To: [email-masked]
Subject: [lmap] Which performance metrics for LMAP?

Dear all,

Another clarifying question: I'm not too sure which performance metric types LMAP targets.
draft-linsner-lmap-use-cases-00 mentions:

   with layer 3 and up

The different drafts use different terms: service, end user experience, quality of experience.
Does Quality of Experience imply performance metrics per application, per 5 tuple (IP addresses, ports, protocol), per DCSP?
Or do we want to stay at layer 3 performance metrics, typically IPPM type of metrics?
If the latter, then LMAP is about large scale deployment of IPPM, right?

While I fully understand the business needs for "and up" in "include the metrics associated with layer 3 and up", let's not boil the ocean.
Performance Metrics at Other Layer (PMOL<http://datatracker.ietf.org/wg/pmol/charter/>), a concluded WG, tried and it proved to be difficult.

Some more discussions, on the mailing list or during the BoF, on this topic would be appreciated.

Regards, Benoit

---

From: "Bugenhagen, Michael K" <[email-masked]>
Date: Thu, 7 Mar 2013 22:16:17 +0000

Benoit,

       I think that the requirement to"future proof" the test framework in order to ensure it will support any future test we throw at it requires that the test framework be able to support both service commissioning (Throughput) type tests, as well as other SLA like tracking tests such as Twamp and the Ethernet OAM, and even service level testing.    So essentially they should all fit inside the framework

M

Sent from my iPhone

On Mar 7, 2013, at 5:28 AM, "Benoit Claise" <[email-masked]<mailto:[email-masked]>> wrote:

Dear all,

Regards, Benoit
_______________________________________________
lmap mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/lmap

---

From: <[email-masked]>
Date: Mon, 11 Mar 2013 18:03:51 +0000

agree with this,

also, i think the work of lmap is on the Controller-MA and MA-Collector interface (MA = MEasurement Agent)
for instance, Controller tells MA "Run test X to MA-2 every hour and report results to Collector-5 once per day"
"test X" is defined in a registry. IPPM will hopefully be re-chartered to define a registry of IPPM tests.
but I can see sometimes you might want "test X" to refer to some non-IPPM test - possibly another stds organisation could define such a registry?

phil
________________________________
From: [email-masked] [[email-masked]] On Behalf Of Bugenhagen, Michael K [[email-masked]]
Sent: 07 March 2013 22:16
To: Benoit Claise
Cc: [email-masked]
Subject: Re: [lmap] Which performance metrics for LMAP?

Benoit,

M

Sent from my iPhone

Dear all,

Regards, Benoit