Subject: [lmap] lmap framework and IPPM registries.

From: MARCELO BAGNULO BRAUN <[email-masked]>
Date: Mon, 14 Apr 2014 16:50:14 +0200

Hi,

One issue i realized is that the current lmap framework states the the 
measurement method is specified as a URN to a registry entry, while the 
current IPPM registry draft do not have a URN as metric identifier but 
a name and a metric identifier.

I think it is very important that these two efforts are aligned so i 
suggest that we discuss whether a URN is the proper way to go or the 
current flat identifier proposed in the ippm registry drafts.

I am uncertain what is the benefit of an URN versus a flat identifier. 
Regards, marcelo

-- 
----
MARCELO BAGNULO BRAUN
WebCartero
Universidad Carlos III de Madrid

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 14 Apr 2014 19:57:47 +0200

On Mon, Apr 14, 2014 at 04:50:14PM +0200, MARCELO BAGNULO BRAUN wrote:
> Hi,
> 
> One issue i realized is that the current lmap framework states the
> the measurement method is specified as a URN to a registry entry,
> while the current IPPM registry draft do not have a URN as metric
> identifier but a name and a metric identifier.

So according to IPPM, which one is the "native" unique thing to refer
to, the name or the metric identifier (a 16-bit number if I understand
right)?

> I am uncertain what is the benefit of an URN versus a flat
> identifier. Regards, marcelo

It should be a URI.

Standard metrics can have their own URN space, e.g., derived from
their unique "thing" (the name or the metric, I can't tell). Or IPPM
even makes their native "identifier" simply a URN. Making this
reference a URI within LMAP makes things easily extensible without
requiring to go through a central registry or to work with potential
clashes in number spaces reserved for private use. Using URIs also
seems rather natural in a web-based world.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: MARCELO BAGNULO BRAUN <[email-masked]>
Date: Mon, 14 Apr 2014 20:41:09 +0200

> to, the name or the metric identifier (a 16-bit number if I understand

the identifier is the unique identifier

metrics names should be unique, but not must be

> seems rather natural in a web-based world.

It is perfectrly possible to reserve a range of identifiers for private 
use so a manager of a measurement domain can locally assign values for 
metrics without going through the registry.

I still dont understand how does the IANA manages a well know set of 
metric identifiers using URN/URIs. I am not saying this is not 
feasible, just that i fail to understand it yet.

Regards, marcelo

---

From: "Bugenhagen, Michael K" <[email-masked]>
Date: Tue, 15 Apr 2014 14:27:58 +0000

Marcelo, Juergen, Al

    Just for clarity - and my making sure I'm tracking this correctly - and we're all on the same page -

The tests conducted by most of the regulators (the one's they are reporting) are modifications of the standard testing methods.
i.e. - the Throughput test results use a delay window or stability window to negate the influence of "burst" or buffering - and report the stable throughput after x seconds, or y seconds of stability.

Question -  Given the Registry method we've described requires a Standard test MIB to become a registry entry
Assumption - Does it follow that we need to write a standalone test standard (RFC) for each unique test method...

I think this is true, if not please correct me.

At the moment we have very few tests that use that modified method, which infers we have more work to do on the IPPM side with new test RFC's?

Thanks,
Mike 		

-----Original Message-----
From: lmap [mailto:[email-masked]] On Behalf Of MARCELO BAGNULO BRAUN
Sent: Monday, April 14, 2014 1:41 PM
To: Juergen Schoenwaelder
Cc: [email-masked]
Subject: Re: [lmap] lmap framework and IPPM registries.

> seems rather natural in a web-based world.

It is perfectrly possible to reserve a range of identifiers for private use so a manager of a measurement domain can locally assign values for metrics without going through the registry.

I still dont understand how does the IANA manages a well know set of metric identifiers using URN/URIs. I am not saying this is not feasible, just that i fail to understand it yet.

Regards, marcelo

_______________________________________________
lmap mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/lmap

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Tue, 15 Apr 2014 10:45:13 -0400

Mike,  I won't be able to expand on details with you,
but see my reply below:

> From: Bugenhagen, Michael K [mailto:[email-masked]]
> Sent: Tuesday, April 15, 2014 10:28 AM
...
> and we're all on the same page -

We are not, we have yet to achieve that state in many cases, 
unfortunately.

> 
> The tests conducted by most of the regulators (the one's they are
> reporting) are modifications of the standard testing methods.
> i.e. - the Throughput test results use a delay window or stability window
> to negate the influence of "burst" or buffering - and report the stable
> throughput after x seconds, or y seconds of stability.

Real bad example, because IPPM does not yet have a solid metric and method RFC
for Throughput or Capacity. Work is in-progress here, of course.

> Assumption - Does it follow that we need to write a standalone test
> standard (RFC) for each unique test method...
> 

We considered this possibility, but practicality caused us to choose the
alternative where one or more existing RFCs are references for a Registered Metric,
and the registry entry itself MUST provide all the details to specify the metric 
and methods *exactly*, without ambiguity.

So, not true about lots of new RFCs. We simply don't see this happening
to support the registry - there's barely enough review energy to get the 
the registry design memos adopted and to consensus.  This is clearly overhead
work and though essential to LMAP, not exciting == few volunteers.

Some RFCs are needed (Throughput) and many very good Registry Entries are needed.

Al

> 
> At the moment we have very few tests that use that modified method, which
> infers we have more work to do on the IPPM side with new test RFC's?
> 
> Thanks,
> Mike
> 
> 
> 
> 
> 
> 
> 
> 
> metric identifiers using URN/URIs. I am not saying this is not feasible,
> just that i fail to understand it yet.
> 
> Regards, marcelo
> 
> 
> 
> > /js
> >
> > --

---

From: "Bugenhagen, Michael K" <[email-masked]>
Date: Tue, 15 Apr 2014 14:55:28 +0000

Al - 

   Thanks!
  
In the BBF meeting we were discussing how to move forward with documenting the tests for some of the U.S. based tests that are currently being reported so we had some apples to apples reference tests.

   We agreed that going through the IETF IPPM / registry group was the way to go- so I wanted to make sure what that process might look like.
I thought it would require new RFC work for defining a special test without ambiguity.... as you indicate.

  So we'll point anyone wanting to standardize tests to the IPPM - Excellent.

BTW - The BBF has some WT-143 broadband tests already defined on throughput testing, I'll discuss getting those converted to RFC's with the original editors of those tests.. that seems a logical first step.

Cheers,
Mike 

-----Original Message-----
From: lmap [mailto:[email-masked]] On Behalf Of MORTON, ALFRED C (AL)
Sent: Tuesday, April 15, 2014 9:45 AM
To: Bugenhagen, Michael K; 'MARCELO BAGNULO BRAUN'; 'Juergen Schoenwaelder'
Cc: [email-masked]'
Subject: Re: [lmap] lmap framework and IPPM registries.

Mike,  I won't be able to expand on details with you, but see my reply below:

> From: Bugenhagen, Michael K 
> [mailto:[email-masked]]
...
> - and we're all on the same page -

We are not, we have yet to achieve that state in many cases, unfortunately.

Al

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Tue, 15 Apr 2014 12:03:25 -0400

Mike,

...
>   So we'll point anyone wanting to standardize tests to the IPPM -

They should start by reviewing
http://tools.ietf.org/html/draft-ietf-ippm-registry-active-00
and the core registry memo it references.  (This is what I asked
at the last IPPM session, that future *users* of the registry,
those preparing entries, implementing metrics and methods, or both,
should see if what we have designed will be sufficient for their
needs.

> 
> BTW - The BBF has some WT-143 broadband tests already defined on
> throughput testing, I'll discuss getting those converted to RFC's with the
> original editors of those tests.. that seems a logical first step.
>

There is an Informational RFC on TCP Throughput testing (not a std metric)
http://tools.ietf.org/html/rfc6349

Diagnostic throughput testing is useful, but again this is 
work-already-in-progress in IPPM. I doubt anything will be rubber-stamped.

Al

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Tue, 15 Apr 2014 16:14:15 -0400

For me and those whose needs I understand,
a URI for metrics seems useful. I would guess that IANA has already been
asked for some form of ID persistence, and if they agree to provide when requested, 
then it would be a (new) requirement in the IANA section.

Also, we need both a metric and a method of measurement, these are
both present in our current version of the metric registry.

>From Framework 5.2.1 Instruction

   o  the Measurement Task configurations, each of which needs:

      *  the Measurement Method, specified as a URN to a registry entry.
         The registry could be defined by the IETF
         [I-D.manyfolks-ippm-metric-registry], locally by the operator
         of the measurement system or perhaps by another standards
         organisation.

So let's fix this as follows:

   o  the Measurement Task configurations, each of which needs:

      *  the Registered Metric, specified as a URI to a registry entry,
         and which includes the specification of a Measurement Method.
         The registry could be defined by the IETF
         [I-D.manyfolks-ippm-metric-registry], locally by the operator
         of the measurement system or perhaps by another standards
         organisation.

(off-list discussion contributed to this suggestion), 
Al

Al

> Sent: Monday, April 14, 2014 10:50 AM
> To: [email-masked]
> Subject: [lmap] lmap framework and IPPM registries.
> 
> Hi,
>

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Tue, 15 Apr 2014 16:16:00 -0400

it seems (from digging around) that this text with URN first appeared in 
Framework 03, and possibly was suggested by Trevor since the
Info model uses URNs extensively.

> From: MORTON, ALFRED C (AL)
> Sent: Tuesday, April 15, 2014 4:14 PM
> To: 'MARCELO BAGNULO BRAUN'; [email-masked]
> Subject: RE: [lmap] lmap framework and IPPM registries.
>

---

From: MARCELO BAGNULO BRAUN <[email-masked]>
Date: Tue, 15 Apr 2014 23:34:36 +0200

So, the question I have is whether IANA can handle URI allocations. I 
dont know the answer to this question. If they can, then i think we are 
ok. If they cant, i think we have a problem.

Let me expand.
I understnad that we want the controller to be able to tell the MA 
using the lmap protocol(s) to execute a measurement task using a given 
metric X.
Metric X can be a locally defined metric or it can be a well know 
metric registered in the IANA registry.

The reason for having registered metrics is so that for instace if an 
isp buys a CPE that is implementing an lmap MA, it can also be 
implementing a set of well defined metrics registered in the IANA 
registry for performance metrics.
If that is the case, then we can have a CPE that off the shelf can be 
integrated into an lmap measurement system and the controller can use 
the lmap protocol so that the MA can execute well know metric X.

So, the lmap control protocol will need to carry a metric identifer to 
identify the metric that the controller is asking the MA to use.
This identifier must be registered in the iana registry for performance 
metrics (for the case of the well known metrics).

Now, i know how to ask iana to create a registry for performance 
metrics and include a metric identifier field in the registry that can 
be used in the control protocol to identify the different metrics.
What i dont know is whether the metric identifier in the iana registry 
can be an URI. That i understand would imply that iana would be able to 
hand over identifiers that would look like something like this: 
http://www.iana.org/ietf/ippm/metrics/37897

or

http://www.iana.org/ietf/ippm/metrics/udp_latency_95th_mean

If iana can handle this, then we are ok. If iana doesnt do this, then 
we have a problem because we wouldnt be able to create the registry for 
well known metrics that is compatible with lmap.

Anyone knows if iana has done this before? Can you provide a pointer?

I apologize in advance if this is obvious.

Regards, marcelo

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 16 Apr 2014 09:33:32 +0200

On Tue, Apr 15, 2014 at 11:34:36PM +0200, MARCELO BAGNULO BRAUN wrote:

> So, the question I have is whether IANA can handle URI allocations.
> I dont know the answer to this question. If they can, then i think
> we are ok. If they cant, i think we have a problem.

IANA already manages URNs for several protocols. I guess they won't
have a problem doing one more. See RFC 3553 (BCP 73) for a starting
point.

/js

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 16 Apr 2014 09:35:56 +0200

On Mon, Apr 14, 2014 at 08:41:09PM +0200, MARCELO BAGNULO BRAUN wrote:
> 
> It is perfectrly possible to reserve a range of identifiers for
> private use so a manager of a measurement domain can locally assign
> values for metrics without going through the registry.
> 

A set of values reserved for private use means the values will not be
unique and clash. If I have an LMAP installation with MA provided by
different vendors, there is a reasonable chance that private use
identifiers shipped with their products clash. By moving to URIs, this
risk can be minimized.

/js

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Wed, 16 Apr 2014 08:35:12 -0400

> From: lmap [mailto:[email-masked]] On Behalf Of Juergen
> Schoenwaelder
> Sent: Wednesday, April 16, 2014 3:36 AM
...
> identifiers shipped with their products clash. By moving to URIs, this

I've asked IANA if support for URIs is possible, on their general contact 
e-mail address (which has spawned a ticket number, etc.). It may take a few
iterations on the question & answer, but I'll report back when the loop 
closes.

Al

---

From: "Bugenhagen, Michael K" <[email-masked]>
Date: Wed, 16 Apr 2014 19:49:33 +0000

Marcelo, Al,

    One gap I think we have here is how to identify a common set of "network environment" attributes that need to be used by the tests.
They aren't identified in the test methodology as a common abstract MIB, or attribute across the board so this needs to be handled elsewhere.

My assumption what that it would be in the registry as some type of "header" or description on how to execute a test, but if I'm following you that assumption is wrong.

   Example -

Cross Traffic, or customer traffic... 

Some tests have assumptions about the presence of this, and when it's acceptable to do the test, or not based on it's existence.
Given there is NO common MIB / attribute for it today... this becomes an issue in defining the "test method"

Kind of a gap... with different ways to close it.

Regards,
Mike   

-----Original Message-----
From: lmap [mailto:[email-masked]] On Behalf Of MORTON, ALFRED C (AL)
Sent: Tuesday, April 15, 2014 3:14 PM
To: MARCELO BAGNULO BRAUN; [email-masked]
Subject: Re: [lmap] lmap framework and IPPM registries.

For me and those whose needs I understand, a URI for metrics seems useful. I would guess that IANA has already been asked for some form of ID persistence, and if they agree to provide when requested, then it would be a (new) requirement in the IANA section.

Also, we need both a metric and a method of measurement, these are both present in our current version of the metric registry.

(off-list discussion contributed to this suggestion), Al

Al

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Wed, 16 Apr 2014 16:10:53 -0400

In the original registry proposal(s) over a year ago now,
we had an Environment (sub) Registry:
http://tools.ietf.org/html/draft-bagnulo-ippm-new-registry-00#section-2.3

We were convinced to take this out of the registry design, 
it's not really part of the metric specifications and there 
likely were other reasons I've forgotten, but it's out now for good.

The Info model draft covers this topic.
http://tools.ietf.org/html/draft-ietf-lmap-information-model-00#section-3.7
...
   The result data rows may optionally include an indication of the
   cross-traffic (e.g., the total number of octets of non-measurement
   traffic passing through the interfaces used by a Measurement Task
   during the measurement period).

Al

> Sent: Wednesday, April 16, 2014 3:50 PM
> To: MORTON, ALFRED C (AL); 'MARCELO BAGNULO BRAUN'; [email-masked]'
>     One gap I think we have here is how to identify a common set of
> "network environment" attributes that need to be used by the tests.
> They aren't identified in the test methodology as a common abstract MIB,
> or attribute across the board so this needs to be handled elsewhere.
> 
> My assumption what that it would be in the registry as some type of
> "header" or description on how to execute a test, but if I'm following you
> that assumption is wrong.
> 
>    Example -
> 
> Some tests have assumptions about the presence of this, and when it's
> acceptable to do the test, or not based on it's existence.
> For me and those whose needs I understand, a URI for metrics seems useful.
> I would guess that IANA has already been asked for some form of ID
> persistence, and if they agree to provide when requested, then it would be
> a (new) requirement in the IANA section.
> 
> Also, we need both a metric and a method of measurement, these are both
> present in our current version of the metric registry.
>

---

From: marcelo bagnulo braun <[email-masked]>
Date: Tue, 22 Apr 2014 09:22:10 +0200

Hi,

I have checked RFC3553 and I guess we can do a similar approach.
In order to do that, the IPPM registry draft (in particular the core 
registry draft) would need to add a mandatory URI column in the 
registry. I will take the issue to IPPM to see if they are ok with that.

Regards, marcelo

---

From: "Bugenhagen, Michael K" <[email-masked]>
Date: Tue, 22 Apr 2014 17:25:17 +0000

Al, Phil,

   In order to execute a test we need consistant MA behavior "about" or when different test environment variables are encountered.

As Al pointed out, we have some "information model" objects that call out different environmental conditions -

First -  We need to identify those with definitions vs. just objects.

Secondly - but very possibly more challenging -

	How do we take those "environmental" attributes and define the test behavior around those when the test being used is in a registry.

Example case ...

Test A = throughput
Test B = latency
Environmental x = cross traffic.

Behaviors of the test with the environmental... 
For Test A ---  terminate a test when environmental "x" is detected, then mark the test as terminated due to "environment x".
For Test B ---  record environmental "x" during test b.

What this indicates to me is that we need a very Standard Test Execution string that details out all the environmental with Codified "behavior" codes as to what the MA does during and after the test when it experiences a environmental..

	More problematically operationally reaction to things like cross traffic generally doesn't use "lab like" absolutes like "zero cross traffic" so we will also have some fuzzy logic (thresholds) in our environmental definitions.

So - Cap statement....
At the end of the day it looks like we still need a standardized (about environmental) test instruction set that is implemented the same across the board.   Both from a Attribute, and behavior aspect.

This constitutes a Gap that needs to be fixed.

Regards,
Mike

-----Original Message-----
From: MORTON, ALFRED C (AL) [mailto:[email-masked]] 
Sent: Wednesday, April 16, 2014 3:11 PM
To: Bugenhagen, Michael K; 'MARCELO BAGNULO BRAUN'; [email-masked]'
Subject: RE: [lmap] lmap framework and IPPM registries.

In the original registry proposal(s) over a year ago now, we had an Environment (sub) Registry:
http://tools.ietf.org/html/draft-bagnulo-ippm-new-registry-00#section-2.3

We were convinced to take this out of the registry design, it's not really part of the metric specifications and there likely were other reasons I've forgotten, but it's out now for good.

Al

> They aren't identified in the test methodology as a common abstract 
> MIB, or attribute across the board so this needs to be handled elsewhere.
> 
> "header" or description on how to execute a test, but if I'm following 
> you that assumption is wrong.
> 
>    Example -
> 
> From: lmap [mailto:[email-masked]] On Behalf Of MORTON, ALFRED 
> C
> (AL)
> persistence, and if they agree to provide when requested, then it 
> would be a (new) requirement in the IANA section.
>