Subject: [lmap] some thoughts about the LMAP activity

From: <[email-masked]>
Date: Thu, 8 Nov 2012 16:31:58 +0000

Couple of thoughts after the LMAP bar bof last night.

Thank-you to Henning and James for organising and initiating the meeting and the I-D about this important work.

I think we should distinguish several topics:

1. Use cases -From an IETF perspective the aim (I think) is motivational and to show the broad scope. they don't have to be completely exhaustive.

2. Functional architecture - I think it's important for us to agree the high level functions (starting from something like Henning's picture in the plenary, or the discussion Marcelo triggered about 'Starting the work description') with enough detail to confirm consensus). Ideally we would agree a functional architecture with other standards bodies working in the area. Maybe this can be achieved informally, via the people who are active in several standards bodies.

3. Which pieces of this functional architecture are
a. standardised by the ietf
b. standardised by some other body
c. not appropriate for standardisation
Ideally other standards bodies would have the same view of this breakdown, to avoid the danger that there end up being 0 or 2 standards where there should be 1.

I think the I-D and discussion are great start on 1 and 2. To get a BoF we need a really clear answer on 3 as well. But we don't need a complete answer before starting any work at the IETF - eg there's already seems good agreement about doing some metrics related activity in ippm. Perhaps we can define that already, and define other work a little later.

Personally I'd find it very useful to have an overview of where other standards bodies have got to and what work that they're planning. For example, I remember that Michael B offered to give an overview of the Broadand forum's framework (= functional architecture?), is it possible to share it please? 802.16 (?) was also mentioned last night, is any info available about that please?

One comment about the I-D. I find the Requirements section a bit premature. For example, I don't think it's that useful for us to agree requirements about pieces of the functional architecture that are in 3b & 3c, ie outside the IETF's remit.

Finally, I was wondering about how Henning and the authors of the I-D want to handle input. Do you want comments that you'll gradually work in, extra authors/editors, split the doc into different bits....?

Best wishes,
phil

---

From: James Miller <[email-masked]>
Date: Thu, 8 Nov 2012 12:12:20 -0500

Thanks to everyone for the contributions on the list so far and the
attendance and interesting feedback at the BAR BOF, and to Philip for the
below.

On Thu, Nov 8, 2012 at 11:31 AM, <[email-masked]> wrote:

> 1. Use cases -From an IETF perspective the aim (I think) is motivational
> and to show the broad scope. they don't have to be completely exhaustive.
>

In answer to your comments point below, I would appreciate specific
references to the documents section 2 discussing use cases.  We did not
intend to provide a highly specific description of examples but wanted to
capture at a high level what sorts of use cases may exist in a way that
would be relevant for understanding how the architcture and metrics would
need to be developed.

If there are categories of use cases that are not captured we would like to
make those changes.  I think it's important that we provide the use cases
at a high-level but more specific discussions may clutter the focus of the
draft if the references aren't necessary to provide background for the goal
of defining the architecture in the document.  Currently we have the
following three cases:

   - Provider network measurements
   - User network diagnostics
   - Multi-provider network measurements

Some specific references on the list and in discussion last night related
to:

   - Can a user perform a test and get data?

Under this formulation, yes it is the "user network diagnostics" case.

   - Could different parties, contracting or volunteering together, operate
   a test together, such as a small ISP contracting out work with a
   third-party?

Yes, we think that the authentication/security and data communications flow
that we map out try to capture this.  In fact the FCC's case is similar is
a variant of this case where we work with many parties including
subscribers who volunteer to join the program, SamKnows who is the
contractor running the test, and measurement lab who also works with us to
support our measurement servers solution.

If people think that more specific examples that would demonstrate the use
cases would be helpful in the categorical sections, we would be happy to
include suggestions.

>
> 2. Functional architecture - I think it's important for us to agree the
> high level functions (starting from something like Henning's picture in the
> plenary, or the discussion Marcelo triggered about 'Starting the work
> description') with enough detail to confirm consensus). Ideally we would
> agree a functional architecture with other standards bodies working in the
> area. Maybe this can be achieved informally, via the people who are active
> in several standards bodies.
>
>

The ID currently maps this in section 3 defining the overview of the
architecture.  The ID text tracks the diagrams and approaches in both
Henning's plenary talk and the approaches that have been employed with the
FCC measurement programs.  Would there be changes to the ID that could
help facilitate the discussion?

  3 <http://tools.ietf.org/html/draft-schulzrinne-lmap-requirements-00#section-3>.
 Architecture Overview  . . . . . . . . . . . . . . . . . . . .  8
<http://tools.ietf.org/html/draft-schulzrinne-lmap-requirements-00#page-8>
     3.1 <http://tools.ietf.org/html/draft-schulzrinne-lmap-requirements-00#section-3.1>.
 Measurement client . . . . . . . . . . . . . . . . . . . .  8
<http://tools.ietf.org/html/draft-schulzrinne-lmap-requirements-00#page-8>
     3.2 <http://tools.ietf.org/html/draft-schulzrinne-lmap-requirements-00#section-3.2>.
 Measurement server . . . . . . . . . . . . . . . . . . . .  8
<http://tools.ietf.org/html/draft-schulzrinne-lmap-requirements-00#page-8>
     3.3 <http://tools.ietf.org/html/draft-schulzrinne-lmap-requirements-00#section-3.3>.
 Measurement controller . . . . . . . . . . . . . . . . . .  8
<http://tools.ietf.org/html/draft-schulzrinne-lmap-requirements-00#page-8>
     3.4 <http://tools.ietf.org/html/draft-schulzrinne-lmap-requirements-00#section-3.4>.
 Data collector . . . . . . . . . . . . . . . . . . . . . .  8
<http://tools.ietf.org/html/draft-schulzrinne-lmap-requirements-00#page-8>
     3.5 <http://tools.ietf.org/html/draft-schulzrinne-lmap-requirements-00#section-3.5>.
 Network parameter server . . . . . . . . . . . . . . . . .  9
<http://tools.ietf.org/html/draft-schulzrinne-lmap-requirements-00#page-9>

On this and the points below, I would like to suggest that anyone
interesting this topic call in or attend our next Generation meeting at FCC
Headquarters currently Scheduled for November 28 in the morning.  We could
have an overview from folks doing work in the IEEE and Broadband Forum and
offer a platform for discussion on the architecture and how groups might
currently be heading.

We typically make these meeting available via WebEx but if folks are in DC,
I'm happy to brew some coffee.

> Ideally other standards bodies would have the same view of this breakdown,
> to avoid the danger that there end up being 0 or 2 standards where there
> should be 1.
>
> I think the I-D and discussion are great start on 1 and 2. To get a BoF we
> need a really clear answer on 3 as well. But we don't need a complete
> answer before starting any work at the IETF - eg there's already seems good
> agreement about doing some metrics related activity in ippm. Perhaps we can
> define that already, and define other work a little later.
>
> Personally I'd find it very useful to have an overview of where other
> standards bodies have got to and what work that they're planning. For
> example, I remember that Michael B offered to give an overview of the
> Broadand forum's framework (= functional architecture?), is it possible to
> share it please? 802.16 (?) was also mentioned last night, is any info
> available about that please?
>
> One comment about the I-D. I find the Requirements section a bit
> premature. For example, I don't think it's that useful for us to agree
> requirements about pieces of the functional architecture that are in 3b &
> 3c, ie outside the IETF's remit.
>
> Finally, I was wondering about how Henning and the authors of the I-D want
> to handle input. Do you want comments that you'll gradually work in, extra
> authors/editors, split the doc into different bits....?
>
> Best wishes,
> phil
>
>
> _______________________________________________
> lmap mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/lmap
>
>

---

From: Brian Trammell <[email-masked]>
Date: Thu, 8 Nov 2012 16:41:59 -0500

Hi, Philip, all,

> I think the I-D and discussion are great start on 1 and 2. To get a BoF we need a really clear answer on 3 as well. But we don't need a complete answer before starting any work at the IETF - eg the…

First I should say that I think we should be careful here: unless it's very tightly scoped, standardization of functional architecture can IMO turn into a bit of a rathole. I may be generalizing a bit much from my experience, but the closer the functional architecture comes to simply being a description of the minimum set of boxes and lines you need to illustrate or explain the terminology and operation of a protocol, the better. The closer it gets to being an implementation report of a particular attempt at solving a particular set of requirements, the less resilient and less useful it is likely to prove.

We're here to specify protocols such that components and systems can interoperate toward meeting a common requirement, 

> One comment about the I-D. I find the Requirements section a bit premature. For example, I don't think it's that useful for us to agree requirements about pieces of the functional architecture that…

Agreed.

---

From: Brian Trammell <[email-masked]>
Date: Thu, 8 Nov 2012 16:44:47 -0500

(Oops, hit send instead of save :/ please stand by for the rest of this message...)

---

From: <[email-masked]>
Date: Thu, 8 Nov 2012 22:04:34 +0000

[cut]

[Brian]

[phil] I agree. For me, a functional architecture can be a good way of generating a common view on what a WG is trying to do in its specification phase. simplicity is generally good as it's easier to understand, so easier to see if there's a common view - but not so simple that it hides where there's actually some fundamental disagreement about what the WG is trying to specify
phil

---

From: Brian Trammell <[email-masked]>
Date: Thu, 8 Nov 2012 17:07:14 -0500

hi Phil, all,

> [phil] I agree. For me, a functional architecture can be a good way of generating a common view on what a WG is trying to do in its specification phase. simplicity is generally good as it's easier…

Absolutely. I think maybe I'm getting caught up on terminology here;  perhaps I'd prefer to call it a "reference architecture", to capture that it's a description of how a protocol or set of protocols are designed to work, as opposed to a proscription for how they must be deployed. 

(The rest of my thought in a moment...)

cheers,

Brian

---

From: Brian Trammell <[email-masked]>
Date: Thu, 8 Nov 2012 17:32:03 -0500

Hi, Phil, all,

Apologies for the early send; my thought got fragmented by lack of sleep, apparently. Here it is reassembled.

We're here to specify protocols such that components and systems can interoperate toward meeting a common requirement: if there's nothing to be gained by having independent implementations working with each other, then a standard protocol is useful as a form of documentation for the community, but is no more useful than the implementors' making their interface specifications available.

I believe in the use cases presented in the draft, and especially the regulatory use case presented in the tsvarea/plenary presentations, standard protocols both for control as well as for metrics data exchange would have a lot of value. For regulatory/verification purposes it is _absolutely_ essential that the definition of the metrics produced by different vendors' equipment and operators' implementations are transparently defined and openly comparable (within a known and reported margin of error); it is also vitally important for troubleshooting of complex problems with multiple partial causes. 

On the control side, this is also vitally important. Whether for regulatory, research, or troubleshooting use cases, measurement becomes much easier to deploy when it's possible to grab a (tested) component off the shelf, whether that be simple active delay probing software on an end system or a box in the core estimating performance characteristics passively, and know that it will interoperate with existing data collection and measurement control already deployed.

So I guess I'd come at this same question from a slightly different, perhaps too IETF-centric angle: what is this venue (the IETF) good for, and how much does work alluded to so far in the LMAP effort, as outlined, overlap with work already underway in the IETF? Does there a exist a venue where there is more overlap? 

Here, it's clear that standard definitions of performance metrics fit very well with IPPM. It's a little less clear where the control interactions would fit. I don't see a lot of overlap here with NETCONF, for example, but at the same time, I would submit that it probably doesn't make sense to standardize measurement control in one venue and metrics in another, especially as they're most probably done at the same layer.

Agreed.

Cheers,

Brian

---

From: Capello Alessandro <[email-masked]>
Date: Fri, 9 Nov 2012 01:04:06 +0100

Hi all,

I support Brian's point on interworking.

Looking at the use cases, you can envision a residential user that:

* has a measurement probe embedded in the home gateway managed by the ISP

* has a dedicated appliance in the home network to run tests for regulators

* has a software probe on the PC that runs tests on behalf of an OTT player

* ...

I think that the controlled exchange of data, requests, etc. between different administrative domains (ISPs, regulators, OTT players, ...) should be part of the functions required (something like peering relationships but applied to measurement instead of routing). In my (probably biased) mind ISPs have a key role here but it looks really like a win-win game to me (larger data set, better accuracy, etc.).

Regards,

Alessandro

Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle persone indicate. La diffusione, copia o qualsiasi altra azione derivante dalla conoscenza di queste informazioni sono rigorosamente vietate. Qualora abbiate ricevuto questo documento per errore siete cortesemente pregati di darne immediata comunicazione al mittente e di provvedere alla sua distruzione, Grazie.

This e-mail and any attachments is confidential and may contain privileged information intended for the addressee(s) only. Dissemination, copying, printing or use by anybody else is unauthorised. If you are not the intended recipient, please delete this message and any attachments and advise the sender by return e-mail, Thanks.

Rispetta l'ambiente. Non stampare questa mail se non ? necessario.