Subject: [lmap] What is broadband?

From: Benoit Claise <[email-masked]>
Date: Thu, 07 Mar 2013 11:47:16 +0100

Dear all,

I started to review the drafts, and I will start posting a few questions 
to the list.
Open questions, clarifying questions, in order to generate some discussions.
Disclaimer: I have not yet read the entire list archive. Apologize in 
advance if some points have been discussed already.

Here is my first question. What is broadband in the LMAP context?
Is it DSL, cable, ETTH, Fiber to the home?  Is LMAP technology independent?
And I see also "enterprise edge router", "cellular data or satellite" in 
draft-schulzrinne-lmap-requirements. In or out?
Or do we have in mind a phase approach, starting with the "enterprise 
edge router" first, and then home network?

Interestingly, I don't know what A stands for in LMAP, if it stands for 
something.
According to http://trac.tools.ietf.org/bof/trac/wiki/WikiStart, the A 
doesn't stand for anything.
However, looking at the different draft titles, there is some confusion.

    draft-linsner-lmap-use-cases-0 title is Large-Scale Broadband Measurement Use Cases
    draft-schulzrinne-lmap-requirements-00.txt title is Large-Scale Measurement of Broadband Performance
    draft-boucadair-lmap-considerations-00,_Large scale Measurement of__Access__network Performance (LMAP)_:
       Requirements and Issues from a Network Provider Perspective

Some more discussions, on the mailing list or during the BoF, on this 
topic would be appreciated.

Regards, Benoit

---

From: Shane Amante <[email-masked]>
Date: Thu, 7 Mar 2013 08:41:11 -0700

Benoit,

On Mar 7, 2013, at 3:47 AM, Benoit Claise <[email-masked]> wrote:
> Dear all,
> 
> I started to review the drafts, and I will start posting a few questions to the list.
> Disclaimer: I have not yet read the entire list archive. Apologize in advance if some points have been discussed already. 
> 
> And I see also "enterprise edge router", "cellular data or satellite" in draft-schulzrinne-lmap-requirements. In or out?
> Or do we have in mind a phase approach, starting with the "enterprise edge router" first, and then home network?

Speaking for the network I operate, I'm very much an advocate of saying that "Enterprise Edge Router" is "in-scope".  We would very much benefit from a standards-based measurement enablement and collection regime vs. mostly proprietary, and non-scalable, approaches that exist today.

This is not to diminish the importance of similar test capabilities for residential broadband use-cases.  We absolutely need to work on those, as well.  

With respect to priority, my hope is that we do not have to choose to prioritize one over the other.  Rather, I would hope that both can be developed in parallel, because both -- at least, IMO -- have a substantially overlapping set of requirements/features. 

My $0.02,

-shane

> Interestingly, I don't know what A stands for in LMAP, if it stands for something.
> According to http://trac.tools.ietf.org/bof/trac/wiki/WikiStart, the A doesn't stand for anything.
>    draft-boucadair-lmap-considerations-00, Large scale Measurement of Access network Performance (LMAP):
> Some more discussions, on the mailing list or during the BoF, on this topic would be appreciated.
> 
> Regards, Benoit
> 
> 
> 
> _______________________________________________
> lmap mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/lmap

---

From: "Bugenhagen, Michael K" <[email-masked]>
Date: Thu, 7 Mar 2013 22:20:13 +0000

The word "access" should be key here as part of the definition provided we are talking about an Internet service, which is the second component.   I don't really  think we are building tests that won't work on smaller pipes so questioning if it really 'broadband' or not is correct IMO.

Sent from my iPhone

On Mar 7, 2013, at 9:41 AM, "Shane Amante" <[email-masked]<mailto:[email-masked]>> wrote:

Benoit,

On Mar 7, 2013, at 3:47 AM, Benoit Claise <[email-masked]<mailto:[email-masked]>> wrote:
Dear all,

My $0.02,

-shane

Regards, Benoit

_______________________________________________
lmap mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/lmap

_______________________________________________
lmap mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/lmap

---

From: James Miller <[email-masked]>
Date: Thu, 7 Mar 2013 21:30:25 -0500

I believe that Henning had commented at some point that the LMAP definition
he contemplated had "architecture" as the 'A' element but certainly access
is an important piece.  I think one of the problems that has been discussed
also on the LMAP and our FCC Next-Gen lists is that a complete view of LMAP
performance measurements would implicate elements from the user's laptop,
through wireless and other local LAN, carriers access networks, Tier 1 and
other peering networks, the application host side and everything in
between.  Clearly there would be a lot of technologies included within
that functional scope.

For reference, in the FCC Measuring Broadband America Program we focused on
measurement from the consumers' broadband modem through the carriers
network to where it connects to a tier one peering point.  LMAP should be
able to address the broad mix of other use cases that would have a mix of
other elements and motivations.

Graphic in Report at page 9.
http://transition.fcc.gov/cgb/measuringbroadbandreport/2Methodology.pdf

On Thu, Mar 7, 2013 at 5:20 PM, Bugenhagen, Michael K <
[email-masked]> wrote:

>  The word "access" should be key here as part of the definition provided
> we are talking about an Internet service, which is the second component.
> I don't really  think we are building tests that won't work on smaller
> pipes so questioning if it really 'broadband' or not is correct IMO.
>
>
>
> Sent from my iPhone
>
> On Mar 7, 2013, at 9:41 AM, "Shane Amante" <[email-masked]> wrote:
>
>  Benoit,
>
>  Speaking for the network I operate, I'm very much an advocate of saying
> that "Enterprise Edge Router" is "in-scope".  We would very much benefit
> from a standards-based measurement enablement and collection regime vs.
> mostly proprietary, and non-scalable, approaches that exist today.
>
>  This is not to diminish the importance of similar test capabilities for
> residential broadband use-cases.  We absolutely need to work on those, as
> well.
>
>  With respect to priority, my hope is that we do not have to choose to
> prioritize one over the other.  Rather, I would hope that both can be
> developed in parallel, because both -- at least, IMO -- have a
> substantially overlapping set of requirements/features.
>
>  My $0.02,
>
>  -shane
>
>
>    draft-boucadair-lmap-considerations-00, *Large scale Measurement of **Access **network Performance (LMAP)*:

---

From: Shane Amante <[email-masked]>
Date: Thu, 7 Mar 2013 20:24:18 -0700

James, All,

On Mar 7, 2013, at 7:30 PM, James Miller <[email-masked]> wrote:
> I believe that Henning had commented at some point that the LMAP definition he contemplated had "architecture" as the 'A' element but certainly access is an important piece.  I think one of the problems that has been discussed also on the LMAP and our FCC Next-Gen lists is that a complete view of LMAP performance measurements would implicate elements from the user's laptop, through wireless and other local LAN, carriers access networks, Tier 1 and other peering networks, the application host side and everything in between.  Clearly there would be a lot of technologies included within that functional scope.

I agree that access is important, but not to the exclusion of everything else that constitutes an Internet end-user customer experience or, alternatively, an Enterprise end-user customer experience -- which is what I believe you're saying above?  That's why it will be important to, at some point, figure out *if*, and then how, to try to segment the portions of the end-to-end path that you describe above so we can attribute good or bad performance to a particular portion of the path so that, ultimately, the correct network operator can be contacted to look into the problem further.  I do not believe that this requires us to break down the end-to-end path on a router-hop by router-hop basis, but rather we need to be able to identify 'sign posts' along the path that can correlate well to end-to-end path.

-shane

> For reference, in the FCC Measuring Broadband America Program we focused on measurement from the consumers' broadband modem through the carriers network to where it connects to a tier one peering point.  LMAP should be able to address the broad mix of other use cases that would have a mix of other elements and motivations.
> 
> Graphic in Report at page 9. http://transition.fcc.gov/cgb/measuringbroadbandreport/2Methodology.pdf
> 
> On Thu, Mar 7, 2013 at 5:20 PM, Bugenhagen, Michael K <[email-masked]> wrote:

---

From: James Miller <[email-masked]>
Date: Thu, 7 Mar 2013 22:57:46 -0500

I agree Shane--was what I was trying to get at. :)

Our early FCC focus was basically the Access network but I think we're all
making the point that the mix-n-match of the network measurement pieces
will be driven by a particular user's use case.

For our FCC work, the focus (discussed in more detail in our reports) was
on understanding the performance in the segment under the control of a
carrier delivering broadband to a given consumer.  Naturally many other
elements affect an end user's experience but our initial interest was on
understanding that segment for the purposes of providing better information
to the consumer about the portion of Internet performance that the carrier
provides.  The FCC's focus and history is on that piece and so it makes
sense that it might begin there, but a particular carrier's focus might
mirror the access network focus, but perhaps with less market-trend
orientation and more diagnostic in flavor.  Maybe LMAP together with other
protocols (TR69 and other BBForum work for example) could benefit all those
use cases.

In recent meetings we've discussed other more targeted mini-studies on
things like CDN or in-home Wifi performance.  Other potential LMAP users
already focus their work on these or other elements of the network.  I
would envision the LMAP work could provide the glue between any of these
elements and different use cases.  Worth noting that much of the initial
discussions in the FCC's Next-Gen group were on the potential value of
standardized interfaces between such different elements of measurement
infrastructure.

Another element that's probably worth noting is that the use cases also may
direct the actual tests that someone wants to run.  I would imagine the
actual tests you might want to run for evaluating in-home LAN, access
network, CDN, mobile networks may vary.  So I think the work on the test
registry and other IPPM work on new tests will also improve our definitions
and understanding of "broadband performance".

As always my personal views and not the FCC's.. possible valued at less
than $.02  :)

On Thu, Mar 7, 2013 at 10:24 PM, Shane Amante <[email-masked]> wrote:

> I believe that Henning had commented at some point that the LMAP
> definition he contemplated had "architecture" as the 'A' element but
> certainly access is an important piece.  I think one of the problems that
> has been discussed also on the LMAP and our FCC Next-Gen lists is that a
> complete view of LMAP performance measurements would implicate elements
> from the user's laptop, through wireless and other local LAN, carriers
> access networks, Tier 1 and other peering networks, the application host
> side and everything in between.  Clearly there would be a lot of
> technologies included within that functional scope.
>
>
> I agree that access is important, but not to the exclusion of everything
> else that constitutes an Internet end-user customer experience or,
> alternatively, an Enterprise end-user customer experience -- which is what
> I believe you're saying above?  That's why it will be important to, at some
> point, figure out *if*, and then how, to try to segment the portions of the
> end-to-end path that you describe above so we can attribute good or bad
> performance to a particular portion of the path so that, ultimately, the
> correct network operator can be contacted to look into the problem further.
>  I do not believe that this requires us to break down the end-to-end path
> on a router-hop by router-hop basis, but rather we need to be able to
> identify 'sign posts' along the path that can correlate well to end-to-end
> path.
>
> -shane
>
>
> For reference, in the FCC Measuring Broadband America Program we focused
> on measurement from the consumers' broadband modem through the carriers

---

From: Henning Schulzrinne <[email-masked]>
Date: Fri, 8 Mar 2013 23:16:09 +0000

I would hope that the protocol components are largely independent of the measurement endpoints, but there is indeed a need for somebody (not necessarily the IETF and not necessarily one entity) to define them if measurements are meant  to be comparable. For some private-use cases, e.g., within a large enterprise network or for internal carrier use, that's probably less important. If you believe that protocol functionality depends on defining these "mileposts", I'd be curious if you have examples in mind.

Henning

From: [email-masked] [mailto:[email-masked]] On Behalf Of Shane Amante
Sent: Thursday, March 07, 2013 10:24 PM
To: James Miller
Cc: Benoit Claise; Bugenhagen, Michael K; [email-masked]
Subject: Re: [lmap] What is broadband?

James, All,

On Mar 7, 2013, at 7:30 PM, James Miller <[email-masked]<mailto:[email-masked]>> wrote:
I believe that Henning had commented at some point that the LMAP definition he contemplated had "architecture" as the 'A' element but certainly access is an important piece.  I think one of the problems that has been discussed also on the LMAP and our FCC Next-Gen lists is that a complete view of LMAP performance measurements would implicate elements from the user's laptop, through wireless and other local LAN, carriers access networks, Tier 1 and other peering networks, the application host side and everything in between.  Clearly there would be a lot of technologies included within that functional scope.

-shane

Graphic in Report at page 9. http://transition.fcc.gov/cgb/measuringbroadbandreport/2Methodology.pdf
On Thu, Mar 7, 2013 at 5:20 PM, Bugenhagen, Michael K <[email-masked]<mailto:[email-masked]>> wrote:
The word "access" should be key here as part of the definition provided we are talking about an Internet service, which is the second component.   I don't really  think we are building tests that won't work on smaller pipes so questioning if it really 'broadband' or not is correct IMO.

Sent from my iPhone

My $0.02,

-shane

Regards, Benoit

---

From: <[email-masked]>
Date: Sun, 10 Mar 2013 19:56:28 +0000

the 'A' in lmap doesnt stand for anything (somewhere lost in the mists of history... it didnt seem sensible to change the abbreviation before the BoF, since people have got used to the term)

consumer and enterprise wired broadband certainly of interest to me. that includes the network beyond the access (BRAS, peering point etc).  so a "registry" of reference points in the network seems like work that needs to be done [tools.ietf.org/html/draft-morton-ippm-lmap-path as first attempt]

phil

________________________________
From: [email-masked] [[email-masked]] On Behalf Of James Miller [[email-masked]]
Sent: 08 March 2013 02:30
To: Bugenhagen, Michael K
Cc: Benoit Claise; Shane Amante; [email-masked]
Subject: Re: [lmap] What is broadband?

Sent from my iPhone

Benoit,

My $0.02,

-shane

Regards, Benoit

---

From: "Sharam Hakimi" <[email-masked]>
Date: Mon, 11 Mar 2013 12:42:55 -0400

James,

I think mixing testing of wireless and wire technologies confuses the
issues. One wireless is WiFi which is the extension of LAN and the other
is the cellular network. They both have different requirements and two
major conditions in both

                Signal Strength

                Roaming

have great effect on the achieved performance which does not exist in
the wire condition . Some of these performance issues might also be
better resolved in layer 2 rather than layer 3 and above. For example,
before I run any throughput test I want to make sure that I have the
highest strength signal possible be it WiFi or Cellular wireless
connection ( and stay stationary during the test). On the other hand if
I am trying to find the best location for signal strength a throughput
test might not be the best tool for the job.

I would suggest to keep the two separate at least for the short term.

Sharam

From: [email-masked] [mailto:[email-masked]] On Behalf Of
James Miller
Sent: Thursday, March 07, 2013 10:58 PM
To: Shane Amante
Cc: Benoit Claise; Bugenhagen,Michael K; [email-masked]
Subject: Re: [lmap] What is broadband?

Our early FCC focus was basically the Access network but I think we're
all making the point that the mix-n-match of the network measurement
pieces will be driven by a particular user's use case.  

For our FCC work, the focus (discussed in more detail in our reports)
was on understanding the performance in the segment under the control of
a carrier delivering broadband to a given consumer.  Naturally many
other elements affect an end user's experience but our initial interest
was on understanding that segment for the purposes of providing better
information to the consumer about the portion of Internet performance
that the carrier provides.  The FCC's focus and history is on that piece
and so it makes sense that it might begin there, but a particular
carrier's focus might mirror the access network focus, but perhaps with
less market-trend orientation and more diagnostic in flavor.  Maybe LMAP
together with other protocols (TR69 and other BBForum work for example)
could benefit all those use cases.

Another element that's probably worth noting is that the use cases also
may direct the actual tests that someone wants to run.  I would imagine
the actual tests you might want to run for evaluating in-home LAN,
access network, CDN, mobile networks may vary.  So I think the work on
the test registry and other IPPM work on new tests will also improve our
definitions and understanding of "broadband performance".

On Thu, Mar 7, 2013 at 10:24 PM, Shane Amante <[email-masked]>
wrote:

James, All,

On Mar 7, 2013, at 7:30 PM, James Miller <[email-masked]>
wrote:

	I believe that Henning had commented at some point that the LMAP
definition he contemplated had "architecture" as the 'A' element but
certainly access is an important piece.  I think one of the problems
that has been discussed also on the LMAP and our FCC Next-Gen lists is
that a complete view of LMAP performance measurements would implicate
elements from the user's laptop, through wireless and other local LAN,
carriers access networks, Tier 1 and other peering networks, the
application host side and everything in between.  Clearly there would be
a lot of technologies included within that functional scope.

I agree that access is important, but not to the exclusion of everything
else that constitutes an Internet end-user customer experience or,
alternatively, an Enterprise end-user customer experience -- which is
what I believe you're saying above?  That's why it will be important to,
at some point, figure out *if*, and then how, to try to segment the
portions of the end-to-end path that you describe above so we can
attribute good or bad performance to a particular portion of the path so
that, ultimately, the correct network operator can be contacted to look
into the problem further.  I do not believe that this requires us to
break down the end-to-end path on a router-hop by router-hop basis, but
rather we need to be able to identify 'sign posts' along the path that
can correlate well to end-to-end path.

-shane

For reference, in the FCC Measuring Broadband America Program we focused
on measurement from the consumers' broadband modem through the carriers
network to where it connects to a tier one peering point.  LMAP should
be able to address the broad mix of other use cases that would have a
mix of other elements and motivations.

On Thu, Mar 7, 2013 at 5:20 PM, Bugenhagen, Michael K
<[email-masked]> wrote:

Sent from my iPhone

On Mar 7, 2013, at 9:41 AM, "Shane Amante" <[email-masked]>
wrote:

	Benoit, 

	On Mar 7, 2013, at 3:47 AM, Benoit Claise <[email-masked]>
wrote:

		Dear all,
		
		I started to review the drafts, and I will start posting
a few questions to the list.
		Open questions, clarifying questions, in order to
generate some discussions.
		Disclaimer: I have not yet read the entire list archive.
Apologize in advance if some points have been discussed already. 
		
		Here is my first question. What is broadband in the LMAP
context?
		Is it DSL, cable, ETTH, Fiber to the home?  Is LMAP
technology independent?
		And I see also "enterprise edge router", "cellular data
or satellite" in draft-schulzrinne-lmap-requirements. In or out?
		Or do we have in mind a phase approach, starting with
the "enterprise edge router" first, and then home network?

	Speaking for the network I operate, I'm very much an advocate of
saying that "Enterprise Edge Router" is "in-scope".  We would very much
benefit from a standards-based measurement enablement and collection
regime vs. mostly proprietary, and non-scalable, approaches that exist
today.

	This is not to diminish the importance of similar test
capabilities for residential broadband use-cases.  We absolutely need to
work on those, as well.  

	With respect to priority, my hope is that we do not have to
choose to prioritize one over the other.  Rather, I would hope that both
can be developed in parallel, because both -- at least, IMO -- have a
substantially overlapping set of requirements/features. 

	My $0.02,

	-shane

	Interestingly, I don't know what A stands for in LMAP, if it
stands for something.
	According to http://trac.tools.ietf.org/bof/trac/wiki/WikiStart,
the A doesn't stand for anything.
	However, looking at the different draft titles, there is some
confusion.

	   draft-linsner-lmap-use-cases-0 title is Large-Scale Broadband
Measurement Use Cases
	   draft-schulzrinne-lmap-requirements-00.txt title is
Large-Scale Measurement of Broadband Performance
	   draft-boucadair-lmap-considerations-00, Large scale
Measurement of Access network Performance (LMAP):
	      Requirements and Issues from a Network Provider
Perspective

	Some more discussions, on the mailing list or during the BoF, on
this topic would be appreciated.
	
	Regards, Benoit

---

From: Marc Linsner <[email-masked]>
Date: Mon, 11 Mar 2013 14:37:14 -0400

Sharam,

IMO, the only difference between wireless and wired are the test metrics
that can be performed.  LMAP is concerned with the management of the
measurement clients, controllers, and collectors.  Until further thought
proves otherwise, I don't see any difference in the management of devices on
any of the layer 1/2s.

The specifics you mention, Signal Strength and Roaming, should be dealt with
in the definition of the test metric.

-Marc-

> James,
> I think mixing testing of wireless and wire technologies confuses the issues.
> One wireless is WiFi which is the extension of LAN and the other is the
> cellular network. They both have different requirements and two major
> conditions in both
>                 Signal Strength
>                 Roaming
>  
> have great effect on the achieved performance which does not exist in the wire
> condition . Some of these performance issues might also be better resolved in
> layer 2 rather than layer 3 and above. For example, before I run any
> throughput test I want to make sure that I have the highest strength signal
> possible be it WiFi or Cellular wireless connection ( and stay stationary
> during the test). On the other hand if I am trying to find the best location
> for signal strength a throughput test might not be the best tool for the job.
>  
> From: [email-masked] [mailto:[email-masked]] On Behalf Of James
> Miller
> making the point that the mix-n-match of the network measurement pieces will
> be driven by a particular user's use case.
> 
>  
> 
> For our FCC work, the focus (discussed in more detail in our reports) was on
> understanding the performance in the segment under the control of a carrier
> delivering broadband to a given consumer.  Naturally many other elements
> affect an end user's experience but our initial interest was on understanding
> that segment for the purposes of providing better information to the consumer
> about the portion of Internet performance that the carrier provides.  The
> FCC's focus and history is on that piece and so it makes sense that it might
> begin there, but a particular carrier's focus might mirror the access network
> focus, but perhaps with less market-trend orientation and more diagnostic in
> flavor.  Maybe LMAP together with other protocols (TR69 and other BBForum work
> for example) could benefit all those use cases.
> 
>  
> 
> In recent meetings we've discussed other more targeted mini-studies on things
> like CDN or in-home Wifi performance.  Other potential LMAP users already
> focus their work on these or other elements of the network.  I would envision
> the LMAP work could provide the glue between any of these elements and
> different use cases.  Worth noting that much of the initial discussions in the
> FCC's Next-Gen group were on the potential value of standardized interfaces
> between such different elements of measurement infrastructure.
> 
>  
> 
> direct the actual tests that someone wants to run.  I would imagine the actual
> tests you might want to run for evaluating in-home LAN, access network, CDN,
> mobile networks may vary.  So I think the work on the test registry and other
> IPPM work on new tests will also improve our definitions and understanding of
> "broadband performance".
> 
>  
> 
> As always my personal views and not the FCC's.. possible valued at less than
> $.02  :)
> 
>> he contemplated had "architecture" as the 'A' element but certainly access is
>> an important piece.  I think one of the problems that has been discussed also
>> on the LMAP and our FCC Next-Gen lists is that a complete view of LMAP
>> other peering networks, the application host side and everything in between.
>> Clearly there would be a lot of technologies included within that functional
>> scope.
> 
>  
> I agree that access is important, but not to the exclusion of everything else
> that constitutes an Internet end-user customer experience or, alternatively,
> an Enterprise end-user customer experience -- which is what I believe you're
> saying above?  That's why it will be important to, at some point, figure out
> *if*, and then how, to try to segment the portions of the end-to-end path that
> you describe above so we can attribute good or bad performance to a particular
> portion of the path so that, ultimately, the correct network operator can be
> contacted to look into the problem further.  I do not believe that this
> requires us to break down the end-to-end path on a router-hop by router-hop
> basis, but rather we need to be able to identify 'sign posts' along the path
> that can correlate well to end-to-end path.
> 
>  
> 
> -shane
> 
>  
> 
> 
> 
> measurement from the consumers' broadband modem through the carriers network
> to where it connects to a tier one peering point.  LMAP should be able to
> address the broad mix of other use cases that would have a mix of other
> elements and motivations.
> 
>  
> 
> The word "access" should be key here as part of the definition provided we are
> talking about an Internet service, which is the second component.   I don't
> really  think we are building tests that won't work on smaller pipes so
> questioning if it really 'broadband' or not is correct IMO.
> 
>  
> 
> 
> 
> Sent from my iPhone
> 
> 
>> Speaking for the network I operate, I'm very much an advocate of saying that
>> "Enterprise Edge Router" is "in-scope".  We would very much benefit from a
>> standards-based measurement enablement and collection regime vs. mostly
>> proprietary, and non-scalable, approaches that exist today.
>> 
>>  
>> 
>> developed in parallel, because both -- at least, IMO -- have a substantially
>> overlapping set of requirements/features.
>> 
>>  
>> 
>> My $0.02,
>> 
>>  
>> 
>> -shane
>> 
>>  
>> 
>> 
>> 
> _______________________________________________ lmap mailing list
> [email-masked] https://www.ietf.org/mailman/listinfo/lmap

---

From: "Sharam Hakimi" <[email-masked]>
Date: Mon, 11 Mar 2013 16:28:39 -0400

Marc,

I think the management agent that a wireless device might use would be
somewhat different than a wire device as there are metrics that a
wireless device has, that do not exist in a wire device. My opinion.

Sharam

From: [email-masked] [mailto:[email-masked]] On Behalf Of
Marc Linsner
Sent: Monday, March 11, 2013 2:37 PM
To: Sharam Hakimi
Cc: [email-masked]
Subject: Re: [lmap] What is broadband?

Sharam,

IMO, the only difference between wireless and wired are the test metrics
that can be performed.  LMAP is concerned with the management of the
measurement clients, controllers, and collectors.  Until further thought
proves otherwise, I don't see any difference in the management of
devices on any of the layer 1/2s.

The specifics you mention, Signal Strength and Roaming, should be dealt
with in the definition of the test metric.

-Marc-

	James,

	I think mixing testing of wireless and wire technologies
confuses the issues. One wireless is WiFi which is the extension of LAN
and the other is the cellular network. They both have different
requirements and two major conditions in both

	                Signal Strength

	                Roaming

	have great effect on the achieved performance which does not
exist in the wire condition . Some of these performance issues might
also be better resolved in layer 2 rather than layer 3 and above. For
example, before I run any throughput test I want to make sure that I
have the highest strength signal possible be it WiFi or Cellular
wireless connection ( and stay stationary during the test). On the other
hand if I am trying to find the best location for signal strength a
throughput test might not be the best tool for the job.

	I would suggest to keep the two separate at least for the short
term.

	Sharam

	From: [email-masked] [mailto:[email-masked]] On
Behalf Of James Miller
	Sent: Thursday, March 07, 2013 10:58 PM
	To: Shane Amante
	Cc: Benoit Claise; Bugenhagen,Michael K; [email-masked]
	Subject: Re: [lmap] What is broadband?

	Our early FCC focus was basically the Access network but I think
we're all making the point that the mix-n-match of the network
measurement pieces will be driven by a particular user's use case.  

	For our FCC work, the focus (discussed in more detail in our
reports) was on understanding the performance in the segment under the
control of a carrier delivering broadband to a given consumer.
Naturally many other elements affect an end user's experience but our
initial interest was on understanding that segment for the purposes of
providing better information to the consumer about the portion of
Internet performance that the carrier provides.  The FCC's focus and
history is on that piece and so it makes sense that it might begin
there, but a particular carrier's focus might mirror the access network
focus, but perhaps with less market-trend orientation and more
diagnostic in flavor.  Maybe LMAP together with other protocols (TR69
and other BBForum work for example) could benefit all those use cases.

	In recent meetings we've discussed other more targeted
mini-studies on things like CDN or in-home Wifi performance.  Other
potential LMAP users already focus their work on these or other elements
of the network.  I would envision the LMAP work could provide the glue
between any of these elements and different use cases.  Worth noting
that much of the initial discussions in the FCC's Next-Gen group were on
the potential value of standardized interfaces between such different
elements of measurement infrastructure.

	Another element that's probably worth noting is that the use
cases also may direct the actual tests that someone wants to run.  I
would imagine the actual tests you might want to run for evaluating
in-home LAN, access network, CDN, mobile networks may vary.  So I think
the work on the test registry and other IPPM work on new tests will also
improve our definitions and understanding of "broadband performance".

	As always my personal views and not the FCC's.. possible valued
at less than $.02  :)

	On Thu, Mar 7, 2013 at 10:24 PM, Shane Amante
<[email-masked]> wrote:

	James, All,

	On Mar 7, 2013, at 7:30 PM, James Miller
<[email-masked]> wrote:

		I believe that Henning had commented at some point that
the LMAP definition he contemplated had "architecture" as the 'A'
element but certainly access is an important piece.  I think one of the
problems that has been discussed also on the LMAP and our FCC Next-Gen
lists is that a complete view of LMAP performance measurements would
implicate elements from the user's laptop, through wireless and other
local LAN, carriers access networks, Tier 1 and other peering networks,
the application host side and everything in between.  Clearly there
would be a lot of technologies included within that functional scope.

	I agree that access is important, but not to the exclusion of
everything else that constitutes an Internet end-user customer
experience or, alternatively, an Enterprise end-user customer experience
-- which is what I believe you're saying above?  That's why it will be
important to, at some point, figure out *if*, and then how, to try to
segment the portions of the end-to-end path that you describe above so
we can attribute good or bad performance to a particular portion of the
path so that, ultimately, the correct network operator can be contacted
to look into the problem further.  I do not believe that this requires
us to break down the end-to-end path on a router-hop by router-hop
basis, but rather we need to be able to identify 'sign posts' along the
path that can correlate well to end-to-end path.

	-shane

	

	For reference, in the FCC Measuring Broadband America Program we
focused on measurement from the consumers' broadband modem through the
carriers network to where it connects to a tier one peering point.  LMAP
should be able to address the broad mix of other use cases that would
have a mix of other elements and motivations.

	The word "access" should be key here as part of the definition
provided we are talking about an Internet service, which is the second
component.   I don't really  think we are building tests that won't work
on smaller pipes so questioning if it really 'broadband' or not is
correct IMO.

	
	Sent from my iPhone

	On Mar 7, 2013, at 9:41 AM, "Shane Amante"
<[email-masked]> wrote:

		Benoit, 

		On Mar 7, 2013, at 3:47 AM, Benoit Claise
<[email-masked]> wrote:

			Dear all,
			
			I started to review the drafts, and I will start
posting a few questions to the list.
			Open questions, clarifying questions, in order
to generate some discussions.
			Disclaimer: I have not yet read the entire list
archive. Apologize in advance if some points have been discussed
already. 
			
			Here is my first question. What is broadband in
the LMAP context?
			Is it DSL, cable, ETTH, Fiber to the home?  Is
LMAP technology independent?
			And I see also "enterprise edge router",
"cellular data or satellite" in draft-schulzrinne-lmap-requirements. In
or out?
			Or do we have in mind a phase approach, starting
with the "enterprise edge router" first, and then home network?

		Speaking for the network I operate, I'm very much an
advocate of saying that "Enterprise Edge Router" is "in-scope".  We
would very much benefit from a standards-based measurement enablement
and collection regime vs. mostly proprietary, and non-scalable,
approaches that exist today.

		With respect to priority, my hope is that we do not have
to choose to prioritize one over the other.  Rather, I would hope that
both can be developed in parallel, because both -- at least, IMO -- have
a substantially overlapping set of requirements/features. 

		My $0.02,

		-shane

		

		Interestingly, I don't know what A stands for in LMAP,
if it stands for something.
		According to
http://trac.tools.ietf.org/bof/trac/wiki/WikiStart, the A doesn't stand
for anything.
		However, looking at the different draft titles, there is
some confusion.

		   draft-linsner-lmap-use-cases-0 title is Large-Scale
Broadband Measurement Use Cases
		   draft-schulzrinne-lmap-requirements-00.txt title is
Large-Scale Measurement of Broadband Performance
		   draft-boucadair-lmap-considerations-00, Large scale
Measurement of Access network Performance (LMAP):
		      Requirements and Issues from a Network Provider
Perspective

		Some more discussions, on the mailing list or during the
BoF, on this topic would be appreciated.
		
		Regards, Benoit

	 

	_______________________________________________ lmap mailing
list [email-masked] https://www.ietf.org/mailman/listinfo/lmap

---

From: Henning Schulzrinne <[email-masked]>
Date: Mon, 11 Mar 2013 20:43:52 +0000

The goal would be that the control and distribution components should be independent of the metrics, so that metrics can be added, both standardized ones and proprietary ones specific to a commercial provider or research network. If the control mechanism doesn't work for wireless metrics, for example, that might be an indication that it is not generic enough.

________________________________
From: [email-masked] [[email-masked]] on behalf of Sharam Hakimi [[email-masked]]
Sent: Monday, March 11, 2013 4:28 PM
To: Marc Linsner
Cc: [email-masked]
Subject: Re: [lmap] What is broadband?

Marc,
I think the management agent that a wireless device might use would be somewhat different than a wire device as there are metrics that a wireless device has, that do not exist in a wire device. My opinion.

Sharam

From: [email-masked] [mailto:[email-masked]] On Behalf Of Marc Linsner
Sent: Monday, March 11, 2013 2:37 PM
To: Sharam Hakimi
Cc: [email-masked]
Subject: Re: [lmap] What is broadband?

Sharam,

IMO, the only difference between wireless and wired are the test metrics that can be performed.  LMAP is concerned with the management of the measurement clients, controllers, and collectors.  Until further thought proves otherwise, I don't see any difference in the management of devices on any of the layer 1/2s.

The specifics you mention, Signal Strength and Roaming, should be dealt with in the definition of the test metric.

-Marc-

James,
I think mixing testing of wireless and wire technologies confuses the issues. One wireless is WiFi which is the extension of LAN and the other is the cellular network. They both have different requirements and two major conditions in both
                Signal Strength
                Roaming

have great effect on the achieved performance which does not exist in the wire condition . Some of these performance issues might also be better resolved in layer 2 rather than layer 3 and above. For example, before I run any throughput test I want to make sure that I have the highest strength signal possible be it WiFi or Cellular wireless connection ( and stay stationary during the test). On the other hand if I am trying to find the best location for signal strength a throughput test might not be the best tool for the job.

Sharam

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]] On Behalf Of James Miller
Sent: Thursday, March 07, 2013 10:58 PM
To: Shane Amante
Cc: Benoit Claise; Bugenhagen,Michael K; [email-masked]<mailto:[email-masked]>
Subject: Re: [lmap] What is broadband?

Our early FCC focus was basically the Access network but I think we're all making the point that the mix-n-match of the network measurement pieces will be driven by a particular user's use case.

For our FCC work, the focus (discussed in more detail in our reports) was on understanding the performance in the segment under the control of a carrier delivering broadband to a given consumer.  Naturally many other elements affect an end user's experience but our initial interest was on understanding that segment for the purposes of providing better information to the consumer about the portion of Internet performance that the carrier provides.  The FCC's focus and history is on that piece and so it makes sense that it might begin there, but a particular carrier's focus might mirror the access network focus, but perhaps with less market-trend orientation and more diagnostic in flavor.  Maybe LMAP together with other protocols (TR69 and other BBForum work for example) could benefit all those use cases.

In recent meetings we've discussed other more targeted mini-studies on things like CDN or in-home Wifi performance.  Other potential LMAP users already focus their work on these or other elements of the network.  I would envision the LMAP work could provide the glue between any of these elements and different use cases.  Worth noting that much of the initial discussions in the FCC's Next-Gen group were on the potential value of standardized interfaces between such different elements of measurement infrastructure.

Another element that's probably worth noting is that the use cases also may direct the actual tests that someone wants to run.  I would imagine the actual tests you might want to run for evaluating in-home LAN, access network, CDN, mobile networks may vary.  So I think the work on the test registry and other IPPM work on new tests will also improve our definitions and understanding of "broadband performance".

As always my personal views and not the FCC's.. possible valued at less than $.02  :)
On Thu, Mar 7, 2013 at 10:24 PM, Shane Amante <[email-masked]<mailto:[email-masked]>> wrote:
James, All,

-shane

Sent from my iPhone

My $0.02,

-shane

Regards, Benoit

_______________________________________________ lmap mailing list [email-masked]<mailto:[email-masked]> https://www.ietf.org/mailman/listinfo/lmap

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Mon, 11 Mar 2013 21:47:35 +0000

> somewhat different than a wire device as there are metrics that a wireless
> device has, that do not exist in a wire device. My opinion.
> 
> Sharam

I agree with Marc. Most wireless providers' cellular devices are managed with OMA-DM. Many consumer broadband devices are managed with TR-069.  Others with SNMP. OMA-DM and TR-069 both use xml data models. SNMP uses MIBs. The proposed registry would be equally mappable on any of these protocols. FWIW BBF and OMA liaise a lot to harmonize various data models. So we know it can be done.
Barbara