Subject: [lmap] Fwd: Re: SV: Pete Resnick's Discuss on draft-ietf-lmap-use-cases-05: (with DISCUSS)

From: Benoit Claise <[email-masked]>
Date: Mon, 15 Dec 2014 17:52:16 +0100

FYI, for our interim today.

Note: the "send notice" for all WG documents in the tracker now includes 
the LMAP WG.
Each update from the IESG (DISCUSS/COMMENT) will now reach the WG. This 
is now the default for now documents, but was not changed for old documents.

Regards, Benoit

-------- Original Message --------
Subject: 	Re: SV: Pete Resnick's Discuss on 
draft-ietf-lmap-use-cases-05: (with DISCUSS)
Date: 	Mon, 15 Dec 2014 10:36:28 -0600
From: 	Pete Resnick <[email-masked]>
To: 	"Sørensen, Frode" <[email-masked]>
CC: 	[email-masked] <[email-masked]>, [email-masked] 
<[email-masked]>, [email-masked] <[email-masked]>, 
[email-masked] <[email-masked]>, 
[email-masked] <[email-masked]>, 
[email-masked] 
<[email-masked]>

Hi Frode,

I'll be joining the interim call today so that we can discuss the issue, 
but let me talk about your suggested text below.

First let me say that I think it's *vital* that regulators protocol 
needs are brought to the table and I have absolutely no concerns at all 
about documenting those needs is this draft. The important issue to me 
is that we're documenting their *protocol needs*, the same way we do 
with any users of any of our protocols. What we don't do is document 
their philosophies and missions; we don't have to know those things to 
provide them with the protocol elements they require. What concerned me 
about a great deal of the text about the regulators in particular 
(though there is some in the ISP use case as well) is that it was framed 
as a discussion of their internal philosophies and political mission 
instead of the actual use case for the protocol. For example, take the 
first paragraph of the ISP section:

    A network operator needs to understand the performance of their
    networks, the performance of the suppliers (downstream and upstream
    networks), the performance of Internet access services, and the
    impact that such performance has on the experience of their
    customers. Largely, the processes that ISPs operate (which are based
    on network measurement) include:

That speaks in terms of the information that the ISP wants to get out of 
measuring the broadband network. Now, compare that to even your 
replacement first paragraph for the regulators:

    Regulators in some jurisdictions around the world are responding to
    consumers' adoption of Internet access services for traditional
    telecommunications and media services by promoting competition among
    providers of electronic communications. These regulators seek to use
    competition to increase the choice, price, and quality for users of
    Internet access services. This is more effective with better
    information, so some regulators have developed large-scale
    measurement programs, e.g. [FCC MBA] and [EC QBS], to gathering data
    to perform analysis and reporting on diverse aspects of broadband
    performance.

That speaks in terms of promotion of competition, choice, and price, not 
what the regulators wish to measure.  In the last sentence, it finally 
starts to head toward what is needed, but it doesn't come right out and 
say it, and then the second paragraph jumps right back into 
philosophical statements about "consumer, industry, and regulatory 
concerns".

Switching to using "many" or "some" does help make the text a bit less 
problematic, but I really think the overall structure of the regulatory 
discussion makes it read as if the document has a political mission. 
Changing things around to describe the kind of information the 
regulators need, as against their political purposes in wanting that 
information, would allay my concerns.

pr

On 12/7/14 11:45 AM, Sørensen, Frode wrote:
>
> Pete, all,
>
> Thank you for your review and comments. I agree that telecoms 
> regulation varies around the globe, but the decision to include the 
> regulator use case in this draft was taken at an early stage. If IESG 
> stick to that conclusion, we need to describe what regulators actually 
> do. But this use case may not be relevant in every country, and this 
> was well-known when it was decided to include regulators in the LMAP work.
>
> I would also like to add that lots of jurisdictions outside US and EU 
> have regulators which encourage competition in the telecoms market. We 
> have REGULATEL <http://www.regulatel.org/> (Latin American Forum of 
> Telecoms Regulators), EMERG <http://www.emergonline.org/> 
> (Euro-Mediterranean Regulators Group) encompassing regulators from 
> South Europe, North Africa and the Middle East, as well as EaP 
> <http://www.easternpartnership.org/> (Eastern Partnership) covering 
> Armenia, Azerbaijan, Belarus, Georgia, Moldova and Ukraine. These are 
> examples I know about based on my work in BEREC, but I assume there 
> may be others as well. But I also agree that in some countries there 
> may not be any regulator.
>
> Also net neutrality is becoming a target of telecoms regulation in 
> many regions of the world. Chile was one of the first countries, at 
> least before any European country, to introduce statutory net 
> neutrality. And I was personally attending a meeting with REGULATEL 
> discussing net neutrality regulation, and recently lawmakers have been 
> active in many countries in Latin America introducing net neutrality 
> laws. Also some African and Asian countries are reported to have or to 
> consider this. This implies that regulators may also need to measure 
> network traffic to assess compliance with net neutrality regulation.
>
> Of course, if IESG don't want to cover the regulator use case, or 
> don't want to cover LMAP use cases at all, I accept this. But I would 
> like to propose an update, which I hope will be acceptable, of the 
> description of the regulator use case which goes like this: We take on 
> board the suggestions from Alissa and Phil, in addition we can also 
> change from "many" to "some" in Phil's version, and we can move the 
> specific US and EU measurement programs to informative references, 
> while generalizing the related text in sections 2.2 and 4.2. This 
> would lead to the version below.
>
> Thanks,
>
> Frode
>
> 2.2 Regulator Use Case
>
> ----------------------------
>
> Regulators in some jurisdictions around the world are responding to
>
> consumers' adoption of Internet access services for traditional
>
> telecommunications and media services by promoting competition among
>
> providers of electronic communications. These regulators seek to use
>
> competition to increase the choice, price, and quality for users of 
> Internet
>
> access services. This is more effective with better information, so some
>
> regulators have developed large-scale measurement programs, e.g.
>
> [FCC MBA] and [EC QBS], to gathering data to perform analysis and
>
> reporting on diverse aspects of broadband performance.
>
> While each jurisdiction responds to distinct consumer, industry, and
>
> regulatory concerns, much commonality exists in the need to produce
>
> datasets that can be used to compare multiple Internet access service
>
> providers, diverse technical solutions, geographic and regional
>
> distributions, and marketed and provisioned levels and combinations
>
> of broadband Internet access services. In some jurisdictions, the
>
> role of measuring is provided by a measurement provider.
>
> Measurement providers measure network performance from users towards
>
> multiple content and application providers, including dedicated test
>
> measurement servers, to show the performance of the actual Internet
>
> access service provided by different ISPs. Users need to know the
>
> performance that they are achieving from their own ISP. In addition,
>
> they need to know the performance of other ISPs of same location as
>
> background information for selecting their ISP. Measurement providers
>
> will provide measurement results with associated measurement methods
>
> and measurement metrics.
>
> From a consumer perspective, the differentiation between fixed and
>
> mobile (cellular) Internet access services is blurring as the
>
> applications used are very similar. Hence, regulators are measuring
>
> both fixed and mobile Internet access services.
>
> A regulator's role in the development and enforcement of broadband
>
> Internet access service policies also requires that the measurement
>
> approaches meet a high level of verifiability, accuracy and provider-
>
> independence to support valid and meaningful comparisons of Internet
>
> access service performance.
>
> LMAP standards could answer regulators shared needs by providing
>
> scalable, cost-effective, scientifically robust solutions to the
>
> measurement and collection of broadband Internet access service
>
> performance information.
>
> 4 Details of Regulator Use Case
>
> --------------------------------------
>
> 4.1 Promoting competition through transparency
>
> ------------------------------------------------------------
>
> Competition plays a vital role in regulation of some electronic
>
> communications markets. For competition to successfully discipline
>
> operators' behavior in the interests of their customers, end users
>
> must be fully aware of the characteristics of the ISPs' access
>
> offers. In some jurisdictions regulators mandate that transparent
>
> information made available about service offers.
>
> End users need effective transparency to be able to make informed
>
> choices throughout the different stages of their relationship with
>
> ISPs, when selecting Internet access service offers, and when
>
> considering switching service offer within an ISP or to an
>
> alternative ISP. Quality information about service offers could
>
> include speed, delay, and jitter. Regulators can publish such
>
> information to facilitate end users' choice of service provider and
>
> offer. It may also encourage ISPs to use the same metrics in their
>
> service level contracts, which would further help end users to choose
>
> an ISP. Finally, transparency may help content, application, service
>
> and device providers develop their Internet offerings.
>
> The published information needs to be:
>
> o Accurate - the measurement results must be correct and not
>
> influenced by errors or side effects. The results should be
>
> reproducible and consistent over time.
>
> o Comparable - common metrics should be used across different
>
> ISPs and service offerings so that measurement results can be
>
> compared.
>
> o Meaningful - the metrics used for measurements need to reflect
>
> what end users value about their broadband Internet access service
>
> o Reliable - the number and distribution of measurement agents,
>
> and the statistical processing of the raw measurement data, needs
>
> to be appropriate
>
> A set of measurement parameters and associated measurement methods
>
> are used over time, e.g. speed, delay, and jitter. Then the
>
> measurement raw data are collected and go through statistical post-
>
> processing before the results can be published in an Internet access
>
> service quality index to facilitate end users' choice of service
>
> provider and offer.
>
> The regulator can also promote competition through transparency by
>
> encouraging end users to monitor the performance of their own
>
> broadband Internet access service. They might use this information to
>
> check that the performance meets that specified in their contract or
>
> to understand whether their current subscription is the most
>
> appropriate.
>
> 4.2 Promoting broadband deployment
>
> -----------------------------------------------
>
> Governments sometimes set strategic goals for high-speed broadband
>
> penetration as an important component of the economic, cultural and
>
> social development of the society. To evaluate the effect of the
>
> stimulated growth over time, broadband Internet access take-up and
>
> penetration of high-speed access can be monitored through measurement
>
> campaigns. An example of such an initiative is [DAE].
>
> To monitor actual broadband Internet access performance in a specific
>
> country or a region, extensive measurement campaigns are needed. A
>
> panel can be built based on operators and packages in the market,
>
> spread over urban, suburban and rural areas. Probes can then be
>
> distributed to the participants of the campaign.
>
> Periodic tests running on the probes can for example measure actual
>
> speed at peak and off-peak hours, but also other detailed quality
>
> metrics like delay and jitter. Collected data goes afterwards through
>
> statistical analysis, deriving estimates for the whole population
>
> which can then be presented and published regularly.
>
> Using a harmonized or standardized measurement methodology, or even a
>
> common quality measurement platform, measurement results could also
>
> be used for benchmarking of providers and/or countries.
>
> 4.3 Monitoring "net neutrality"
>
> --------------------------------------
>
> Regulatory approaches related to net neutrality and the open Internet
>
> has been introduced in some jurisdictions. Examples of such efforts
>
> are the Internet policy as outlined by the Body of European
>
> Regulators for Electronic Communications Guidelines for quality of
>
> service [BEREC Guidelines] and US FCC Preserving the Open Internet
>
> Report and Order [FCC R&O]. Although legal challenges can change the
>
> status of policy, the take-away for LMAP purposes is that 
> policy-makers are looking for
>
> measurement solutions to assist them in discovering biased treatment
>
> of traffic flows. The exact definitions and requirements vary from
>
> one jurisdiction to another; the comments below provide some hints
>
> about the potential role of measurements.
>
> A regulator may want to monitor traffic management practices or compare
>
> the performance of Internet access service with specialized services 
> offered
>
> in parallel to but separate from Internet access service  (for example 
> IPTV).
>
> A regulator could monitor departures from application agnosticism
>
> such as blocking or throttling of traffic from specific applications,
>
> and preferential treatment of specific applications. A measurement
>
> system could send, or passively monitor, application-specific traffic
>
> and then measure in detail the transfer of the different packets.
>
> Whilst it is relatively easy to measure port blocking, it is a
>
> research topic how to detect other types of differentiated treatment.
>
> The paper, "Glasnost: Enabling End Users to Detect Traffic
>
> Differentiation" [M-Labs NSDI 2010] and follow-on tool "Glasnost"
>
> [Glasnost] are examples of work in this area.
>
> A regulator could also monitor the performance of the broadband
>
> service over time, to try and detect if the specialized service is
>
> provided at the expense of the Internet access service. Comparison
>
> between ISPs or between different countries may also be relevant for
>
> this kind of evaluation.
>
> *Fra:*Sørensen, Frode [mailto:[email-masked]]
> *Sendt:* 4. desember 2014 10:23
> *Til:* [email-masked]; [email-masked]; 
> [email-masked]; [email-masked]
> *Kopi:* [email-masked]; 
> [email-masked]; [email-masked]
> *Emne:* SV: Pete Resnick's Discuss on draft-ietf-lmap-use-cases-05: 
> (with DISCUSS)
>
> Pete, all,
>
> Thanks for clarifying the background of the use case document, Blaise. 
> And thanks for your rephrasing, Phil, I support these proposals. I am 
> of course open to rephrase other parts as well, if needed. The way I 
> understand Pete's response, it is principally against the regulator 
> use case's style. Personally, I find this stuff relevant to the use 
> case, since it describes the motivation behind the use case, but of 
> course the appropriateness of this has to be decided by IESG.
>
> Best,
>
> Frode
>
> *Fra:[email-masked] 
> <mailto:[email-masked]>[mailto:[email-masked]]
> *Sendt:* 4. desember 2014 09:57
> *Til:* [email-masked] <mailto:[email-masked]>; 
> [email-masked] <mailto:[email-masked]>; 
> [email-masked] <mailto:[email-masked]>
> *Kopi:* [email-masked] 
> <mailto:[email-masked]>; 
> [email-masked] <mailto:[email-masked]>; 
> [email-masked] <mailto:[email-masked]>
> *Emne:* RE: Pete Resnick's Discuss on draft-ietf-lmap-use-cases-05: 
> (with DISCUSS)
>
> Pete,
>
> I can see your point. To take your specific examples, I have a couple 
> of proposals. Frode -- feel free to disagree!
>
> OLD
>
>     Competition plays a vital role in regulation of the electronic
>     communications markets.
>   
> NEW
>     Competition plays a vital role in regulation of many electronic
>     communications markets.
>   
> [I take your point that not all regulators will rely on competition to play a vital role]
>   
>
> <<  Governments sometimes set strategic goals for high-speed broadband
>     penetration as an important component of the economic, cultural and
>
>    social development of the society.>>
>
> I think "sometimes" makes this sentence ok -- isn't it just a 
> statement of fact?
>
> Similarly, suggest we slightly tone down things like this:-
>
> OLD
>
> Regulators in jurisdictions around the world are responding to
>     consumers' adoption of Internet access services for traditional
>     telecommunications and media services by promoting competition among
>     providers of electronic communications, to ensure that users derive
>     maximum benefit in terms of choice, price, and quality.
>
> NEW
>
> Regulators in many jurisdictions around the world are responding to
>     consumers' adoption of Internet access services for traditional
>     telecommunications and media services by promoting competition among
>     providers of electronic communications. These regulators seek to use competition to increase the choice, price, and quality for users of Internet access services.
>
> Best wishes
>
> phil
>
> *From:*Benoit Claise [mailto:[email-masked]]
> *Sent:* 04 December 2014 08:16
> *To:* Pete Resnick; The IESG
> *Cc:* [email-masked] 
> <mailto:[email-masked]>; 
> [email-masked] <mailto:[email-masked]>; 'Brian 
> Haberman'
> *Subject:* Re: Pete Resnick's Discuss on draft-ietf-lmap-use-cases-05: 
> (with DISCUSS)
>
> Pete, Brian,
>
> [Let me cover Brian's abstain as well: 
> http://datatracker.ietf.org/doc/draft-ietf-lmap-use-cases/ballot/#brian-haberman]
>
> Let's go back to the BoF and chartering.
> >From the the BoF, two primary use cases were identified: the ISP and 
> the regulators.
> The charter has been created to make sure that both use cases are 
> taken into account. It's like doing the homework before specifying the 
> solution.
> While ISPs typically participate to the IETF, it's not the case for 
> the regulators.
>
> What is the appropriate way to work with regulators? Not sure it's a WIKI.
> This is the reason why the charter clearly mentions:
>
> The LMAP WG will produce the following work items:
> 1. ...
> 2. The LMAP Use Cases - provides the motivating use cases as a basis 
> for the work
>
> The LMAP community did a good job to reach out to regulators. Having a 
> regulator in the author list, who has been acting as a point of 
> contact for many regulators in Europe, is an achievement
>
> While I support the recent trend in the IETF/IESG to move away from 
> requirements and use cases when we have a clear problem statement, I 
> believe that:
>     1. having a use cases document for LMAP is appropriate
>     2. the IESG should push back on this type of document at charter 
> creation time,
>         not when the WG provides the document.
>
> Regarding the specific wording of some paragraph, it could be 
> improved, some paragraphs might be removed but ... I'm not shocked by 
> the text.
>
> Let's discuss during the IESG telechat.
>
> Regards, Benoit
>
>     Pete Resnick has entered the following ballot position for
>
>     draft-ietf-lmap-use-cases-05: Discuss
>
>       
>
>     When responding, please keep the subject line intact and reply to all
>
>     email addresses included in the To and CC lines. (Feel free to cut this
>
>     introductory paragraph, however.)
>
>       
>
>       
>
>     Please refer tohttp://www.ietf.org/iesg/statement/discuss-criteria.html
>
>     for more information about IESG DISCUSS and COMMENT positions.
>
>       
>
>       
>
>     The document, along with other ballot positions, can be found here:
>
>     http://datatracker.ietf.org/doc/draft-ietf-lmap-use-cases/
>
>       
>
>       
>
>       
>
>     ----------------------------------------------------------------------
>
>     DISCUSS:
>
>     ----------------------------------------------------------------------
>
>       
>
>     In the end, I'm not going to stand in the way of this document if there
>
>     is consensus on its content; it is just Informational. But on our agenda,
>
>     we are specifically asked regarding Informational documents: "Is this
>
>     document a reasonable contribution to the area of Internet engineering
>
>     which it covers?" And I'd like to take a moment to DISCUSS that
>
>     question.
>
>       
>
>     There's a significant amount of the discussion of the ISP Use Case that
>
>     contains a lot of fluff about business and operational models that I
>
>     think probably doesn't belong in an IETF document, though it's nothing
>
>     out of hand. But I'm having some real trouble with some of the discussion
>
>     on the Regulator Use Case. Some examples:
>
>       
>
>     2.2 Regulator Use Case
>
>       
>
>         Regulators in jurisdictions around the world are responding to
>
>         consumers' adoption of Internet access services for traditional
>
>         telecommunications and media services by promoting competition among
>
>         providers of electronic communications, to ensure that users derive
>
>         maximum benefit in terms of choice, price, and quality.
>
>       
>
>         Competition is more effective with better information, so some
>
>         regulators have developed large-scale measurement programs. For
>
>         example, programs such as the U.S. Federal Communications
>
>         Commission's (FCC) Measuring Broadband America (MBA)...
>
>         
>
>     4.1 Promoting competition through transparency
>
>       
>
>         Competition plays a vital role in regulation of the electronic
>
>         communications markets. For competition to successfully discipline
>
>         operators' behavior in the interests of their customers, end users
>
>         must be fully aware of the characteristics of the ISPs' access
>
>         offers. In some jurisdictions regulators mandate that transparent
>
>         information imade available about service offers.
>
>         
>
>         [...]
>
>       
>
>     4.2 Promoting broadband deployment
>
>       
>
>         Governments sometimes set strategic goals for high-speed broadband
>
>         penetration as an important component of the economic, cultural and
>
>         social development of the society. To evaluate the effect of the
>
>         stimulated growth over time, broadband Internet access take-up and
>
>         penetration of high-speed access can be monitored through measurement
>
>         campaigns.
>
>       
>
>         An example of such an initiative is the "Digital Agenda for
>
>     Europe"...
>
>       
>
>     It seems completely inappropriate for an Internet *Engineering* Task
>
>     Force Working Group to be making proclamations about economic competition
>
>     and the merits (and no notable demerits!) thereof, talking about
>
>     competitions's "vital role in the regulation of electronic communications
>
>     markets", and touting programs for "economic, cultural and social
>
>     development of the society." I think it's a bit unseemly, and at some
>
>     level (and certainly for folks from different cultures) potentially
>
>     offensive. (And I haven't mentioned the whole net neutrality discussion
>
>     that Alissa points out.)
>
>       
>
>     Is this really appropriate material for an IETF consensus document? I can
>
>     see it as an IAB document, I can see it as an independent submission, but
>
>     as an IETF WG document?
>
>       
>
>     I'd really like to hear other IESG members' opinions on this. As I said,
>
>     in the end I won't stand in the way, but before I ABSTAIN, I'd like to
>
>     hear what others think.
>
>       
>
>       
>
>       
>
>       
>
>     .
>
>       
>

-- 
Pete Resnick<http://www.qualcomm.com/~presnick/>
Qualcomm Technologies, Inc. - +1 (858)651-4478