Subject: Re: [lmap] incoming liaison statement from IEEE Project P802.16.3

From: Jérôme Benoit <[email-masked]>
Date: Fri, 5 Apr 2013 14:22:45 +0200

On Thu, 28 Mar 2013 08:17:46 +0000 in
<[email-masked]>,
Romascanu, Dan (Dan) "Romascanu, Dan (Dan)" <[email-masked]> wrote:

> Hi,
Hello, 
> Please note the following incoming liaison statement from IEEE Project P802.16.3 concerning the LMAP activity. 
> 
> https://datatracker.ietf.org/liaison/1244/
> 
> Comments and discussions are welcome. 

There's no point in a private server to help anonymity. A cryptographic
system is enough that can for example one time key pair with a
timestamp as a pass-phrase- fire and forget crypto -. The main
advantage of such a system is that you do not need to cypher the
transmission, the cyphering is done at the payload level, the keys are
deleted once the measurement session is done. The key exchange rate
could be high (a downside). The server keep the private key of a
session to de-cypher. I can dig the design if the LMAP WG is in a hurry
but I will do it anyway later. I call this kind of cryptographic system
"time based crypto". 

I really would like the LMAP WG to not throw away the ISP's cheats on a
measurement campaign, it 's not an coding issue, it's a design issue
of the measurement by itself : if a measurement design can't cheat a
DPI, it's just not a good measurement design. 

Cheers. 

-- 
Jérôme Benoit aka fraggle
La Météo du Net - http://grenouille.com
OpenPGP Key ID : 9FE9161D
Key fingerprint : 9CA4 0249 AF57 A35B 34B3 AC15 FAA0 CB50 9FE9 161D

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Fri, 5 Apr 2013 16:02:18 +0000

> > Comments and discussions are welcome.

1. Private vs. Public definition
I'm really unclear on how the liaison authors are defining "Public" and "Private". I can think of 4 different ways in which these terms might be defined, and would be curious as to which was actually meant (or whether it's a 5th definition I hadn't considered).
A. Public: owned by the government; Private: owned by a private entity [this is the definition I'm used to, but it doesn't really seem to be the one that is being used, since currently all of the test servers and data collection systems are privately owned, including those used by SamKnows and service providers]
B. Public: Internet facing and no authentication required; Private: not Internet-facing or requiring authentication [doesn't seem to be what's implied regarding Measurement Servers, but does seem to be implied by the Data Collector usage, which really confuses me]
C. Public: dedicated for regulatory data measurement/collection; Private: everything else [note that employees of regulators who have participated in this debate have indicated that in order to make the test systems more cost effective, they don't think it's necessary to dedicate systems -- rather the systems used should be able to meet the needs of multiple use cases; therefore, IMO, this isn't a useful definition]
D. Public: used (but not necessarily dedicated) for regulatory data measurement/collection; Private: everything else [for example, if a regulator somewhere in the world decided they wanted a ping test run against www.google.com, that would then be labeled a "public" measurement server; SamKnows servers would be "public"; servers used by ISPs for this purpose (and other purposes) would be public; this seems to be closest to what's implied]

I'm thinking the words "Public" and "Private" perhaps shouldn't be used, because maybe there isn't a good definition that really conveys what's meant.

2. "Public Data Collectors"
First, I think we need to distinguish between the Data Collector (which gets raw data from the Measurement Agent) and any database/system/server that is used to make collected data available to people or other systems. The liaison seems to suggest that these are one and the same. That strikes me as a very unusual way to architect operation support systems. It certainly isn't how I would architect things. Personally, I consider the interfaces that a ("private") company (ISP or "company that a regulator contracts with to do measurements") has between their Data Collector and their systems that provide people/systems access to data to be outside the scope of any IETF WG. Data providing and data storage are very distinct from data collection. What does or does not constitute "private" data and how secure access to the data-that-is-made-available needs to be is probably best determined at a national level. Who gets access to the data is probably also best determined at a national level. For an ISP to create a system that allows an end user to log in and get data collected about their access line (collected for the ISP use case, the regulator use case, or explicitly for the end user case) is very do-able and doesn't need IETF help. But whether or not to do that is up to the ISP (unless the regulator/government mandates something). A contracted company could also create a mechanism that gives test participants access to their data. For an ISP or contracted company to remove private elements from data that gets provided to others is easily accomplished. What data elements to provide, whether to do such data scrubbing, and what data gets scrubbed, is a matter to be discussed at a national level. Therefore, I see absolutely no reason to discuss "Public" vs. "Private" Data Collectors. By Definition A -- unless the government owns the Data Collectors, they are all "Private", and what data ultimately gets supplied to others is determined by the owner of the data and, for the regulatory use case, the regulator. By Definition B -- there shouldn't be a Data Collector that allows for its raw, collected data to be accessed without authentication over the Internet. By Definition C -- it's certainly possible to have certain measurement data sent to a regulatory-dedicated Data Collector (and potentially also sent to a non-regulatory-dedicated Data Collector if the data is to be used for other use cases, as well), but this isn't how I would design things; I would split the data out after it gets to the Data Collector, and not complicate the MA by making it know which data gets used for what use case. Definition D doesn't seem to apply, given how the Data Collector is discussed in the liaison.

Barbara

---

From: Jérôme Benoit <[email-masked]>
Date: Sun, 7 Apr 2013 06:30:37 +0200

On Fri, 5 Apr 2013 16:02:18 +0000 in
<[email-masked]>,

> D. Public: used (but not necessarily dedicated) for regulatory data measurement/collection; Private: everything else [for example, if a regulator somewhere in the world decided they wanted a ping t…

ok, just let's me make all of this crystal clear in term of
definition : 

The measurement results content's level of privacy is what the *user*
choose to define as private or public. It's just as simple as that.
Default level is complete and total anonymity. The implementation of
"complete and total anonymity" is what is what we *NEED* to talk about.
Everything else is ... bullshit.  

> I'm thinking the words "Public" and "Private" perhaps shouldn't be used, because maybe there isn't a good definition that really conveys what's meant.

I'm in total agreement.

> First, I think we need to distinguish between the Data Collector (which gets raw data from the Measurement Agent) and any database/system/server that is used to make collected data available to people or other systems. The liaison seems to suggest that these are one and the same. That strikes me as a very unusual way to architect operation support systems. It certainly isn't how I would architect things. Personally, I consider the interfaces that a ("private") company (ISP or "company that a regulator contracts with to do measurements") has between their Data Collector and their systems that provide people/systems access to data to be outside the scope of any IETF WG. Data providing and data storage are very distinct from data collection. What does or does not constitute "private" data and how secure access to the data-that-is-made-available needs to be is probably best determined at a national level. Who gets access to the data is probably also best determined at a national 
>  level. For an ISP to create a system that allows an end user to log in and get data collected about their access line (collected for the ISP use case, the regulator use case, or explicitly for the end user case) is very do-able and doesn't need IETF help. But whether or not to do that is up to the ISP (unless the regulator/government mandates something). A contracted company could also create a mechanism that gives test participants access to their data. For an ISP or contracted company to remove private elements from data that gets provided to others is easily accomplished. What data elements to provide, whether to do such data scrubbing, and what data gets scrubbed, is a matter to be discussed at a national level. Therefore, I see absolutely no reason to discuss "Public" vs. "Private" Data Collectors. By Definition A -- unless the government owns the Data Collectors, they are all "Private", and what data ultimately gets supplied to others is determined by the owner of the 
>  data and, for the regulatory use case, the regulator. By Definition B -- there shouldn't be a Data Collector that allows for its raw, collected data to be accessed without authentication over the Internet. By Definition C -- it's certainly possible to have certain measurement data sent to a regulatory-dedicated Data Collector (and potentially also sent to a non-regulatory-dedicated Data Collector if the data is to be used for other use cases, as well), but this isn't how I would design things; I would split the data out after it gets to the Data Collector, and not complicate the MA by making it know which data gets used for what use case. Definition D doesn't seem to apply, given how the Data Collector is discussed in the liaison.
> 
> Barbara

Well ... you're just missing the point : how do you ensure the
anonymity of the measurement results ? the question is *HOW*. Period. 

The split of the data collector level of privacy is not the right
answer. It's just a bandaid on a bad design. The right question is : 

How do you guarantee a level of privacy AND anonymity of the measurement
results in your non-very-centralized design ? 

One time private or session key is just one of the answer -with
overheads-. 

The job now is to find a really good way to ensure both. I can really
help the LMAP WG but please, do use this mailing list of a primary 
tool to build the design from scratch or zero. Do not thrown dump idea
out of the box here of future IETF or IEEE standard without a
primary discussion here. 

The end result will be just like it looks right : please guys or girls,
go back to your white board and think deeper. 

Regards,

---

From: <[email-masked]>
Date: Mon, 8 Apr 2013 10:10:37 +0100

I think our visions of the LMAP framework may be rather different.

For myself (and I think Barbara) things are fairly simple. _All_ data collection is private. Someone installs Measurement Agents and collects the results. This communication will be secured. Outside the technical system there will be an agreement of what data is collected and what can be done with it. We certainly don't want to be in the business of a user creating electronic use policies for the data.

The Measurement Agent will have an identifier that can be linked with a living subject by the Controller (although the Measurement Agent is unlikely to know either the details of this person itself). This has to be the case since the Controller installed/configured this measurement agent in the first place. The framework would allow the Collector to be a party who does not have this information - practically though the data can be collected by the same organisation as the Controller of the Measurement Agents, processed to whatever standards of blurring/blending/anonymity is required before passing to other parties.

Nowhere in our use cases do we seem to have a requirement to have Measurement Agents reporting directly to a publically accessible datastore.

Trevor.

>-----Original Message-----
>From: [email-masked] [mailto:[email-masked]] On Behalf Of
>Jérôme Benoit
>Sent: 07 April 2013 05:31
>To: [email-masked]
>Subject: Re: [lmap] incoming liaison statement from IEEE Project P802.16.3
>
>> I'm really unclear on how the liaison authors are defining "Public" and
>"Private". I can think of 4 different ways in which these terms might be
>defined, and would be curious as to which was actually meant (or whether
>it's a 5th definition I hadn't considered).
>> A. Public: owned by the government; Private: owned by a private entity
>> [this is the definition I'm used to, but it doesn't really seem to be
>> the one that is being used, since currently all of the test servers
>> and data collection systems are privately owned, including those used
>> by SamKnows and service providers] B. Public: Internet facing and no
>> authentication required; Private: not Internet-facing or requiring
>> authentication [doesn't seem to be what's implied regarding
>> Measurement Servers, but does seem to be implied by the Data Collector
>> usage, which really confuses me] C. Public: dedicated for regulatory
>> data measurement/collection; Private: everything else [note that
>> employees of regulators who have participated in this debate have
>> indicated that in order to make the test systems more cost effective,
>> they don't think it's necessary to dedicate systems -- rather the
>> systems used should be able to meet the needs of multiple use cases;
>> therefore, IMO, this isn't a useful definition] D. Public: used (but
>> not necessarily dedicated) for regulatory data measurement/collection;
>> Private: everything else [for example, if a regulator somewhere in the
>> world decided they wanted a ping test run against www.google.com, that
>> would then be labeled a "public" measurement server; SamKnows servers
>> would be "public"; servers used by ISPs for this purpose (and other
>> purposes) would be public; this seems to be closest to what's implied]
>
>ok, just let's me make all of this crystal clear in term of definition :
>
>> I'm thinking the words "Public" and "Private" perhaps shouldn't be used,
>because maybe there isn't a good definition that really conveys what's
>meant.
>
>I'm in total agreement.
>
>>
>> First, I think we need to distinguish between the Data Collector
>> (which gets raw data from the Measurement Agent) and any
>> database/system/server that is used to make collected data available to
>people or other systems. The liaison seems to suggest that these are one and
>the same. That strikes me as a very unusual way to architect operation
>support systems. It certainly isn't how I would architect things. Personally, I
>consider the interfaces that a ("private") company (ISP or "company that a
>regulator contracts with to do measurements") has between their Data
>Collector and their systems that provide people/systems access to data to
>be outside the scope of any IETF WG. Data providing and data storage are
>very distinct from data collection. What does or does not constitute
>"private" data and how secure access to the data-that-is-made-available
>needs to be is probably best determined at a national level. Who gets access
>to the data is probably also best determined at a national  level. For an ISP to
>create a system that allows an end user to log in and get data collected
>about their access line (collected for the ISP use case, the regulator use case,
>or explicitly for the end user case) is very do-able and doesn't need IETF help.
>But whether or not to do that is up to the ISP (unless the
>regulator/government mandates something). A contracted company could
>also create a mechanism that gives test participants access to their data. For
>an ISP or contracted company to remove private elements from data that
>gets provided to others is easily accomplished. What data elements to
>provide, whether to do such data scrubbing, and what data gets scrubbed, is
>a matter to be discussed at a national level. Therefore, I see absolutely no
>reason to discuss "Public" vs. "Private" Data Collectors. By Definition A --
>unless the government owns the Data Collectors, they are all "Private", and
>what data ultimately gets supplied to others is determined by the owner of
>the  data and, for the regulatory use case, the regulator. By Definition B --
>there shouldn't be a Data Collector that allows for its raw, collected data to
>be accessed without authentication over the Internet. By Definition C -- it's
>certainly possible to have certain measurement data sent to a regulatory-
>dedicated Data Collector (and potentially also sent to a non-regulatory-
>dedicated Data Collector if the data is to be used for other use cases, as well),
>but this isn't how I would design things; I would split the data out after it
>gets to the Data Collector, and not complicate the MA by making it know
>which data gets used for what use case. Definition D doesn't seem to apply,
>given how the Data Collector is discussed in the liaison.
>>
>> Barbara
>
>
>Well ... you're just missing the point : how do you ensure the anonymity of
>the measurement results ? the question is *HOW*. Period.
>
>The split of the data collector level of privacy is not the right answer. It's just
>a bandaid on a bad design. The right question is :
>
>How do you guarantee a level of privacy AND anonymity of the
>measurement results in your non-very-centralized design ?
>
>One time private or session key is just one of the answer -with overheads-.
>
>The job now is to find a really good way to ensure both. I can really help the
>LMAP WG but please, do use this mailing list of a primary tool to build the
>design from scratch or zero. Do not thrown dump idea out of the box here of
>future IETF or IEEE standard without a primary discussion here.
>
>The end result will be just like it looks right : please guys or girls, go back to
>your white board and think deeper.
>
>Regards,
>
>--

---

From: Jérôme Benoit <[email-masked]>
Date: Mon, 8 Apr 2013 19:33:39 +0200

On Mon, 8 Apr 2013 10:10:37 +0100 in
<[email-masked]>,
[email-masked] <[email-masked]> wrote:

> For myself (and I think Barbara) things are fairly simple. _All_ data collection is private. 

Why ? 
If I want to do data mining in order to classify ISP, I do not want
to have to ask : 

* for a permission to access the data; 
* for a permission from all users that run the measurement agent to
  avoid some legal issue. 

The system must ensure technically that the data will stay anonymous
and because of that, freely available to anyone that want to dig
them.   

> Someone installs Measurement Agents and collects the results. This communication will be secured. Outside the technical system there will be an agreement of what data is collected and what can be done with it. We certainly don't want to be in the business of a user creating electronic use policies for the data.

It's not a matter of business, it's a matter of ethics : it's the users
data, it's not the collectors data and the measurement system must be
thoughts with that in mind. 

> The Measurement Agent will have an identifier that can be linked with a living subject by the Controller (although the Measurement Agent is unlikely to know either the details of this person itself…

OK, but you move the pb to a another level in the architecture : Can
the measurement agent trust all the data collectors ?
Can the user behind the measurement agent (that can perform passive
measurement) just blindly throw raw results to a third party data
collector runt by people he do not know ? 
    
> Nowhere in our use cases do we seem to have a requirement to have Measurement Agents reporting directly to a publically accessible datastore.

Great, so you design a measurement system that directly express : hey
users, trust blindly your private data-store, even if
we can capture your passwords, trust all the people running the
system ... active or passive measurement, the owner of the data is
the end-user and it's just common sense to let's the user choose what
he want other people will do with his data and It must be covered not
only legally but by design.

The measurement system your trying the design if for private
interest, the measurement system I will like to see designed is
for public interest.

---

From: Brian Trammell <[email-masked]>
Date: Tue, 9 Apr 2013 09:40:56 +1200

hi Jérôme, all,

Apologies for the derail, but this thread has touched on an important point on anonymity in network measurement.

> and because of that, freely available to anyone that want to dig

Depending on the exact schema of the data collected, relying on technical means to ensure anonymity while still providing for differentiation of individual nodes is not realistic. The utility of a given data set for a given question is inherently related to the utility of the dataset for the "question" of identity, and there is a deep and unavoidable tradeoff between risk of deanonymization and dataset utility. Given that the networks being measured (1) have a discernible and well-known structure and (2) link PII (an address/time tuple) with that structure, there exist practical attacks against structured-mapping approaches to data anonymization.

Best practices in measurement system design (including discarding identifying information as soon as possible in the process, i.e. at the MA or first collector; aggregation of results built into the measurement agents or mediating collectors; and so on) can mitigate these issues somewhat. Requiring a measurement system to control and audit access to measurement specifications and results, limiting access to trusted actors who can be contractually or legally punished in the case of abuse, must be part of the solution as well. (It need not be part of a standard measurement architecture, unless the authentication, authorization, and auditing must interact across domain boundaries).

Open publication of results with anonymous _access_ would require the _complete_ removal of node- or network-level addressing information, publication only of aggregated results verified to contain information from enough individual nodes per data point to ensure anonymity. 

(Back on topic, I agree that the IEEE choice of "public" and "private" for terminology apparently meaning "external" and "internal" is regrettable, and would suggest as part of the LMAP response to this liaison be that they reconsider this terminological choice, because it would seem to have the potential for a great deal of confusion.)

Best regards,

Brian

---

From: Henning Schulzrinne <[email-masked]>
Date: Mon, 8 Apr 2013 22:22:28 +0000

I think it's important to distinguish between active data (e.g., ping delays) and passive data. Knowing the source (IP address) of pings on a fixed connection doesn't appear to divulge PII. For mobile data, location information is an obvious concern, even with active measurements. This discussion has been had in great detail as part of the mobile Measurement Broadband America project; you might find the resulting solutions and trade-offs to be of interest.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Brian Trammell
Sent: Monday, April 08, 2013 5:41 PM
To: Jérôme Benoit
Cc: [email-masked]
Subject: Re: [lmap] incoming liaison statement from IEEE Project P802.16.3

hi Jérôme, all,

On 9 Apr 2013, at 5:33, Jérôme Benoit <[email-masked]> wrote:

> <[email-masked]
> ost.net>, [email-masked] <[email-masked]> wrote:
> 

Best regards,

Brian
_______________________________________________
lmap mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/lmap

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Mon, 8 Apr 2013 22:32:02 +0000

> If I want to do data mining in order to classify ISP, I do not want to have to ask
> :
> 

<bhs> Case 1: MA and Data Collector managed by ISP. The ISP will be bound by national privacy laws. Any data that the ISP is required, by law, to make available to others, it will. Who gets access and whether they have to ask permission (from the regulatory agency) to access the legally-mandated data is something that needs to be determined at a national level. Requiring someone who wants access to the legally-mandated data to ask permission from end users is unrealistic. In this case (where the ISP collects the data), the ISP can be expected to get that end user permission. Sharing of any other (not covered by legal mandates) data is at the discretion of the ISP, so long as the ISP abides by laws, and legally binding customer agreements. It's also a good idea to take cultural norms and opinions of privacy advocates into account. For this case, I consider the interface between the MA and the Data Collector to be "in scope" of an lmap architecture. All other interfaces to the Data Collector are "out of scope" (again, IMO). The tests and test endpoints will meet nationally-determined legal requirements. The regulatory agency may decide to require tests recommended by IETF, or some other org.

Case 2: MA managed by ISP, Data Collector managed by regulatorily-identified entity. The ISP will make sure the MA only provides the mandated data. All else is as above.

Case 3: MA and Data Collector managed by regulatorily-identified entity. Whatever the regulator decides is what happens. If the regulator mandates that the ISP provide test endpoints, then that's what will occur. If the regulator has this other entity provide test endpoints, then that's what will happen.

Case 4: MA managed by the end user. This is appropriate for an end user collecting their own measurement data, but I would recommend against trusting such data for any regulatory purpose. What data the end user collects and who they send it to is up to them. The test endpoints they test against is up to them and/or whoever provided them with the MA.

Summary: For regulatory-mandated data collection, the regulator makes the rules. Hopefully they do this together with ISPs and interested parties and make sure that end user privacy is taken into account. For all other data, whoever collects it gets to say what happens to it (while abiding by all applicable laws).

> The system must ensure technically that the data will stay anonymous and
> because of that, freely available to anyone that want to dig
> them.

<bhs> Whoever has access to data must obey applicable laws concerning that data. Whether there will exist any set of "freely available" data is a matter of national policy. If national policy requires that access to all collected data be restricted, then there will be no "freely available" data. If there is no regulatory requirement for any data to be made available, then those who have data have the ability to decide whether, to whom, what, and how to make data available to others, within legal bounds and customer agreements. An "lmap architecture" will not know what the applicable laws are; therefore the "system" is *not* required technically to ensure data anonymity and availability. That is the responsibility of the entities who have the data.

> > Someone installs Measurement Agents and collects the results. This
> communication will be secured. Outside the technical system there will be an
> agreement of what data is collected and what can be done with it. We
> certainly don't want to be in the business of a user creating electronic use
> policies for the data.
> 
> It's not a matter of business, it's a matter of ethics : it's the users data, it's not
> the collectors data and the measurement system must be thoughts with that
> in mind.
<bhs> No, it absolutely is the collector's data. Once someone has the data, the only thing that guides them are laws. Ethics guide laws. Different nations have different standards of ethics. I agree that it's not a matter of business. But the only thing protecting the user, once the data has left the user's premises, are laws.

> > The Measurement Agent will have an identifier that can be linked with a
> living subject by the Controller (although the Measurement Agent is unlikely
> to know either the details of this person itself). This has to be the case since
> the Controller installed/configured this measurement agent in the first place.
> The framework would allow the Collector to be a party who does not have
> this information - practically though the data can be collected by the same
> organisation as the Controller of the Measurement Agents, processed to
> whatever standards of blurring/blending/anonymity is required before
> passing to other parties.
> 
> OK, but you move the pb to a another level in the architecture : Can the
> measurement agent trust all the data collectors ?
> measurement) just blindly throw raw results to a third party data collector
> run by people he do not know ?

<bhs> That is for the user to decide, before agreeing to have that MA (that sends data to whatever Data Collector) in his/her home network or device. But once the user has agreed to have that MA, everything else is outside the user's control. The only control the user has is whether or not to allow the MA to be there.

> > Nowhere in our use cases do we seem to have a requirement to have
> Measurement Agents reporting directly to a publically accessible datastore.
> 
> users, trust blindly your private data-store, even if we can capture your
> passwords, trust all the people running the system ... active or passive
> measurement, the owner of the data is the end-user and it's just common
> sense to let's the user choose what he want other people will do with his
> data and It must be covered not only legally but by design.
<bhs> No, it's only the user's data if it's the user's MA and the user's Data Collector. Once the data has left the user's hands, someone else has possession of it, and only laws are really able to govern what happens to it. If a user doesn't feel they can trust whoever receives the data, then they won't agree to participate. Which should be their right. But once they've agreed, they've agreed. Unless they revoke their agreement. It's a binary condition that doesn't require a complex technical solution to manage.

> The measurement system your trying the design if for private interest, the
> measurement system I will like to see designed is for public interest.

<bhs> No, the architecture that I am working towards is an architecture that I believe it is reasonable to expect ISPs to be willing to implement and deploy. If someone wants to create an architecture that is only useful if an ISP implements/deploys it, but which no ISP can reasonably be expected to willingly agree to, then I have no interest in such an architecture. If someone wants to create an architecture that doesn't rely on the ISP doing anything, at all, that's fine with me, too. That's a "don't care" for me. 

Barbara

---

From: Brian Trammell <[email-masked]>
Date: Tue, 9 Apr 2013 10:50:18 +1200

Hi, Henning, all,

This is a good point: passively observed data is _far_ more sensitive to these sorts of concerns. 

However, since active measurement traffic on (fixed) broadband links necessarily shares a bottleneck link with user traffic*, information about the volume and timing of that user traffic may be inferred from the delay and bandwidth numbers derivable from measurement traffic. In cases in which the measurement system suspends active measurements during periods of user-generated cross traffic, you get less information, but the details of the measurement schedules themselves may leak information about user activity periods and (very rough estimates of) volume.

Unlike such inference attacks on anonymized passive measurement data, I haven't actually looked into this in any detail. But it seems like information in the timing patterns could at _least_ serve to sort users into "interactive-dominated" versus "bulk-dominated" (i.e. P2P) bins. Though, indeed, that's almost certainly not enough information to break address anonymization.

It's still not safe to assume that _just_ because it's not user traffic that there's no risk. Published data still needs to be evaluated for its utility to determining user identity and activity, though on thinking about it I'm much less pessimistic about the prospect of a reasonable tradeoff existing in the active measurement case. :)

Best regards,

Brian

---

From: "Bugenhagen, Michael K" <[email-masked]>
Date: Tue, 9 Apr 2013 06:00:13 +0000

Brian, Henning,

As Brian mentioned - If the test framework is "open and flexible," and contains a probe placed within a "home network LAN" can expose a great deal more information than the performance of the Access Service itself.

1) What type of information is accessible (just about all of it depending upon where the probe is placed) 
2) Who's information is it (the customer - aka yours and mine)
 
  We have to take these into the "how to preserve the rights of the individual" bucket as part of the solution IMHO.  

My suggestion here is to avoid trying to solve a legal problem in a technical standard, and try to find a technical solution to enable the customer to control what happens to their data, vs. assuming it will be made public and trying to solve that.

Otherwise we may be doomed to solve the problems associated with outer realms of data security, privacy rules, and defining what exactly is anonymity. 
Like  -  How long can someone beyond the customer themselves "hold" data that contains private information ?
	-  What are the precautions for securing data, are there reporting responsibilities if exposed ?
	-  Who makes the rules on what's considered private, who gets access to that and why   
....

Regards,
Mike

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Brian Trammell
Sent: Monday, April 08, 2013 5:50 PM
To: Henning Schulzrinne
Cc: Jérôme Benoit; [email-masked]
Subject: [lmap] on anonymity and active measurements Re: incoming liaison statement from IEEE Project P802.16.3

Hi, Henning, all,

Best regards,

Brian

On 9 Apr 2013, at 10:22, Henning Schulzrinne <[email-masked]> wrote:

>> <[email-masked]
>> h ost.net>, [email-masked] <[email-masked]> wrote:
>> 
> (Back on topic, I agree that the IEEE choice of "public" and "private" 
> for terminology apparently meaning "external" and "internal" is 
> regrettable, and would suggest as part of the LMAP response to this 
> liaison be that they reconsider this terminological choice, because it 
> would seem to have the potential for a great deal of confusion.)
> 
> Best regards,
> 
> Brian

---

From: <[email-masked]>
Date: Tue, 9 Apr 2013 09:37:40 +0100

The use cases we have require the identity of the individual and/or line. For a regulator they need to link this to ISP and product, location, market, line length, marketed speed and other attributes. This information will not be publically available to any measurement agent. If there is a problem with the measurement device they need to contact individuals involved.

An ISP operating a measurement platform needs the line identifier to be able to diagnose and resolve network problems. We don't currently have primary use cases which would drive deployment of the measurement devices and would require anonymised data. If there is a secondary use (e.g. by a party who does not operate the measurement framework) then the first party can apply anonymisation etc.

Trevor.

>Sent: 08 April 2013 18:34
>To: [email-masked]
>If I want to do data mining in order to classify ISP, I do not want to have to
>ask :
>
>It's not a matter of business, it's a matter of ethics : it's the users data, it's
>not the collectors data and the measurement system must be thoughts with
>that in mind.
>
>>
>runt by people he do not know ?
>

---

From: Jérôme Benoit <[email-masked]>
Date: Tue, 9 Apr 2013 16:37:12 +0200

On Mon, 8 Apr 2013 22:32:02 +0000 in
<[email-masked]>,
STARK, BARBARA H "STARK, BARBARA H" <[email-masked]> wrote:

> <bhs> Case 1: MA and Data Collector managed by ISP. The ISP will be bound by national privacy laws. Any data that the ISP is required, by law, to make available to others, it will. Who gets access and whether they have to ask permission (from the regulatory agency) to access the legally-mandated data is something that needs to be determined at a national level. Requiring someone who wants access to the legally-mandated data to ask permission from end users is unrealistic. In this case (where the ISP collects the data), the ISP can be expected to get that end user permission. Sharing of any other (not covered by legal mandates) data is at the discretion of the ISP, so long as the ISP abides by laws, and legally binding customer agreements. It's also a good idea to take cultural norms and opinions of privacy advocates into account. For this case, I consider the interface between the MA and the Data Collector to be "in scope" of an lmap architecture. All other interfaces to the 
>  Data Collector are "out of scope" (again, IMO). The tests and test endpoints will meet nationally-determined legal requirements. The regulatory agency may decide to require tests recommended by IETF, or some other org.
>
> 

Case 5 : MA run by end user; landmarks run by ISP, end-users (the MA
embed landmark functionalities), non-profit organisation, who knows;
data collectors run a non-profit organisation, ISP, national
agency that regulate "communication", who knows; data mining done by
ISP, non-profit organization, etc.  

Internet is designed as an end to end open commutated network and the
use case LMAP you describe do not fellow at least the same policy. It's
rather a bad idea that will just end to people's trust in the LMAP
results.  

> Summary: For regulatory-mandated data collection, the regulator makes the rules. Hopefully they do this together with ISPs and interested parties and make sure that end user privacy is taken into a…

I do not care about the rules made by people I can't get in touch with, I care about 
the level of privacy, the level of trust *people* will give to the LMAP
and the technical side of these problematics to implement it in the
LMAP.    

> <bhs> Whoever has access to data must obey applicable laws concerning that data. Whether there will exist any set of "freely available" data is a matter of national policy. If national policy requi…

I strongly disagree with your opinions here : The whole point of the
LMAP is to offer to the people a measurement system that they can
*TRUST*. 
You're just building the opposite : measurement done with
opacity, measurement results collected and "stolen" from the people,
mining that can't be verified by anyone by default. 
That kind of measurement system is so away from the Internet way of
designing protocols that you will just kill the baby before he grows
up.   

> <bhs> No, it absolutely is the collector's data. Once someone has the data, the only thing that guides them are laws. Ethics guide laws. Different nations have different standards of ethics. I agre…

I'm an engineer and most the people here probably so. Laws are for
lawyers. 
If you're engineering an LMAP to do not offer the minimum privacy
requirement to enforce people trust in the system with measurements' : 

* transparency; 
* openness; 
* privacy; 
* neutrality. 

Well, ... you're just missing the point of an LMAP made by the people
and for the people that will last and last and last and last. 

> <bhs> That is for the user to decide, before agreeing to have that MA (that sends data to whatever Data Collector) in his/her home network or device. But once the user has agreed to have that MA, e…

I propose a design where people have control over their data. You must
really have a good argument to throw the idea away. So far, you've not
raised a single good argument against it : the legalese is out of the
scope of an engineering decision that can ensure a good level of
privacy in the measurement system in regard to the very high level
of private raw datas that will go in a datastore for passive
measurement.  

> <bhs> No, it's only the user's data if it's the user's MA and the user's Data Collector. Once the data has left the user's hands, someone else has possession of it, and only laws are really able to…

Sure : let's me make you understand one basic condition of user participation in
such a system : no single user will give to anyone a raw packets capture
without strong technical guarantee that the third party will not be
able to associate the capture to the user (and even with technical
guarantee, user will still be hesitating on giving away raw packets
capture). So either the future WG acknowledge this very simple fact,
either all passive measurement that need mining on raw packets capture
will just never happen is the real world. 

> <bhs> No, the architecture that I am working towards is an architecture that I believe it is reasonable to expect ISPs to be willing to implement and deploy. If someone wants to create an architect…

See Case 5

Regards,

---

From: <[email-masked]>
Date: Tue, 9 Apr 2013 16:08:33 +0100

>Sent: 09 April 2013 15:37
>To: STARK, BARBARA H
>Cc: [email-masked]
...
>embed landmark functionalities), non-profit organisation, who knows; data
>collectors run a non-profit organisation, ISP, national agency that regulate
>"communication", who knows; data mining done by ISP, non-profit
>organization, etc.

If you have a use case that you clearly think needs to be accommodated in the design of the LMAP framework, then write it up. Then we can have the discussion of whether this use case can be done with a certain design. "who-knows" is not good enough - we can't design a system to requirements that may or may not exist. In my opinion this use case needs to be for the benefit of someone who might actually pay for a measurement framework - i.e. there needs to be a deployment motivation. At the end of the day if these additional use cases push the design to be too complex, costly, late or not fit for certain other use cases then it may never get deployed.

Trevor.

---

From: Jérôme Benoit <[email-masked]>
Date: Tue, 9 Apr 2013 17:14:05 +0200

On Tue, 9 Apr 2013 09:40:56 +1200 in
<[email-masked]>, Brian Trammell

> Apologies for the derail, but this thread has touched on an important point on anonymity in network measurement.

I hope so :)  

> Depending on the exact schema of the data collected, relying on technical means to ensure anonymity while still providing for differentiation of individual nodes is not realistic. The utility of a…

The trade-off on anonymity is probably unavoidable (but I must still
think about it some more) but it must exist in the design. So far, 
the RFCs do not enforce a policy on it.  

> Best practices in measurement system design (including discarding identifying information as soon as possible in the process, i.e. at the MA or first collector; aggregation of results built into the measurement agents or mediating collectors; and so on) can mitigate these issues somewhat. Requiring a measurement system to control and audit access to measurement specifications and results, limiting access to trusted actors who can be contractually or legally punished in the case of abuse, must be part of the solution as well.

I agree. So time to put this BCPs in the RFC ?
All the technical details will go in further RFCs but pretty please
state that this issue on privacy and anonymity will be take into
account. 

> (It need not be part of a standard measurement architecture, unless
> the authentication, authorization, and auditing must interact across
> domain boundaries).

I disagree :)
It's not that hard to design a cryptographic system inside the LMAP
that can trade-off between the crypto load in the system and the level
of privacy.

---

From: Jérôme Benoit <[email-masked]>
Date: Tue, 9 Apr 2013 17:49:04 +0200

On Tue, 9 Apr 2013 16:08:33 +0100 in
<[email-masked]>,

> If you have a use case that you clearly think needs to be accommodated in the design of the LMAP framework, then write it up. Then we can have the discussion of whether this use case can be done wi…

The non-profit organisation I participate as a volunteer have landmarks
inside the ISP backbone, outside the ISP backbone runt by volunteers,
peering association, hosting entreprise. The MA are run on end-user
computer. The MA is designed and implemented to send data to different
data collector because ISP, researchers, advanced end-user want the
data. The MA is responsible of the data obfuscation (the new one, not
the old one). It's as simple as it is : the use case do already exist
and the implementation is on it's way to reflect the use case. 

  / Data Collector
MA- Data Collector  
  \ Data Collector

 - is a connector, default is a REST protocol but the code is modular. 

Why not do data collector synchronization ? 
Because we want the MA user to choose to whom he want to send the data. 

Why we choose to anonymize the data ?
Because the personal user data are belonging to the user and not to
anyone else. 

Where we choose to anonymize ? 
The MA. It's in the hand of the user with good default values.  

How ? 
Under discussion. Results are expressed in JSON syntax. JSONSEC
extension (like XMLSEC) is under evaluation, spec to be written as well
as other way to anonymize the data : discard one octet of IP
addresses, discard fields in datagram, constant hash where possible for
certain measurement that do not the know the content if the content is
different, etc. 

Cheers.

---

From: <[email-masked]>
Date: Tue, 9 Apr 2013 17:04:29 +0100

I don't see why this isn't covered by the envisaged framework - user owns the controller and MA, schedules their own tests and specifies the collectors of the data for the various tests as part of the test configuration. The result report will include a random UUID for example (necessary to correlate multiple reports from the same probe). In your case we would not perform MA authentication I guess. If the user wants to send these results to the collector via a proxy or TOR network or similar to hide their IP address this would be possible.

Trevor.

Trevor Burbridge
Network Infrastructure & Innovation | BT Innovate & Design
Tel: 01473 645115
Fax: 01473 640929

This email contains BT information, which may be privileged or confidential. It's meant only for the individual(s) or entity named above. If you're not the intended recipient, note that disclosing, copying, distributing or using this information is prohibited. If you've received this email in error, please let me know immediately on the email address above. Thank you.
We monitor our email system, and may record your emails. 
British Telecommunications plc
Registered office: 81 Newgate Street London EC1A 7AJ
Registered in England no: 1800000 

>Sent: 09 April 2013 16:49
>To: [email-masked]
>> >data collectors run a non-profit organisation, ISP, national agency
>> >that regulate "communication", who knows; data mining done by ISP,
>> >non-profit organization, etc.
>>
>> If you have a use case that you clearly think needs to be accommodated in
>the design of the LMAP framework, then write it up. Then we can have the
>discussion of whether this use case can be done with a certain design. "who-
>knows" is not good enough - we can't design a system to requirements that
>may or may not exist. In my opinion this use case needs to be for the benefit
>of someone who might actually pay for a measurement framework - i.e.
>there needs to be a deployment motivation. At the end of the day if these
>additional use cases push the design to be too complex, costly, late or not fit
>for certain other use cases then it may never get deployed.
>
>
>data collector because ISP, researchers, advanced end-user want the data.
>The MA is responsible of the data obfuscation (the new one, not the old
>one). It's as simple as it is : the use case do already exist and the
>implementation is on it's way to reflect the use case.
>
>  / Data Collector
>MA- Data Collector
>  \ Data Collector
>
>
>Because the personal user data are belonging to the user and not to anyone
>else.
>
>Under discussion. Results are expressed in JSON syntax. JSONSEC extension
>(like XMLSEC) is under evaluation, spec to be written as well as other way to
>anonymize the data : discard one octet of IP addresses, discard fields in
>datagram, constant hash where possible for certain measurement that do
>not the know the content if the content is different, etc.
>
>Cheers.
>
>--

---

From: Jérôme Benoit <[email-masked]>
Date: Tue, 9 Apr 2013 19:33:42 +0200

On Tue, 9 Apr 2013 17:04:29 +0100 in
<[email-masked]>,
[email-masked] <[email-masked]> wrote:

> I don't see why this isn't covered by the envisaged framework - user owns the controller and MA, schedules their own tests and specifies the collectors of the data for the various tests as part of the test configuration. The result report will include a random UUID for example (necessary to correlate multiple reports from the same probe).

The level of privacy of measurement data or meta-data is non stated
anywhere in the RFCs, not even taken into account. And it's not too
late to define different level of privacy in the RFCs

> In your case we would not perform MA authentication I guess. If the user wants to send these results to the collector via a proxy or TOR network or similar to hide their IP address this would be possible.

I will but with the measurement agent coordinator, not with the data
collector as long as a level of privacy is defined on measurement
results with an corresponding implementation of the level of privacy.
It can be a measurement based policy or a global policy.
Measurement are either user-defined or coordinated.

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 9 Apr 2013 20:49:07 +0200

> late to define different level of privacy in the RFCs

So please, write an I-D discussing the privacy of measurement data
and meta-data and how it can be protected.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Jérôme Benoit <[email-masked]>
Date: Tue, 9 Apr 2013 22:24:24 +0200

On Tue, 9 Apr 2013 20:49:07 +0200 in
<[email-masked]>, Juergen Schoenwaelder Juergen

> and meta-data and how it can be protected.

Can it wait for a week ? 
I'm travelling right now and should be able to write only something at
the end of the next week.

---

From: Hannes Tschofenig <[email-masked]>
Date: Tue, 9 Apr 2013 23:30:50 +0300

Hi Dan, 

thanks for sending the liaison around. I read through it and I have a few remarks. 

It might be useful to highlight that the IETF#86 discussion was a BOF and the purpose of the BOF is not to work out the solution but rather to discuss whether a working group should be formed. As such, it is quite naturual that not all design aspects have been discussed during that meeting. 

With regard to the questions:

Question 1: "we wonder if the LMAP discussions have considered the use of both public and private Measurement Servers

Protocol work, as it is usually done in the IETF, does not imply that a specific entity runs the server. As such, any work that comes out of a (yet-to-be-created) LMAP working group will most likely be run by public and privacy entities. I would in general say that there is a distinction between the protocol aspects and the deployment aspect. Who runs a specific service is a deployment question and not necessarily a protocol design question. 

"Private" in this context is a also a bit confusing because it does not necessarily mean that private only refers to a server run by the same entity that runs the client, particularly if the client is located on the end device (under the control of the end user).

Question 2: "From our perspective, privacy is a core issue and helped to motivate our architecture at the most basic level."

Security and privacy are certainly dealt with in IETF protocols although I have to say that the range of what an acceptable level of privacy and security protection constitutes varies quite significantly between different participants. Even the understanding of what privacy and security is varies quite a lot. I am sure the IEEE in the meanwhile has noticed that as well with their experience in WEP. There are two documents that are worthwhile to look at to make sure that we actually talk the same language: 

For security have a look at RFC 3552 (see http://tools.ietf.org/html/rfc3552) and for privacy take a look at http://tools.ietf.org/html/draft-iab-privacy-considerations-07. 

In Section 7 of the privacy document you can find a couple of questions that are, of course, also applicable to the LMAP work, namely: 

 - Data Minimization
 - User Participation
 - Security
 - General

Have a look at this document since there are not too many documents around that give privacy guidelines for people working in a standards developing context. 

One of the most important parts in this work is certainly the user participation, which typically is easier to accomplish when the architecture allows the end user to actually give consent to the data collection selectively and also allows the end user to revoke their consent at any time. In general, such consent is more difficult to provide when the functionality is provided purely network internally, i.e., when the user does not have any new software running on their end devices. Luckily the measurements are most meaningful when they are truely end-to-end tests rather than testing a sub-set of the end-to-end communication path (which had been earlier in other measurement projects). 

Another important point is certainly what information is collected and provided to third parties. Anonymizing data might be a possibility but truely anonymized data are less meaningful. A future working group will have to certainly look at the trade-offs between the different identifiers involved. 

Ciao
Hannes

PS: I may not know enough about the IEEE but the statement "our project is targeted directly at mobile users (independent of air interface technology)" raised some questions for me. I believe the IEEE could best provide their expertise when they work on radio related technology. 

On Mar 28, 2013, at 10:17 AM, Romascanu, Dan (Dan) wrote:

> Dan

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 9 Apr 2013 23:15:34 +0200

> the end of the next week. 

There is no working group yet, there are no milestones yet, so take
all the time you need.

/js

---

From: Jérôme Benoit <[email-masked]>
Date: Wed, 10 Apr 2013 21:54:34 +0200

> The use cases we have require the identity of the individual and/or line. For a regulator they need to link this to ISP and product, location, market, line length, marketed speed and other attribut…

The user have the measurement agent and have the information that will
permit to access the link caracteristics, it's measurement meta-data. 
"wild" measurement agent or "wild" measurement are not a pb. They are a
problem if you deliberatly choose to not include them by design. 

> An ISP operating a measurement platform needs the line identifier to be able to diagnose and resolve network problems. We don't currently have primary use cases which would drive deployment of the…

ISPs operate the whole measurement system that will permit to evaluate themselves 
with no mechanisms that will permit to ensure the neutrality of the
measurement system ... I hope you're kidding ? If not, such a
measurement system where judge are also the party is just a big joke
that will cost money to the people and where people will be able to
participate, review, trust ... fantastic ! No ?

++

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 10 Apr 2013 22:36:39 +0200

On Wed, Apr 10, 2013 at 09:54:34PM +0200, Jérôme Benoit wrote:
> 
> ISPs operate the whole measurement system that will permit to
> evaluate themselves with no mechanisms that will permit to ensure
> the neutrality of the measurement system ... I hope you're kidding ?
> If not, such a measurement system where judge are also the party is
> just a big joke that will cost money to the people and where people
> will be able to participate, review, trust ... fantastic ! No ?

There are multiple use cases with different motivations and different
goals and we should not mess them up. The ISP use case has a rather
detailed description:

http://tools.ietf.org/html/draft-linsner-lmap-use-cases-02

Perhaps it helps if we discuss concrete parts of the text in this
draft so that we can improve the document.

/js