Subject: RE: [ipfix-reqs] Indicating Anonymization

From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" <[email-masked]>
Date: Mon, 30 Jun 2003 12:19:05 -0700

Tal,

  I think your last statement sums it up:  "Is the anonymization requirement
well defined?"

  In general, if the requirement is to prevent data which is considered
private being used by anyone, then I think the policy would take place at
the IPFIX Observer level itself.  In this case any correlation is
meaningless, and runs counter to the expectation of privacty.

  One particular technique could be the use of a 32-bit random number
calculated by the observer each time it starts, and not available to an
external system.  This value could then simply be XOR'd with the actual IP
address.  This would provide a one way anonymization and guarantee that the
same IP address is always given the same id over the period the exporter is
up.

  I think one complicating factor is that you may want the IP address of
users anonymized, but NOT the IP address of servers on the Internet.  What
it means to configure anonymization behavior is probably an area of further
exploration.

  I'm unclear on why anonymizing ports is interesting.

  I'd prefer to see a separate Flow Attribute in the Information Model to
effectively inform any downstream consumer of the fact that what is being
delivered is NOT the IP address of the actual system, but some scrambled
value.  Semantically, this is a different piece of information, i.e. it is a
unique, but non-correlateable identifier.

Regards,

  Jeff Meyer

> -----Original Message-----
> From: Tal Givoly [mailto:[email-masked]]
> Sent: Sunday, June 29, 2003 3:52 PM
> To: Simon Leinen; [email-masked]
> Subject: RE: [ipfix-reqs] Indicating Anonymization
> 
> 
> Simon,
> 
> It may be that that anonymization is perhaps even more 
> complex than this.
> 
> I assume that the original intent was that even without any form of
> encryption, the legitimate collector, or any party 
> eavesdropping on the
> communication, would not be able to correlate individual 
> user's usage back
> to that user. However, any form of anonymization that is a 
> 1-to-1 function
> of the original address space, and is completely specified, 
> implies that
> anybody could go back and discover the true identity of the 
> "anonymized"
> user. This would clearly be the case if one knows the 
> function applied to
> anonymize the address, and it is a 1-to-1 function (because you may be
> using, for instance, a 32-bit field to represent the "anonymized" IP
> address). One would know the function, if as you suggest, the function
> itself is completely specified.
> 
> Why should it be a 1-to-1 function, you may ask? Well, it is 
> important that
> the collector still be able to distinguish the identity of 
> the individual
> user sessions - although not necessarily the user itself. But 
> it shouldn't
> get mixed with other sessions.
> 
> Why should you use the same field type for the anonymized 
> values? Well,
> that's not mandatory - but it would make it possible for some 
> tools that are
> designed for operating on IP addresses, for instance, to operate on
> "anonymized" IP addresses. This is not mandatory, but may be useful.
> 
> Now, anonymization at the device level does introduce another 
> problem - any
> system downstream from the collector may not be able to 
> correlate back to
> individual user session. Supposing, for instance, IP addresses are
> dynamically assigned via DHCP or RADIUS (depending on access 
> technology) -
> then another source of information will actually have the 
> user's identity
> (be it MAC address for DHCP or user name/realm for RADIUS). 
> It will also be
> the source that allows for distinguishing two users sessions 
> sharing the
> same IP but at different times (from 9-10 user1 used IP1 
> while the same IP1
> was used between 10:05 and 11 by user2). So any anonymization 
> in the device
> would prevent downstream systems from recognizing that 
> different records
> represent different users traffic altogether.
> 
> This raises the questions:
> - Is anonymization requirement well-defined?
> - Is it even an important requirement for IPFIX?
> 
> I don't recall that there was much discussion ever about this 
> requirement.
> Wouldn't encryption of the links to avoid eavesdropping serve 
> most of the
> intended purpose? Why do we require the devices to support this?
> 
> Tal
> 
> -----Original Message-----
> From: majordomo listserver 
> [mailto:[email-masked]]On Behalf
> Of Simon Leinen
> Sent: Sunday, June 29, 2003 1:40 PM
> To: [email-masked]
> Subject: [ipfix-reqs] Indicating Anonymization
> 
> 
> While updating my protocol evaluation I-D, I found the requirements
> I-D section on anonymization somewhat problematic.  I apologize
> bringing this up so late in the process.
> 
> The current revision (-10) has the following to say about
> anonymization:
> 
>   6.7.  Anonymization
> 
>      The exporting process MAY be capable of anonymizing source and
>      destination IP addresses in flow data before exporting them. It
>      MAY support anonymization of port numbers and other
>      fields. Please note that anonymization is not originally an
>      application requirement, but derived from general requirements
>      for treatment of measured traffic data within a network.
> 
>      When anonymized flow data is exported, this MUST be clearly
>      indicated to all receiving collecting processes, such that they
>      can distinguish anonymized data from non-anonymized data.
> 
> Note that there is no single generally accepted method for anonymizing
> flow export data (or even individual fields such as an IP address),
> and different situations may require different levels of
> anonymization.
> 
> Because of this, I think that it is not sufficient that the collector
> be able to "distinguish anonymized data from non-anonymized data", but
> it must also know the method and "degree" of anonymization.  This is
> similar to sampling, where it is not sufficient to know whether there
> is sampling going on or not, but one is also interested in sampling
> parameters such as sampling method (e.g. pseudo-random,
> packet/byte/time/flow-based) and parameters (e.g. sampling interval).
> 
> On the other hand, whether and how anonymization is performed will be
> purely a matter of configuration of the exporting device, and will not
> change during normal operation.  This is in contrast to sampling,
> where the parameters may change to adapt to the offered load.  So one
> could argue that it's not even necessary to report anonymization,
> because of the general assumption that the collector can know the
> static configuration of the exporting device by some other means.
> 
> If we agree that collectors can obtain the necessary knowledge about
> the configuration of anonymization by other means, then I suggest the
> following replacement text.  This is blatantly stolen from the current
> text about sampling:
> 
>   6.7.  Anonymization
> 
>      The metering process MAY be capable of anonymizing source and
>      destination IP addresses in flow data before exporting them. It
>      MAY support anonymization of port numbers and other
>      fields. Please note that anonymization is not originally an
>      application requirement, but derived from general requirements
>      for treatment of measured traffic data within a network.
> 
>      If anonymization is supported, the anonymization configuration
>      MUST be well defined. The anonymization configuration includes
>      the anonymization method and all its parameters.
> 
> However, if we really think that the anonymization configuration must
> be conveyed to the collector through the IPFIX protocol, I would
> suggest the following wording (also stolen from the text on sampling,
> but from the part on what happens when the sampling configuration
> changes during operation):
> 
>   6.7.  Anonymization
> 
>      The metering process MAY be capable of anonymizing source and
>      destination IP addresses in flow data before exporting them. It
>      MAY support anonymization of port numbers and other
>      fields. Please note that anonymization is not originally an
>      application requirement, but derived from general requirements
>      for treatment of measured traffic data within a network.
> 
>      If anonymization is supported, the anonymization configuration
>      MUST be indicated to the collector. The anonymization
>      configuration includes the anonymization method and all its
>      parameters.
> --
> Simon.
> 
> --
> Help        mailto:[email-masked] and say "help" 
> in message
> body
> Unsubscribe mailto:[email-masked] and say
> "unsubscribe ipfix" in message body
> Archive     http://ipfix.doit.wisc.edu/archive/
> 
> 
> --
> Help        mailto:[email-masked] and say "help" 
> in message body
> Unsubscribe mailto:[email-masked] and say
> "unsubscribe ipfix" in message body
> Archive     http://ipfix.doit.wisc.edu/archive/
> 

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: "Tal Givoly" <[email-masked]>
Date: Mon, 30 Jun 2003 15:12:04 -0700

Jeff,

That, indeed, was the main point I was trying to make.

I'll explain why it seems that the anonymization algorithm you suggest is
too simple - if someone can come up with an effective alternative, the
requirement may have some more merit than it has now.

Why is XOR with a random number too simple a function if one wants to
anonymize traffic?

If the same 32-bit field is XORed for all instances of the same IP address,
then it would be very simple to identify that pattern. For instance, traffic
flowing to or from port 53 would be DNS traffic. It would be very easy to
figure out which are these DNS servers, as there aren't that many associated
with an organization. Same thing for SMTP/POP3 servers and any other server
with well-known ports. So, basically, it wouldn't be difficult to identify
the constant with which all IP addresses are XORed.

This may have lead people to think about anonymization of other fields, such
as port numbers... I believe that that too is a futile effort if it is just
a fixed XOR pattern. Not many protocols have the same source/destination
port - the ones that do stand out in the crowd. DNS is one of them. Most DNS
traffic between servers shares port 53 for both client and server. So once
you identify the few protocols that identify this pattern and solve a few
simple equations, you know the mask that was applied.

Getting back to the bigger picture - can anybody whom contributed to the
original requirements provide some rationale and perhaps even propose at
least one transformation function that servers the purpose?

Tal

-----Original Message-----
From: MEYER,JEFFREY D (HP-Cupertino,ex1) [mailto:[email-masked]]
Sent: Monday, June 30, 2003 12:19 PM
To: 'Tal Givoly'; Simon Leinen; [email-masked]
Subject: RE: [ipfix-reqs] Indicating Anonymization

Tal,

Regards,

  Jeff Meyer