From bclaise@cisco.com Fri Apr 2 04:42:16 2010 Return-Path: X-Original-To: ipfix@core3.amsl.com Delivered-To: ipfix@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 859653A67E7 for ; Fri, 2 Apr 2010 04:42:16 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 1.452 X-Spam-Level: * X-Spam-Status: No, score=1.452 tagged_above=-999 required=5 tests=[AWL=-2.083, BAYES_50=0.001, DNS_FROM_OPENWHOIS=1.13, HTML_MESSAGE=0.001, J_CHICKENPOX_37=0.6, LONGWORDS=1.803] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Iqo7Hz4KslTv for ; Fri, 2 Apr 2010 04:42:05 -0700 (PDT) Received: from av-tac-bru.cisco.com (odd-brew.cisco.com [144.254.15.119]) by core3.amsl.com (Postfix) with ESMTP id CC0453A677E for ; Fri, 2 Apr 2010 04:42:04 -0700 (PDT) X-TACSUNS: Virus Scanned Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id o32Bgb6I021383 for ; Fri, 2 Apr 2010 13:42:37 +0200 (CEST) Received: from [10.55.43.58] (ams-bclaise-8719.cisco.com [10.55.43.58]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id o32BgVvi021329 for ; Fri, 2 Apr 2010 13:42:31 +0200 (CEST) Message-ID: <4BB5D827.1060406@cisco.com> Date: Fri, 02 Apr 2010 14:42:31 +0300 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3 MIME-Version: 1.0 To: "ipfix@ietf.org" References: <4BB5C86D.7020107@cisco.com> In-Reply-To: <4BB5C86D.7020107@cisco.com> Content-Type: multipart/alternative; boundary="------------040401070009040802010204" Subject: [IPFIX] draft-ietf-ipfix-anon-02.txt: WG last call review X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Apr 2010 11:42:16 -0000 This is a multi-part message in MIME format. --------------040401070009040802010204 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Dear authors, I review this draft. You should consider the comments in sequence. Regards, Benoit. > IPFIX Working Group E. Boschi > Internet-Draft B. Trammell > Intended status: Experimental Hitachi Europe > Expires: August 19, 2010 February 15, 2010 > > IP Flow Anonymisation Support > draft-ietf-ipfix-anon-02.txt > > Abstract > > This document describes anonymisation techniques for IP flow data and > the export of anonymised data using the IPFIX protocol. It > categorizes common anonymisation schemes and defines the parameters > needed to describe them. It provides guidelines for the > implementation of anonymised data export and storage over IPFIX, and > describes an information model and Options-based method for > anonymisation technique metadata export within the IPFIX protocol or > storage in IPFIX Files. > > Status of this Memo > > This Internet-Draft is submitted to IETF in full conformance with the > provisions of BCP 78 and BCP 79. > > Internet-Drafts are working documents of the Internet Engineering > Task Force (IETF), its areas, and its working groups. Note that > other groups may also distribute working documents as Internet- > Drafts. > > Internet-Drafts are draft documents valid for a maximum of six months > and may be updated, replaced, or obsoleted by other documents at any > time. It is inappropriate to use Internet-Drafts as reference > material or to cite them other than as "work in progress." > > The list of current Internet-Drafts can be accessed at > http://www.ietf.org/ietf/1id-abstracts.txt. > > The list of Internet-Draft Shadow Directories can be accessed at > http://www.ietf.org/shadow.html. > > This Internet-Draft will expire on August 19, 2010. > > Copyright Notice > > Copyright (c) 2010 IETF Trust and the persons identified as the > document authors. All rights reserved. > > Boschi& Trammell Expires August 19, 2010 [Page 1] > Internet-Draft IP Flow Anonymisation Support February 2010 > > This document is subject to BCP 78 and the IETF Trust's Legal > Provisions Relating to IETF Documents > (http://trustee.ietf.org/license-info) in effect on the date of > publication of this document. Please review these documents > carefully, as they describe your rights and restrictions with respect > to this document. Code Components extracted from this document must > include Simplified BSD License text as described in Section 4.e of > the Trust Legal Provisions and are provided without warranty as > described in the BSD License. > > Table of Contents > > 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 > 1.1. IPFIX Protocol Overview . . . . . . . . . . . . . . . . . 4 > 1.2. IPFIX Documents Overview . . . . . . . . . . . . . . . . . 5 > 1.3. Anonymisation within the IPFIX Architecture . . . . . . . 5 > 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 7 > 3. Categorisation of Anonymisation Techniques . . . . . . . . . . 7 > 4. Anonymisation of IP Flow Data . . . . . . . . . . . . . . . . 8 > 4.1. IP Address Anonymisation . . . . . . . . . . . . . . . . . 10 > 4.1.1. Truncation . . . . . . . . . . . . . . . . . . . . . . 10 > 4.1.2. Reverse Truncation . . . . . . . . . . . . . . . . . . 11 > 4.1.3. Permutation . . . . . . . . . . . . . . . . . . . . . 11 > 4.1.4. Prefix-preserving Pseudonymisation . . . . . . . . . . 11 > 4.2. Hardware Address Anonymisation . . . . . . . . . . . . . . 12 > 4.2.1. Reverse Truncation . . . . . . . . . . . . . . . . . . 12 > 4.2.2. Permutation . . . . . . . . . . . . . . . . . . . . . 13 > 4.2.3. Structured Pseudonymisation . . . . . . . . . . . . . 13 > 4.3. Timestamp Anonymisation . . . . . . . . . . . . . . . . . 13 > 4.3.1. Precision Degradation . . . . . . . . . . . . . . . . 13 > 4.3.2. Enumeration . . . . . . . . . . . . . . . . . . . . . 14 > 4.3.3. Random Shifts . . . . . . . . . . . . . . . . . . . . 14 > 4.4. Counter Anonymisation . . . . . . . . . . . . . . . . . . 14 > 4.4.1. Precision Degradation . . . . . . . . . . . . . . . . 15 > 4.4.2. Binning . . . . . . . . . . . . . . . . . . . . . . . 15 > 4.4.3. Random Noise Addition . . . . . . . . . . . . . . . . 15 > 4.5. Anonymisation of Other Flow Fields . . . . . . . . . . . . 16 > 4.5.1. Binning . . . . . . . . . . . . . . . . . . . . . . . 16 > 4.5.2. Permutation . . . . . . . . . . . . . . . . . . . . . 16 > 5. Parameters for the Description of Anonymisation Techniques . . 16 > 5.1. Stability . . . . . . . . . . . . . . . . . . . . . . . . 17 > 5.2. Truncation Length . . . . . . . . . . . . . . . . . . . . 17 > > 5.3. Bin Map . . . . . . . . . . . . . . . . . . . . . . . . . 17 > 5.4. Permutation . . . . . . . . . . . . . . . . . . . . . . . 18 > 5.5. Shift Amount . . . . . . . . . . . . . . . . . . . . . . . 18 > 6. Anonymisation Export Support in IPFIX . . . . . . . . . . . . 18 > 6.1. Anonymisation Options Template . . . . . . . . . . . . . . 18 > > Boschi& Trammell Expires August 19, 2010 [Page 2] > Internet-Draft IP Flow Anonymisation Support February 2010 > > 6.2. Recommended Information Elements for Anonymisation > Metadata . . . . . . . . . . . . . . . . . . . . . . . . . 20 > 6.2.1. informationElementIndex . . . . . . . . . . . . . . . 20 > 6.2.2. anonymisationFlags . . . . . . . . . . . . . . . . . . 20 > 6.2.3. anonymisationTechnique . . . . . . . . . . . . . . . . 22 > 7. Applying Anonymisation Techniques to IPFIX Export and > Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 > 7.1. Arrangement of Processes in IPFIX Anonymisation . . . . . 24 > 7.2. IPFIX-Specific Anonymisation Guidelines . . . . . . . . . 27 > 7.2.1. Appropriate Use of Information Elements for > Anonymised Data . . . . . . . . . . . . . . . . . . . 27 > 7.2.2. Export of Perimeter-Based Anonymisation Policies . . . 28 > 7.2.3. Anonymisation of Header Data . . . . . . . . . . . . . 28 > 7.2.4. Anonymisation of Options Data . . . . . . . . . . . . 29 > 7.2.5. Special-Use Address Space Considerations . . . . . . . 30 > 8. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 > 9. Security Considerations . . . . . . . . . . . . . . . . . . . 33 > 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 34 > 11. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 35 > 12. References . . . . . . . . . . . . . . . . . . . . . . . . . . 35 > 12.1. Normative References . . . . . . . . . . . . . . . . . . . 35 > 12.2. Informative References . . . . . . . . . . . . . . . . . . 35 > Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 36 > > Boschi& Trammell Expires August 19, 2010 [Page 3] > Internet-Draft IP Flow Anonymisation Support February 2010 > > 1. Introduction > > The standardisation of an IP flow information export protocol > [RFC5101] and associated representations removes a technical barrier > to the sharing of IP flow data across organizational boundaries and > with network operations, security, and research communities for a > wide variety of purposes. However, with wider dissemination comes > greater risks to the privacy of the users of networks under > measurement, and to the security of those networks. While it is not > a complete solution to the issues posed by distribution of IP flow > information, anonymisation (i.e., the deletion or transformation of > information that is considered sensitive and could be used to reveal > the identity of subjects involved in a communication) is an important > tool for the protection of privacy within network measurement > infrastructures. > > This document presents a mechanism for representing anonymised data > within IPFIX and guidelines for using it. It begins with a > categorization of anonymisation techniques. It then describes > applicability of each technique to commonly anonymisable fields of IP > flow data, organized by information element data type and semantics > as in [RFC5102]; enumerates the parameters required by each of the > applicable anonymisation techniques; and provides guidelines for the > use of each of these techniques in accordance with best practices in > data protection. Finally, it specifies a mechanism for exporting > anonymised data and binding anonymisation metadata to templates using > IPFIX Options. > > 1.1. IPFIX Protocol Overview > > In the IPFIX protocol, { type, length, value } tuples are expressed > in templates containing { type, length } pairs, specifying which { > value } fields are present in data records conforming to the > Template, giving great flexibility as to what data is transmitted. > Since Templates are sent very infrequently compared with Data > Records, this results in significant bandwidth savings. Various > different data formats may be transmitted simply by sending new > Templates specifying the { type, length } pairs for the new data > format. See [RFC5101] for more information. > > The IPFIX information model [RFC5102] defines a large number of > standard Information Elements which provide the necessary { type } > information for Templates. The use of standard elements enables > interoperability among different vendors' implementations. > Additionally, non-standard enterprise-specific elements may be > defined for private use. > > Boschi& Trammell Expires August 19, 2010 [Page 4] > Internet-Draft IP Flow Anonymisation Support February 2010 > > 1.2. IPFIX Documents Overview > > "Specification of the IPFIX Protocol for the Exchange of IP Traffic > Flow Information" [RFC5101] and its associated documents define the > IPFIX Protocol, which provides network engineers and administrators > with access to IP traffic flow information. > > "Architecture for IP Flow Information Export" [RFC5470] defines the > architecture for the export of measured IP flow information out of an > IPFIX Exporting Process to an IPFIX Collecting Process, and the basic > terminology used to describe the elements of this architecture, per > the requirements defined in "Requirements for IP Flow Information > Export" [RFC3917]. The IPFIX Protocol document [RFC5101] then covers > the details of the method for transporting IPFIX Data Records and > Templates via a congestion-aware transport protocol from an IPFIX > Exporting Process to an IPFIX Collecting Process. > > "Information Model for IP Flow Information Export" [RFC5102] > describes the Information Elements used by IPFIX, including details > on Information Element naming, numbering, and data type encoding. > Finally, "IPFIX Applicability" [RFC5472] describes the various > applications of the IPFIX protocol and their use of information > exported via IPFIX, and relates the IPFIX architecture to other > measurement architectures and frameworks. > > Additionally, "Specification of the IPFIX File Format" [RFC5655] > describes a file format based upon the IPFIX Protocol for the storage > of flow data. > > This document references the Protocol and Architecture documents for > terminology, and extends the IPFIX Information Model to provide new > Information Elements for anonymisation metadata. The anonymisation > techniques described herein are equally applicable to the IPFIX > Protocol and data stored in IPFIX Files. > > 1.3. Anonymisation within the IPFIX Architecture > > "Architecture for IP Flow Information Export" [RFC5470] defines the > functions performed in sequence by the various functional blocks in > an IPFIX Device as in the figure below. > > Packet(s) coming into Observation Point(s) > | | > v v > +----------------+-------------------------+ +-----+-------+ > | Metering Process on an | | | > | Observation Point | | | > > Boschi& Trammell Expires August 19, 2010 [Page 5] > Internet-Draft IP Flow Anonymisation Support February 2010 > > | | | | > | packet header capturing | | | > | | |...| Metering | > | timestamping | | Process N | > | | | | | > | +----->+ | | | > | | | | | | > | | sampling Si (1:1 in case of no | | | > | | | sampling) | | | > | | filtering Fi (select all when | | | > | | | no criteria) | | | > | +------+ | | | > | | | | | > | | Timing out Flows | | | > | | Handle resource overloads | | | > +--------|---------------------------------+ +-----|-------+ > | | > Flow Records (identified by Observation Domain) Flow Records > | | > +---------+---------------------------------+ > | > +--------------------|----------------------------------------------+ > | | Exporting Process | > |+-------------------|-------------------------------------------+ | > || v IPFIX Protocol | | > ||+-----------------------------+ +----------------------------+| | > |||Rules for | |Functions || | > ||| Picking/sending Templates | |-Packetise selected Control || | > ||| Picking/sending Flow Records|->|& data Information into || | > ||| Encoding Template& data | | IPFIX export packets. || | > ||| Selecting Flows to export(*)| |-Handle export errors || | > ||+-----------------------------+ +----------------------------+| | > |+----------------------------+----------------------------------+ | > | | | > | exported IPFIX Messages | > | | | > | +------------+-----------------+ | > | | Anonymise export packet(*) | | > | +------------+-----------------+ | > | | | > | +------------+-----------------+ | > | | Transport Protocol | | > | +------------+-----------------+ | > | | | > +-----------------------------+-------------------------------------+ > | > v > IPFIX export packet to Collector > > Boschi& Trammell Expires August 19, 2010 [Page 6] > Internet-Draft IP Flow Anonymisation Support February 2010 > > (*) indicates that the block is optional. > In the general architecture, but not in this spec. right? > Figure 1: IPFIX Device functional blocks > I was hoping to get the reference model from the IPFIX Mediator framework. If the reference model was from the IPFIX Mediator framework, then we would not have the issue with optional. > Note that, according to the original architecture specification, > What is this? RFC5470? > IPFIX Message anonymisation is optionally performed as the final > operation before handing the Message to the transport protocol for > export. While no provision is made in the architecture for > anonymisation metadata as in Section 6, this arrangement does allow > for the message rewriting necessary for comprehensive anonymisation > of IPFIX export as in Section 7. The development of the IPFIX > Mediation [I-D.ietf-ipfix-mediators-framework] framework and the > IPFIX File Format [RFC5655] expand upon this initial architectural > allowance for anonymisation by adding to the list of places that > anonymisation may be applied. The former specifies IPFIX Mediators, > which rewrite existing IPFIX messages, and the latter specifies a > IPFIX Message? It would be nice to say in the terminology section which terms you use from other RFCs. For example, data set vs dataset versus Data Set For example, flow records is lower case, but I see IPFIX File, Options Template Please be consistent with capitalization > method for storage of IPFIX data in files. > > More detail on the applicable architectural arrangements of > anonymisation can be found in Section 7.1 > > 2. Terminology > > Terms used in this document that are defined in the Terminology > section of the IPFIX Protocol [RFC5101] document are to be > interpreted as defined there. > > The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", > "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this > document are to be interpreted as described in RFC 2119 [RFC2119]. > > 3. Categorisation of Anonymisation Techniques > > Anonymisation modifies a data set in order to protect the identity of > the people or entities described by the data set from disclosure. > With respect to network traffic data, anonymisation generally > attempts to preserve some set of properties of the network traffic > useful for a given application or applications, while ensuring the > data cannot be traced back to the specific networks, hosts, or users > generating the traffic. > > Anonymisation may be broadly classified according to two properties: > recoverability and countability. All anonymisation techniques map > the real space of identifiers or values into a separate, anonymised > space, according to some function. A technique is said to be > > Boschi& Trammell Expires August 19, 2010 [Page 7] > Internet-Draft IP Flow Anonymisation Support February 2010 > > recoverable when the function used is invertible or can otherwise be > reversed and a real identifier can be recovered from a given > replacement identifier. > > Countability compares the dimension of the anonymised space (N) to > the dimension of the real space (M), and denotes how the count of > unique values is preserved by the anonymisation function. If the > anonymised space is smaller than the real space, then the function is > said to generalise the input, mapping more than one input point to > each anonymous value (e.g., as with aggregation). By definition, > generalisation is not recoverable. > > If the dimensions of the anonymised and real spaces are the same, > such that the count of unique values is preserved, then the function > is said to be a direct substitution function. If the dimension of > the anonymised space is larger, such that each real value maps to a > set of anonymised values, then the function is said to be a set > substitution function. Note that with set substitution functions, > the sets of anonymised values are not necessarily disjoint. Either > direct or set substitution functions are said to be one-way if there > exists no practical method for recovering the real data point from an > anonymised one. > > This classification is summarised in the table below. > > +------------------------+-----------------+------------------------+ > | Recoverability / | Recoverable | Non-recoverable | > | Countability | | | > +------------------------+-----------------+------------------------+ > | N< M | N.A. | Generalisation | > | N = M | Direct | One-way Direct | > | | Substitution | Substitution | > | N> M | Set | One-way Set | > | | Substitution | Substitution | > +------------------------+-----------------+------------------------+ > > 4. Anonymisation of IP Flow Data > > Due to the restricted semantics of IP flow data, there is a > relatively limited set of specific anonymisation techniques available > on flow data, though each falls into the broad categories above. > Each type of field that may commonly appear in a flow record may have > its own applicable specific techniques. > > While anonymisation is generally applied at the resolution of single > fields within a flow record, attacks against anonymisation use entire > flows and relationships between hosts and flows within a given data > > Boschi& Trammell Expires August 19, 2010 [Page 8] > Internet-Draft IP Flow Anonymisation Support February 2010 > > set. Therefore, fields which may not necessarily be identifying by > themselves may be anonymised in order to increase the anonymity of > the data set as a whole. > > Of all the fields in an IP flow record, IP addresses are the most > likely to be used to directly identify entities in the real world. > Each IP address is associated with an interface on a network host, > and can potentially be identified with a single user. Additionally, > IP addresses are structured identifiers; that is, partial IP address > prefixes may be used to identify networks just as full IP addresses > identify hosts. This makes anonymisation of IP addresses > particularly important. > > Hardware addresses uniquely identify devices on the network; while > they are not often available in traffic data collected at Layer 3, > and cannot be used to locate devices within the network, some traces > may contain sub-IP data including hardware address data. Hardware > addresses may be mappable to device serial numbers, and to the > entities or individuals who purchased the devices, when combined with > external databases. They may also leak via IPv6 addresses in certain > circumstances. Therefore, hardware address anonymisation is also > important. > I would clearly mention MAC address > Port numbers identify abstract entities (applications) as opposed to > real-world entities, but they can be used to classify hosts and user > behavior. Passive port fingerprinting, both of well-known and > ephemeral ports, can be used to determine the operating system > running on a host. Relative data volumes by port can also be used to > determine the host's function (workstation, web server, etc.); this > information can be used to identify hosts and users. > > While not identifiers in and of themselves, timestamps and counters > can reveal the behavior of the hosts and users on a network. Any > given network activity is recognizable by a pattern of relative time > differences and data volumes in the associated sequence of flows, > even without host address information. They can therefore be used to > identify hosts and users. Timestamps and counters are also > vulnerable to traffic injection attacks, where traffic with a known > pattern is injected into a network under measurement, and this > pattern is later identified in the anonymised data set. > > The simplest and most extreme form of anonymisation, which can be > applied to any field of a flow record, is black-marker anonymisation, > or complete deletion of a given field. Note that black-marker > anonymisation is equivalent to simply not exporting the field(s) in > question. > > While black-marker anonymisation completely protects the data in the > > Boschi& Trammell Expires August 19, 2010 [Page 9] > Internet-Draft IP Flow Anonymisation Support February 2010 > > deleted fields from the risk of disclosure, it also reduces the > utility of the anonymised data set as a whole. Techniques that > retain some information while reducing (though not eliminating) the > disclosure risk will be extensively discussed in the following > sections; note that the techniques specifically applicable to IP > addresses, timestamps, ports, and counters will be discussed in > separate sections. > > 4.1. IP Address Anonymisation > > Since IP addresses are the most common identifiers within flow data > that can be used to directly identify a person, organization, or > host, most of the work on flow and trace data anonymisation has gone > into IP address anonymisation techniques. Indeed, the aim of most > attacks against anonymisation is to recover the map from anonymised > IP addresses to original IP addresses thereby identifying the > identified hosts. There is therefore a wide range of IP address > anonymisation schemes that fit into the following categories. > > +------------------------------------+---------------------+ > | Scheme | Action | > +------------------------------------+---------------------+ > | Truncation | Generalisation | > | Reverse Truncation | Generalisation | > | Permutation | Direct Substitution | > | Prefix-preserving Pseudonymisation | Direct Substitution | > +------------------------------------+---------------------+ > > 4.1.1. Truncation > > Truncation removes "n" of the least significant bits from an IP > address, replacing them with zeroes. In effect, it replaces a host > address with a network address for some fixed netblock; for IPv4 > addresses, 8-bit truncation corresponds to replacement with a /24 > network address. Truncation is a non-reversible generalisation > scheme. Note that while truncation is effective for making hosts > non-identifiable, it preserves information which can be used to > identify an organization, a geographic region, a country, or a > continent (or RIR region of responsibility). > what is RIR? > Truncation to an address length of 0 is equivalent to black-marker > anonymisation. Complete removal of IP address information is only > recommended for analysis tasks which have no need to separate flow > data by host or network; e.g. as a first stage to per-application > (port) or time-series total volume analyses. > > Boschi& Trammell Expires August 19, 2010 [Page 10] > Internet-Draft IP Flow Anonymisation Support February 2010 > > 4.1.2. Reverse Truncation > > Reverse truncation removes "n" of the most significant bits from an > IP address, replacing them with zeroes. Reverse truncation is a non- > reversible generalisation scheme. Reverse truncation is effective > for making networks unidentifiable, partially or completely removing > information which can be used to identify an organization, a > geographic region, a country, or a continent (or RIR region of > responsibility). However, it may cause ambiguity when applied to > data collected from more than one network, since it treats all the > hosts with the same address on different networks as if they are the > same host. It is not particularly useful when publishing data where > the network of origin is known or can be easily guessed by virtue of > the identity of the publisher. > > Like truncation, reverse truncation to an address length of 0 is > equivalent to black-marker anonymisation. > > 4.1.3. Permutation > > Permutation is a direct substitution technique, replacing each IP > address with an address selected from the set of possible IP > addresses, guaranteeing that each anonymised address represents a > unique original address. The selection function is often random, > though it is not necessarily so. Permutation does not preserve any > structural information about a network, but it does preserve the > unique count of IP addresses. Any application that requires more > structure than host-uniqueness will not be able to use permuted IP > addresses. > > 4.1.4. Prefix-preserving Pseudonymisation > > Prefix-preserving pseudonymisation is a direct substitution > technique, like permutation but further restricted such that the > structure of subnets is preserved at each level while anonymising IP > addresses. If two real IP addresses match on a prefix of "n" bits, > the two anonymised IP addresses will match on a prefix of "n" bits as > well. This is useful when relationships among networks must be > preserved for a given analysis task, but introduces structure into > the anonymised data which can be exploited in attacks against the > anonymisation technique. > > Scanning in Internet background traffic can cause particular problems > with this technique: if a scanner uses a predictable and known > sequence of addresses, this information can be used to reverse the > substitution. The low order portion of the address can be left > unanonymized as a partial defense against this attack. > > Boschi& Trammell Expires August 19, 2010 [Page 11] > Internet-Draft IP Flow Anonymisation Support February 2010 > > 4.2. Hardware Address Anonymisation > > Flow data containing sub-IP information can also contain identifying > information in the form of the hardware (MAC) address. While > hardware address information cannot be used to locate a node within a > network, it can be used to directly uniquely identify a specific > device. Vendors or organizations within the supply chain may then > have the information necessary to identify the entity or individual > that purchased the device. > > Hardware address information is not as structured as IP address > information. EUI-48 and EUI-64 hardware addresses contain an > Organizational Unique Identifier in the three most significant bytes > of the address; this OUI additionally contains bits noting whether > the address is locally or globally administered. Beyond this, the > address is unstructured, and there is no particular relationship > among the OUIs assigned to a given vendor. > > Note that hardware address information also appear within IPv6 > addresses, as the EAP-64 address, or EAP-48 address encoded as an > EAP-64 address, is used as the least significant 64 bits of the IPv6 > address in the case of link local addressing or stateless > autoconfiguration; the considerations and techniques in this section > may then apply to such IPv6 addresses as well. > > +-----------------------------+---------------------+ > | Scheme | Action | > +-----------------------------+---------------------+ > | Reverse Truncation | Generalisation | > | Permutation | Direct Substitution | > | Structured Pseudonymisation | Direct Substitution | > +-----------------------------+---------------------+ > > 4.2.1. Reverse Truncation > > Reverse truncation removes "n" of the most significant bits from an > MAC address, replacing them with zeroes. Reverse truncation is a > non-reversible generalisation scheme. This has the effect of > removing bits of the OUI, which identify manufacturers, before > removing the least significant bits. Reverse truncation of 24 bits > zeroes out the OUI. > > Reverse truncation is effective for making device manufacturers > partially or completely unidentifiable within a dataset. However, it > may cause ambiguity by introducing the possibility of truncated MAC > address collision. Also note that the utility or removing > manufacturer information is dubious, and not particularly well- > covered by the literature. > > Boschi& Trammell Expires August 19, 2010 [Page 12] > Internet-Draft IP Flow Anonymisation Support February 2010 > > Reverse truncation to an address length of 0 is equivalent to black- > marker anonymisation. > > 4.2.2. Permutation > > Permutation is a direct substitution technique, replacing each MAC > address with an address selected from the set of possible MAC > addresses, guaranteeing that each anonymised address represents a > unique original address. The selection function is often random, > though it is not necessarily so. Permutation does not preserve any > structural information about a network, but it does preserve the > unique count of devices on the network. Any application that > requires more structure than host-uniqueness will not be able to use > permuted MAC addresses. > > 4.2.3. Structured Pseudonymisation > > Structured pseudonymisation for MAC addresses is a direct > substitution technique, like permutation, but restricted such that > the OUI (the most significant three bytes) is permuted separately > from the node identifier, the remainder. This is useful when the > uniqueness of OUIs must be preserved for a given analysis task, but > introduces structure into the anonymised data which can be exploited > in attacks against the anonymisation technique. > > 4.3. Timestamp Anonymisation > > The particular time at which a flow began or ended is not > particularly identifiable information, but it can be used as part of > attacks against other anonymisation techniques or for user profiling. > Presice timestamps can be used in injected-traffic fingerprinting > precise Yo might want to describe what fingerprinting is > attacks as well as to identify certain activity by response delay and > size fingerprinting. Therefore, timestamp information may be > anonymised in order to ensure the protection of the entire dataset. > > +-----------------------+----------------------------+ > | Scheme | Action | > +-----------------------+----------------------------+ > | Precision Degradation | Generalisation | > | Enumeration | Direct or Set Substitution | > | Random Shifts | Direct Substitution | > +-----------------------+----------------------------+ > > 4.3.1. Precision Degradation > it's like truncation, but specific to time and counter, right? > Precision Degradation is a generalisation technique that removes the > most precise components of a timestamp, accounting all events > occurring in each given interval (e.g. one millisecond for > > Boschi& Trammell Expires August 19, 2010 [Page 13] > Internet-Draft IP Flow Anonymisation Support February 2010 > > millisecond level degradation) as simultaneous. This has the effect > of potentially collapsing many timestamps into one. With this > technique time precision is reduced, and sequencing may be lost, but > the information at which time the event occurred is preserved. The > anonymised data may not be generally useful for applications which > require strict sequencing of flows. > > Note that flow meters with low time precision (e.g. second precision, > or millisecond precision on high-capacity networks) perform the > equivalent of precision degradation anonymisation by their design. > > Note also that degradation to a very low precision (e.g. on the order > of minutes, hours, or days) is commonly used in analyses operating on > time-series aggregated data, and may also be described as binning; > though the time scales are longer and applicability more restricted, > this is in principle the same operation. > > Precision degradation to infinitely low precision is equivalent to > black-marker anonymisation. Removal of timestamp information is only > recommended for analysis tasks which have no need to separate flows > in time, for example for counting total volumes or unique occurrences > of other flow keys in an entire dataset. > > 4.3.2. Enumeration > > Enumeration is a substitution function that retains the chronological > order in which events occurred while eliminating time information. > Timestamps are substituted by equidistant timestamps (or numbers) > starting from a randomly chosen start value. The resulting data is > useful for applications requiring strict sequencing, but not for > those requiring good timing information (e.g. delay- or jitter- > measurement for QoS applications or SLA validation). > > 4.3.3. Random Shifts > > Random time shifts add a random offset to every timestamp within a > dataset. This reversible substitution technique therefore retains > duration and inter-event interval information as well as > chronological order of flows. It is primarily intended to defeat > traffic injection fingerprinting attacks. > > 4.4. Counter Anonymisation > > Counters (such as packet and octet volumes per flow) are subject to > fingerprinting and injection attacks against anonymisation, or for > user profiling as timestamps are. Counter anonymisation can help > defeat these attacks, but are only usable for analysis tasks for > which relative or imprecise magnitudes of activity are useful. > > Boschi& Trammell Expires August 19, 2010 [Page 14] > Internet-Draft IP Flow Anonymisation Support February 2010 > > Counter information can also be completely removed, but this is only > recommended for analysis tasks which have no need to evaluate the > removed counter, for example for counting only unique occurrences of > other flow keys. > > +-----------------------+----------------------------+ > | Scheme | Action | > +-----------------------+----------------------------+ > | Precision Degradation | Generalisation | > | Binning | Generalisation | > | Random noise addition | Direct or Set Substitution | > +-----------------------+----------------------------+ > > 4.4.1. Precision Degradation > > As with precision degradation in timestamps, precision degradation of > counters removes lower-order bits of the counters, treating all the > counters in a given range as having the same value. Depending on the > precision reduction, this loses information about the relationships > between sizes of similarly-sized flows, but keeps relative magnitude > information. Precision degradation to an infinitely low precision is > equivalent to black-marker anonymisation. > > 4.4.2. Binning > > Binning can be seen as a special case of precision degradation; the > operation is identical, except for in precision degradation the > counter ranges are uniform, and in binning they need not be. For > example, a common counter binning scheme for packet counters could be > to bin values 1-2 together, and 3-infinity together, thereby > separating potentially completely-opened TCP connections from > unopened ones. Binning schemes are generally chosen to keep > precisely the amount of information required in a counter for a given > analysis task. Note that, also unlike precision degradation, the bin > label need not be within the bin's range. Binning counters to a > single bin is equivalent to black-marker anonymisation. > > 4.4.3. Random Noise Addition > > Random noise addition adds a random amount to a counter in each flow; > this is used to keep relative magnitude information and minimize the > disruption to size relationship information while avoiding > fingerprinting attacks against anonymisation. Note that there is no > guarantee that random noise addition will maintain ranking order by a > counter among members of a set. Random noise addition is > particularly useful when the derived analysis data will not be > presented in such a way as to require the lower-order bits of the > counters. > > Boschi& Trammell Expires August 19, 2010 [Page 15] > Internet-Draft IP Flow Anonymisation Support February 2010 > > 4.5. Anonymisation of Other Flow Fields > > Other fields, particularly port numbers and protocol numbers, can be > used to partially identify the applications that generated the > traffic in a a given flow trace. This information can be used in > fingerprinting attacks, and may be of interest on its own (e.g., to > reveal that a certain application with suspected vulnerabilities is > running on a given network). These fields are generally anonymised > using one of two techniques. > > +-------------+---------------------+ > | Scheme | Action | > +-------------+---------------------+ > | Binning | Generalisation | > | Permutation | Direct Substitution | > +-------------+---------------------+ > > 4.5.1. Binning > > Binning is a generalisation technique mapping a set of potentially > non-uniform ranges into a set of arbitrarily labeled bins. Common > bin arrangements depend on the field type and the analysis > application. For example, an IP protocol bin arrangement may > preserve 1, 6, and 17 for ICMP, UDP, and TCP traffic, and bin all > other protocols into a single bin, to mitigate the use of uncommon > protocols in fingerprinting attacks. Another example arrangement may > bin source and destination ports into low (0-1023) and high (1024- > 65535) bins in order to tell service from ephemeral ports without > identifying individual applications. > > Binning other flow key fields to a single bin is equivalent to black- > marker anonymisation. Removal of other flow key information is only > recommended for analysis tasks which have no need to differentiate > flows on the removed keys, for example for total traffic counts or > unique counts of other flow keys. > > 4.5.2. Permutation > > Permutation is a direct substitution technique, replacing each value > with an value selected from the set of possible range, guaranteeing > that each anonymised value represents a unique original value. This > is used to preserve the count of unique values without preserving > information about, or the ordering of, the values themselves. > > 5. Parameters for the Description of Anonymisation Techniques > > This section details the abstract parameters used to describe the > > Boschi& Trammell Expires August 19, 2010 [Page 16] > Internet-Draft IP Flow Anonymisation Support February 2010 > > anonymisation techniques examined in the previous section, on a per- > parameter basis. These parameters and their export safety inform the > design of the IPFIX anonymisation metadata export specified in the > following section. > you use different terms for metadata IPFIX anonymisation metadata export anonymisation technique metadata export anonymisation metadata metadata about exported flows and the flow collection infrastructure ... You might want to be consistent > 5.1. Stability > > Any given anonymisation technique may be applied with a varying range > of stability. Stability is important for assessing the comparability > of anonymised information in different data sets, or in the same data > set over different time periods. In general, stability ranges from > completely stable to completely unstable; however, note that the > completely unstable case is indistinguishable from black-marker > anonymisation. A completely stable anonymisation will always map a > given value in the real space to the same value in the anonymised > space. In practice, an anonymisation may also be stable for every > data set published by an a particular producer to a particular > consumer, stable for a stated time period within a dataset or across > datasets, or stable only for a single data set. > > If no information about stability is available, users of anonymised > data may assume that the techniques used are stable across the entire > dataset, but unstable across datasets. Note that stability presents > a risk-utility tradeoff, as completely stable anonymisation can be > used for longer-term trend analysis tasks but also presents more risk > of attack given the stable mapping. > In this section, you don't say if you must export the content of this section, while you do for all the other 5.x sections > 5.2. Truncation Length > > Truncation and precision degradation are described by the truncation > length, or the amount of data still remaining in the anonymised field > after anonymisation. > > Truncation length can be inferred from a given data set, and need not > be specially exported or protected. > > 5.3. Bin Map > > Binning is described by the specification of a bin mapping function. > This function can be generally expressed in terms of an associative > array that maps each point in the original space to a bin, although > from an implementation standpoint most bin functions are much simpler > and more efficient. > > Since knowledge of the bin mapping function can be used to partially > deanonymise binned data, depending on the degree of generalisation, > no information about the bin mapping function should be exported. > > Boschi& Trammell Expires August 19, 2010 [Page 17] > Internet-Draft IP Flow Anonymisation Support February 2010 > > 5.4. Permutation > > Like binning, permutation is described by the specification of a > permutation function. In the general case, this can be expressed in > terms of an associative array that maps each point in the original > space to a point in the anonymised space. Unlike binning, each point > in the anonymised space must correspond to a single, unique point in > the original space. > > Since knowledge of the permutation function can be used to completely > deanonymise permuted data, no information about the permutation > function or its parameters should be exported. > > 5.5. Shift Amount > > Shifting requires an amount to shift each value by. Since the shift > amount can be used to deanonymise data protected by shifting, no > information about the shift amount should be exported. > > 6. Anonymisation Export Support in IPFIX > > Anonymised data exported via IPFIX SHOULD be annotated with > anonymisation metadata, which details which fields described by which > Templates are anonymised, and provides appropriate information on the > anonymisation techniques used. This metadata SHOULD be exported in > Data Records described by the recommended Options Templates described > in this section; these Options Templates use the additional > Information Elements described in the following subsection. > > Note that fields anonymised using the black-marker (removal) > technique do not require any special metadata support. Black-marker > anonymised fields SHOULD NOT be exported at all; the absence of the > MUST NOT? > field in a given Data Set is implicitly declared by not including the > corresponding Information Element in the Template describing that > Data Set. > > 6.1. Anonymisation Options Template > > The Anonymisation Options Template describes anonymisation records, > which allow anonymisation metadata to be exported inline over IPFIX > or stored in an IPFIX File, by binding information about > anonymisation techniques to Information Elements within defined > Templates. IPFIX Exporting Processes SHOULD export anonymisation > records for any Template describing exported anonymised Data Records; > an anonymization record is actually a data record of the Anonymization Options Template Record, as opposed to an anonymized record. It would be better to define those terms, no > IPFIX Collecting Processes and processes downstream from them MAY use > anonymisation records to treat anonymised data differently depending > on the applied technique. > > Boschi& Trammell Expires August 19, 2010 [Page 18] > Internet-Draft IP Flow Anonymisation Support February 2010 > > An Exporting Process SHOULD export anonymisation records after the > Templates they describe have been exported, And MUST send the anonymization record before the anomized records? Do you want to benefit from the per-stream SCTP draft. That would solve most of your problems: order, reliable Otherwise, if the collector doesn't have received the anonimization records, then it might be treating the flow records wrongly. > and SHOULD export > anonymisation records reliably. > > Anonymisation records, like Templates, MUST be handled by Collecting > Processes as scoped to the Transport Session in which they are sent. > While the Stability Class within the anonymisationFlags IE can be > used to declare that a given anonymisation technique's mapping will > remain stable across multiple sessions, each session MUST re-export > the anonymisation Records along with the templates. > > +-------------------------+-----------------------------------------+ > | IE | Description | > +-------------------------+-----------------------------------------+ > | templateId [scope] | The Template ID of the Template | > | | containing the Information Element | > | | described by this anonymisation record. | > | | This Information Element MUST be | > | | defined as a Scope Field. | > | informationElementId | The Information Element identifier of | > | [scope] | the Information Element described by | > | | this anonymisation record. This | > | | Information Element MUST be defined as | > | | a Scope Field. | > | informationElementId | The Private Enterprise Number of the | > | [scope] [optional] | enterprise-specific Information Element | > | | described by this anonymisation record. | > | | This Information Element MUST be | > | | defined as a Scope Field if present. | > Isn't it privateEnterpriseNumber instead of informationElementId? Actually, the example and the description mention privateEnterpriseNumber However, I'm wondering why you need this privateEnterpriseNumber at all, because the "E" bit is already part of informationElementId 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |E| Information Element ident. | Field Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Enterprise Number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ I'm confused. > | informationElementIndex | The Information Element index of the | > | [scope] [optional] | instance of the Information Element | > | | described by this anonymisation record | > | | identified by the informationElementId | > | | within the Template. Optional; need | > | | only be present when describing | > | | Templates that have multiple instances | > | | of the same Information Element. This | > | | Information Element MUST be defined as | > | | a Scope Field if present. This | > | | Information Element is defined in | > | | Section 6.2, below. | > | anonymisationFlags | Flags describing the mapping stability | > | | and specialized modifications to the | > | | Anonymisation Technique in use. SHOULD | > | | be present. This Information Element | > | | is defined in Section 6.2, below. | > > Boschi& Trammell Expires August 19, 2010 [Page 19] > Internet-Draft IP Flow Anonymisation Support February 2010 > > | anonymisationTechnique | The technique used to anonymise the | > | | data. MUST be present. This | > | | Information Element is defined in | > | | Section 6.2, below. | > +-------------------------+-----------------------------------------+ > > 6.2. Recommended Information Elements for Anonymisation Metadata > > 6.2.1. informationElementIndex > > Description: A zero-based index of an Information Element > referenced by informationElementId within a Template referenced by > templateId; used to disambiguate scope for templates containing > multiple identical Information Elements. > > Abstract Data Type: unsigned16 > > ElementId: TBD3 > > Status: Proposed > > 6.2.2. anonymisationFlags > > Description: A flag word describing specialized modifications to > the anonymisation policy in effect for the anonymisation technique > applied to a referenced Information Element within a referenced > Template. When flags are clear (0), the normal policy (as > described by anonymisationTechnique) applies without modification. > > MSB 14 13 12 11 10 9 8 7 6 5 4 3 2 1 LSB > +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+ > | Reserved |LOR|PmA| SC | > +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+ > > anonymisationFlags IE > > +--------+----------+-----------------------------------------------+ > | bit(s) | name | description | > | (LSB = | | | > | 0) | | | > +--------+----------+-----------------------------------------------+ > | 0-1 | SC | Stability Class: see the Stability Class | > | | | table below, and section Section 5.1. | > > Boschi& Trammell Expires August 19, 2010 [Page 20] > Internet-Draft IP Flow Anonymisation Support February 2010 > > | 2 | PmA | Perimeter Anonymisation: when set (1), source | > source based on what? I remember the lengthy discussion in biflow. > | | | address Information Elements are interpreted | > | | | as external addresses, and destination | > | | | address Information Elements are interpreted | > | | | as internal addresses, for the purposes of | > | | | associating anonymisationTechnique to | > | | | Information Elements. MUST NOT be set when | > | | | associated with a non-endpoint (i.e., source- | > | | | or destination-) Information Element. SHOULD | > | | | be consistent within a record (i.e., if a | > | | | source- Information Element has this flag | > | | | set, the corresponding destination- element | > | | | SHOULD have this flag set, and vice-versa.) | > | 3 | LOR | Low-Order Unchanged: when set (1), the | > | | | low-order bits of the anonymised Information | > | | | Element contain real data. This modification | > | | | is intended for the anonymisation of | > | | | network-level addresses while leaving | > | | | host-level addresses intact in order to | > | | | preserve host level-structure, which could | > | | | otherwise be used to reverse anonymisation. | > | | | MUST NOT be set when associated with a | > | | | truncation-based anonymisationTechnique. | > | 4-15 | Reserved | Reserved for future use: SHOULD be cleared | > | | | (0) by the Exporting Process and MUST be | > | | | ignored by the Collecting Process. | > +--------+----------+-----------------------------------------------+ > > The Stability Class portion of this flags word describes the > stability class of the anonymisation technique applied to a > referenced Information Element within a referenced Template. > Stability classes refer to the stability of the parameters of the > anonymisation technique, and therefore the comparability of the > mapping between the real and anonymised values over time. This > determines which anonymised datasets may be compared with each > other. Values are as follows: > > Boschi& Trammell Expires August 19, 2010 [Page 21] > Internet-Draft IP Flow Anonymisation Support February 2010 > > +-----+-----+-------------------------------------------------------+ > | Bit | Bit | Description | > | 1 | 0 | | > +-----+-----+-------------------------------------------------------+ > | 0 | 0 | Undefined: the Exporting Process makes no | > | | | representation as to how stable the mapping is, or | > | | | over what time period values of this field will | > | | | remain comparable; while the Collecting Process MAY | > | | | assume Session level stability, Session level | > | | | stability is not guaranteed. Processes SHOULD assume | > | | | this is the case in the absence of stability class | > | | | information; this is the default stability class. | > | 0 | 1 | Session: the Exporting Process will ensure that the | > | | | parameters of the anonymisation technique are stable | > | | | during the Transport Session. All the values of the | > | | | described Information Element for each Record | > | | | described by the referenced Template within the | > | | | Transport Session are comparable. The Exporting | > | | | Process SHOULD endeavour to ensure at least this | > | | | stability class. | > | 1 | 0 | Exporter-Collector Pair: the Exporting Process will | > | | | ensure that the parameters of the anonymisation | > | | | technique are stable across Transport Sessions over | > | | | time with the given Collecting Process, but may use | > | | | different parameters for different Collecting | > | | | Processes. Data exported to different Collecting | > | | | Processes is not comparable. | > | 1 | 1 | Stable: the Exporting Process will ensure that the | > | | | parameters of the anonymisation technique are stable | > | | | across Transport Sessions over time, regardless of | > | | | the Collecting Process to which it is sent. | > +-----+-----+-------------------------------------------------------+ > > Abstract Data Type: unsigned16 > > ElementId: TBD1 > > Status: Proposed > > 6.2.3. anonymisationTechnique > While reading, it looks more logical to have the anonymisationTechnique before the anonymizationFlags IE > Description: A description of the anonymisation technique applied > to a referenced Information Element within a referenced Template. > And options template as well? For example, in my company, we export a options template record with the sampling rates. This is something I missed completely, and is valid for all IEs > Each technique may be applicable only to certain Information > Elements and recommended only for certain Infomation Elements; > these restrictions are noted in the table below. > > Boschi& Trammell Expires August 19, 2010 [Page 22] > Internet-Draft IP Flow Anonymisation Support February 2010 > > +-------+---------------------------+-----------------+-------------+ > | Value | Description | Applicable to | Recommended | > | | | | for | > +-------+---------------------------+-----------------+-------------+ > | 0 | Undefined: the Exporting | all | all | > | | Process makes no | | | > | | representation as to | | | > | | whether the defined field | | | > | | is anonymised or not. | | | > | | While the Collecting | | | > | | Process MAY assume that | | | > | | the field is not | | | > | | anonymised, it is not | | | > | | guaranteed not to be. | | | > | | This is the default | | | > | | anonymisation technique. | | | > | 1 | None: the values exported | all | all | > | | are real. | | | > | 2 | Precision | all | all | > | | Degradation/Truncation: | | | > | | the values exported are | | | > | | anonymised using simple | | | > | | precision degradation or | | | > | | truncation. The new | | | > | | precision or number of | | | > | | truncated bits is | | | > | | implicit in the exported | | | > | | data, and can be deduced | | | > | | by the Collecting | | | > | | Process. | | | > | 3 | Binning: the values | all | all | > | | exported are anonymised | | | > | | into bins. | | | > | 4 | Enumeration: the values | all | timestamps | > | | exported are anonymised | | | > | | by enumeration. | | | > | 5 | Permutation: the values | all | identifiers | > | | exported are anonymised | | | > | | by random permutation. | | | > > Boschi& Trammell Expires August 19, 2010 [Page 23] > Internet-Draft IP Flow Anonymisation Support February 2010 > > | 6 | Structured Permutation: | addresses | | > | | the values exported are | | | > | | anonymised by random | | | > | | permutation, preserving | | | > | | bit-level structure as | | | > | | appropriate; this | | | > | | represents | | | > | | prefix-preserving IP | | | > | | address anonymisation or | | | > | | structured MAC address | | | > | | anonymisation. | | | > | 7 | Reverse Truncation: the | addresses | | > | | values exported are | | | > | | anonymised using reverse | | | > | | truncation. The number | | | > | | of truncated bits is | | | > | | implicit in the exported | | | > | | data, and can be deduced | | | > | | by the Collecting | | | > | | Process. | | | > | 8 | Noise: the values | non-identifiers | counters | > | | exported are anonymised | | | > | | by adding random noise to | | | > | | each value. | | | > | 9 | Offset: the values | all | timestamps | > | | exported are anonymised | | | > | | by adding a single offset | | | > | | to all values. | | | > +-------+---------------------------+-----------------+-------------+ > > Abstract Data Type: unsigned16 > > ElementId: TBD2 > > Status: Proposed > > 7. Applying Anonymisation Techniques to IPFIX Export and Storage > > When exporting or storing anonymised flow data using IPFIX, certain > interactions between the IPFIX Protocol and the anonymisation > techniques in use must be considered; these are treated in the > subsections below. > > 7.1. Arrangement of Processes in IPFIX Anonymisation > > Anonymisation may be applied to IPFIX data at three stages within the > collection infrastructure: on initial export, at a mediator, or after > > Boschi& Trammell Expires August 19, 2010 [Page 24] > Internet-Draft IP Flow Anonymisation Support February 2010 > > collection, as shown in Figure 2. Each of these locations has > specific considerations and applicability. > > +==========================================+ > | Exporting Process | > +==========================================+ > | | > | (Anonymised at Original Exporter) | > V | > +=============================+ | > | Mediator | | > +=============================+ | > | | > | (Anonymising Mediator) | > V V > +==========================================+ > | Collecting Process | > +==========================================+ > | > | (Anonymising CP/File Writer) > V > +--------------------+ > | IPFIX File Storage | > +--------------------+ > > Figure 2: Potential Anonymisation Locations > > Anonymisation is generally performed before the wider dissemination > or repurposing of a flow data set, e.g., adapting operational > measurement data for research. Therefore, direct anonymisation of > flow data on initial export is only applicable in certain restricted > circumstances: when the Exporting Process is "publishing" data to a > Collecting Process directly, and the Exporting Process and Collecting > Process are operated by different entities. Note that certain > guidelines in Section 7.2.3 with respect to timestamp anonymisation > may not apply in this case, as the Collecting Process may be able to > deduce certain timing information from the time at which each Message > IPFIX Message > is received. > > A much more flexible arrangement is to anonymise data within a > Mediator [I-D.ietf-ipfix-mediators-framework]. Here, original data > is sent to a Mediator, which performs the anonymisation function and > re-exports the anonymised data. Such a Mediator could be located at > the administrative domain boundary of the initial Exporting Process > operator, exporting anonymised data to other consumers outside the > organisation. In this case, the original Exporter SHOULD use TLS as > specified in [RFC5101] to secure the channel to the Mediator, and the > Mediator should follow the guidelines in Section 7.2, to mitigate the > > Boschi& Trammell Expires August 19, 2010 [Page 25] > Internet-Draft IP Flow Anonymisation Support February 2010 > > risk of original data disclosure. > > When data is to be published as an anonymised data set in an IPFIX > File [RFC5655], the anonymisation may be done at the final Collecting > Process before storage and dissemination, as well. For me, it's a different problem. IPFIX File could be done on routers. Basically, this paragraph is about the relationship with IPFIX File. Btw, we're alos missing the relationship with reducing redundancy, with IPFIX structured data For the later, does it apply to all instances of a template? So, some kind of recursivity is implied > In this case, the > Collector should follow the guidelines in Section 7.2, especially as > regards File-specific Options in Section 7.2.4 > > In each of these data flows, the anonymisation of records is > undertaken by an Intermediate Anonymisation Process (IAP); the data > flows into and out of this IAP are shown in Figure 3 below. > > packets --+ +- IPFIX Messages -+ > | | | > V V V > +==================+ +====================+ +=============+ > | Metering Process | | Collecting Process | | File Reader | > +==================+ +====================+ +=============+ > | Non-anonymised | Records | > V V V > +=========================================================+ > | Intermediate Anonymisation Process (IAP) | > +=========================================================+ > | Anonymised ^ Anonymised | > | Records | Records | > V | V > +===================+ Anonymisation +=============+ > | Exporting Process |<--- Parameters ------>| File Writer | > +===================+ +=============+ > | | > +------------> IPFIX Messages<----------+ > > Figure 3: Data flows through the anonymisation process > This is exactly what I was hoping to see in figure1, i.e. this draft applying to the Intermediate Process in [IFPIX-MD-FMWK] ... where the IP can be an independent boxe, or inside the router. I completely missed that second part in the draft. > Anonymisation parameters must also be available to the Exporting > Process and/or File Writer in order to ensure header data is also > appropriately anonymised as in Section 7.2.3. > > Following each of the data flows through the IAP, we describe five > basic types of anonymisation arrangements within this framework in > Figure 4. In addition to the three arrangements described in detail > above, anonymisation can also be done at a collocated Metering > Process and File Writer (see section 7.3.2 of [RFC5655]), or at a > file manipulator (see section 7.3.7 of [RFC5655]). > > Boschi& Trammell Expires August 19, 2010 [Page 26] > Internet-Draft IP Flow Anonymisation Support February 2010 > > +----+ +-----+ +----+ > pkts -> | MP |->| IAP |->| EP |-> anonymisation on Original Exporter > +----+ +-----+ +----+ > +----+ +-----+ +----+ > pkts -> | MP |->| IAP |->| FW |-> Anonymising collocated MP/File Writer > +----+ +-----+ +----+ > +----+ +-----+ +----+ > IPFIX -> | CP |->| IAP |->| EP |-> Anonymising Mediator (Masquerading Proxy) > +----+ +-----+ +----+ > +----+ +-----+ +----+ > IPFIX -> | CP |->| IAP |->| FW |-> Anonymising collocated CP/File Writer > +----+ +-----+ +----+ > +----+ +-----+ +----+ > IPFIX -> | FR |->| IAP |->| FW |-> Anonymising file manipulator > File +----+ +-----+ +----+ > > Figure 4: Possible anonymisation arrangements in the IPFIX > architecture > > Note that anonymisation may occur at more than one location within a > given collection infrastructure, to provide varying levels of > anonymisation, disclosure risk, or data utility for specific > purposes. > > 7.2. IPFIX-Specific Anonymisation Guidelines > > In implementing and deploying the anonymisation techniques described > in this document, implementors should note that IPFIX already > provides features that support anonymised data export, and use these > where appropriate. Care must also be taken that data structures > supporting the operation of the protocol itself do not leak data that > could be used to reverse the anonymisation applied to the flow data. > Such data structures may appear in the header, or within the data > stream itself, especially as options data. Each of these and their > impact on specific anonymisation techniques is noted in a separate > subsection below. > > 7.2.1. Appropriate Use of Information Elements for Anonymised Data > > Note, as in Section 6 above, that black-marker anonymised fields > SHOULD NOT be exported at all; the absence of the field in a given > Data Set is implicitly declared by not including the corresponding > Information Element in the Template describing that Data Set. > > When using precision degradation of timestamps, Exporting Processes > SHOULD export timing information using Information Elements of an > appropriate precision, as explained in Section 4.5 of [RFC5153]. For > example, timestamps measured in millisecond-level precision and > > Boschi& Trammell Expires August 19, 2010 [Page 27] > Internet-Draft IP Flow Anonymisation Support February 2010 > > degraded to second-level precision should use flowStartSeconds and > flowEndSeconds, not flowStartMilliseconds and flowEndMilliseconds. > > When exporting anonymised data and anonymisation metadata, Exporting > Processes SHOULD ensure that the combination of Information Element > and declared anonymisation technique are compatible. Specifically, > the applicable and recommended Information Element types and > semantics for each technique are noted in the description of the > anonymisationTechnique Information Element in Section 6.2.3. In this > description, a timestamp is an Information Element with the data type > dateTimeSeconds, dataTimeMilliseconds, dateTimeMicroseconds, or > dateTimeNanoseconds; an address is an Information Element with the > data type ipv4Address, ipv6Address, or macAddress; and an identifier > is an Information Element with identifier data type semantics. > Exporting Process MUST NOT export Anonymisation Options records > binding techniques to Information Elements to which they are not > applicable, and SHOULD NOT export Anonymisation Options records > binding techniques to Information Elements for which they are not > recommended. > > 7.2.2. Export of Perimeter-Based Anonymisation Policies > > Data collected from a single network may require different > anonymisation policies for addresses internal and external to the > network. For example, internal addresses could be subject to simple > permutation, while external addresses could be aggregated into > networks by truncation. When exporting anonymised perimeter > bidirectional flow (biflow) data as in section 5.2 of [RFC5103], this > arrangement may be easily represented by specifying one technique for > source endpoint information (which represents the external endpoint > in a perimeter biflow) and one technique for destination endpoint > information (which represents the internal address in a perimeter > biflow). > > However, it can also be useful to represent perimeter-based > anonymisation policies with unidirectional flow (uniflow), or non- > perimeter biflow data. In this case, the Perimeter Anonymisation bit > (bit 2) in the anonymisationFlags Information Element describing the > anonymised address Information Elements can be set to change the > meaning of "source" and "destination" of Information Elements to mean > "external" and "internal" as with perimeter biflows, but only with > respect to anonymisation policies. > > 7.2.3. Anonymisation of Header Data > > Each IPFIX Message contains a Message Header; within this Message > Header are contained two fields which may be used to break certain > anonymisation techniques: the Export Time, and the Observation Domain > > Boschi& Trammell Expires August 19, 2010 [Page 28] > Internet-Draft IP Flow Anonymisation Support February 2010 > > ID > > Export of IPFIX Messages containing anonymised timestamp data where > the original Export Time Message header has some relationship to the > anonymised timestamps SHOULD anonymise the Export Time header field > using an equivalent technique, if possible. Otherwise, relationships > between export and flow time could be used to partially or totally > reverse timestamp anonymisation. > What if the time is in the future, the collector would probably discard the information. You might want to say which methods of 4.3.* applies. > The similarity in size between an Observation Domain ID and an IPv4 > address (32 bits) may lead to a temptation to use an IPv4 interface > address on the Metering or Exporting Process as the Observation > Domain ID. If this address bears some relation to the IP addresses > in the flow data (e.g., shares a network prefix with internal > addresses) and the IP addresses in the flow data are anonymised in a > structure-preserving way, then the Observation Domain ID may be used > to break the IP address anonymisation. Use of an IPv4 interface > address on the Metering or Exporting Process as the Observation > Domain ID is NOT RECOMMENDED in this case. > > 7.2.4. Anonymisation of Options Data > > IPFIX uses the Options mechanism to export, among other things, > metadata about exported flows and the flow collection infrastructure. > As with the IPFIX Message Header, certain Options recommended in > [RFC5101] and [RFC5655] containing flow timestamps and network > addresses of Exporting and Collecting Processes may be used to break > certain anonymisation techniques; care should be taken while using > them with anonymised data export and storage. > > The Exporting Process Reliability Statistics Options Template, > recommended in [RFC5101], contains an Exporting Process ID field, > which may be an exportingProcessIPv4Address Information Element or an > exportingProcessIPv6Address Information Element. If the Exporting > Process address bears some relation to the IP addresses in the flow > data (e.g., shares a network prefix with internal addresses) and the > IP addresses in the flow data are anonymised in a structure- > preserving way, then the Exporting Process address may be used to > break the IP address anonymisation. Exporting Processes exporting > anonymised data in this situation SHOULD mitigate the risk of attack > either by omitting Options described by the Exporting Process > Reliability Statistics Options Template, or by anonymising the > Exporting Process address using a similar technique to that used to > anonymise the IP addresses in the exported data. > > Similarly, the Export Session Details Options Template and Message > Details Options Template specified for the IPFIX File Format > [RFC5655] may contain the exportingProcessIPv4Address Information > > Boschi& Trammell Expires August 19, 2010 [Page 29] > Internet-Draft IP Flow Anonymisation Support February 2010 > > Element or the exportingProcessIPv6Address Information Element to > identify an Exporting Process from which a flow record was received, > and the collectingProcessIPv4Address Information Element or the > collectingProcessIPv6Address Information Element to identify the > Collecting Process which received it. If the Exporting Process or > Collecting Process address bears some relation to the IP addresses in > the flow data (e.g., shares a network prefix with internal addresses) > and the IP addresses in the flow data are anonymised in a structure- > preserving way, then the Exporting Process or Collecting Process > address may be used to break the IP address anonymisation. Since > these Options Templates are primarily intended for storing IPFIX > Transport Session data for auditing, replay, and testing purposes, it > is NOT RECOMMENDED that storage of anonymised data include these > Options Templates in order to mitigate the risk of attack. > > The Message Details Options Template specified for the IPFIX File > Format [RFC5655] also contains the collectionTimeMilliseconds > Information Element. As with the Export Time Message Header field, > if the exported flow data contains anonymised timestamp information, > and the collectionTimeMilliseconds Information Element in a given > Message has some relationship to the anonymised timestamp > information, then this relationship can be exploited to reverse the > timestamp anonymisation. Since this Options Template is primarily > intended for storing IPFIX Transport Session data for auditing, > replay, and testing purposes, it is NOT RECOMMENDED that storage of > anonymised data include this Options Template in order to mitigate > the risk of attack. > > Since the Time Window Options Template specified for the IPFIX File > Format [RFC5655] refers to the timestamps within the flow data to > provide partial table of contents information for an IPFIX File, care > must be taken to ensure that Options described by this template are > written using the anonymised timestamps instead of the original ones. > Is there something in the IPFIX-MIB or IPFIX-CONF that can be used to deduce something about the anonymization technique? > 7.2.5. Special-Use Address Space Considerations > > When anonymising data for transport or storage using IPFIX containing > anonymised IP addresses, and the analysis purpose permits doing so, > it is recommended to filter out or leave unanonymised data containing > the special-use IPv4 addresses enumerated in [RFC3330] or the > special-use IPv6 addresses enumerated in [RFC5153]. Data containing > these addresses (e.g. 0.0.0.0 and 169.254.0.0/16 for link-local > autoconfiguration in IPv4 space) are often associated with specific, > well-known behavioral patterns. Detection of these patterns in > anonymised data can lead to deanonymisation of these special-use > addresses, which increases the chance of a complete reversal of > anonymisation by an attacker, especially of prefix-preserving > techniques. > > Boschi& Trammell Expires August 19, 2010 [Page 30] > Internet-Draft IP Flow Anonymisation Support February 2010 > > 8. Examples > A nice to have would be the flow records before and after the anonymization. > In this example, consider the export or storage of an anonymised IPv4 > dataset from a single network described by a simple template > containing a timestamp in seconds, a five-tuple, and packet and octet > counters. The template describing each record in this dataset is > shown in figure Figure 5. > > 1 2 3 > 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Set ID = 2 | Length = 40 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Template ID = 256 | Field Count = 8 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > |0| flowStartSeconds 150 | Field Length = 4 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > |0| sourceIPv4Address 8 | Field Length = 4 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > |0| destinationIPv4Address 12 | Field Length = 4 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > |0| sourceTransportPort 7 | Field Length = 2 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > |0| destinationTransportPort 11 | Field Length = 2 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > |0| packetDeltaCount 2 | Field Length = 4 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > |0| octetDeltaCount 1 | Field Length = 4 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > |0| protocolIdentifier 4 | Field Length = 1 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > Figure 5: Example Flow Template > > Suppose that this dataset is anonymised according to the following > policy: > > o IP addresses within the network are protected by reverse > truncation. > > o IP addresses outside the network are protected by prefix- > preserving anonymisation. > > o Octet counts are exported using degraded precision in order to > provide minimal protection against fingerprinting attacks. > > o All other fields are exported unanonymised. > > Boschi& Trammell Expires August 19, 2010 [Page 31] > Internet-Draft IP Flow Anonymisation Support February 2010 > > In order to export anonymisation records for this template and > policy, first, the Anonymisation Options Template shown in figure > Figure 6 is exported. For this example, the optional > privateEnterpriseNumber and informationElementIndex Information > Elements are omitted, because they are not used. > > 1 2 3 > 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Set ID = 3 | Length = 26 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Template ID = 257 | Field Count = 4 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Scope Field Count = 2 |0| templateID 346 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Field Length = 2 |0| informationElementId 303 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Field Length = 2 |0| anonymisationFlags 339 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Field Length = 2 |0| anonymisationTechnique 344 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Field Length = 2 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > Figure 6: Example Anonymisation Options Template > > Following the Anonymisation Options Template comes a Data Set > containing Anonymisation Records. This data set has an entry for > each Information Element Specifier in Template 256 describing the > flow records. This Data Set is shown in figure Figure 7. Note that > sourceIPv4Address and destinationIPv4Address have the Perimeter > Anonymisation (0x0004) flag set in anonymisationFlags, meaning that > source address should be treated as network-external, and the > destination address as network-internal. > > Boschi& Trammell Expires August 19, 2010 [Page 32] > Internet-Draft IP Flow Anonymisation Support February 2010 > > 1 2 3 > 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Set ID = 257 | Length = 68 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Template 256 | flowStartSeconds IE 150 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | no flags 0x0000 | Not Anonymised 1 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > Maybe I missed it, but is there a specification in this document that expresses: MUST have a record for each IE of a template, even if no anonymization is applied. Is there a specification that expresses: MUST have a record for each IE of each template, even if no IE of a particular template is anonymized. > | Template 256 | sourceIPv4Address IE 8 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Perimeter, Session SC 0x0005 | Structured Permutation 6 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Template 256 | destinationIPv4Address IE 12 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Perimeter, Stable 0x0005 | Reverse Truncation 7 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Template 256 | sourceTransportPort IE 7 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | no flags 0x0000 | Not Anonymised 1 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Template 256 | dest.TransportPort IE 11 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | no flags 0x0000 | Not Anonymised 1 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Template 256 | packetDeltaCount IE 2 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | no flags 0x0000 | Not Anonymised 1 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Template 256 | octetDeltaCount IE 1 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Stable 0x0003 | Precision Degradation 2 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Template 256 | protocolIdentifier IE 4 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | no flags 0x0000 | Not Anonymised 1 | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > Figure 7: Example Anonymisation Records > > Following the Anonymisation Records come the data sets containing the > anonymised data, exported according to the template in figure > Figure 5 > > 9. Security Considerations > > This document provides guidelines for exporting metadata about > > Boschi& Trammell Expires August 19, 2010 [Page 33] > Internet-Draft IP Flow Anonymisation Support February 2010 > > anonymised data in IPFIX, or storing metadata about anonymised data > in IPFIX Files. It is not intended as a general statement on the > applicability of specific flow data anonymisation techniques. > Exporters or publishers of anonymised data must take care that the > applied anonymisation technique is appropriate for the data source, > the purpose, and the risk of deanonymisation of a given application. > > We note specifically that anonymisation is not a replacement for > encryption for confidentiality. It is only appropriate for > protecting identifying information in data to be used for purposes in > which the protected data is irrelevant. Confidentiality in export is > best served by using TLS or DTLS as in the Security Considerations > section of [RFC5101], and in long-term storage by implementation- > specific protection applied as in the Security Considerations section > of [RFC5655]. Indeed, confidentiality and anonymisation are not > mutually exclusive, as encryption for confidentiality may be applied > to anonymised data export or storage, as well, when the anonymised > data is not intended for public release. > > When using pseudonymisation techniques that have a mutable mapping, > there is an inherent tradeoff in the stability of the map between > long-term comparability and security of the dataset against > deanonymisation. In general, deanonymisation attacks are more > effective given more information, so the longer a given mapping is > valid, the more information can be applied to deanonymisation. The > specific details of this are technique-dependent and therefore out of > the scope of this document. > > When releasing anonymised data, publishers need to ensure that data > that could be used in deanonymisation is not leaked through the > export protocol; guidelines for addressing this risk are provided in > Section 7.2. > > Note as well that the Security Considerations section of [RFC5101] > applies as well to the export of anonymised data, and the Security > Considerations section of [RFC5655] to the storage of anonymised > data, or the publication of anonymised traces. > > 10. IANA Considerations > > This document specifies the creation of several new IPFIX Information > Elements in the IPFIX Information Element registry located at > http://www.iana.org/assignments/ipfix, as defined in Section 6.2 > above. IANA has assigned the following Information Element numbers > for their respective Information Elements as specified below: > > Boschi& Trammell Expires August 19, 2010 [Page 34] > Internet-Draft IP Flow Anonymisation Support February 2010 > > o Information Element number TBD1 for the anonymisationFlags > Information Element. > > o Information Element number TBD2 for the anonymisationTechnique > Information Element. > > o Information Element number TBD3 for the informationElementIndex > Information Element. > > [NOTE for IANA: The text TBDn should be replaced with the respective > assigned Information Element numbers where they appear in this > document.] > > 11. Acknowledgments > > We thank Paul Aitken and John McHugh for their comments and insight, > and Carsten Schmoll for his review. Special thanks to the ICT-PRISM > project for its material support of this work. > > 12. References > > 12.1. Normative References > > [RFC5101] Claise, B., "Specification of the IP Flow Information > Export (IPFIX) Protocol for the Exchange of IP Traffic > Flow Information", RFC 5101, January 2008. > > [RFC5102] Quittek, J., Bryant, S., Claise, B., Aitken, P., and J. > Meyer, "Information Model for IP Flow Information Export", > RFC 5102, January 2008. > > [RFC5610] Boschi, E., Trammell, B., Mark, L., and T. Zseby, > "Exporting Type Information for IP Flow Information Export > (IPFIX) Information Elements", RFC 5610, July 2009. > > [RFC5655] Trammell, B., Boschi, E., Mark, L., Zseby, T., and A. > Wagner, "Specification of the IP Flow Information Export > (IPFIX) File Format", RFC 5655, October 2009. > > [RFC3330] IANA, "Special-Use IPv4 Addresses", RFC 3330, > September 2002. > > 12.2. Informative References > > [RFC5103] Trammell, B. and E. Boschi, "Bidirectional Flow Export > Using IP Flow Information Export (IPFIX)", RFC 5103, > > Boschi& Trammell Expires August 19, 2010 [Page 35] > Internet-Draft IP Flow Anonymisation Support February 2010 > > January 2008. > > [RFC5472] Zseby, T., Boschi, E., Brownlee, N., and B. Claise, "IP > Flow Information Export (IPFIX) Applicability", RFC 5472, > March 2009. > > [RFC5470] Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek, > "Architecture for IP Flow Information Export", RFC 5470, > March 2009. > > [I-D.ietf-ipfix-mediators-framework] > Kobayashi, A., Claise, B., and K. Ishibashi, "IPFIX > Mediation: Framework", > draft-ietf-ipfix-mediators-framework-04 (work in > progress), October 2009. > > [I-D.ietf-ipfix-mediators-problem-statement] > Kobayashi, A., Claise, B., Nishida, H., Sommer, C., > Dressler, F., and E. Stephan, "IPFIX Mediation: Problem > Statement", > draft-ietf-ipfix-mediators-problem-statement-07 (work in > progress), December 2009. > > [RFC5153] Boschi, E., Mark, L., Quittek, J., Stiemerling, M., and P. > Aitken, "IP Flow Information Export (IPFIX) Implementation > Guidelines", RFC 5153, April 2008. > > [RFC3917] Quittek, J., Zseby, T., Claise, B., and S. Zander, > "Requirements for IP Flow Information Export (IPFIX)", > RFC 3917, October 2004. > > [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate > Requirement Levels", BCP 14, RFC 2119, March 1997. > > Authors' Addresses > > Elisa Boschi > Hitachi Europe > c/o ETH Zurich > Gloriastrasse 35 > 8092 Zurich > Switzerland > > Phone: +41 44 632 70 57 > Email:elisa.boschi@hitachi-eu.com > > Boschi& Trammell Expires August 19, 2010 [Page 36] > Internet-Draft IP Flow Anonymisation Support February 2010 > > Brian Trammell > Hitachi Europe > c/o ETH Zurich > Gloriastrasse 35 > 8092 Zurich > Switzerland > > Phone: +41 44 632 70 13 > Email:brian.trammell@hitachi-eu.com > > Boschi& Trammell Expires August 19, 2010 --------------040401070009040802010204 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Dear authors,

I review this draft.
You should consider the comments in sequence.

Regards, Benoit.
IPFIX Working Group                                            E. Boschi
Internet-Draft                                               B. Trammell
Intended status: Experimental                             Hitachi Europe
Expires: August 19, 2010                               February 15, 2010

                     IP Flow Anonymisation Support
                      draft-ietf-ipfix-anon-02.txt

Abstract

   This document describes anonymisation techniques for IP flow data and
   the export of anonymised data using the IPFIX protocol.  It
   categorizes common anonymisation schemes and defines the parameters
   needed to describe them.  It provides guidelines for the
   implementation of anonymised data export and storage over IPFIX, and
   describes an information model and Options-based method for
   anonymisation technique metadata export within the IPFIX protocol or
   storage in IPFIX Files.

Status of this Memo

   This Internet-Draft is submitted to IETF in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt.

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.

   This Internet-Draft will expire on August 19, 2010.

Copyright Notice

   Copyright (c) 2010 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

Boschi & Trammell        Expires August 19, 2010                [Page 1]
Internet-Draft        IP Flow Anonymisation Support        February 2010

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the BSD License.

Table of Contents

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  4
     1.1.  IPFIX Protocol Overview  . . . . . . . . . . . . . . . . .  4
     1.2.  IPFIX Documents Overview . . . . . . . . . . . . . . . . .  5
     1.3.  Anonymisation within the IPFIX Architecture  . . . . . . .  5
   2.  Terminology  . . . . . . . . . . . . . . . . . . . . . . . . .  7
   3.  Categorisation of Anonymisation Techniques . . . . . . . . . .  7
   4.  Anonymisation of IP Flow Data  . . . . . . . . . . . . . . . .  8
     4.1.  IP Address Anonymisation . . . . . . . . . . . . . . . . . 10
       4.1.1.  Truncation . . . . . . . . . . . . . . . . . . . . . . 10
       4.1.2.  Reverse Truncation . . . . . . . . . . . . . . . . . . 11
       4.1.3.  Permutation  . . . . . . . . . . . . . . . . . . . . . 11
       4.1.4.  Prefix-preserving Pseudonymisation . . . . . . . . . . 11
     4.2.  Hardware Address Anonymisation . . . . . . . . . . . . . . 12
       4.2.1.  Reverse Truncation . . . . . . . . . . . . . . . . . . 12
       4.2.2.  Permutation  . . . . . . . . . . . . . . . . . . . . . 13
       4.2.3.  Structured Pseudonymisation  . . . . . . . . . . . . . 13
     4.3.  Timestamp Anonymisation  . . . . . . . . . . . . . . . . . 13
       4.3.1.  Precision Degradation  . . . . . . . . . . . . . . . . 13
       4.3.2.  Enumeration  . . . . . . . . . . . . . . . . . . . . . 14
       4.3.3.  Random Shifts  . . . . . . . . . . . . . . . . . . . . 14
     4.4.  Counter Anonymisation  . . . . . . . . . . . . . . . . . . 14
       4.4.1.  Precision Degradation  . . . . . . . . . . . . . . . . 15
       4.4.2.  Binning  . . . . . . . . . . . . . . . . . . . . . . . 15
       4.4.3.  Random Noise Addition  . . . . . . . . . . . . . . . . 15
     4.5.  Anonymisation of Other Flow Fields . . . . . . . . . . . . 16
       4.5.1.  Binning  . . . . . . . . . . . . . . . . . . . . . . . 16
       4.5.2.  Permutation  . . . . . . . . . . . . . . . . . . . . . 16
   5.  Parameters for the Description of Anonymisation Techniques . . 16
     5.1.  Stability  . . . . . . . . . . . . . . . . . . . . . . . . 17
     5.2.  Truncation Length  . . . . . . . . . . . . . . . . . . . . 17
  
     5.3.  Bin Map  . . . . . . . . . . . . . . . . . . . . . . . . . 17
     5.4.  Permutation  . . . . . . . . . . . . . . . . . . . . . . . 18
     5.5.  Shift Amount . . . . . . . . . . . . . . . . . . . . . . . 18
   6.  Anonymisation Export Support in IPFIX  . . . . . . . . . . . . 18
     6.1.  Anonymisation Options Template . . . . . . . . . . . . . . 18

Boschi & Trammell        Expires August 19, 2010                [Page 2]
Internet-Draft        IP Flow Anonymisation Support        February 2010

     6.2.  Recommended Information Elements for Anonymisation
           Metadata . . . . . . . . . . . . . . . . . . . . . . . . . 20
       6.2.1.  informationElementIndex  . . . . . . . . . . . . . . . 20
       6.2.2.  anonymisationFlags . . . . . . . . . . . . . . . . . . 20
       6.2.3.  anonymisationTechnique . . . . . . . . . . . . . . . . 22
   7.  Applying Anonymisation Techniques to IPFIX Export and
       Storage  . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
     7.1.  Arrangement of Processes in IPFIX Anonymisation  . . . . . 24
     7.2.  IPFIX-Specific Anonymisation Guidelines  . . . . . . . . . 27
       7.2.1.  Appropriate Use of Information Elements for
               Anonymised Data  . . . . . . . . . . . . . . . . . . . 27
       7.2.2.  Export of Perimeter-Based Anonymisation Policies . . . 28
       7.2.3.  Anonymisation of Header Data . . . . . . . . . . . . . 28
       7.2.4.  Anonymisation of Options Data  . . . . . . . . . . . . 29
       7.2.5.  Special-Use Address Space Considerations . . . . . . . 30
   8.  Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
   9.  Security Considerations  . . . . . . . . . . . . . . . . . . . 33
   10. IANA Considerations  . . . . . . . . . . . . . . . . . . . . . 34
   11. Acknowledgments  . . . . . . . . . . . . . . . . . . . . . . . 35
   12. References . . . . . . . . . . . . . . . . . . . . . . . . . . 35
     12.1. Normative References . . . . . . . . . . . . . . . . . . . 35
     12.2. Informative References . . . . . . . . . . . . . . . . . . 35
   Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 36

Boschi & Trammell        Expires August 19, 2010                [Page 3]
Internet-Draft        IP Flow Anonymisation Support        February 2010

1.  Introduction

   The standardisation of an IP flow information export protocol
   [RFC5101] and associated representations removes a technical barrier
   to the sharing of IP flow data across organizational boundaries and
   with network operations, security, and research communities for a
   wide variety of purposes.  However, with wider dissemination comes
   greater risks to the privacy of the users of networks under
   measurement, and to the security of those networks.  While it is not
   a complete solution to the issues posed by distribution of IP flow
   information, anonymisation (i.e., the deletion or transformation of
   information that is considered sensitive and could be used to reveal
   the identity of subjects involved in a communication) is an important
   tool for the protection of privacy within network measurement
   infrastructures.

   This document presents a mechanism for representing anonymised data
   within IPFIX and guidelines for using it.  It begins with a
   categorization of anonymisation techniques.  It then describes
   applicability of each technique to commonly anonymisable fields of IP
   flow data, organized by information element data type and semantics
   as in [RFC5102]; enumerates the parameters required by each of the
   applicable anonymisation techniques; and provides guidelines for the
   use of each of these techniques in accordance with best practices in
   data protection.  Finally, it specifies a mechanism for exporting
   anonymised data and binding anonymisation metadata to templates using
   IPFIX Options.

1.1.  IPFIX Protocol Overview

   In the IPFIX protocol, { type, length, value } tuples are expressed
   in templates containing { type, length } pairs, specifying which {
   value } fields are present in data records conforming to the
   Template, giving great flexibility as to what data is transmitted.
   Since Templates are sent very infrequently compared with Data
   Records, this results in significant bandwidth savings.  Various
   different data formats may be transmitted simply by sending new
   Templates specifying the { type, length } pairs for the new data
   format.  See [RFC5101] for more information.

   The IPFIX information model [RFC5102] defines a large number of
   standard Information Elements which provide the necessary { type }
   information for Templates.  The use of standard elements enables
   interoperability among different vendors' implementations.
   Additionally, non-standard enterprise-specific elements may be
   defined for private use.

Boschi & Trammell        Expires August 19, 2010                [Page 4]
Internet-Draft        IP Flow Anonymisation Support        February 2010

1.2.  IPFIX Documents Overview

   "Specification of the IPFIX Protocol for the Exchange of IP Traffic
   Flow Information" [RFC5101] and its associated documents define the
   IPFIX Protocol, which provides network engineers and administrators
   with access to IP traffic flow information.

   "Architecture for IP Flow Information Export" [RFC5470] defines the
   architecture for the export of measured IP flow information out of an
   IPFIX Exporting Process to an IPFIX Collecting Process, and the basic
   terminology used to describe the elements of this architecture, per
   the requirements defined in "Requirements for IP Flow Information
   Export" [RFC3917].  The IPFIX Protocol document [RFC5101] then covers
   the details of the method for transporting IPFIX Data Records and
   Templates via a congestion-aware transport protocol from an IPFIX
   Exporting Process to an IPFIX Collecting Process.

   "Information Model for IP Flow Information Export" [RFC5102]
   describes the Information Elements used by IPFIX, including details
   on Information Element naming, numbering, and data type encoding.
   Finally, "IPFIX Applicability" [RFC5472] describes the various
   applications of the IPFIX protocol and their use of information
   exported via IPFIX, and relates the IPFIX architecture to other
   measurement architectures and frameworks.

   Additionally, "Specification of the IPFIX File Format" [RFC5655]
   describes a file format based upon the IPFIX Protocol for the storage
   of flow data.

   This document references the Protocol and Architecture documents for
   terminology, and extends the IPFIX Information Model to provide new
   Information Elements for anonymisation metadata.  The anonymisation
   techniques described herein are equally applicable to the IPFIX
   Protocol and data stored in IPFIX Files.

1.3.  Anonymisation within the IPFIX Architecture

   "Architecture for IP Flow Information Export" [RFC5470] defines the
   functions performed in sequence by the various functional blocks in
   an IPFIX Device as in the figure below.

                    Packet(s) coming into Observation Point(s)
                      |                                   |
                      v                                   v
     +----------------+-------------------------+   +-----+-------+
     |          Metering Process on an          |   |             |
     |             Observation Point            |   |             |

Boschi & Trammell        Expires August 19, 2010                [Page 5]
Internet-Draft        IP Flow Anonymisation Support        February 2010

     |                                          |   |             |
     |   packet header capturing                |   |             |
     |        |                                 |...| Metering    |
     |   timestamping                           |   | Process N   |
     |        |                                 |   |             |
     | +----->+                                 |   |             |
     | |      |                                 |   |             |
     | |   sampling Si (1:1 in case of no       |   |             |
     | |      |          sampling)              |   |             |
     | |   filtering Fi (select all when        |   |             |
     | |      |          no criteria)           |   |             |
     | +------+                                 |   |             |
     |        |                                 |   |             |
     |        |        Timing out Flows         |   |             |
     |        |    Handle resource overloads    |   |             |
     +--------|---------------------------------+   +-----|-------+
              |                                           |
      Flow Records (identified by Observation Domain)  Flow Records
              |                                           |
              +---------+---------------------------------+
                        |
   +--------------------|----------------------------------------------+
   |                    |     Exporting Process                        |
   |+-------------------|-------------------------------------------+  |
   ||                   v       IPFIX Protocol                      |  |
   ||+-----------------------------+  +----------------------------+|  |
   |||Rules for                    |  |Functions                   ||  |
   ||| Picking/sending Templates   |  |-Packetise selected Control ||  |
   ||| Picking/sending Flow Records|->|  & data Information into   ||  |
   ||| Encoding Template & data    |  |  IPFIX export packets.     ||  |
   ||| Selecting Flows to export(*)|  |-Handle export errors       ||  |
   ||+-----------------------------+  +----------------------------+|  |
   |+----------------------------+----------------------------------+  |
   |                             |                                     |
   |                    exported IPFIX Messages                        |
   |                             |                                     |
   |                +------------+-----------------+                   |
   |                |  Anonymise export packet(*)  |                   |
   |                +------------+-----------------+                   |
   |                             |                                     |
   |                +------------+-----------------+                   |
   |                |       Transport  Protocol    |                   |
   |                +------------+-----------------+                   |
   |                             |                                     |
   +-----------------------------+-------------------------------------+
                                 |
                                 v
                    IPFIX export packet to Collector

Boschi & Trammell        Expires August 19, 2010                [Page 6]
Internet-Draft        IP Flow Anonymisation Support        February 2010

   (*) indicates that the block is optional.
In the general architecture, but not in this spec. right?
                 Figure 1: IPFIX Device functional blocks
I was hoping to get the reference model from the IPFIX Mediator framework.
If the reference model was from the IPFIX Mediator framework, then we would not have the issue with optional.
   Note that, according to the original architecture specification,
What is this? RFC5470?
   IPFIX Message anonymisation is optionally performed as the final
   operation before handing the Message to the transport protocol for
   export.  While no provision is made in the architecture for
   anonymisation metadata as in Section 6, this arrangement does allow
   for the message rewriting necessary for comprehensive anonymisation
   of IPFIX export as in Section 7.  The development of the IPFIX
   Mediation [I-D.ietf-ipfix-mediators-framework] framework and the
   IPFIX File Format [RFC5655] expand upon this initial architectural
   allowance for anonymisation by adding to the list of places that
   anonymisation may be applied.  The former specifies IPFIX Mediators,
   which rewrite existing IPFIX messages, and the latter specifies a
IPFIX Message?
It would be nice to say in the terminology section which terms you use from other RFCs.
For example, data set vs dataset versus Data Set
For example, flow records is lower case, but I see IPFIX File, Options Template
Please be consistent with capitalization

   method for storage of IPFIX data in files.

   More detail on the applicable architectural arrangements of
   anonymisation can be found in Section 7.1

2.  Terminology

   Terms used in this document that are defined in the Terminology
   section of the IPFIX Protocol [RFC5101] document are to be
   interpreted as defined there.

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC 2119 [RFC2119].

3.  Categorisation of Anonymisation Techniques

   Anonymisation modifies a data set in order to protect the identity of
   the people or entities described by the data set from disclosure.
   With respect to network traffic data, anonymisation generally
   attempts to preserve some set of properties of the network traffic
   useful for a given application or applications, while ensuring the
   data cannot be traced back to the specific networks, hosts, or users
   generating the traffic.

   Anonymisation may be broadly classified according to two properties:
   recoverability and countability.  All anonymisation techniques map
   the real space of identifiers or values into a separate, anonymised
   space, according to some function.  A technique is said to be

Boschi & Trammell        Expires August 19, 2010                [Page 7]
Internet-Draft        IP Flow Anonymisation Support        February 2010

   recoverable when the function used is invertible or can otherwise be
   reversed and a real identifier can be recovered from a given
   replacement identifier.

   Countability compares the dimension of the anonymised space (N) to
   the dimension of the real space (M), and denotes how the count of
   unique values is preserved by the anonymisation function.  If the
   anonymised space is smaller than the real space, then the function is
   said to generalise the input, mapping more than one input point to
   each anonymous value (e.g., as with aggregation).  By definition,
   generalisation is not recoverable.

   If the dimensions of the anonymised and real spaces are the same,
   such that the count of unique values is preserved, then the function
   is said to be a direct substitution function.  If the dimension of
   the anonymised space is larger, such that each real value maps to a
   set of anonymised values, then the function is said to be a set
   substitution function.  Note that with set substitution functions,
   the sets of anonymised values are not necessarily disjoint.  Either
   direct or set substitution functions are said to be one-way if there
   exists no practical method for recovering the real data point from an
   anonymised one.

   This classification is summarised in the table below.

   +------------------------+-----------------+------------------------+
   | Recoverability /       | Recoverable     | Non-recoverable        |
   | Countability           |                 |                        |
   +------------------------+-----------------+------------------------+
   | N < M                  | N.A.            | Generalisation         |
   | N = M                  | Direct          | One-way Direct         |
   |                        | Substitution    | Substitution           |
   | N > M                  | Set             | One-way Set            |
   |                        | Substitution    | Substitution           |
   +------------------------+-----------------+------------------------+

4.  Anonymisation of IP Flow Data

   Due to the restricted semantics of IP flow data, there is a
   relatively limited set of specific anonymisation techniques available
   on flow data, though each falls into the broad categories above.
   Each type of field that may commonly appear in a flow record may have
   its own applicable specific techniques.

   While anonymisation is generally applied at the resolution of single
   fields within a flow record, attacks against anonymisation use entire
   flows and relationships between hosts and flows within a given data

Boschi & Trammell        Expires August 19, 2010                [Page 8]
Internet-Draft        IP Flow Anonymisation Support        February 2010

   set.  Therefore, fields which may not necessarily be identifying by
   themselves may be anonymised in order to increase the anonymity of
   the data set as a whole.

   Of all the fields in an IP flow record, IP addresses are the most
   likely to be used to directly identify entities in the real world.
   Each IP address is associated with an interface on a network host,
   and can potentially be identified with a single user.  Additionally,
   IP addresses are structured identifiers; that is, partial IP address
   prefixes may be used to identify networks just as full IP addresses
   identify hosts.  This makes anonymisation of IP addresses
   particularly important.

   Hardware addresses uniquely identify devices on the network; while
   they are not often available in traffic data collected at Layer 3,
   and cannot be used to locate devices within the network, some traces
   may contain sub-IP data including hardware address data.  Hardware
   addresses may be mappable to device serial numbers, and to the
   entities or individuals who purchased the devices, when combined with
   external databases.  They may also leak via IPv6 addresses in certain
   circumstances.  Therefore, hardware address anonymisation is also
   important.
I would clearly mention MAC address
   Port numbers identify abstract entities (applications) as opposed to
   real-world entities, but they can be used to classify hosts and user
   behavior.  Passive port fingerprinting, both of well-known and
   ephemeral ports, can be used to determine the operating system
   running on a host.  Relative data volumes by port can also be used to
   determine the host's function (workstation, web server, etc.); this
   information can be used to identify hosts and users.

   While not identifiers in and of themselves, timestamps and counters
   can reveal the behavior of the hosts and users on a network.  Any
   given network activity is recognizable by a pattern of relative time
   differences and data volumes in the associated sequence of flows,
   even without host address information.  They can therefore be used to
   identify hosts and users.  Timestamps and counters are also
   vulnerable to traffic injection attacks, where traffic with a known
   pattern is injected into a network under measurement, and this
   pattern is later identified in the anonymised data set.

   The simplest and most extreme form of anonymisation, which can be
   applied to any field of a flow record, is black-marker anonymisation,
   or complete deletion of a given field.  Note that black-marker
   anonymisation is equivalent to simply not exporting the field(s) in
   question.

   While black-marker anonymisation completely protects the data in the

Boschi & Trammell        Expires August 19, 2010                [Page 9]
Internet-Draft        IP Flow Anonymisation Support        February 2010

   deleted fields from the risk of disclosure, it also reduces the
   utility of the anonymised data set as a whole.  Techniques that
   retain some information while reducing (though not eliminating) the
   disclosure risk will be extensively discussed in the following
   sections; note that the techniques specifically applicable to IP
   addresses, timestamps, ports, and counters will be discussed in
   separate sections.

4.1.  IP Address Anonymisation

   Since IP addresses are the most common identifiers within flow data
   that can be used to directly identify a person, organization, or
   host, most of the work on flow and trace data anonymisation has gone
   into IP address anonymisation techniques.  Indeed, the aim of most
   attacks against anonymisation is to recover the map from anonymised
   IP addresses to original IP addresses thereby identifying the
   identified hosts.  There is therefore a wide range of IP address
   anonymisation schemes that fit into the following categories.

       +------------------------------------+---------------------+
       | Scheme                             | Action              |
       +------------------------------------+---------------------+
       | Truncation                         | Generalisation      |
       | Reverse Truncation                 | Generalisation      |
       | Permutation                        | Direct Substitution |
       | Prefix-preserving Pseudonymisation | Direct Substitution |
       +------------------------------------+---------------------+

4.1.1.  Truncation

   Truncation removes "n" of the least significant bits from an IP
   address, replacing them with zeroes.  In effect, it replaces a host
   address with a network address for some fixed netblock; for IPv4
   addresses, 8-bit truncation corresponds to replacement with a /24
   network address.  Truncation is a non-reversible generalisation
   scheme.  Note that while truncation is effective for making hosts
   non-identifiable, it preserves information which can be used to
   identify an organization, a geographic region, a country, or a
   continent (or RIR region of responsibility).
what is RIR?
   Truncation to an address length of 0 is equivalent to black-marker
   anonymisation.  Complete removal of IP address information is only
   recommended for analysis tasks which have no need to separate flow
   data by host or network; e.g. as a first stage to per-application
   (port) or time-series total volume analyses.

Boschi & Trammell        Expires August 19, 2010               [Page 10]
Internet-Draft        IP Flow Anonymisation Support        February 2010

4.1.2.  Reverse Truncation

   Reverse truncation removes "n" of the most significant bits from an
   IP address, replacing them with zeroes.  Reverse truncation is a non-
   reversible generalisation scheme.  Reverse truncation is effective
   for making networks unidentifiable, partially or completely removing
   information which can be used to identify an organization, a
   geographic region, a country, or a continent (or RIR region of
   responsibility).  However, it may cause ambiguity when applied to
   data collected from more than one network, since it treats all the
   hosts with the same address on different networks as if they are the
   same host.  It is not particularly useful when publishing data where
   the network of origin is known or can be easily guessed by virtue of
   the identity of the publisher.

   Like truncation, reverse truncation to an address length of 0 is
   equivalent to black-marker anonymisation.

4.1.3.  Permutation

   Permutation is a direct substitution technique, replacing each IP
   address with an address selected from the set of possible IP
   addresses, guaranteeing that each anonymised address represents a
   unique original address.  The selection function is often random,
   though it is not necessarily so.  Permutation does not preserve any
   structural information about a network, but it does preserve the
   unique count of IP addresses.  Any application that requires more
   structure than host-uniqueness will not be able to use permuted IP
   addresses.

4.1.4.  Prefix-preserving Pseudonymisation

   Prefix-preserving pseudonymisation is a direct substitution
   technique, like permutation but further restricted such that the
   structure of subnets is preserved at each level while anonymising IP
   addresses.  If two real IP addresses match on a prefix of "n" bits,
   the two anonymised IP addresses will match on a prefix of "n" bits as
   well.  This is useful when relationships among networks must be
   preserved for a given analysis task, but introduces structure into
   the anonymised data which can be exploited in attacks against the
   anonymisation technique.

   Scanning in Internet background traffic can cause particular problems
   with this technique: if a scanner uses a predictable and known
   sequence of addresses, this information can be used to reverse the
   substitution.  The low order portion of the address can be left
   unanonymized as a partial defense against this attack.

Boschi & Trammell        Expires August 19, 2010               [Page 11]
Internet-Draft        IP Flow Anonymisation Support        February 2010

4.2.  Hardware Address Anonymisation

   Flow data containing sub-IP information can also contain identifying
   information in the form of the hardware (MAC) address.  While
   hardware address information cannot be used to locate a node within a
   network, it can be used to directly uniquely identify a specific
   device.  Vendors or organizations within the supply chain may then
   have the information necessary to identify the entity or individual
   that purchased the device.

   Hardware address information is not as structured as IP address
   information.  EUI-48 and EUI-64 hardware addresses contain an
   Organizational Unique Identifier in the three most significant bytes
   of the address; this OUI additionally contains bits noting whether
   the address is locally or globally administered.  Beyond this, the
   address is unstructured, and there is no particular relationship
   among the OUIs assigned to a given vendor.

   Note that hardware address information also appear within IPv6
   addresses, as the EAP-64 address, or EAP-48 address encoded as an
   EAP-64 address, is used as the least significant 64 bits of the IPv6
   address in the case of link local addressing or stateless
   autoconfiguration; the considerations and techniques in this section
   may then apply to such IPv6 addresses as well.

           +-----------------------------+---------------------+
           | Scheme                      | Action              |
           +-----------------------------+---------------------+
           | Reverse Truncation          | Generalisation      |
           | Permutation                 | Direct Substitution |
           | Structured Pseudonymisation | Direct Substitution |
           +-----------------------------+---------------------+

4.2.1.  Reverse Truncation

   Reverse truncation removes "n" of the most significant bits from an
   MAC address, replacing them with zeroes.  Reverse truncation is a
   non-reversible generalisation scheme.  This has the effect of
   removing bits of the OUI, which identify manufacturers, before
   removing the least significant bits.  Reverse truncation of 24 bits
   zeroes out the OUI.

   Reverse truncation is effective for making device manufacturers
   partially or completely unidentifiable within a dataset.  However, it
   may cause ambiguity by introducing the possibility of truncated MAC
   address collision.  Also note that the utility or removing
   manufacturer information is dubious, and not particularly well-
   covered by the literature.

Boschi & Trammell        Expires August 19, 2010               [Page 12]
Internet-Draft        IP Flow Anonymisation Support        February 2010

   Reverse truncation to an address length of 0 is equivalent to black-
   marker anonymisation.

4.2.2.  Permutation

   Permutation is a direct substitution technique, replacing each MAC
   address with an address selected from the set of possible MAC
   addresses, guaranteeing that each anonymised address represents a
   unique original address.  The selection function is often random,
   though it is not necessarily so.  Permutation does not preserve any
   structural information about a network, but it does preserve the
   unique count of devices on the network.  Any application that
   requires more structure than host-uniqueness will not be able to use
   permuted MAC addresses.

4.2.3.  Structured Pseudonymisation

   Structured pseudonymisation for MAC addresses is a direct
   substitution technique, like permutation, but restricted such that
   the OUI (the most significant three bytes) is permuted separately
   from the node identifier, the remainder.  This is useful when the
   uniqueness of OUIs must be preserved for a given analysis task, but
   introduces structure into the anonymised data which can be exploited
   in attacks against the anonymisation technique.

4.3.  Timestamp Anonymisation

   The particular time at which a flow began or ended is not
   particularly identifiable information, but it can be used as part of
   attacks against other anonymisation techniques or for user profiling.
   Presice timestamps can be used in injected-traffic fingerprinting
precise

Yo might want to describe what fingerprinting is
   attacks as well as to identify certain activity by response delay and
   size fingerprinting.  Therefore, timestamp information may be
   anonymised in order to ensure the protection of the entire dataset.

          +-----------------------+----------------------------+
          | Scheme                | Action                     |
          +-----------------------+----------------------------+
          | Precision Degradation | Generalisation             |
          | Enumeration           | Direct or Set Substitution |
          | Random Shifts         | Direct Substitution        |
          +-----------------------+----------------------------+

4.3.1.  Precision Degradation
it's like truncation, but specific to time and counter, right?
   Precision Degradation is a generalisation technique that removes the
   most precise components of a timestamp, accounting all events
   occurring in each given interval (e.g. one millisecond for

Boschi & Trammell        Expires August 19, 2010               [Page 13]
Internet-Draft        IP Flow Anonymisation Support        February 2010

   millisecond level degradation) as simultaneous.  This has the effect
   of potentially collapsing many timestamps into one.  With this
   technique time precision is reduced, and sequencing may be lost, but
   the information at which time the event occurred is preserved.  The
   anonymised data may not be generally useful for applications which
   require strict sequencing of flows.

   Note that flow meters with low time precision (e.g. second precision,
   or millisecond precision on high-capacity networks) perform the
   equivalent of precision degradation anonymisation by their design.

   Note also that degradation to a very low precision (e.g. on the order
   of minutes, hours, or days) is commonly used in analyses operating on
   time-series aggregated data, and may also be described as binning;
   though the time scales are longer and applicability more restricted,
   this is in principle the same operation.

   Precision degradation to infinitely low precision is equivalent to
   black-marker anonymisation.  Removal of timestamp information is only
   recommended for analysis tasks which have no need to separate flows
   in time, for example for counting total volumes or unique occurrences
   of other flow keys in an entire dataset.

4.3.2.  Enumeration

   Enumeration is a substitution function that retains the chronological
   order in which events occurred while eliminating time information.
   Timestamps are substituted by equidistant timestamps (or numbers)
   starting from a randomly chosen start value.  The resulting data is
   useful for applications requiring strict sequencing, but not for
   those requiring good timing information (e.g. delay- or jitter-
   measurement for QoS applications or SLA validation).

4.3.3.  Random Shifts

   Random time shifts add a random offset to every timestamp within a
   dataset.  This reversible substitution technique therefore retains
   duration and inter-event interval information as well as
   chronological order of flows.  It is primarily intended to defeat
   traffic injection fingerprinting attacks.

4.4.  Counter Anonymisation

   Counters (such as packet and octet volumes per flow) are subject to
   fingerprinting and injection attacks against anonymisation, or for
   user profiling as timestamps are.  Counter anonymisation can help
   defeat these attacks, but are only usable for analysis tasks for
   which relative or imprecise magnitudes of activity are useful.

Boschi & Trammell        Expires August 19, 2010               [Page 14]
Internet-Draft        IP Flow Anonymisation Support        February 2010

   Counter information can also be completely removed, but this is only
   recommended for analysis tasks which have no need to evaluate the
   removed counter, for example for counting only unique occurrences of
   other flow keys.

          +-----------------------+----------------------------+
          | Scheme                | Action                     |
          +-----------------------+----------------------------+
          | Precision Degradation | Generalisation             |
          | Binning               | Generalisation             |
          | Random noise addition | Direct or Set Substitution |
          +-----------------------+----------------------------+

4.4.1.  Precision Degradation

   As with precision degradation in timestamps, precision degradation of
   counters removes lower-order bits of the counters, treating all the
   counters in a given range as having the same value.  Depending on the
   precision reduction, this loses information about the relationships
   between sizes of similarly-sized flows, but keeps relative magnitude
   information.  Precision degradation to an infinitely low precision is
   equivalent to black-marker anonymisation.

4.4.2.  Binning

   Binning can be seen as a special case of precision degradation; the
   operation is identical, except for in precision degradation the
   counter ranges are uniform, and in binning they need not be.  For
   example, a common counter binning scheme for packet counters could be
   to bin values 1-2 together, and 3-infinity together, thereby
   separating potentially completely-opened TCP connections from
   unopened ones.  Binning schemes are generally chosen to keep
   precisely the amount of information required in a counter for a given
   analysis task.  Note that, also unlike precision degradation, the bin
   label need not be within the bin's range.  Binning counters to a
   single bin is equivalent to black-marker anonymisation.

4.4.3.  Random Noise Addition

   Random noise addition adds a random amount to a counter in each flow;
   this is used to keep relative magnitude information and minimize the
   disruption to size relationship information while avoiding
   fingerprinting attacks against anonymisation.  Note that there is no
   guarantee that random noise addition will maintain ranking order by a
   counter among members of a set.  Random noise addition is
   particularly useful when the derived analysis data will not be
   presented in such a way as to require the lower-order bits of the
   counters.

Boschi & Trammell        Expires August 19, 2010               [Page 15]
Internet-Draft        IP Flow Anonymisation Support        February 2010

4.5.  Anonymisation of Other Flow Fields

   Other fields, particularly port numbers and protocol numbers, can be
   used to partially identify the applications that generated the
   traffic in a a given flow trace.  This information can be used in
   fingerprinting attacks, and may be of interest on its own (e.g., to
   reveal that a certain application with suspected vulnerabilities is
   running on a given network).  These fields are generally anonymised
   using one of two techniques.

                   +-------------+---------------------+
                   | Scheme      | Action              |
                   +-------------+---------------------+
                   | Binning     | Generalisation      |
                   | Permutation | Direct Substitution |
                   +-------------+---------------------+

4.5.1.  Binning

   Binning is a generalisation technique mapping a set of potentially
   non-uniform ranges into a set of arbitrarily labeled bins.  Common
   bin arrangements depend on the field type and the analysis
   application.  For example, an IP protocol bin arrangement may
   preserve 1, 6, and 17 for ICMP, UDP, and TCP traffic, and bin all
   other protocols into a single bin, to mitigate the use of uncommon
   protocols in fingerprinting attacks.  Another example arrangement may
   bin source and destination ports into low (0-1023) and high (1024-
   65535) bins in order to tell service from ephemeral ports without
   identifying individual applications.

   Binning other flow key fields to a single bin is equivalent to black-
   marker anonymisation.  Removal of other flow key information is only
   recommended for analysis tasks which have no need to differentiate
   flows on the removed keys, for example for total traffic counts or
   unique counts of other flow keys.

4.5.2.  Permutation

   Permutation is a direct substitution technique, replacing each value
   with an value selected from the set of possible range, guaranteeing
   that each anonymised value represents a unique original value.  This
   is used to preserve the count of unique values without preserving
   information about, or the ordering of, the values themselves.

5.  Parameters for the Description of Anonymisation Techniques

   This section details the abstract parameters used to describe the

Boschi & Trammell        Expires August 19, 2010               [Page 16]
Internet-Draft        IP Flow Anonymisation Support        February 2010

   anonymisation techniques examined in the previous section, on a per-
   parameter basis.  These parameters and their export safety inform the
   design of the IPFIX anonymisation metadata export specified in the
   following section.
you use different terms for metadata
IPFIX anonymisation metadata export
anonymisation technique metadata export
anonymisation metadata
metadata about exported flows and the flow collection infrastructure
...
You might want to be consistent

5.1.  Stability

   Any given anonymisation technique may be applied with a varying range
   of stability.  Stability is important for assessing the comparability
   of anonymised information in different data sets, or in the same data
   set over different time periods.  In general, stability ranges from
   completely stable to completely unstable; however, note that the
   completely unstable case is indistinguishable from black-marker
   anonymisation.  A completely stable anonymisation will always map a
   given value in the real space to the same value in the anonymised
   space.  In practice, an anonymisation may also be stable for every
   data set published by an a particular producer to a particular
   consumer, stable for a stated time period within a dataset or across
   datasets, or stable only for a single data set.

   If no information about stability is available, users of anonymised
   data may assume that the techniques used are stable across the entire
   dataset, but unstable across datasets.  Note that stability presents
   a risk-utility tradeoff, as completely stable anonymisation can be
   used for longer-term trend analysis tasks but also presents more risk
   of attack given the stable mapping.
In this section, you don't say if you must export the content of this section, while you do for all the other 5.x sections
5.2.  Truncation Length

   Truncation and precision degradation are described by the truncation
   length, or the amount of data still remaining in the anonymised field
   after anonymisation.

   Truncation length can be inferred from a given data set, and need not
   be specially exported or protected.

5.3.  Bin Map

   Binning is described by the specification of a bin mapping function.
   This function can be generally expressed in terms of an associative
   array that maps each point in the original space to a bin, although
   from an implementation standpoint most bin functions are much simpler
   and more efficient.

   Since knowledge of the bin mapping function can be used to partially
   deanonymise binned data, depending on the degree of generalisation,
   no information about the bin mapping function should be exported.

Boschi & Trammell        Expires August 19, 2010               [Page 17]
Internet-Draft        IP Flow Anonymisation Support        February 2010

5.4.  Permutation

   Like binning, permutation is described by the specification of a
   permutation function.  In the general case, this can be expressed in
   terms of an associative array that maps each point in the original
   space to a point in the anonymised space.  Unlike binning, each point
   in the anonymised space must correspond to a single, unique point in
   the original space.

   Since knowledge of the permutation function can be used to completely
   deanonymise permuted data, no information about the permutation
   function or its parameters should be exported.

5.5.  Shift Amount

   Shifting requires an amount to shift each value by.  Since the shift
   amount can be used to deanonymise data protected by shifting, no
   information about the shift amount should be exported.

6.  Anonymisation Export Support in IPFIX

   Anonymised data exported via IPFIX SHOULD be annotated with
   anonymisation metadata, which details which fields described by which
   Templates are anonymised, and provides appropriate information on the
   anonymisation techniques used.  This metadata SHOULD be exported in
   Data Records described by the recommended Options Templates described
   in this section; these Options Templates use the additional
   Information Elements described in the following subsection.

   Note that fields anonymised using the black-marker (removal)
   technique do not require any special metadata support.  Black-marker
   anonymised fields SHOULD NOT be exported at all; the absence of the
MUST NOT?
   field in a given Data Set is implicitly declared by not including the
   corresponding Information Element in the Template describing that
   Data Set.

6.1.  Anonymisation Options Template

   The Anonymisation Options Template describes anonymisation records,
   which allow anonymisation metadata to be exported inline over IPFIX
   or stored in an IPFIX File, by binding information about
   anonymisation techniques to Information Elements within defined
   Templates.  IPFIX Exporting Processes SHOULD export anonymisation
   records for any Template describing exported anonymised Data Records;
an anonymization record is actually a data record of the Anonymization Options Template Record, as opposed to an anonymized record.
It would be better to define those terms, no

   IPFIX Collecting Processes and processes downstream from them MAY use
   anonymisation records to treat anonymised data differently depending
   on the applied technique.

Boschi & Trammell        Expires August 19, 2010               [Page 18]
Internet-Draft        IP Flow Anonymisation Support        February 2010

   An Exporting Process SHOULD export anonymisation records after the
   Templates they describe have been exported,
And MUST send the anonymization record before the anomized records?

Do you want to benefit from the per-stream SCTP draft.
That would solve most of your problems: order, reliable
Otherwise, if the collector doesn't have received the anonimization records, then it might be treating the flow records wrongly.
 and SHOULD export
   anonymisation records reliably.

   Anonymisation records, like Templates, MUST be handled by Collecting
   Processes as scoped to the Transport Session in which they are sent.
   While the Stability Class within the anonymisationFlags IE can be
   used to declare that a given anonymisation technique's mapping will
   remain stable across multiple sessions, each session MUST re-export
   the anonymisation Records along with the templates.

   +-------------------------+-----------------------------------------+
   | IE                      | Description                             |
   +-------------------------+-----------------------------------------+
   | templateId [scope]      | The Template ID of the Template         |
   |                         | containing the Information Element      |
   |                         | described by this anonymisation record. |
   |                         | This Information Element MUST be        |
   |                         | defined as a Scope Field.               |
   | informationElementId    | The Information Element identifier of   |
   | [scope]                 | the Information Element described by    |
   |                         | this anonymisation record.  This        |
   |                         | Information Element MUST be defined as  |
   |                         | a Scope Field.                          |
   | informationElementId    | The Private Enterprise Number of the    |
   | [scope] [optional]      | enterprise-specific Information Element |
   |                         | described by this anonymisation record. |
   |                         | This Information Element MUST be        |
   |                         | defined as a Scope Field if present.    |
Isn't it privateEnterpriseNumber instead of informationElementId?
Actually, the example and the description mention privateEnterpriseNumber
However, I'm wondering why you need this privateEnterpriseNumber at all,  because the "E" bit is already part of informationElementId 
 0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |E|  Information Element ident. |        Field Length           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                      Enterprise Number                        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
I'm confused. 



  
   | informationElementIndex | The Information Element index of the    |
   | [scope] [optional]      | instance of the Information Element     |
   |                         | described by this anonymisation record  |
   |                         | identified by the informationElementId  |
   |                         | within the Template.  Optional; need    |
   |                         | only be present when describing         |
   |                         | Templates that have multiple instances  |
   |                         | of the same Information Element.  This  |
   |                         | Information Element MUST be defined as  |
   |                         | a Scope Field if present.  This         |
   |                         | Information Element is defined in       |
   |                         | Section 6.2, below.                     |
   | anonymisationFlags      | Flags describing the mapping stability  |
   |                         | and specialized modifications to the    |
   |                         | Anonymisation Technique in use.  SHOULD |
   |                         | be present.  This Information Element   |
   |                         | is defined in Section 6.2, below.       |

Boschi & Trammell        Expires August 19, 2010               [Page 19]
Internet-Draft        IP Flow Anonymisation Support        February 2010

   | anonymisationTechnique  | The technique used to anonymise the     |
   |                         | data.  MUST be present.  This           |
   |                         | Information Element is defined in       |
   |                         | Section 6.2, below.                     |
   +-------------------------+-----------------------------------------+

6.2.  Recommended Information Elements for Anonymisation Metadata

6.2.1.  informationElementIndex

   Description:   A zero-based index of an Information Element
      referenced by informationElementId within a Template referenced by
      templateId; used to disambiguate scope for templates containing
      multiple identical Information Elements.

   Abstract Data Type:   unsigned16

   ElementId:   TBD3

   Status:   Proposed

6.2.2.  anonymisationFlags

   Description:   A flag word describing specialized modifications to
      the anonymisation policy in effect for the anonymisation technique
      applied to a referenced Information Element within a referenced
      Template.  When flags are clear (0), the normal policy (as
      described by anonymisationTechnique) applies without modification.

      MSB   14  13  12  11  10   9   8   7   6   5   4   3   2   1  LSB
      +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
      |                Reserved                       |LOR|PmA|   SC  |
      +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+

                            anonymisationFlags IE

   +--------+----------+-----------------------------------------------+
   | bit(s) | name     | description                                   |
   | (LSB = |          |                                               |
   | 0)     |          |                                               |
   +--------+----------+-----------------------------------------------+
   | 0-1    | SC       | Stability Class: see the Stability Class      |
   |        |          | table below, and section Section 5.1.         |

Boschi & Trammell        Expires August 19, 2010               [Page 20]
Internet-Draft        IP Flow Anonymisation Support        February 2010

   | 2      | PmA      | Perimeter Anonymisation: when set (1), source |
  



source based on what? I remember the lengthy discussion in biflow.



  
   |        |          | address Information Elements are interpreted  |
   |        |          | as external addresses, and destination        |
   |        |          | address Information Elements are interpreted  |
   |        |          | as internal addresses, for the purposes of    |
   |        |          | associating anonymisationTechnique to         |
   |        |          | Information Elements.  MUST NOT be set when   |
   |        |          | associated with a non-endpoint (i.e., source- |
   |        |          | or destination-) Information Element.  SHOULD |
   |        |          | be consistent within a record (i.e., if a     |
   |        |          | source- Information Element has this flag     |
   |        |          | set, the corresponding destination- element   |
   |        |          | SHOULD have this flag set, and vice-versa.)   |
   | 3      | LOR      | Low-Order Unchanged: when set (1), the        |
   |        |          | low-order bits of the anonymised Information  |
   |        |          | Element contain real data.  This modification |
   |        |          | is intended for the anonymisation of          |
   |        |          | network-level addresses while leaving         |
   |        |          | host-level addresses intact in order to       |
   |        |          | preserve host level-structure, which could    |
   |        |          | otherwise be used to reverse anonymisation.   |
   |        |          | MUST NOT be set when associated with a        |
   |        |          | truncation-based anonymisationTechnique.      |
   | 4-15   | Reserved | Reserved for future use: SHOULD be cleared    |
   |        |          | (0) by the Exporting Process and MUST be      |
   |        |          | ignored by the Collecting Process.            |
   +--------+----------+-----------------------------------------------+

      The Stability Class portion of this flags word describes the
      stability class of the anonymisation technique applied to a
      referenced Information Element within a referenced Template.
      Stability classes refer to the stability of the parameters of the
      anonymisation technique, and therefore the comparability of the
      mapping between the real and anonymised values over time.  This
      determines which anonymised datasets may be compared with each
      other.  Values are as follows:

Boschi & Trammell        Expires August 19, 2010               [Page 21]
Internet-Draft        IP Flow Anonymisation Support        February 2010

   +-----+-----+-------------------------------------------------------+
   | Bit | Bit | Description                                           |
   | 1   | 0   |                                                       |
   +-----+-----+-------------------------------------------------------+
   | 0   | 0   | Undefined: the Exporting Process makes no             |
   |     |     | representation as to how stable the mapping is, or    |
   |     |     | over what time period values of this field will       |
   |     |     | remain comparable; while the Collecting Process MAY   |
   |     |     | assume Session level stability, Session level         |
   |     |     | stability is not guaranteed.  Processes SHOULD assume |
   |     |     | this is the case in the absence of stability class    |
   |     |     | information; this is the default stability class.     |
   | 0   | 1   | Session: the Exporting Process will ensure that the   |
   |     |     | parameters of the anonymisation technique are stable  |
   |     |     | during the Transport Session.  All the values of the  |
   |     |     | described Information Element for each Record         |
   |     |     | described by the referenced Template within the       |
   |     |     | Transport Session are comparable.  The Exporting      |
   |     |     | Process SHOULD endeavour to ensure at least this      |
   |     |     | stability class.                                      |
   | 1   | 0   | Exporter-Collector Pair: the Exporting Process will   |
   |     |     | ensure that the parameters of the anonymisation       |
   |     |     | technique are stable across Transport Sessions over   |
   |     |     | time with the given Collecting Process, but may use   |
   |     |     | different parameters for different Collecting         |
   |     |     | Processes.  Data exported to different Collecting     |
   |     |     | Processes is not comparable.                          |
   | 1   | 1   | Stable: the Exporting Process will ensure that the    |
   |     |     | parameters of the anonymisation technique are stable  |
   |     |     | across Transport Sessions over time, regardless of    |
   |     |     | the Collecting Process to which it is sent.           |
   +-----+-----+-------------------------------------------------------+

   Abstract Data Type:   unsigned16

   ElementId:   TBD1

   Status:   Proposed

6.2.3.  anonymisationTechnique
  



While reading, it looks more logical to have the anonymisationTechnique
before the anonymizationFlags IE


  
   Description:   A description of the anonymisation technique applied
      to a referenced Information Element within a referenced Template.
  



And options template as well? For example, in my company, we export a
options template record with the sampling rates.

This is something I missed completely, and is valid for all IEs



  
      Each technique may be applicable only to certain Information
      Elements and recommended only for certain Infomation Elements;
      these restrictions are noted in the table below.

Boschi & Trammell        Expires August 19, 2010               [Page 22]
Internet-Draft        IP Flow Anonymisation Support        February 2010

   +-------+---------------------------+-----------------+-------------+
   | Value | Description               | Applicable to   | Recommended |
   |       |                           |                 | for         |
   +-------+---------------------------+-----------------+-------------+
   | 0     | Undefined: the Exporting  | all             | all         |
   |       | Process makes no          |                 |             |
   |       | representation as to      |                 |             |
   |       | whether the defined field |                 |             |
   |       | is anonymised or not.     |                 |             |
   |       | While the Collecting      |                 |             |
   |       | Process MAY assume that   |                 |             |
   |       | the field is not          |                 |             |
   |       | anonymised, it is not     |                 |             |
   |       | guaranteed not to be.     |                 |             |
   |       | This is the default       |                 |             |
   |       | anonymisation technique.  |                 |             |
   | 1     | None: the values exported | all             | all         |
   |       | are real.                 |                 |             |
   | 2     | Precision                 | all             | all         |
   |       | Degradation/Truncation:   |                 |             |
   |       | the values exported are   |                 |             |
   |       | anonymised using simple   |                 |             |
   |       | precision degradation or  |                 |             |
   |       | truncation.  The new      |                 |             |
   |       | precision or number of    |                 |             |
   |       | truncated bits is         |                 |             |
   |       | implicit in the exported  |                 |             |
   |       | data, and can be deduced  |                 |             |
   |       | by the Collecting         |                 |             |
   |       | Process.                  |                 |             |
   | 3     | Binning: the values       | all             | all         |
   |       | exported are anonymised   |                 |             |
   |       | into bins.                |                 |             |
   | 4     | Enumeration: the values   | all             | timestamps  |
   |       | exported are anonymised   |                 |             |
   |       | by enumeration.           |                 |             |
   | 5     | Permutation: the values   | all             | identifiers |
   |       | exported are anonymised   |                 |             |
   |       | by random permutation.    |                 |             |

Boschi & Trammell        Expires August 19, 2010               [Page 23]
Internet-Draft        IP Flow Anonymisation Support        February 2010

   | 6     | Structured Permutation:   | addresses       |             |
   |       | the values exported are   |                 |             |
   |       | anonymised by random      |                 |             |
   |       | permutation, preserving   |                 |             |
   |       | bit-level structure as    |                 |             |
   |       | appropriate; this         |                 |             |
   |       | represents                |                 |             |
   |       | prefix-preserving IP      |                 |             |
   |       | address anonymisation or  |                 |             |
   |       | structured MAC address    |                 |             |
   |       | anonymisation.            |                 |             |
   | 7     | Reverse Truncation: the   | addresses       |             |
   |       | values exported are       |                 |             |
   |       | anonymised using reverse  |                 |             |
   |       | truncation.  The number   |                 |             |
   |       | of truncated bits is      |                 |             |
   |       | implicit in the exported  |                 |             |
   |       | data, and can be deduced  |                 |             |
   |       | by the Collecting         |                 |             |
   |       | Process.                  |                 |             |
   | 8     | Noise: the values         | non-identifiers | counters    |
   |       | exported are anonymised   |                 |             |
   |       | by adding random noise to |                 |             |
   |       | each value.               |                 |             |
   | 9     | Offset: the values        | all             | timestamps  |
   |       | exported are anonymised   |                 |             |
   |       | by adding a single offset |                 |             |
   |       | to all values.            |                 |             |
   +-------+---------------------------+-----------------+-------------+

   Abstract Data Type:   unsigned16

   ElementId:   TBD2

   Status:   Proposed

7.  Applying Anonymisation Techniques to IPFIX Export and Storage

   When exporting or storing anonymised flow data using IPFIX, certain
   interactions between the IPFIX Protocol and the anonymisation
   techniques in use must be considered; these are treated in the
   subsections below.

7.1.  Arrangement of Processes in IPFIX Anonymisation

   Anonymisation may be applied to IPFIX data at three stages within the
   collection infrastructure: on initial export, at a mediator, or after
  







  
Boschi & Trammell        Expires August 19, 2010               [Page 24]
Internet-Draft        IP Flow Anonymisation Support        February 2010

   collection, as shown in Figure 2.  Each of these locations has
   specific considerations and applicability.

               +==========================================+
               | Exporting Process                        |
               +==========================================+
                 |                                      |
                 |    (Anonymised at Original Exporter) |
                 V                                      |
               +=============================+          |
               | Mediator                    |          |
               +=============================+          |
                 |                                      |
                 | (Anonymising Mediator)               |
                 V                                      V
               +==========================================+
               | Collecting Process                       |
               +==========================================+
                       |
                       | (Anonymising CP/File Writer)
                       V
               +--------------------+
               | IPFIX File Storage |
               +--------------------+

                Figure 2: Potential Anonymisation Locations

   Anonymisation is generally performed before the wider dissemination
   or repurposing of a flow data set, e.g., adapting operational
   measurement data for research.  Therefore, direct anonymisation of
   flow data on initial export is only applicable in certain restricted
   circumstances: when the Exporting Process is "publishing" data to a
   Collecting Process directly, and the Exporting Process and Collecting
   Process are operated by different entities.  Note that certain
   guidelines in Section 7.2.3 with respect to timestamp anonymisation
   may not apply in this case, as the Collecting Process may be able to
   deduce certain timing information from the time at which each Message
  



IPFIX Message



  
   is received.

   A much more flexible arrangement is to anonymise data within a
   Mediator [I-D.ietf-ipfix-mediators-framework].  Here, original data
   is sent to a Mediator, which performs the anonymisation function and
   re-exports the anonymised data.  Such a Mediator could be located at
   the administrative domain boundary of the initial Exporting Process
   operator, exporting anonymised data to other consumers outside the
   organisation.  In this case, the original Exporter SHOULD use TLS as
   specified in [RFC5101] to secure the channel to the Mediator, and the
   Mediator should follow the guidelines in Section 7.2, to mitigate the

Boschi & Trammell        Expires August 19, 2010               [Page 25]
Internet-Draft        IP Flow Anonymisation Support        February 2010

   risk of original data disclosure.

   When data is to be published as an anonymised data set in an IPFIX
   File [RFC5655], the anonymisation may be done at the final Collecting
   Process before storage and dissemination, as well. 



For me, it's a different problem. IPFIX File could be done on routers.

Basically, this paragraph is about the relationship with IPFIX File.

Btw, we're alos missing the relationship with reducing redundancy, with
IPFIX structured data

For the later, does it apply to all instances of a template? So, some
kind of recursivity is implied



  
 In this case, the
   Collector should follow the guidelines in Section 7.2, especially as
   regards File-specific Options in Section 7.2.4

   In each of these data flows, the anonymisation of records is
   undertaken by an Intermediate Anonymisation Process (IAP); the data
   flows into and out of this IAP are shown in Figure 3 below.

   packets --+                     +- IPFIX Messages -+
             |                     |                  |
             V                     V                  V
   +==================+ +====================+ +=============+
   | Metering Process | | Collecting Process | | File Reader |
   +==================+ +====================+ +=============+
             |      Non-anonymised | Records          |
             V                     V                  V
   +=========================================================+
   |          Intermediate Anonymisation Process (IAP)       |
   +=========================================================+
             | Anonymised     ^            Anonymised |
             | Records        |               Records |
             V                |                       V
   +===================+    Anonymisation      +=============+
   | Exporting Process |<--- Parameters ------>| File Writer |
   +===================+                       +=============+
             |                                        |
             +------------> IPFIX Messages <----------+

          Figure 3: Data flows through the anonymisation process
  



This is exactly what I was hoping to see in figure1, i.e. this draft
applying to the Intermediate Process in [IFPIX-MD-FMWK]

... where the IP can be an independent boxe, or inside the router.

I completely missed that second part in the draft.


  
   Anonymisation parameters must also be available to the Exporting
   Process and/or File Writer in order to ensure header data is also
   appropriately anonymised as in Section 7.2.3.

   Following each of the data flows through the IAP, we describe five
   basic types of anonymisation arrangements within this framework in
   Figure 4.  In addition to the three arrangements described in detail
   above, anonymisation can also be done at a collocated Metering
   Process and File Writer (see section 7.3.2 of [RFC5655]), or at a
   file manipulator (see section 7.3.7 of [RFC5655]).

Boschi & Trammell        Expires August 19, 2010               [Page 26]
Internet-Draft        IP Flow Anonymisation Support        February 2010

         +----+  +-----+  +----+
 pkts -> | MP |->| IAP |->| EP |-> anonymisation on Original Exporter
         +----+  +-----+  +----+
         +----+  +-----+  +----+
 pkts -> | MP |->| IAP |->| FW |-> Anonymising collocated MP/File Writer
         +----+  +-----+  +----+
         +----+  +-----+  +----+
IPFIX -> | CP |->| IAP |->| EP |-> Anonymising Mediator (Masquerading Proxy)
         +----+  +-----+  +----+
         +----+  +-----+  +----+
IPFIX -> | CP |->| IAP |->| FW |-> Anonymising collocated CP/File Writer
         +----+  +-----+  +----+
         +----+  +-----+  +----+
IPFIX -> | FR |->| IAP |->| FW |-> Anonymising file manipulator
 File    +----+  +-----+  +----+

        Figure 4: Possible anonymisation arrangements in the IPFIX
                               architecture

   Note that anonymisation may occur at more than one location within a
   given collection infrastructure, to provide varying levels of
   anonymisation, disclosure risk, or data utility for specific
   purposes.

7.2.  IPFIX-Specific Anonymisation Guidelines

   In implementing and deploying the anonymisation techniques described
   in this document, implementors should note that IPFIX already
   provides features that support anonymised data export, and use these
   where appropriate.  Care must also be taken that data structures
   supporting the operation of the protocol itself do not leak data that
   could be used to reverse the anonymisation applied to the flow data.
   Such data structures may appear in the header, or within the data
   stream itself, especially as options data.  Each of these and their
   impact on specific anonymisation techniques is noted in a separate
   subsection below.

7.2.1.  Appropriate Use of Information Elements for Anonymised Data

   Note, as in Section 6 above, that black-marker anonymised fields
   SHOULD NOT be exported at all; the absence of the field in a given
   Data Set is implicitly declared by not including the corresponding
   Information Element in the Template describing that Data Set.

   When using precision degradation of timestamps, Exporting Processes
   SHOULD export timing information using Information Elements of an
   appropriate precision, as explained in Section 4.5 of [RFC5153].  For
   example, timestamps measured in millisecond-level precision and

Boschi & Trammell        Expires August 19, 2010               [Page 27]
Internet-Draft        IP Flow Anonymisation Support        February 2010

   degraded to second-level precision should use flowStartSeconds and
   flowEndSeconds, not flowStartMilliseconds and flowEndMilliseconds.

   When exporting anonymised data and anonymisation metadata, Exporting
   Processes SHOULD ensure that the combination of Information Element
   and declared anonymisation technique are compatible.  Specifically,
   the applicable and recommended Information Element types and
   semantics for each technique are noted in the description of the
   anonymisationTechnique Information Element in Section 6.2.3.  In this
   description, a timestamp is an Information Element with the data type
   dateTimeSeconds, dataTimeMilliseconds, dateTimeMicroseconds, or
   dateTimeNanoseconds; an address is an Information Element with the
   data type ipv4Address, ipv6Address, or macAddress; and an identifier
   is an Information Element with identifier data type semantics.
   Exporting Process MUST NOT export Anonymisation Options records
   binding techniques to Information Elements to which they are not
   applicable, and SHOULD NOT export Anonymisation Options records
   binding techniques to Information Elements for which they are not
   recommended.

7.2.2.  Export of Perimeter-Based Anonymisation Policies

   Data collected from a single network may require different
   anonymisation policies for addresses internal and external to the
   network.  For example, internal addresses could be subject to simple
   permutation, while external addresses could be aggregated into
   networks by truncation.  When exporting anonymised perimeter
   bidirectional flow (biflow) data as in section 5.2 of [RFC5103], this
   arrangement may be easily represented by specifying one technique for
   source endpoint information (which represents the external endpoint
   in a perimeter biflow) and one technique for destination endpoint
   information (which represents the internal address in a perimeter
   biflow).

   However, it can also be useful to represent perimeter-based
   anonymisation policies with unidirectional flow (uniflow), or non-
   perimeter biflow data.  In this case, the Perimeter Anonymisation bit
   (bit 2) in the anonymisationFlags Information Element describing the
   anonymised address Information Elements can be set to change the
   meaning of "source" and "destination" of Information Elements to mean
   "external" and "internal" as with perimeter biflows, but only with
   respect to anonymisation policies.

7.2.3.  Anonymisation of Header Data

   Each IPFIX Message contains a Message Header; within this Message
   Header are contained two fields which may be used to break certain
   anonymisation techniques: the Export Time, and the Observation Domain

Boschi & Trammell        Expires August 19, 2010               [Page 28]
Internet-Draft        IP Flow Anonymisation Support        February 2010

   ID

   Export of IPFIX Messages containing anonymised timestamp data where
   the original Export Time Message header has some relationship to the
   anonymised timestamps SHOULD anonymise the Export Time header field
   using an equivalent technique, if possible.  Otherwise, relationships
   between export and flow time could be used to partially or totally
   reverse timestamp anonymisation.
  



What if the time is in the future, the collector would probably discard
the information.

You might want to say which methods of 4.3.* applies. 





  
   The similarity in size between an Observation Domain ID and an IPv4
   address (32 bits) may lead to a temptation to use an IPv4 interface
   address on the Metering or Exporting Process as the Observation
   Domain ID.  If this address bears some relation to the IP addresses
   in the flow data (e.g., shares a network prefix with internal
   addresses) and the IP addresses in the flow data are anonymised in a
   structure-preserving way, then the Observation Domain ID may be used
   to break the IP address anonymisation.  Use of an IPv4 interface
   address on the Metering or Exporting Process as the Observation
   Domain ID is NOT RECOMMENDED in this case.

7.2.4.  Anonymisation of Options Data

   IPFIX uses the Options mechanism to export, among other things,
   metadata about exported flows and the flow collection infrastructure.
   As with the IPFIX Message Header, certain Options recommended in
   [RFC5101] and [RFC5655] containing flow timestamps and network
   addresses of Exporting and Collecting Processes may be used to break
   certain anonymisation techniques; care should be taken while using
   them with anonymised data export and storage.

   The Exporting Process Reliability Statistics Options Template,
   recommended in [RFC5101], contains an Exporting Process ID field,
   which may be an exportingProcessIPv4Address Information Element or an
   exportingProcessIPv6Address Information Element.  If the Exporting
   Process address bears some relation to the IP addresses in the flow
   data (e.g., shares a network prefix with internal addresses) and the
   IP addresses in the flow data are anonymised in a structure-
   preserving way, then the Exporting Process address may be used to
   break the IP address anonymisation.  Exporting Processes exporting
   anonymised data in this situation SHOULD mitigate the risk of attack
   either by omitting Options described by the Exporting Process
   Reliability Statistics Options Template, or by anonymising the
   Exporting Process address using a similar technique to that used to
   anonymise the IP addresses in the exported data.

   Similarly, the Export Session Details Options Template and Message
   Details Options Template specified for the IPFIX File Format
   [RFC5655] may contain the exportingProcessIPv4Address Information

Boschi & Trammell        Expires August 19, 2010               [Page 29]
Internet-Draft        IP Flow Anonymisation Support        February 2010

   Element or the exportingProcessIPv6Address Information Element to
   identify an Exporting Process from which a flow record was received,
   and the collectingProcessIPv4Address Information Element or the
   collectingProcessIPv6Address Information Element to identify the
   Collecting Process which received it.  If the Exporting Process or
   Collecting Process address bears some relation to the IP addresses in
   the flow data (e.g., shares a network prefix with internal addresses)
   and the IP addresses in the flow data are anonymised in a structure-
   preserving way, then the Exporting Process or Collecting Process
   address may be used to break the IP address anonymisation.  Since
   these Options Templates are primarily intended for storing IPFIX
   Transport Session data for auditing, replay, and testing purposes, it
   is NOT RECOMMENDED that storage of anonymised data include these
   Options Templates in order to mitigate the risk of attack.

   The Message Details Options Template specified for the IPFIX File
   Format [RFC5655] also contains the collectionTimeMilliseconds
   Information Element.  As with the Export Time Message Header field,
   if the exported flow data contains anonymised timestamp information,
   and the collectionTimeMilliseconds Information Element in a given
   Message has some relationship to the anonymised timestamp
   information, then this relationship can be exploited to reverse the
   timestamp anonymisation.  Since this Options Template is primarily
   intended for storing IPFIX Transport Session data for auditing,
   replay, and testing purposes, it is NOT RECOMMENDED that storage of
   anonymised data include this Options Template in order to mitigate
   the risk of attack.

   Since the Time Window Options Template specified for the IPFIX File
   Format [RFC5655] refers to the timestamps within the flow data to
   provide partial table of contents information for an IPFIX File, care
   must be taken to ensure that Options described by this template are
   written using the anonymised timestamps instead of the original ones.
  



Is there something in the IPFIX-MIB or IPFIX-CONF that can be used to
deduce something about the anonymization technique?



  
7.2.5.  Special-Use Address Space Considerations

   When anonymising data for transport or storage using IPFIX containing
   anonymised IP addresses, and the analysis purpose permits doing so,
   it is recommended to filter out or leave unanonymised data containing
   the special-use IPv4 addresses enumerated in [RFC3330] or the
   special-use IPv6 addresses enumerated in [RFC5153].  Data containing
   these addresses (e.g. 0.0.0.0 and 169.254.0.0/16 for link-local
   autoconfiguration in IPv4 space) are often associated with specific,
   well-known behavioral patterns.  Detection of these patterns in
   anonymised data can lead to deanonymisation of these special-use
   addresses, which increases the chance of a complete reversal of
   anonymisation by an attacker, especially of prefix-preserving
   techniques.

Boschi & Trammell        Expires August 19, 2010               [Page 30]
Internet-Draft        IP Flow Anonymisation Support        February 2010

8.  Examples
  



A nice to have would be the flow records before and after the
anonymization.



  
   In this example, consider the export or storage of an anonymised IPv4
   dataset from a single network described by a simple template
   containing a timestamp in seconds, a five-tuple, and packet and octet
   counters.  The template describing each record in this dataset is
   shown in figure Figure 5.

                            1                   2                   3
        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |          Set ID = 2           |          Length =  40         |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |      Template ID = 256        |        Field Count = 8        |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |0| flowStartSeconds        150 |       Field Length =  4       |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |0| sourceIPv4Address         8 |       Field Length =  4       |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |0| destinationIPv4Address   12 |       Field Length =  4       |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |0| sourceTransportPort       7 |       Field Length =  2       |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |0| destinationTransportPort 11 |       Field Length =  2       |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |0| packetDeltaCount          2 |       Field Length =  4       |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |0| octetDeltaCount           1 |       Field Length =  4       |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |0| protocolIdentifier        4 |       Field Length =  1       |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                      Figure 5: Example Flow Template

   Suppose that this dataset is anonymised according to the following
   policy:

   o  IP addresses within the network are protected by reverse
      truncation.

   o  IP addresses outside the network are protected by prefix-
      preserving anonymisation.

   o  Octet counts are exported using degraded precision in order to
      provide minimal protection against fingerprinting attacks.

   o  All other fields are exported unanonymised.

Boschi & Trammell        Expires August 19, 2010               [Page 31]
Internet-Draft        IP Flow Anonymisation Support        February 2010

   In order to export anonymisation records for this template and
   policy, first, the Anonymisation Options Template shown in figure
   Figure 6 is exported.  For this example, the optional
   privateEnterpriseNumber and informationElementIndex Information
   Elements are omitted, because they are not used.

                              1                   2                   3
          0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
         +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
         |          Set ID = 3           |          Length =  26         |
         +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
         |      Template ID = 257        |        Field Count = 4        |
         +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
         |    Scope Field Count = 2      |0| templateID              346 |
         +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
         |       Field Length = 2        |0| informationElementId    303 |
         +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
         |       Field Length = 2        |0| anonymisationFlags      339 |
         +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
         |       Field Length = 2        |0| anonymisationTechnique  344 |
         +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
         |       Field Length = 2        |
         +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

             Figure 6: Example Anonymisation Options Template

   Following the Anonymisation Options Template comes a Data Set
   containing Anonymisation Records.  This data set has an entry for
   each Information Element Specifier in Template 256 describing the
   flow records.  This Data Set is shown in figure Figure 7.  Note that
   sourceIPv4Address and destinationIPv4Address have the Perimeter
   Anonymisation (0x0004) flag set in anonymisationFlags, meaning that
   source address should be treated as network-external, and the
   destination address as network-internal.

Boschi & Trammell        Expires August 19, 2010               [Page 32]
Internet-Draft        IP Flow Anonymisation Support        February 2010

                            1                   2                   3
        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |          Set ID = 257         |          Length =  68         |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |          Template 256         | flowStartSeconds       IE 150 |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       | no flags               0x0000 | Not Anonymised              1 |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  



Maybe I missed it, but is there a specification in this document that
expresses: MUST have a record for each IE of a template, even if no
anonymization is applied.

Is there a specification that expresses: MUST have a record for each IE
of each template, even if no IE of a particular template is anonymized.







  
       |          Template 256         | sourceIPv4Address        IE 8 |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       | Perimeter, Session SC 0x0005  | Structured Permutation      6 |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |          Template 256         | destinationIPv4Address  IE 12 |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       | Perimeter, Stable     0x0005  | Reverse Truncation          7 |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |          Template 256         | sourceTransportPort      IE 7 |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       | no flags               0x0000 | Not Anonymised              1 |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |          Template 256         | dest.TransportPort      IE 11 |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       | no flags               0x0000 | Not Anonymised              1 |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |          Template 256         | packetDeltaCount         IE 2 |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       | no flags               0x0000 | Not Anonymised              1 |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |          Template 256         | octetDeltaCount          IE 1 |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       | Stable                 0x0003 | Precision Degradation       2 |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |          Template 256         | protocolIdentifier      IE 4  |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       | no flags               0x0000 | Not Anonymised              1 |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                  Figure 7: Example Anonymisation Records

   Following the Anonymisation Records come the data sets containing the
   anonymised data, exported according to the template in figure
   Figure 5

9.  Security Considerations

   This document provides guidelines for exporting metadata about

Boschi & Trammell        Expires August 19, 2010               [Page 33]
Internet-Draft        IP Flow Anonymisation Support        February 2010

   anonymised data in IPFIX, or storing metadata about anonymised data
   in IPFIX Files.  It is not intended as a general statement on the
   applicability of specific flow data anonymisation techniques.
   Exporters or publishers of anonymised data must take care that the
   applied anonymisation technique is appropriate for the data source,
   the purpose, and the risk of deanonymisation of a given application.

   We note specifically that anonymisation is not a replacement for
   encryption for confidentiality.  It is only appropriate for
   protecting identifying information in data to be used for purposes in
   which the protected data is irrelevant.  Confidentiality in export is
   best served by using TLS or DTLS as in the Security Considerations
   section of [RFC5101], and in long-term storage by implementation-
   specific protection applied as in the Security Considerations section
   of [RFC5655].  Indeed, confidentiality and anonymisation are not
   mutually exclusive, as encryption for confidentiality may be applied
   to anonymised data export or storage, as well, when the anonymised
   data is not intended for public release.

   When using pseudonymisation techniques that have a mutable mapping,
   there is an inherent tradeoff in the stability of the map between
   long-term comparability and security of the dataset against
   deanonymisation.  In general, deanonymisation attacks are more
   effective given more information, so the longer a given mapping is
   valid, the more information can be applied to deanonymisation.  The
   specific details of this are technique-dependent and therefore out of
   the scope of this document.

   When releasing anonymised data, publishers need to ensure that data
   that could be used in deanonymisation is not leaked through the
   export protocol; guidelines for addressing this risk are provided in
   Section 7.2.

   Note as well that the Security Considerations section of [RFC5101]
   applies as well to the export of anonymised data, and the Security
   Considerations section of [RFC5655] to the storage of anonymised
   data, or the publication of anonymised traces.

10.  IANA Considerations

   This document specifies the creation of several new IPFIX Information
   Elements in the IPFIX Information Element registry located at
   http://www.iana.org/assignments/ipfix, as defined in Section 6.2
   above.  IANA has assigned the following Information Element numbers
   for their respective Information Elements as specified below:

Boschi & Trammell        Expires August 19, 2010               [Page 34]
Internet-Draft        IP Flow Anonymisation Support        February 2010

   o  Information Element number TBD1 for the anonymisationFlags
      Information Element.

   o  Information Element number TBD2 for the anonymisationTechnique
      Information Element.

   o  Information Element number TBD3 for the informationElementIndex
      Information Element.

   [NOTE for IANA: The text TBDn should be replaced with the respective
   assigned Information Element numbers where they appear in this
   document.]

11.  Acknowledgments

   We thank Paul Aitken and John McHugh for their comments and insight,
   and Carsten Schmoll for his review.  Special thanks to the ICT-PRISM
   project for its material support of this work.

12.  References

12.1.  Normative References

   [RFC5101]  Claise, B., "Specification of the IP Flow Information
              Export (IPFIX) Protocol for the Exchange of IP Traffic
              Flow Information", RFC 5101, January 2008.

   [RFC5102]  Quittek, J., Bryant, S., Claise, B., Aitken, P., and J.
              Meyer, "Information Model for IP Flow Information Export",
              RFC 5102, January 2008.

   [RFC5610]  Boschi, E., Trammell, B., Mark, L., and T. Zseby,
              "Exporting Type Information for IP Flow Information Export
              (IPFIX) Information Elements", RFC 5610, July 2009.

   [RFC5655]  Trammell, B., Boschi, E., Mark, L., Zseby, T., and A.
              Wagner, "Specification of the IP Flow Information Export
              (IPFIX) File Format", RFC 5655, October 2009.

   [RFC3330]  IANA, "Special-Use IPv4 Addresses", RFC 3330,
              September 2002.

12.2.  Informative References

   [RFC5103]  Trammell, B. and E. Boschi, "Bidirectional Flow Export
              Using IP Flow Information Export (IPFIX)", RFC 5103,

Boschi & Trammell        Expires August 19, 2010               [Page 35]
Internet-Draft        IP Flow Anonymisation Support        February 2010

              January 2008.

   [RFC5472]  Zseby, T., Boschi, E., Brownlee, N., and B. Claise, "IP
              Flow Information Export (IPFIX) Applicability", RFC 5472,
              March 2009.

   [RFC5470]  Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek,
              "Architecture for IP Flow Information Export", RFC 5470,
              March 2009.

   [I-D.ietf-ipfix-mediators-framework]
              Kobayashi, A., Claise, B., and K. Ishibashi, "IPFIX
              Mediation: Framework",
              draft-ietf-ipfix-mediators-framework-04 (work in
              progress), October 2009.

   [I-D.ietf-ipfix-mediators-problem-statement]
              Kobayashi, A., Claise, B., Nishida, H., Sommer, C.,
              Dressler, F., and E. Stephan, "IPFIX Mediation: Problem
              Statement",
              draft-ietf-ipfix-mediators-problem-statement-07 (work in
              progress), December 2009.

   [RFC5153]  Boschi, E., Mark, L., Quittek, J., Stiemerling, M., and P.
              Aitken, "IP Flow Information Export (IPFIX) Implementation
              Guidelines", RFC 5153, April 2008.

   [RFC3917]  Quittek, J., Zseby, T., Claise, B., and S. Zander,
              "Requirements for IP Flow Information Export (IPFIX)",
              RFC 3917, October 2004.

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119, March 1997.

Authors' Addresses

   Elisa Boschi
   Hitachi Europe
   c/o ETH Zurich
   Gloriastrasse 35
   8092 Zurich
   Switzerland

   Phone: +41 44 632 70 57
   Email: elisa.boschi@hitachi-eu.com

Boschi & Trammell        Expires August 19, 2010               [Page 36]
Internet-Draft        IP Flow Anonymisation Support        February 2010

   Brian Trammell
   Hitachi Europe
   c/o ETH Zurich
   Gloriastrasse 35
   8092 Zurich
   Switzerland

   Phone: +41 44 632 70 13
   Email: brian.trammell@hitachi-eu.com

Boschi & Trammell        Expires August 19, 2010               








--------------040401070009040802010204--

From trammell@tik.ee.ethz.ch  Thu Apr  8 02:47:00 2010
Return-Path: 
X-Original-To: ipfix@core3.amsl.com
Delivered-To: ipfix@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 5DBAD3A6A2B for ; Thu,  8 Apr 2010 02:47:00 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.603
X-Spam-Level: 
X-Spam-Status: No, score=-2.603 tagged_above=-999 required=5 tests=[BAYES_50=0.001, MIME_QP_LONG_LINE=1.396, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PFGy+ItaWE7r for ; Thu,  8 Apr 2010 02:46:57 -0700 (PDT)
Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by core3.amsl.com (Postfix) with ESMTP id 6333A3A6A26 for ; Thu,  8 Apr 2010 02:46:56 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 46C16D9385; Thu,  8 Apr 2010 11:46:52 +0200 (MEST)
X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch
Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id YvYOMQoeDr+u; Thu,  8 Apr 2010 11:46:51 +0200 (MEST)
Received: from public-docking-hg-3-197.ethz.ch (public-docking-hg-3-197.ethz.ch [129.132.246.197]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id C874BD9324; Thu,  8 Apr 2010 11:46:51 +0200 (MEST)
Mime-Version: 1.0 (Apple Message framework v1078)
Content-Type: text/plain; charset=us-ascii
From: Brian Trammell 
In-Reply-To: <4BB5D827.1060406@cisco.com>
Date: Thu, 8 Apr 2010 11:46:51 +0200
Content-Transfer-Encoding: quoted-printable
Message-Id: <643E6408-8BDF-48F5-888D-EA95C812730E@tik.ee.ethz.ch>
References: <4BB5C86D.7020107@cisco.com> <4BB5D827.1060406@cisco.com>
To: Benoit Claise 
X-Mailer: Apple Mail (2.1078)
Cc: "ipfix@ietf.org" 
Subject: Re: [IPFIX] draft-ietf-ipfix-anon-02.txt: WG last call review
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Thu, 08 Apr 2010 09:47:00 -0000

hi, Benoit,

thank you very much for the review! Comments inline, irrelevant parts =
snipped. These changes will be applied to future anon-03, which will =
appear after we get Lothar's WGLC comments.

Cheers,

Brian

On Apr 2, 2010, at 1:42 PM, Benoit Claise wrote:

> Dear authors,
>=20
> I review this draft.
> You should consider the comments in sequence.
>=20
> Regards, Benoit.
>> IPFIX Working Group                                            E. =
Boschi
>> Internet-Draft                                               B. =
Trammell
>> Intended status: Experimental                             Hitachi =
Europe
>> Expires: August 19, 2010                               February 15, =
2010
>>=20
>>                      IP Flow Anonymisation Support
>>                       draft-ietf-ipfix-anon-02.txt



>> 1.3.  Anonymisation within the IPFIX Architecture
>>=20
>>=20
>>    "Architecture for IP Flow Information Export" [RFC5470] defines =
the
>>    functions performed in sequence by the various functional blocks =
in
>>    an IPFIX Device as in the figure below.
>>=20
>>                     Packet(s) coming into Observation Point(s)
>>                       |                                   |
>>                       v                                   v
>>      +----------------+-------------------------+   +-----+-------+
>>      |          Metering Process on an          |   |             |
>>      |             Observation Point            |   |             |
>>=20
>>=20
>> Boschi & Trammell        Expires August 19, 2010                [Page =
5]
>> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>>=20
>>=20
>>      |                                          |   |             |
>>      |   packet header capturing                |   |             |
>>      |        |                                 |...| Metering    |
>>      |   timestamping                           |   | Process N   |
>>      |        |                                 |   |             |
>>      | +----->+                                 |   |             |
>>      | |      |                                 |   |             |
>>      | |   sampling Si (1:1 in case of no       |   |             |
>>      | |      |          sampling)              |   |             |
>>      | |   filtering Fi (select all when        |   |             |
>>      | |      |          no criteria)           |   |             |
>>      | +------+                                 |   |             |
>>      |        |                                 |   |             |
>>      |        |        Timing out Flows         |   |             |
>>      |        |    Handle resource overloads    |   |             |
>>      +--------|---------------------------------+   +-----|-------+
>>               |                                           |
>>       Flow Records (identified by Observation Domain)  Flow Records
>>               |                                           |
>>               +---------+---------------------------------+
>>                         |
>>    =
+--------------------|----------------------------------------------+
>>    |                    |     Exporting Process                       =
 |
>>    |+-------------------|-------------------------------------------+ =
 |
>>    ||                   v       IPFIX Protocol                      | =
 |
>>    ||+-----------------------------+  +----------------------------+| =
 |
>>    |||Rules for                    |  |Functions                   || =
 |
>>    ||| Picking/sending Templates   |  |-Packetise selected Control || =
 |
>>    ||| Picking/sending Flow Records|->|  & data Information into   || =
 |
>>    ||| Encoding Template & data    |  |  IPFIX export packets.     || =
 |
>>    ||| Selecting Flows to export(*)|  |-Handle export errors       || =
 |
>>    ||+-----------------------------+  +----------------------------+| =
 |
>>    |+----------------------------+----------------------------------+ =
 |
>>    |                             |                                    =
 |
>>    |                    exported IPFIX Messages                       =
 |
>>    |                             |                                    =
 |
>>    |                +------------+-----------------+                  =
 |
>>    |                |  Anonymise export packet(*)  |                  =
 |
>>    |                +------------+-----------------+                  =
 |
>>    |                             |                                    =
 |
>>    |                +------------+-----------------+                  =
 |
>>    |                |       Transport  Protocol    |                  =
 |
>>    |                +------------+-----------------+                  =
 |
>>    |                             |                                    =
 |
>>    =
+-----------------------------+-------------------------------------+
>>                                  |
>>                                  v
>>                     IPFIX export packet to Collector
>>=20
>>=20
>> Boschi & Trammell        Expires August 19, 2010                [Page =
6]
>> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>>=20
>>=20
>>    (*) indicates that the block is optional.
>>  =20
>>=20
> In the general architecture, but not in this spec. right?

Indeed. This is a restatement of a part of 5470, intended mainly to =
provide background for why anonymisation support is crucial for IPFIX. =
However, the diagram is as you point out kind of confusing here. Its =
primary function is to frame the work within the larger, longer work of =
the WG.. which we do in the paragraph below anyway. So we're inclined to =
cut it.

>>                  Figure 1: IPFIX Device functional blocks
>>  =20
>>=20
> I was hoping to get the reference model from the IPFIX Mediator =
framework.
> If the reference model was from the IPFIX Mediator framework, then we =
would not have the issue with optional.
>>    Note that, according to the original architecture specification,
>>  =20
>>=20
> What is this? RFC5470?

Yes. This reference should be here (instead of "original architecture =
specification").

>>    IPFIX Message anonymisation is optionally performed as the final
>>    operation before handing the Message to the transport protocol for
>>    export.  While no provision is made in the architecture for
>>    anonymisation metadata as in Section 6, this arrangement does =
allow
>>    for the message rewriting necessary for comprehensive =
anonymisation
>>    of IPFIX export as in Section 7.  The development of the IPFIX
>>    Mediation [I-D.ietf-ipfix-mediators-framework] framework and the
>>    IPFIX File Format [RFC5655] expand upon this initial architectural
>>    allowance for anonymisation by adding to the list of places that
>>    anonymisation may be applied.  The former specifies IPFIX =
Mediators,
>>    which rewrite existing IPFIX messages, and the latter specifies a
>>  =20
>>=20
> IPFIX Message?
> It would be nice to say in the terminology section which terms you use =
from other RFCs.
> For example, data set vs dataset versus Data Set

This is an unfortunate consequence of IPFIX having something called a =
"Data Set". We need a term within this WG that means "some data" but =
that does not conflict with the defined terminology. Suggestions?

> For example, flow records is lower case, but I see IPFIX File, Options =
Template
> Please be consistent with capitalization

Will review. Indeed, here "messages" should be "Messages", however, this =
sentence may need to be rewritten anyway, because IPFIX Mediators don't =
rewrite Messages (in the general case) as defined.

>>    method for storage of IPFIX data in files.
>>=20
>>    More detail on the applicable architectural arrangements of
>>    anonymisation can be found in Section 7.1
>>=20
>>=20
>> 2.  Terminology
>>=20
>>=20
>>    Terms used in this document that are defined in the Terminology
>>    section of the IPFIX Protocol [RFC5101] document are to be
>>    interpreted as defined there.
>>=20
>>    The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL =
NOT",
>>    "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in =
this
>>    document are to be interpreted as described in RFC 2119 [RFC2119].
>>=20
>>=20
>> 3.  Categorisation of Anonymisation Techniques
>>=20
>>=20
>>    Anonymisation modifies a data set in order to protect the identity =
of

Here is a perfect example of the point above. This is "some data", not =
an IPFIX Data Set. Can we agree that "data set" is not "Data Set", or =
should we invent some more terminology here?

>>    the people or entities described by the data set from disclosure.
>>    With respect to network traffic data, anonymisation generally
>>    attempts to preserve some set of properties of the network traffic
>>    useful for a given application or applications, while ensuring the
>>    data cannot be traced back to the specific networks, hosts, or =
users
>>    generating the traffic.



>> 4.  Anonymisation of IP Flow Data



>>    Hardware addresses uniquely identify devices on the network; while
>>    they are not often available in traffic data collected at Layer 3,
>>    and cannot be used to locate devices within the network, some =
traces
>>    may contain sub-IP data including hardware address data.  Hardware
>>    addresses may be mappable to device serial numbers, and to the
>>    entities or individuals who purchased the devices, when combined =
with
>>    external databases.  They may also leak via IPv6 addresses in =
certain
>>    circumstances.  Therefore, hardware address anonymisation is also
>>    important.
>>  =20
>>=20
> I would clearly mention MAC address

Okay... will s/hardware address/MAC address/gi then.

>> 4.1.1.  Truncation
>>=20
>>=20
>>    Truncation removes "n" of the least significant bits from an IP
>>    address, replacing them with zeroes.  In effect, it replaces a =
host
>>    address with a network address for some fixed netblock; for IPv4
>>    addresses, 8-bit truncation corresponds to replacement with a /24
>>    network address.  Truncation is a non-reversible generalisation
>>    scheme.  Note that while truncation is effective for making hosts
>>    non-identifiable, it preserves information which can be used to
>>    identify an organization, a geographic region, a country, or a
>>    continent (or RIR region of responsibility).
>>  =20
>>=20
> what is RIR?

Regional Internet Registry [RFC1466], [RFC2050]. Shall we add these as =
informatives, or drop the reference to RIRs?



>> 4.3.  Timestamp Anonymisation
>>=20
>>=20
>>    The particular time at which a flow began or ended is not
>>    particularly identifiable information, but it can be used as part =
of
>>    attacks against other anonymisation techniques or for user =
profiling.
>>    Presice timestamps can be used in injected-traffic fingerprinting
>>  =20
>>=20
> precise

Oops. An embarrassing word to typo. :)

> Yo might want to describe what fingerprinting is

Will do.

>>    attacks as well as to identify certain activity by response delay =
and
>>    size fingerprinting.  Therefore, timestamp information may be
>>    anonymised in order to ensure the protection of the entire =
dataset.
>>=20
>>           +-----------------------+----------------------------+
>>           | Scheme                | Action                     |
>>           +-----------------------+----------------------------+
>>           | Precision Degradation | Generalisation             |
>>           | Enumeration           | Direct or Set Substitution |
>>           | Random Shifts         | Direct Substitution        |
>>           +-----------------------+----------------------------+
>>=20
>>=20
>> 4.3.1.  Precision Degradation
>>=20
>>  =20
>>=20
> it's like truncation, but specific to time and counter, right?

Truncation is for identifiers, precision degradation is for numbers, but =
yes. This is why they have the same anon technique number in metadata =
export.



>> 5.  Parameters for the Description of Anonymisation Techniques
>>=20
>>=20
>>    This section details the abstract parameters used to describe the
>>=20
>>=20
>> Boschi & Trammell        Expires August 19, 2010               [Page =
16]
>> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>>=20
>>=20
>>    anonymisation techniques examined in the previous section, on a =
per-
>>    parameter basis.  These parameters and their export safety inform =
the
>>    design of the IPFIX anonymisation metadata export specified in the
>>    following section.
>>  =20
>>=20
> you use different terms for metadata
> IPFIX anonymisation metadata export
> anonymisation technique metadata export
> anonymisation metadata

will normalize these

> metadata about exported flows and the flow collection infrastructure

but this one means something different (anon metadata =3D "what are the =
properties of the anonymisation applied to this data?", while this is =
"anything in an options record (that might be useful to =
deanonymisation)").

> You might want to be consistent

Perhaps, yes. ;) Will fix these, definitely.

>> 5.1.  Stability
>>=20
>>=20
>>    Any given anonymisation technique may be applied with a varying =
range
>>    of stability.  Stability is important for assessing the =
comparability
>>    of anonymised information in different data sets, or in the same =
data
>>    set over different time periods.  In general, stability ranges =
from
>>    completely stable to completely unstable; however, note that the
>>    completely unstable case is indistinguishable from black-marker
>>    anonymisation.  A completely stable anonymisation will always map =
a
>>    given value in the real space to the same value in the anonymised
>>    space.  In practice, an anonymisation may also be stable for every
>>    data set published by an a particular producer to a particular
>>    consumer, stable for a stated time period within a dataset or =
across
>>    datasets, or stable only for a single data set.
>>=20
>>    If no information about stability is available, users of =
anonymised
>>    data may assume that the techniques used are stable across the =
entire
>>    dataset, but unstable across datasets.  Note that stability =
presents
>>    a risk-utility tradeoff, as completely stable anonymisation can be
>>    used for longer-term trend analysis tasks but also presents more =
risk
>>    of attack given the stable mapping.
>>  =20
>>=20
> In this section, you don't say if you must export the content of this =
section, while you do for all the other 5.x sections

Good point. Will mention this (i.e., that we SHOULD).



>> 6.  Anonymisation Export Support in IPFIX
>>=20
>>=20
>>    Anonymised data exported via IPFIX SHOULD be annotated with
>>    anonymisation metadata, which details which fields described by =
which
>>    Templates are anonymised, and provides appropriate information on =
the
>>    anonymisation techniques used.  This metadata SHOULD be exported =
in
>>    Data Records described by the recommended Options Templates =
described
>>    in this section; these Options Templates use the additional
>>    Information Elements described in the following subsection.
>>=20
>>    Note that fields anonymised using the black-marker (removal)
>>    technique do not require any special metadata support.  =
Black-marker
>>    anonymised fields SHOULD NOT be exported at all; the absence of =
the
>>  =20
>>=20
> MUST NOT?

A MUST NOT here would conflict with the parallel guidance in RFC 5103 =
(section 4, para 6, page 7) regarding empty reverse IEs (SHOULD not), =
and in the configuration draft, which allows fields which can be =
unambiguously null in context to be exported as null (MAY). What about =
anonymisation in an environment with hardcoded templates? No reason to =
take freedom away from implementors in this situation.=20

>>    field in a given Data Set is implicitly declared by not including =
the
>>    corresponding Information Element in the Template describing that
>>    Data Set.
>>=20
>>=20
>> 6.1.  Anonymisation Options Template
>>=20
>>=20
>>    The Anonymisation Options Template describes anonymisation =
records,
>>    which allow anonymisation metadata to be exported inline over =
IPFIX
>>    or stored in an IPFIX File, by binding information about
>>    anonymisation techniques to Information Elements within defined
>>    Templates.  IPFIX Exporting Processes SHOULD export anonymisation
>>    records for any Template describing exported anonymised Data =
Records;
>>  =20
>>=20
> an anonymization record is actually a data record of the Anonymization =
Options Template Record, as opposed to an anonymized record.
> It would be better to define those terms, no

Yep. So "Anonymisation Record" and "Anonymised Data Record" will go into =
terminology then.

>=20
>>    IPFIX Collecting Processes and processes downstream from them MAY =
use
>>    anonymisation records to treat anonymised data differently =
depending
>>    on the applied technique.
>>=20
>>=20
>> Boschi & Trammell        Expires August 19, 2010               [Page =
18]
>> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>>=20
>>=20
>>    An Exporting Process SHOULD export anonymisation records after the
>>    Templates they describe have been exported,
>>=20
> And MUST send the anonymization record before the anomized records?
>=20
> Do you want to benefit from the per-stream SCTP draft.
> That would solve most of your problems: order, reliable
> Otherwise, if the collector doesn't have received the anonimization =
records, then it might be treating the flow records wrongly.

Yeah, but only if per-stream is anyway compatible with the underlying =
application. Worth a mention and an informative reference, probably.

>>  and SHOULD export
>>    anonymisation records reliably.
>>=20
>>    Anonymisation records, like Templates, MUST be handled by =
Collecting
>>    Processes as scoped to the Transport Session in which they are =
sent.
>>    While the Stability Class within the anonymisationFlags IE can be
>>    used to declare that a given anonymisation technique's mapping =
will
>>    remain stable across multiple sessions, each session MUST =
re-export
>>    the anonymisation Records along with the templates.
>>=20
>>    =
+-------------------------+-----------------------------------------+
>>    | IE                      | Description                            =
 |
>>    =
+-------------------------+-----------------------------------------+
>>    | templateId [scope]      | The Template ID of the Template        =
 |
>>    |                         | containing the Information Element     =
 |
>>    |                         | described by this anonymisation =
record. |
>>    |                         | This Information Element MUST be       =
 |
>>    |                         | defined as a Scope Field.              =
 |
>>    | informationElementId    | The Information Element identifier of  =
 |
>>    | [scope]                 | the Information Element described by   =
 |
>>    |                         | this anonymisation record.  This       =
 |
>>    |                         | Information Element MUST be defined as =
 |
>>    |                         | a Scope Field.                         =
 |
>>    | informationElementId    | The Private Enterprise Number of the   =
 |
>>    | [scope] [optional]      | enterprise-specific Information =
Element |
>>    |                         | described by this anonymisation =
record. |
>>    |                         | This Information Element MUST be       =
 |
>>    |                         | defined as a Scope Field if present.   =
 |
>>  =20
>>=20
> Isn't it privateEnterpriseNumber instead of informationElementId?

Yep, typo. Will fix.

> Actually, the example and the description mention =
privateEnterpriseNumber
> However, I'm wondering why you need this privateEnterpriseNumber at =
all,  because the "E" bit is already part of informationElementId=20
>  0                   1                   2                   3
>    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>    |E|  Information Element ident. |        Field Length           |
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>    |                      Enterprise Number                        |
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>=20
> I'm confused.

The enterprise bit only says that it's got a PEN, not what the PEN is, =
so we still need it.

>>    | informationElementIndex | The Information Element index of the   =
 |
>>    | [scope] [optional]      | instance of the Information Element    =
 |
>>    |                         | described by this anonymisation record =
 |
>>    |                         | identified by the informationElementId =
 |
>>    |                         | within the Template.  Optional; need   =
 |
>>    |                         | only be present when describing        =
 |
>>    |                         | Templates that have multiple instances =
 |
>>    |                         | of the same Information Element.  This =
 |
>>    |                         | Information Element MUST be defined as =
 |
>>    |                         | a Scope Field if present.  This        =
 |
>>    |                         | Information Element is defined in      =
 |
>>    |                         | Section 6.2, below.                    =
 |
>>    | anonymisationFlags      | Flags describing the mapping stability =
 |
>>    |                         | and specialized modifications to the   =
 |
>>    |                         | Anonymisation Technique in use.  =
SHOULD |
>>    |                         | be present.  This Information Element  =
 |
>>    |                         | is defined in Section 6.2, below.      =
 |
>>=20
>>=20
>> Boschi & Trammell        Expires August 19, 2010               [Page =
19]
>> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>>=20
>>=20
>>    | anonymisationTechnique  | The technique used to anonymise the    =
 |
>>    |                         | data.  MUST be present.  This          =
 |
>>    |                         | Information Element is defined in      =
 |
>>    |                         | Section 6.2, below.                    =
 |
>>    =
+-------------------------+-----------------------------------------+



>> 6.2.2.  anonymisationFlags
>>=20
>>=20
>>    Description:   A flag word describing specialized modifications to
>>       the anonymisation policy in effect for the anonymisation =
technique
>>       applied to a referenced Information Element within a referenced
>>       Template.  When flags are clear (0), the normal policy (as
>>       described by anonymisationTechnique) applies without =
modification.
>>=20
>>       MSB   14  13  12  11  10   9   8   7   6   5   4   3   2   1  =
LSB
>>       =
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
>>       |                Reserved                       |LOR|PmA|   SC  =
|
>>       =
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
>>=20
>>                             anonymisationFlags IE
>>=20
>>    =
+--------+----------+-----------------------------------------------+
>>    | bit(s) | name     | description                                  =
 |
>>    | (LSB =3D |          |                                            =
   |
>>    | 0)     |          |                                              =
 |
>>    =
+--------+----------+-----------------------------------------------+
>>    | 0-1    | SC       | Stability Class: see the Stability Class     =
 |
>>    |        |          | table below, and section Section 5.1.        =
 |
>>=20
>>=20
>> Boschi & Trammell        Expires August 19, 2010               [Page =
20]
>> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>>=20
>>=20
>>    | 2      | PmA      | Perimeter Anonymisation: when set (1), =
source |
>>  =20
>>=20
> source based on what? I remember the lengthy discussion in biflow.

See section 7.2.2. If we still need to clarify here, we can.

>>    |        |          | address Information Elements are interpreted =
 |
>>    |        |          | as external addresses, and destination       =
 |
>>    |        |          | address Information Elements are interpreted =
 |
>>    |        |          | as internal addresses, for the purposes of   =
 |
>>    |        |          | associating anonymisationTechnique to        =
 |
>>    |        |          | Information Elements.  MUST NOT be set when  =
 |
>>    |        |          | associated with a non-endpoint (i.e., =
source- |
>>    |        |          | or destination-) Information Element.  =
SHOULD |
>>    |        |          | be consistent within a record (i.e., if a    =
 |
>>    |        |          | source- Information Element has this flag    =
 |
>>    |        |          | set, the corresponding destination- element  =
 |
>>    |        |          | SHOULD have this flag set, and vice-versa.)  =
 |
>>    | 3      | LOR      | Low-Order Unchanged: when set (1), the       =
 |
>>    |        |          | low-order bits of the anonymised Information =
 |
>>    |        |          | Element contain real data.  This =
modification |
>>    |        |          | is intended for the anonymisation of         =
 |
>>    |        |          | network-level addresses while leaving        =
 |
>>    |        |          | host-level addresses intact in order to      =
 |
>>    |        |          | preserve host level-structure, which could   =
 |
>>    |        |          | otherwise be used to reverse anonymisation.  =
 |
>>    |        |          | MUST NOT be set when associated with a       =
 |
>>    |        |          | truncation-based anonymisationTechnique.     =
 |
>>    | 4-15   | Reserved | Reserved for future use: SHOULD be cleared   =
 |
>>    |        |          | (0) by the Exporting Process and MUST be     =
 |
>>    |        |          | ignored by the Collecting Process.           =
 |
>>    =
+--------+----------+-----------------------------------------------+



>> 6.2.3.  anonymisationTechnique
>>=20
>>  =20
>>=20
> While reading, it looks more logical to have the =
anonymisationTechnique before the anonymizationFlags IE

For IEs, the sections in the document (as in 5610, 5655) are =
alphabetized by IE name. However, you're right, it makes more sense to =
do them in "logical" order when there aren't so many of them.

>>    Description:   A description of the anonymisation technique =
applied
>>       to a referenced Information Element within a referenced =
Template.
>>  =20
>>=20
> And options template as well? For example, in my company, we export a =
options template record with the sampling rates.
> This is something I missed completely, and is valid for all IEs

Absolutely. Will add.

>>       Each technique may be applicable only to certain Information
>>       Elements and recommended only for certain Infomation Elements;
>>       these restrictions are noted in the table below.



>> 7.1.  Arrangement of Processes in IPFIX Anonymisation



>>    When data is to be published as an anonymised data set in an IPFIX
>>    File [RFC5655], the anonymisation may be done at the final =
Collecting
>>    Process before storage and dissemination, as well.=20
>>=20
> For me, it's a different problem. IPFIX File could be done on routers.
> Basically, this paragraph is about the relationship with IPFIX File.

Mainly because this section is about the arrangement of processes, and a =
File Writer is something=20

> Btw, we're alos missing the relationship with reducing redundancy

I'm not sure I get what's special about reducing redundancy with respect =
to anon.

> with IPFIX structured data
> For the later, does it apply to all instances of a template? So, some =
kind of recursivity is implied

This is probably something that should be addressed in Structured Data =
(the only hard question here is how meta-template information is handled =
with basicList) as opposed to the other way around, because of the =
chronology, and because anon is not the only meta-template we may have =
to address. In general we should avoid normative references that go =
forward in time...

Since this is metatemplate information, it should be stored by the CP as =
if it belongs to the template. So, yes, it applies to any instance of an =
template active at the time. This implies that we should point out =
somewhere that this information expires when the template is withdrawn =
or expires (ouch!).

>>  In this case, the
>>    Collector should follow the guidelines in Section 7.2, especially =
as
>>    regards File-specific Options in Section 7.2.4
>>=20
>>    In each of these data flows, the anonymisation of records is
>>    undertaken by an Intermediate Anonymisation Process (IAP); the =
data
>>    flows into and out of this IAP are shown in Figure 3 below.
>>=20
>>    packets --+                     +- IPFIX Messages -+
>>              |                     |                  |
>>              V                     V                  V
>>    +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ =
+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ =
+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+
>>    | Metering Process | | Collecting Process | | File Reader |
>>    +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ =
+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ =
+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+
>>              |      Non-anonymised | Records          |
>>              V                     V                  V
>>    +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D+
>>    |          Intermediate Anonymisation Process (IAP)       |
>>    +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D+
>>              | Anonymised     ^            Anonymised |
>>              | Records        |               Records |
>>              V                |                       V
>>    +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+    =
Anonymisation      +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+
>>    | Exporting Process |<--- Parameters ------>| File Writer |
>>    +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+        =
               +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+
>>              |                                        |
>>              +------------> IPFIX Messages <----------+
>>=20
>>           Figure 3: Data flows through the anonymisation process
>>  =20
>>=20
> This is exactly what I was hoping to see in figure1, i.e. this draft =
applying to the Intermediate Process in [IFPIX-MD-FMWK]
> ... where the IP can be an independent boxe, or inside the router.
> I completely missed that second part in the draft.

Another reason simply to drop figure 1.

>>    Anonymisation parameters must also be available to the Exporting
>>    Process and/or File Writer in order to ensure header data is also
>>    appropriately anonymised as in Section 7.2.3.
>>=20
>>    Following each of the data flows through the IAP, we describe five
>>    basic types of anonymisation arrangements within this framework in
>>    Figure 4.  In addition to the three arrangements described in =
detail
>>    above, anonymisation can also be done at a collocated Metering
>>    Process and File Writer (see section 7.3.2 of [RFC5655]), or at a
>>    file manipulator (see section 7.3.7 of [RFC5655]).



>> 7.2.3.  Anonymisation of Header Data
>>=20
>>=20
>>    Each IPFIX Message contains a Message Header; within this Message
>>    Header are contained two fields which may be used to break certain
>>    anonymisation techniques: the Export Time, and the Observation =
Domain
>>=20
>>=20
>> Boschi & Trammell        Expires August 19, 2010               [Page =
28]
>> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>>=20
>>=20
>>    ID
>>=20
>>    Export of IPFIX Messages containing anonymised timestamp data =
where
>>    the original Export Time Message header has some relationship to =
the
>>    anonymised timestamps SHOULD anonymise the Export Time header =
field
>>    using an equivalent technique, if possible.  Otherwise, =
relationships
>>    between export and flow time could be used to partially or totally
>>    reverse timestamp anonymisation.
>>  =20
>>=20
> What if the time is in the future, the collector would probably =
discard the information.

Why? This seems highly implementation dependent.

> You might want to say which methods of 4.3.* applies.=20

Good point.


>>=20
>> 7.2.4.  Anonymisation of Options Data



>>    Since the Time Window Options Template specified for the IPFIX =
File
>>    Format [RFC5655] refers to the timestamps within the flow data to
>>    provide partial table of contents information for an IPFIX File, =
care
>>    must be taken to ensure that Options described by this template =
are
>>    written using the anonymised timestamps instead of the original =
ones.
>>  =20
>>=20
> Is there something in the IPFIX-MIB or IPFIX-CONF that can be used to =
deduce something about the anonymization technique?

Not that would be exported, no. If the device supports configuration =
extensions for anonymisation, clearly you shouldn't export that, but =
it's also out of band, so not really applicable here.



>> 8.  Examples
>>=20
>>  =20
>>=20
> A nice to have would be the flow records before and after the =
anonymization.

Yep. A fair amount of extra work, but, yep, completely agree.=20



>>                             1                   2                   3
>>         0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 =
1
>>        =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>>        |          Set ID =3D 257         |          Length =3D  68    =
     |
>>        =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>>        |          Template 256         | flowStartSeconds       IE =
150 |
>>        =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>>        | no flags               0x0000 | Not Anonymised              =
1 |
>>        =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>>  =20
>>=20
> Maybe I missed it, but is there a specification in this document that =
expresses: MUST have a record for each IE of a template, even if no =
anonymization is applied.
> Is there a specification that expresses: MUST have a record for each =
IE of each template, even if no IE of a particular template is =
anonymized.

Nope. If an IE in a template isn't specified, it's "default" which is =
"maybe not anonymized, but we don't really know." We'll clarify this =
above.



From sujay.ietf@gmail.com  Fri Apr  9 07:31:07 2010
Return-Path: 
X-Original-To: ipfix@core3.amsl.com
Delivered-To: ipfix@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 46F013A68D5 for ; Fri,  9 Apr 2010 07:31:07 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.599
X-Spam-Level: 
X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YClxsB9Jbkcg for ; Fri,  9 Apr 2010 07:31:04 -0700 (PDT)
Received: from mail-gy0-f172.google.com (mail-gy0-f172.google.com [209.85.160.172]) by core3.amsl.com (Postfix) with ESMTP id B24B03A6358 for ; Fri,  9 Apr 2010 07:31:04 -0700 (PDT)
Received: by gyh4 with SMTP id 4so1833915gyh.31 for ; Fri, 09 Apr 2010 07:30:58 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:received:message-id :subject:from:to:cc:content-type; bh=DYheBCpdo6jWX3t96I2VPV7TAwz/+o7pvTyXNBG0PRo=; b=HUUOZU1nkZYfsY43qd5XF+WehYlxam6gf0PO2TO2P0Jcrp+H2b+cFbUyQifK5bTUqo sls03IUAU3SDcH26HbKOlA8PmzOpphJ+w0MEpI+JS+B4JG6BDwiMdSsacNyLlKej3Jy8 BZg9aY0p1SVn68r/65g5mgDbd0TMI3ebZZ1VM=
DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:cc:content-type; b=QeiDp8w0DIuGZv+vH+8XGwsFthkjCC7OyaiQTKVn1t3fg4Q+7bRt9GIDUW/v9jhuq9 qRHbQx8sebNHiAWwyddwvIhiBs9mxNDWt2rdTm9iQ5ee9am+5mz+uFE9TvF2+0a1wuEh pvHCeDRL6zWUahYuO+ABCHCMI5og5DmhEWhFs=
MIME-Version: 1.0
Received: by 10.90.89.16 with HTTP; Fri, 9 Apr 2010 07:30:57 -0700 (PDT)
Date: Fri, 9 Apr 2010 10:30:57 -0400
Received: by 10.90.61.32 with SMTP id j32mr94555aga.25.1270823457751; Fri, 09  Apr 2010 07:30:57 -0700 (PDT)
Message-ID: 
From: sujay gupta 
To: IETF IPFIX Working Group 
Content-Type: text/plain; charset=ISO-8859-1
Cc: Nevil Brownlee , Juergen Quittek 
Subject: [IPFIX]  Follow-up on Reliability Draft
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Fri, 09 Apr 2010 14:31:07 -0000

Group,

This is a follow up on the IETF77 meet, where the following draft was
presented & discussed;

http://tools.ietf.org/html/draft-gupta-ipfix-reliability-template-ext-00

The draft proposes of sending device state as an IE added to the
Optional Reliability Template.

This information is clearly required in practical scenarios.

A suggestion is to keep this IE in a new "Device State & Information
Optional Template", this template
can be used to carry further device specific information ( like vendor
strings, id's etc.) later.

Any suggestions/comments?

Thanks,
-Sujay

From bclaise@cisco.com  Mon Apr 12 08:15:09 2010
Return-Path: 
X-Original-To: ipfix@core3.amsl.com
Delivered-To: ipfix@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id F1FAD3A6A4D for ; Mon, 12 Apr 2010 08:15:09 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.649
X-Spam-Level: 
X-Spam-Status: No, score=-0.649 tagged_above=-999 required=5 tests=[AWL=-0.650, BAYES_50=0.001]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id A7Jd5KZ2guO0 for ; Mon, 12 Apr 2010 08:15:08 -0700 (PDT)
Received: from av-tac-bru.cisco.com (odd-brew.cisco.com [144.254.15.119]) by core3.amsl.com (Postfix) with ESMTP id 48AD03A6A5B for ; Mon, 12 Apr 2010 08:13:07 -0700 (PDT)
X-TACSUNS: Virus Scanned
Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id o3CFD0oJ007945; Mon, 12 Apr 2010 17:13:00 +0200 (CEST)
Received: from [10.55.43.59] (ams-bclaise-87110.cisco.com [10.55.43.59]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id o3CFD0Gp007929; Mon, 12 Apr 2010 17:13:00 +0200 (CEST)
Message-ID: <4BC3387B.3010807@cisco.com>
Date: Mon, 12 Apr 2010 17:12:59 +0200
From: Benoit Claise 
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3
MIME-Version: 1.0
To: Brian Trammell 
References: <4BB5C86D.7020107@cisco.com> <4BB5D827.1060406@cisco.com> <643E6408-8BDF-48F5-888D-EA95C812730E@tik.ee.ethz.ch>
In-Reply-To: <643E6408-8BDF-48F5-888D-EA95C812730E@tik.ee.ethz.ch>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Cc: "ipfix@ietf.org" 
Subject: Re: [IPFIX] draft-ietf-ipfix-anon-02.txt: WG last call review
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Mon, 12 Apr 2010 15:15:10 -0000

Hi Brian,
>
>    
>>>     IPFIX Message anonymisation is optionally performed as the final
>>>     operation before handing the Message to the transport protocol for
>>>     export.  While no provision is made in the architecture for
>>>     anonymisation metadata as in Section 6, this arrangement does allow
>>>     for the message rewriting necessary for comprehensive anonymisation
>>>     of IPFIX export as in Section 7.  The development of the IPFIX
>>>     Mediation [I-D.ietf-ipfix-mediators-framework] framework and the
>>>     IPFIX File Format [RFC5655] expand upon this initial architectural
>>>     allowance for anonymisation by adding to the list of places that
>>>     anonymisation may be applied.  The former specifies IPFIX Mediators,
>>>     which rewrite existing IPFIX messages, and the latter specifies a
>>>
>>>
>>>        
>> IPFIX Message?
>> It would be nice to say in the terminology section which terms you use from other RFCs.
>> For example, data set vs dataset versus Data Set
>>      
> This is an unfortunate consequence of IPFIX having something called a "Data Set". We need a term within this WG that means "some data" but that does not conflict with the defined terminology. Suggestions?
>    
If you have a terminology section, including Data Set, that would solve 
it. It would be clear what data set would (not) be.
>    
>>>     method for storage of IPFIX data in files.
>>>
>>>     More detail on the applicable architectural arrangements of
>>>     anonymisation can be found in Section 7.1
>>>
>>>
>>> 2.  Terminology
>>>
>>>
>>>     Terms used in this document that are defined in the Terminology
>>>     section of the IPFIX Protocol [RFC5101] document are to be
>>>     interpreted as defined there.
>>>
>>>     The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
>>>     "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
>>>     document are to be interpreted as described in RFC 2119 [RFC2119].
>>>
>>>
>>> 3.  Categorisation of Anonymisation Techniques
>>>
>>>
>>>     Anonymisation modifies a data set in order to protect the identity of
>>>        
> Here is a perfect example of the point above. This is "some data", not an IPFIX Data Set. Can we agree that "data set" is not "Data Set", or should we invent some more terminology here?
>    
See my point above.
>
>    
>>> 6.  Anonymisation Export Support in IPFIX
>>>
>>>
>>>     Anonymised data exported via IPFIX SHOULD be annotated with
>>>     anonymisation metadata, which details which fields described by which
>>>     Templates are anonymised, and provides appropriate information on the
>>>     anonymisation techniques used.  This metadata SHOULD be exported in
>>>     Data Records described by the recommended Options Templates described
>>>     in this section; these Options Templates use the additional
>>>     Information Elements described in the following subsection.
>>>
>>>     Note that fields anonymised using the black-marker (removal)
>>>     technique do not require any special metadata support.  Black-marker
>>>     anonymised fields SHOULD NOT be exported at all; the absence of the
>>>
>>>
>>>        
>> MUST NOT?
>>      
> A MUST NOT here would conflict with the parallel guidance in RFC 5103 (section 4, para 6, page 7) regarding empty reverse IEs (SHOULD not), and in the configuration draft, which allows fields which can be unambiguously null in context to be exported as null (MAY). What about anonymisation in an environment with hardcoded templates? No reason to take freedom away from implementors in this situation.
>    

I'm unclear how black-marker fields should be exported: with zero-length 
(bad idea, because 0 might mean something else) or with zero (in that 
case, we lose some bandwidth for all Flow Records)?
Also I was wondering if we could have an Anonymization Options Template 
for a field that does not appear in the Template Record?
Maybe clarify the points first
Whatever you specify, this must be inline with RFC5101:

    If a specific Information Element is required by a Template, but is
    not available in observed packets, the Exporting Process MAY choose
    to export Flow Records without this Information Element in a Data
    Record defined by a new Template.




>
>
>    
>
>    
>> Actually, the example and the description mention privateEnterpriseNumber
>> However, I'm wondering why you need this privateEnterpriseNumber at all,  because the "E" bit is already part of informationElementId
>>   0                   1                   2                   3
>>     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
>>     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>>     |E|  Information Element ident. |        Field Length           |
>>     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>>     |                      Enterprise Number                        |
>>     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>>
>> I'm confused.
>>      
> The enterprise bit only says that it's got a PEN, not what the PEN is, so we still need it.
>    

I was thinking that informationElementId would contain the E bit. 
Checking with RFC5610 and IPFIX IANA, this is not the case.
This should be clarified.
Also expresses the two cases where this is needed.
1. two different PENs with the same informationElementId
2. One PEN and one non PEN with the same informationElementId

>    
>>>     | informationElementIndex | The Information Element index of the    |
>>>     | [scope] [optional]      | instance of the Information Element     |
>>>     |                         | described by this anonymisation record  |
>>>     |                         | identified by the informationElementId  |
>>>     |                         | within the Template.  Optional; need    |
>>>     |                         | only be present when describing         |
>>>     |                         | Templates that have multiple instances  |
>>>     |                         | of the same Information Element.  This  |
>>>     |                         | Information Element MUST be defined as  |
>>>     |                         | a Scope Field if present.  This         |
>>>     |                         | Information Element is defined in       |
>>>     |                         | Section 6.2, below.                     |
>>>     | anonymisationFlags      | Flags describing the mapping stability  |
>>>     |                         | and specialized modifications to the    |
>>>     |                         | Anonymisation Technique in use.  SHOULD |
>>>     |                         | be present.  This Information Element   |
>>>     |                         | is defined in Section 6.2, below.       |
>>>
>>>
>>> Boschi&  Trammell        Expires August 19, 2010               [Page 19]
>>> Internet-Draft        IP Flow Anonymisation Support        February 2010
>>>
>>>
>>>     | anonymisationTechnique  | The technique used to anonymise the     |
>>>     |                         | data.  MUST be present.  This           |
>>>     |                         | Information Element is defined in       |
>>>     |                         | Section 6.2, below.                     |
>>>     +-------------------------+-----------------------------------------+
>>>        
>
>
>>>     When data is to be published as an anonymised data set in an IPFIX
>>>     File [RFC5655], the anonymisation may be done at the final Collecting
>>>     Process before storage and dissemination, as well.
>>>
>>>        
>> For me, it's a different problem. IPFIX File could be done on routers.
>> Basically, this paragraph is about the relationship with IPFIX File.
>>      
> Mainly because this section is about the arrangement of processes, and a File Writer is something
>
>    
>> Btw, we're alos missing the relationship with reducing redundancy
>>      
> I'm not sure I get what's special about reducing redundancy with respect to anon.
>    

>    
>> with IPFIX structured data
>> For the later, does it apply to all instances of a template? So, some kind of recursivity is implied
>>      
> This is probably something that should be addressed in Structured Data (the only hard question here is how meta-template information is handled with basicList) as opposed to the other way around, because of the chronology, and because anon is not the only meta-template we may have to address. In general we should avoid normative references that go forward in time...
>    
Agreed on the last point.
> Since this is metatemplate information, it should be stored by the CP as if it belongs to the template. So, yes, it applies to any instance of an template active at the time. This implies that we should point out somewhere that this information expires when the template is withdrawn or expires (ouch!).
>    
Yes!
Next to the expiration and scope conditions, you might have one sentence 
such as: once the Anonymization Options Template Record is specified, 
it's valid for all instances of Flow Records conforming to this definition.
This would take care of IPFIX-RED-RED and IPFIX-STRUCT
I'm sure you will find some better wording, but you get my point.
>    
>>> 7.2.3.  Anonymisation of Header Data
>>>
>>>
>>>     Each IPFIX Message contains a Message Header; within this Message
>>>     Header are contained two fields which may be used to break certain
>>>     anonymisation techniques: the Export Time, and the Observation Domain
>>>
>>>
>>> Boschi&  Trammell        Expires August 19, 2010               [Page 28]
>>> Internet-Draft        IP Flow Anonymisation Support        February 2010
>>>
>>>
>>>     ID
>>>
>>>     Export of IPFIX Messages containing anonymised timestamp data where
>>>     the original Export Time Message header has some relationship to the
>>>     anonymised timestamps SHOULD anonymise the Export Time header field
>>>     using an equivalent technique, if possible.  Otherwise, relationships
>>>     between export and flow time could be used to partially or totally
>>>     reverse timestamp anonymisation.
>>>
>>>
>>>        
>> What if the time is in the future, the collector would probably discard the information.
>>      
> Why? This seems highly implementation dependent.
>    
Sure, but it was never specified in RFC5101.
Maybe add two sentences such as:
     exporter: SHOULD NOT send in future times, due to anyonymation 
techniques
     collector, MAY accept future times in connection with Anonymization 
Options Templates , but MUST log a warning.
>
>    
>>>     Since the Time Window Options Template specified for the IPFIX File
>>>     Format [RFC5655] refers to the timestamps within the flow data to
>>>     provide partial table of contents information for an IPFIX File, care
>>>     must be taken to ensure that Options described by this template are
>>>     written using the anonymised timestamps instead of the original ones.
>>>
>>>
>>>        
>> Is there something in the IPFIX-MIB or IPFIX-CONF that can be used to deduce something about the anonymization technique?
>>      
> Not that would be exported, no. If the device supports configuration extensions for anonymisation, clearly you shouldn't export that, but it's also out of band, so not really applicable here.
>    

If I'm an attacker, wanting to decipher your anonymization technique, 
everything will help.
If you do support the IPFIX-MIB, PSAMP-MIB, and you use SNMPv1 or v2c, I 
can sniff the community string, and get all information in the MIBs. 
Will it help me? Maybe I'm not sure!
If I can do the same with the configuration, I might help in some cases. 
For example, in case of a permanent cache exported every hour UTC, a 
simple correlation of the time in the anoymized flow records with the 
config time will get me what I need.
So maybe a few sentences would clarify what to pay attention to.

Regards, Benoit.


From trammell@tik.ee.ethz.ch  Wed Apr 14 00:47:18 2010
Return-Path: 
X-Original-To: ipfix@core3.amsl.com
Delivered-To: ipfix@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 053653A689D for ; Wed, 14 Apr 2010 00:47:18 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.999
X-Spam-Level: 
X-Spam-Status: No, score=-3.999 tagged_above=-999 required=5 tests=[BAYES_50=0.001, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ncLkr0NeKJY9 for ; Wed, 14 Apr 2010 00:47:16 -0700 (PDT)
Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by core3.amsl.com (Postfix) with ESMTP id 4992A3A6863 for ; Wed, 14 Apr 2010 00:47:15 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 60831D9345; Wed, 14 Apr 2010 09:47:08 +0200 (MEST)
X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch
Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id kA+JPuwywr2B; Wed, 14 Apr 2010 09:47:08 +0200 (MEST)
Received: from [192.168.0.195] (77-57-164-35.dclient.hispeed.ch [77.57.164.35]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id BFCBED9324; Wed, 14 Apr 2010 09:47:07 +0200 (MEST)
Mime-Version: 1.0 (Apple Message framework v1078)
Content-Type: text/plain; charset=us-ascii
From: Brian Trammell 
In-Reply-To: <4BC3387B.3010807@cisco.com>
Date: Wed, 14 Apr 2010 09:47:07 +0200
Content-Transfer-Encoding: quoted-printable
Message-Id: <18F76C40-E0FD-486E-8189-913CC323FB6C@tik.ee.ethz.ch>
References: <4BB5C86D.7020107@cisco.com> <4BB5D827.1060406@cisco.com> <643E6408-8BDF-48F5-888D-EA95C812730E@tik.ee.ethz.ch> <4BC3387B.3010807@cisco.com>
To: Benoit Claise 
X-Mailer: Apple Mail (2.1078)
Cc: "ipfix@ietf.org" 
Subject: Re: [IPFIX] draft-ietf-ipfix-anon-02.txt: WG last call review
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Wed, 14 Apr 2010 07:47:18 -0000

Hi, Benoit,

Per our telephone conversation, we've addressed these comments in the =
working copy of -03, which I'll submit once I've received and integrated =
Lothar's comments. Many thanks for your review!

Cheers,

Brian


On Apr 12, 2010, at 5:12 PM, Benoit Claise wrote:

> Hi Brian,
>>=20
>>  =20
>>>>    IPFIX Message anonymisation is optionally performed as the final
>>>>    operation before handing the Message to the transport protocol =
for
>>>>    export.  While no provision is made in the architecture for
>>>>    anonymisation metadata as in Section 6, this arrangement does =
allow
>>>>    for the message rewriting necessary for comprehensive =
anonymisation
>>>>    of IPFIX export as in Section 7.  The development of the IPFIX
>>>>    Mediation [I-D.ietf-ipfix-mediators-framework] framework and the
>>>>    IPFIX File Format [RFC5655] expand upon this initial =
architectural
>>>>    allowance for anonymisation by adding to the list of places that
>>>>    anonymisation may be applied.  The former specifies IPFIX =
Mediators,
>>>>    which rewrite existing IPFIX messages, and the latter specifies =
a
>>>>=20
>>>>=20
>>>>      =20
>>> IPFIX Message?
>>> It would be nice to say in the terminology section which terms you =
use from other RFCs.
>>> For example, data set vs dataset versus Data Set
>>>    =20
>> This is an unfortunate consequence of IPFIX having something called a =
"Data Set". We need a term within this WG that means "some data" but =
that does not conflict with the defined terminology. Suggestions?
>>  =20
> If you have a terminology section, including Data Set, that would =
solve it. It would be clear what data set would (not) be.
>>  =20
>>>>    method for storage of IPFIX data in files.
>>>>=20
>>>>    More detail on the applicable architectural arrangements of
>>>>    anonymisation can be found in Section 7.1
>>>>=20
>>>>=20
>>>> 2.  Terminology
>>>>=20
>>>>=20
>>>>    Terms used in this document that are defined in the Terminology
>>>>    section of the IPFIX Protocol [RFC5101] document are to be
>>>>    interpreted as defined there.
>>>>=20
>>>>    The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL =
NOT",
>>>>    "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in =
this
>>>>    document are to be interpreted as described in RFC 2119 =
[RFC2119].
>>>>=20
>>>>=20
>>>> 3.  Categorisation of Anonymisation Techniques
>>>>=20
>>>>=20
>>>>    Anonymisation modifies a data set in order to protect the =
identity of
>>>>      =20
>> Here is a perfect example of the point above. This is "some data", =
not an IPFIX Data Set. Can we agree that "data set" is not "Data Set", =
or should we invent some more terminology here?
>>  =20
> See my point above.
>>=20
>>  =20
>>>> 6.  Anonymisation Export Support in IPFIX
>>>>=20
>>>>=20
>>>>    Anonymised data exported via IPFIX SHOULD be annotated with
>>>>    anonymisation metadata, which details which fields described by =
which
>>>>    Templates are anonymised, and provides appropriate information =
on the
>>>>    anonymisation techniques used.  This metadata SHOULD be exported =
in
>>>>    Data Records described by the recommended Options Templates =
described
>>>>    in this section; these Options Templates use the additional
>>>>    Information Elements described in the following subsection.
>>>>=20
>>>>    Note that fields anonymised using the black-marker (removal)
>>>>    technique do not require any special metadata support.  =
Black-marker
>>>>    anonymised fields SHOULD NOT be exported at all; the absence of =
the
>>>>=20
>>>>=20
>>>>      =20
>>> MUST NOT?
>>>    =20
>> A MUST NOT here would conflict with the parallel guidance in RFC 5103 =
(section 4, para 6, page 7) regarding empty reverse IEs (SHOULD not), =
and in the configuration draft, which allows fields which can be =
unambiguously null in context to be exported as null (MAY). What about =
anonymisation in an environment with hardcoded templates? No reason to =
take freedom away from implementors in this situation.
>>  =20
>=20
> I'm unclear how black-marker fields should be exported: with =
zero-length (bad idea, because 0 might mean something else) or with zero =
(in that case, we lose some bandwidth for all Flow Records)?
> Also I was wondering if we could have an Anonymization Options =
Template for a field that does not appear in the Template Record?
> Maybe clarify the points first
> Whatever you specify, this must be inline with RFC5101:
>=20
>   If a specific Information Element is required by a Template, but is
>   not available in observed packets, the Exporting Process MAY choose
>   to export Flow Records without this Information Element in a Data
>   Record defined by a new Template.
>=20
>=20
>=20
>=20
>>=20
>>=20
>>  =20
>>  =20
>>> Actually, the example and the description mention =
privateEnterpriseNumber
>>> However, I'm wondering why you need this privateEnterpriseNumber at =
all,  because the "E" bit is already part of informationElementId
>>>  0                   1                   2                   3
>>>    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
>>>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>>>    |E|  Information Element ident. |        Field Length           |
>>>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>>>    |                      Enterprise Number                        |
>>>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>>>=20
>>> I'm confused.
>>>    =20
>> The enterprise bit only says that it's got a PEN, not what the PEN =
is, so we still need it.
>>  =20
>=20
> I was thinking that informationElementId would contain the E bit. =
Checking with RFC5610 and IPFIX IANA, this is not the case.
> This should be clarified.
> Also expresses the two cases where this is needed.
> 1. two different PENs with the same informationElementId
> 2. One PEN and one non PEN with the same informationElementId
>=20
>>  =20
>>>>    | informationElementIndex | The Information Element index of the =
   |
>>>>    | [scope] [optional]      | instance of the Information Element  =
   |
>>>>    |                         | described by this anonymisation =
record  |
>>>>    |                         | identified by the =
informationElementId  |
>>>>    |                         | within the Template.  Optional; need =
   |
>>>>    |                         | only be present when describing      =
   |
>>>>    |                         | Templates that have multiple =
instances  |
>>>>    |                         | of the same Information Element.  =
This  |
>>>>    |                         | Information Element MUST be defined =
as  |
>>>>    |                         | a Scope Field if present.  This      =
   |
>>>>    |                         | Information Element is defined in    =
   |
>>>>    |                         | Section 6.2, below.                  =
   |
>>>>    | anonymisationFlags      | Flags describing the mapping =
stability  |
>>>>    |                         | and specialized modifications to the =
   |
>>>>    |                         | Anonymisation Technique in use.  =
SHOULD |
>>>>    |                         | be present.  This Information =
Element   |
>>>>    |                         | is defined in Section 6.2, below.    =
   |
>>>>=20
>>>>=20
>>>> Boschi&  Trammell        Expires August 19, 2010               =
[Page 19]
>>>> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>>>>=20
>>>>=20
>>>>    | anonymisationTechnique  | The technique used to anonymise the  =
   |
>>>>    |                         | data.  MUST be present.  This        =
   |
>>>>    |                         | Information Element is defined in    =
   |
>>>>    |                         | Section 6.2, below.                  =
   |
>>>>    =
+-------------------------+-----------------------------------------+
>>>>      =20
>>=20
>>=20
>>>>    When data is to be published as an anonymised data set in an =
IPFIX
>>>>    File [RFC5655], the anonymisation may be done at the final =
Collecting
>>>>    Process before storage and dissemination, as well.
>>>>=20
>>>>      =20
>>> For me, it's a different problem. IPFIX File could be done on =
routers.
>>> Basically, this paragraph is about the relationship with IPFIX File.
>>>    =20
>> Mainly because this section is about the arrangement of processes, =
and a File Writer is something
>>=20
>>  =20
>>> Btw, we're alos missing the relationship with reducing redundancy
>>>    =20
>> I'm not sure I get what's special about reducing redundancy with =
respect to anon.
>>  =20
>=20
>>  =20
>>> with IPFIX structured data
>>> For the later, does it apply to all instances of a template? So, =
some kind of recursivity is implied
>>>    =20
>> This is probably something that should be addressed in Structured =
Data (the only hard question here is how meta-template information is =
handled with basicList) as opposed to the other way around, because of =
the chronology, and because anon is not the only meta-template we may =
have to address. In general we should avoid normative references that go =
forward in time...
>>  =20
> Agreed on the last point.
>> Since this is metatemplate information, it should be stored by the CP =
as if it belongs to the template. So, yes, it applies to any instance of =
an template active at the time. This implies that we should point out =
somewhere that this information expires when the template is withdrawn =
or expires (ouch!).
>>  =20
> Yes!
> Next to the expiration and scope conditions, you might have one =
sentence such as: once the Anonymization Options Template Record is =
specified, it's valid for all instances of Flow Records conforming to =
this definition.
> This would take care of IPFIX-RED-RED and IPFIX-STRUCT
> I'm sure you will find some better wording, but you get my point.
>>  =20
>>>> 7.2.3.  Anonymisation of Header Data
>>>>=20
>>>>=20
>>>>    Each IPFIX Message contains a Message Header; within this =
Message
>>>>    Header are contained two fields which may be used to break =
certain
>>>>    anonymisation techniques: the Export Time, and the Observation =
Domain
>>>>=20
>>>>=20
>>>> Boschi&  Trammell        Expires August 19, 2010               =
[Page 28]
>>>> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>>>>=20
>>>>=20
>>>>    ID
>>>>=20
>>>>    Export of IPFIX Messages containing anonymised timestamp data =
where
>>>>    the original Export Time Message header has some relationship to =
the
>>>>    anonymised timestamps SHOULD anonymise the Export Time header =
field
>>>>    using an equivalent technique, if possible.  Otherwise, =
relationships
>>>>    between export and flow time could be used to partially or =
totally
>>>>    reverse timestamp anonymisation.
>>>>=20
>>>>=20
>>>>      =20
>>> What if the time is in the future, the collector would probably =
discard the information.
>>>    =20
>> Why? This seems highly implementation dependent.
>>  =20
> Sure, but it was never specified in RFC5101.
> Maybe add two sentences such as:
>    exporter: SHOULD NOT send in future times, due to anyonymation =
techniques
>    collector, MAY accept future times in connection with Anonymization =
Options Templates , but MUST log a warning.
>>=20
>>  =20
>>>>    Since the Time Window Options Template specified for the IPFIX =
File
>>>>    Format [RFC5655] refers to the timestamps within the flow data =
to
>>>>    provide partial table of contents information for an IPFIX File, =
care
>>>>    must be taken to ensure that Options described by this template =
are
>>>>    written using the anonymised timestamps instead of the original =
ones.
>>>>=20
>>>>=20
>>>>      =20
>>> Is there something in the IPFIX-MIB or IPFIX-CONF that can be used =
to deduce something about the anonymization technique?
>>>    =20
>> Not that would be exported, no. If the device supports configuration =
extensions for anonymisation, clearly you shouldn't export that, but =
it's also out of band, so not really applicable here.
>>  =20
>=20
> If I'm an attacker, wanting to decipher your anonymization technique, =
everything will help.
> If you do support the IPFIX-MIB, PSAMP-MIB, and you use SNMPv1 or v2c, =
I can sniff the community string, and get all information in the MIBs. =
Will it help me? Maybe I'm not sure!
> If I can do the same with the configuration, I might help in some =
cases. For example, in case of a permanent cache exported every hour =
UTC, a simple correlation of the time in the anoymized flow records with =
the config time will get me what I need.
> So maybe a few sentences would clarify what to pay attention to.
>=20
> Regards, Benoit.


From braun@net.in.tum.de  Mon Apr 19 06:47:26 2010
Return-Path: 
X-Original-To: ipfix@core3.amsl.com
Delivered-To: ipfix@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id A79773A6ABE for ; Mon, 19 Apr 2010 06:47:26 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: 0.951
X-Spam-Level: 
X-Spam-Status: No, score=0.951 tagged_above=-999 required=5 tests=[BAYES_50=0.001, HELO_EQ_DE=0.35, J_CHICKENPOX_37=0.6]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qoIYjrId-Xgb for ; Mon, 19 Apr 2010 06:47:20 -0700 (PDT)
Received: from mail-out1.informatik.tu-muenchen.de (mail-out1.informatik.tu-muenchen.de [131.159.0.8]) by core3.amsl.com (Postfix) with ESMTP id DC21D28C117 for ; Mon, 19 Apr 2010 06:46:54 -0700 (PDT)
Received: from honshu.net.in.tum.de (honshu.net.in.tum.de [131.159.20.100]) by mail.net.in.tum.de (Postfix) with ESMTPSA id 158782062E3E for ; Mon, 19 Apr 2010 15:47:24 +0200 (CEST)
From: Lothar Braun 
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
Date: Mon, 19 Apr 2010 15:46:30 +0200
Message-Id: 
To: IETF IPFIX Working Group 
Mime-Version: 1.0 (Apple Message framework v1078)
X-Mailer: Apple Mail (2.1078)
X-Virus-Scanned: ClamAV using ClamSMTP
Subject: [IPFIX] WG last call on draft-ietf-ipfix-anon-02.txt
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Mon, 19 Apr 2010 13:47:26 -0000

Hi,

I finally found some time to review the draft. Please find my comments =
inline.

Best regards,
  Lothar=20

>=20
> IPFIX Working Group                                            E. =
Boschi
> Internet-Draft                                               B. =
Trammell
> Intended status: Experimental                             Hitachi =
Europe
> Expires: August 19, 2010                               February 15, =
2010
>=20
>=20
>                      IP Flow Anonymisation Support
>                       draft-ietf-ipfix-anon-02.txt
>=20
> Abstract
>=20
>    This document describes anonymisation techniques for IP flow data =
and
>    the export of anonymised data using the IPFIX protocol.  It
>    categorizes common anonymisation schemes and defines the parameters
>    needed to describe them.  It provides guidelines for the
>    implementation of anonymised data export and storage over IPFIX, =
and
>    describes an information model and Options-based method for
>    anonymisation technique metadata export within the IPFIX protocol =
or
>    storage in IPFIX Files.
>=20
> Status of this Memo
>=20
>    This Internet-Draft is submitted to IETF in full conformance with =
the
>    provisions of BCP 78 and BCP 79.
>=20
>    Internet-Drafts are working documents of the Internet Engineering
>    Task Force (IETF), its areas, and its working groups.  Note that
>    other groups may also distribute working documents as Internet-
>    Drafts.
>=20
>    Internet-Drafts are draft documents valid for a maximum of six =
months
>    and may be updated, replaced, or obsoleted by other documents at =
any
>    time.  It is inappropriate to use Internet-Drafts as reference
>    material or to cite them other than as "work in progress."
>=20
>    The list of current Internet-Drafts can be accessed at
>    http://www.ietf.org/ietf/1id-abstracts.txt.
>=20
>    The list of Internet-Draft Shadow Directories can be accessed at
>    http://www.ietf.org/shadow.html.
>=20
>    This Internet-Draft will expire on August 19, 2010.
>=20
> Copyright Notice
>=20
>    Copyright (c) 2010 IETF Trust and the persons identified as the
>    document authors.  All rights reserved.
>=20
>=20
>=20
>=20
> Boschi & Trammell        Expires August 19, 2010                [Page =
1]
> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>=20
>=20
>    This document is subject to BCP 78 and the IETF Trust's Legal
>    Provisions Relating to IETF Documents
>    (http://trustee.ietf.org/license-info) in effect on the date of
>    publication of this document.  Please review these documents
>    carefully, as they describe your rights and restrictions with =
respect
>    to this document.  Code Components extracted from this document =
must
>    include Simplified BSD License text as described in Section 4.e of
>    the Trust Legal Provisions and are provided without warranty as
>    described in the BSD License.
>=20
>=20
> Table of Contents
>=20
>    1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  =
4
>      1.1.  IPFIX Protocol Overview  . . . . . . . . . . . . . . . . .  =
4
>      1.2.  IPFIX Documents Overview . . . . . . . . . . . . . . . . .  =
5
>      1.3.  Anonymisation within the IPFIX Architecture  . . . . . . .  =
5
>    2.  Terminology  . . . . . . . . . . . . . . . . . . . . . . . . .  =
7
>    3.  Categorisation of Anonymisation Techniques . . . . . . . . . .  =
7
>    4.  Anonymisation of IP Flow Data  . . . . . . . . . . . . . . . .  =
8
>      4.1.  IP Address Anonymisation . . . . . . . . . . . . . . . . . =
10
>        4.1.1.  Truncation . . . . . . . . . . . . . . . . . . . . . . =
10
>        4.1.2.  Reverse Truncation . . . . . . . . . . . . . . . . . . =
11
>        4.1.3.  Permutation  . . . . . . . . . . . . . . . . . . . . . =
11
>        4.1.4.  Prefix-preserving Pseudonymisation . . . . . . . . . . =
11
>      4.2.  Hardware Address Anonymisation . . . . . . . . . . . . . . =
12
>        4.2.1.  Reverse Truncation . . . . . . . . . . . . . . . . . . =
12
>        4.2.2.  Permutation  . . . . . . . . . . . . . . . . . . . . . =
13
>        4.2.3.  Structured Pseudonymisation  . . . . . . . . . . . . . =
13
>      4.3.  Timestamp Anonymisation  . . . . . . . . . . . . . . . . . =
13
>        4.3.1.  Precision Degradation  . . . . . . . . . . . . . . . . =
13
>        4.3.2.  Enumeration  . . . . . . . . . . . . . . . . . . . . . =
14
>        4.3.3.  Random Shifts  . . . . . . . . . . . . . . . . . . . . =
14
>      4.4.  Counter Anonymisation  . . . . . . . . . . . . . . . . . . =
14
>        4.4.1.  Precision Degradation  . . . . . . . . . . . . . . . . =
15
>        4.4.2.  Binning  . . . . . . . . . . . . . . . . . . . . . . . =
15
>        4.4.3.  Random Noise Addition  . . . . . . . . . . . . . . . . =
15
>      4.5.  Anonymisation of Other Flow Fields . . . . . . . . . . . . =
16
>        4.5.1.  Binning  . . . . . . . . . . . . . . . . . . . . . . . =
16
>        4.5.2.  Permutation  . . . . . . . . . . . . . . . . . . . . . =
16
>    5.  Parameters for the Description of Anonymisation Techniques . . =
16
>      5.1.  Stability  . . . . . . . . . . . . . . . . . . . . . . . . =
17
>      5.2.  Truncation Length  . . . . . . . . . . . . . . . . . . . . =
17
>      5.3.  Bin Map  . . . . . . . . . . . . . . . . . . . . . . . . . =
17
>      5.4.  Permutation  . . . . . . . . . . . . . . . . . . . . . . . =
18
>      5.5.  Shift Amount . . . . . . . . . . . . . . . . . . . . . . . =
18
>    6.  Anonymisation Export Support in IPFIX  . . . . . . . . . . . . =
18
>      6.1.  Anonymisation Options Template . . . . . . . . . . . . . . =
18
>=20
>=20
>=20
> Boschi & Trammell        Expires August 19, 2010                [Page =
2]
> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>=20
>=20
>      6.2.  Recommended Information Elements for Anonymisation
>            Metadata . . . . . . . . . . . . . . . . . . . . . . . . . =
20
>        6.2.1.  informationElementIndex  . . . . . . . . . . . . . . . =
20
>        6.2.2.  anonymisationFlags . . . . . . . . . . . . . . . . . . =
20
>        6.2.3.  anonymisationTechnique . . . . . . . . . . . . . . . . =
22
>    7.  Applying Anonymisation Techniques to IPFIX Export and
>        Storage  . . . . . . . . . . . . . . . . . . . . . . . . . . . =
24
>      7.1.  Arrangement of Processes in IPFIX Anonymisation  . . . . . =
24
>      7.2.  IPFIX-Specific Anonymisation Guidelines  . . . . . . . . . =
27
>        7.2.1.  Appropriate Use of Information Elements for
>                Anonymised Data  . . . . . . . . . . . . . . . . . . . =
27
>        7.2.2.  Export of Perimeter-Based Anonymisation Policies . . . =
28
>        7.2.3.  Anonymisation of Header Data . . . . . . . . . . . . . =
28
>        7.2.4.  Anonymisation of Options Data  . . . . . . . . . . . . =
29
>        7.2.5.  Special-Use Address Space Considerations . . . . . . . =
30
>    8.  Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . =
31
>    9.  Security Considerations  . . . . . . . . . . . . . . . . . . . =
33
>    10. IANA Considerations  . . . . . . . . . . . . . . . . . . . . . =
34
>    11. Acknowledgments  . . . . . . . . . . . . . . . . . . . . . . . =
35
>    12. References . . . . . . . . . . . . . . . . . . . . . . . . . . =
35
>      12.1. Normative References . . . . . . . . . . . . . . . . . . . =
35
>      12.2. Informative References . . . . . . . . . . . . . . . . . . =
35
>    Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . =
36
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
> Boschi & Trammell        Expires August 19, 2010                [Page =
3]
> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>=20
>=20
> 1.  Introduction
>=20
>    The standardisation of an IP flow information export protocol
>    [RFC5101] and associated representations removes a technical =
barrier
>    to the sharing of IP flow data across organizational boundaries and
>    with network operations, security, and research communities for a
>    wide variety of purposes.  However, with wider dissemination comes
>    greater risks to the privacy of the users of networks under
>    measurement, and to the security of those networks.  While it is =
not
>    a complete solution to the issues posed by distribution of IP flow
>    information, anonymisation (i.e., the deletion or transformation of
>    information that is considered sensitive and could be used to =
reveal
>    the identity of subjects involved in a communication) is an =
important
>    tool for the protection of privacy within network measurement
>    infrastructures.
>=20
>    This document presents a mechanism for representing anonymised data
>    within IPFIX and guidelines for using it.  It begins with a
>    categorization of anonymisation techniques.  It then describes
>    applicability of each technique to commonly anonymisable fields of =
IP
>    flow data, organized by information element data type and semantics
>    as in [RFC5102]; enumerates the parameters required by each of the
>    applicable anonymisation techniques; and provides guidelines for =
the
>    use of each of these techniques in accordance with best practices =
in
>    data protection.  Finally, it specifies a mechanism for exporting
>    anonymised data and binding anonymisation metadata to templates =
using
>    IPFIX Options.
>=20
> 1.1.  IPFIX Protocol Overview
>=20
>    In the IPFIX protocol, { type, length, value } tuples are expressed
>    in templates containing { type, length } pairs, specifying which {
>    value } fields are present in data records conforming to the
>    Template, giving great flexibility as to what data is transmitted.
>    Since Templates are sent very infrequently compared with Data
>    Records, this results in significant bandwidth savings.  Various
>    different data formats may be transmitted simply by sending new
>    Templates specifying the { type, length } pairs for the new data
>    format.  See [RFC5101] for more information.
>=20
>    The IPFIX information model [RFC5102] defines a large number of
>    standard Information Elements which provide the necessary { type }
>    information for Templates.  The use of standard elements enables
>    interoperability among different vendors' implementations.
>    Additionally, non-standard enterprise-specific elements may be
>    defined for private use.
>=20
>=20
>=20
>=20
>=20
> Boschi & Trammell        Expires August 19, 2010                [Page =
4]
> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>=20
>=20
> 1.2.  IPFIX Documents Overview
>=20
>    "Specification of the IPFIX Protocol for the Exchange of IP Traffic
>    Flow Information" [RFC5101] and its associated documents define the
>    IPFIX Protocol, which provides network engineers and administrators
>    with access to IP traffic flow information.
>=20
>    "Architecture for IP Flow Information Export" [RFC5470] defines the
>    architecture for the export of measured IP flow information out of =
an
>    IPFIX Exporting Process to an IPFIX Collecting Process, and the =
basic
>    terminology used to describe the elements of this architecture, per
>    the requirements defined in "Requirements for IP Flow Information
>    Export" [RFC3917].  The IPFIX Protocol document [RFC5101] then =
covers
>    the details of the method for transporting IPFIX Data Records and
>    Templates via a congestion-aware transport protocol from an IPFIX
>    Exporting Process to an IPFIX Collecting Process.
>=20
>    "Information Model for IP Flow Information Export" [RFC5102]
>    describes the Information Elements used by IPFIX, including details
>    on Information Element naming, numbering, and data type encoding.
>    Finally, "IPFIX Applicability" [RFC5472] describes the various
>    applications of the IPFIX protocol and their use of information
>    exported via IPFIX, and relates the IPFIX architecture to other
>    measurement architectures and frameworks.
>=20
>    Additionally, "Specification of the IPFIX File Format" [RFC5655]
>    describes a file format based upon the IPFIX Protocol for the =
storage
>    of flow data.
>=20
>    This document references the Protocol and Architecture documents =
for
>    terminology, and extends the IPFIX Information Model to provide new
>    Information Elements for anonymisation metadata.  The anonymisation
>    techniques described herein are equally applicable to the IPFIX
>    Protocol and data stored in IPFIX Files.
>=20
> 1.3.  Anonymisation within the IPFIX Architecture
>=20
>    "Architecture for IP Flow Information Export" [RFC5470] defines the
>    functions performed in sequence by the various functional blocks in
>    an IPFIX Device as in the figure below.
>=20
>=20
>                     Packet(s) coming into Observation Point(s)
>                       |                                   |
>                       v                                   v
>      +----------------+-------------------------+   +-----+-------+
>      |          Metering Process on an          |   |             |
>      |             Observation Point            |   |             |
>=20
>=20
>=20
> Boschi & Trammell        Expires August 19, 2010                [Page =
5]
> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>=20
>=20
>      |                                          |   |             |
>      |   packet header capturing                |   |             |
>      |        |                                 |...| Metering    |
>      |   timestamping                           |   | Process N   |
>      |        |                                 |   |             |
>      | +----->+                                 |   |             |
>      | |      |                                 |   |             |
>      | |   sampling Si (1:1 in case of no       |   |             |
>      | |      |          sampling)              |   |             |
>      | |   filtering Fi (select all when        |   |             |
>      | |      |          no criteria)           |   |             |
>      | +------+                                 |   |             |
>      |        |                                 |   |             |
>      |        |        Timing out Flows         |   |             |
>      |        |    Handle resource overloads    |   |             |
>      +--------|---------------------------------+   +-----|-------+
>               |                                           |
>       Flow Records (identified by Observation Domain)  Flow Records
>               |                                           |
>               +---------+---------------------------------+
>                         |
>    =
+--------------------|----------------------------------------------+
>    |                    |     Exporting Process                        =
|
>    |+-------------------|-------------------------------------------+  =
|
>    ||                   v       IPFIX Protocol                      |  =
|
>    ||+-----------------------------+  +----------------------------+|  =
|
>    |||Rules for                    |  |Functions                   ||  =
|
>    ||| Picking/sending Templates   |  |-Packetise selected Control ||  =
|
>    ||| Picking/sending Flow Records|->|  & data Information into   ||  =
|
>    ||| Encoding Template & data    |  |  IPFIX export packets.     ||  =
|
>    ||| Selecting Flows to export(*)|  |-Handle export errors       ||  =
|
>    ||+-----------------------------+  +----------------------------+|  =
|
>    |+----------------------------+----------------------------------+  =
|
>    |                             |                                     =
|
>    |                    exported IPFIX Messages                        =
|
>    |                             |                                     =
|
>    |                +------------+-----------------+                   =
|
>    |                |  Anonymise export packet(*)  |                   =
|
>    |                +------------+-----------------+                   =
|
>    |                             |                                     =
|
>    |                +------------+-----------------+                   =
|
>    |                |       Transport  Protocol    |                   =
|
>    |                +------------+-----------------+                   =
|
>    |                             |                                     =
|
>    =
+-----------------------------+-------------------------------------+
>                                  |
>                                  v
>                     IPFIX export packet to Collector
>=20
>=20
>=20
> Boschi & Trammell        Expires August 19, 2010                [Page =
6]
> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>=20
>=20
>    (*) indicates that the block is optional.
>=20
>=20
>                  Figure 1: IPFIX Device functional blocks
>=20
>    Note that, according to the original architecture specification,
>    IPFIX Message anonymisation is optionally performed as the final
>    operation before handing the Message to the transport protocol for
>    export.  While no provision is made in the architecture for
>    anonymisation metadata as in Section 6, this arrangement does allow
>    for the message rewriting necessary for comprehensive anonymisation
>    of IPFIX export as in Section 7.  The development of the IPFIX
>    Mediation [I-D.ietf-ipfix-mediators-framework] framework and the
>    IPFIX File Format [RFC5655] expand upon this initial architectural
>    allowance for anonymisation by adding to the list of places that
>    anonymisation may be applied.  The former specifies IPFIX =
Mediators,
>    which rewrite existing IPFIX messages, and the latter specifies a
>    method for storage of IPFIX data in files.
>=20
>    More detail on the applicable architectural arrangements of
>    anonymisation can be found in Section 7.1
>=20
>=20
> 2.  Terminology
>=20
>    Terms used in this document that are defined in the Terminology
>    section of the IPFIX Protocol [RFC5101] document are to be
>    interpreted as defined there.
>=20
>    The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
>    "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in =
this
>    document are to be interpreted as described in RFC 2119 [RFC2119].
>=20
>=20
> 3.  Categorisation of Anonymisation Techniques
>=20
>    Anonymisation modifies a data set in order to protect the identity =
of
>    the people or entities described by the data set from disclosure.
>    With respect to network traffic data, anonymisation generally
>    attempts to preserve some set of properties of the network traffic
>    useful for a given application or applications, while ensuring the
>    data cannot be traced back to the specific networks, hosts, or =
users
>    generating the traffic.
>=20
>    Anonymisation may be broadly classified according to two =
properties:
>    recoverability and countability.  All anonymisation techniques map
>    the real space of identifiers or values into a separate, anonymised
>    space, according to some function.  A technique is said to be
>=20
>=20
>=20
> Boschi & Trammell        Expires August 19, 2010                [Page =
7]
> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>=20
>=20
>    recoverable when the function used is invertible or can otherwise =
be
>    reversed and a real identifier can be recovered from a given
>    replacement identifier.
>=20
>    Countability compares the dimension of the anonymised space (N) to
>    the dimension of the real space (M), and denotes how the count of
>    unique values is preserved by the anonymisation function.  If the
>    anonymised space is smaller than the real space, then the function =
is
>    said to generalise the input, mapping more than one input point to
>    each anonymous value (e.g., as with aggregation).  By definition,
>    generalisation is not recoverable.

It might be possible to recover the original generalised values by =
combining the anonymised data with other data. Example: The =
autoconfigured IPv6 address are truncated by 48 bits, but the flows =
contain the original MAC addresses.

Maybe this should be stated here?

>    If the dimensions of the anonymised and real spaces are the same,
>    such that the count of unique values is preserved, then the =
function
>    is said to be a direct substitution function.  If the dimension of
>    the anonymised space is larger, such that each real value maps to a
>    set of anonymised values, then the function is said to be a set
>    substitution function.  Note that with set substitution functions,
>    the sets of anonymised values are not necessarily disjoint.  Either
>    direct or set substitution functions are said to be one-way if =
there
>    exists no practical method for recovering the real data point from =
an
>    anonymised one.

Same as before.

>    This classification is summarised in the table below.
>=20
>    =
+------------------------+-----------------+------------------------+
>    | Recoverability /       | Recoverable     | Non-recoverable        =
|
>    | Countability           |                 |                        =
|
>    =
+------------------------+-----------------+------------------------+
>    | N < M                  | N.A.            | Generalisation         =
|
>    | N =3D M                  | Direct          | One-way Direct       =
  |
>    |                        | Substitution    | Substitution           =
|
>    | N > M                  | Set             | One-way Set            =
|
>    |                        | Substitution    | Substitution           =
|
>    =
+------------------------+-----------------+------------------------+
>=20
>=20
> 4.  Anonymisation of IP Flow Data
>=20
>    Due to the restricted semantics of IP flow data, there is a
>    relatively limited set of specific anonymisation techniques =
available
>    on flow data, though each falls into the broad categories above.
>    Each type of field that may commonly appear in a flow record may =
have
>    its own applicable specific techniques.
>=20
>    While anonymisation is generally applied at the resolution of =
single
>    fields within a flow record, attacks against anonymisation use =
entire
>    flows and relationships between hosts and flows within a given data
>=20
>=20
>=20
> Boschi & Trammell        Expires August 19, 2010                [Page =
8]
> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>=20
>=20
>    set.  Therefore, fields which may not necessarily be identifying by
>    themselves may be anonymised in order to increase the anonymity of
>    the data set as a whole.
>=20
>    Of all the fields in an IP flow record, IP addresses are the most
>    likely to be used to directly identify entities in the real world.
>    Each IP address is associated with an interface on a network host,
>    and can potentially be identified with a single user.  =
Additionally,
>    IP addresses are structured identifiers; that is, partial IP =
address
>    prefixes may be used to identify networks just as full IP addresses
>    identify hosts.  This makes anonymisation of IP addresses
>    particularly important.
>=20
>    Hardware addresses uniquely identify devices on the network; while
>    they are not often available in traffic data collected at Layer 3,
>    and cannot be used to locate devices within the network, some =
traces
>    may contain sub-IP data including hardware address data.  Hardware
>    addresses may be mappable to device serial numbers, and to the
>    entities or individuals who purchased the devices, when combined =
with
>    external databases.  They may also leak via IPv6 addresses in =
certain
>    circumstances.  Therefore, hardware address anonymisation is also
>    important.

The other way around could be even worse (see my example above).

>    Port numbers identify abstract entities (applications) as opposed =
to
>    real-world entities, but they can be used to classify hosts and =
user
>    behavior.  Passive port fingerprinting, both of well-known and
>    ephemeral ports, can be used to determine the operating system
>    running on a host.  Relative data volumes by port can also be used =
to
>    determine the host's function (workstation, web server, etc.); this
>    information can be used to identify hosts and users.
>=20
>    While not identifiers in and of themselves, timestamps and counters
>    can reveal the behavior of the hosts and users on a network.  Any
>    given network activity is recognizable by a pattern of relative =
time
>    differences and data volumes in the associated sequence of flows,
>    even without host address information.  They can therefore be used =
to
>    identify hosts and users.  Timestamps and counters are also
>    vulnerable to traffic injection attacks, where traffic with a known
>    pattern is injected into a network under measurement, and this
>    pattern is later identified in the anonymised data set.
>=20
>    The simplest and most extreme form of anonymisation, which can be
>    applied to any field of a flow record, is black-marker =
anonymisation,
>    or complete deletion of a given field.  Note that black-marker
>    anonymisation is equivalent to simply not exporting the field(s) in
>    question.
>=20
>    While black-marker anonymisation completely protects the data in =
the
>=20
>=20
>=20
> Boschi & Trammell        Expires August 19, 2010                [Page =
9]
> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>=20
>=20
>    deleted fields from the risk of disclosure, it also reduces the
>    utility of the anonymised data set as a whole.  Techniques that
>    retain some information while reducing (though not eliminating) the
>    disclosure risk will be extensively discussed in the following
>    sections; note that the techniques specifically applicable to IP
>    addresses, timestamps, ports, and counters will be discussed in
>    separate sections.
>=20
> 4.1.  IP Address Anonymisation
>=20
>    Since IP addresses are the most common identifiers within flow data
>    that can be used to directly identify a person, organization, or
>    host, most of the work on flow and trace data anonymisation has =
gone
>    into IP address anonymisation techniques.  Indeed, the aim of most
>    attacks against anonymisation is to recover the map from anonymised
>    IP addresses to original IP addresses thereby identifying the
>    identified hosts.  There is therefore a wide range of IP address
>    anonymisation schemes that fit into the following categories.
>=20
>        +------------------------------------+---------------------+
>        | Scheme                             | Action              |
>        +------------------------------------+---------------------+
>        | Truncation                         | Generalisation      |
>        | Reverse Truncation                 | Generalisation      |
>        | Permutation                        | Direct Substitution |
>        | Prefix-preserving Pseudonymisation | Direct Substitution |
>        +------------------------------------+---------------------+
>=20
> 4.1.1.  Truncation
>=20
>    Truncation removes "n" of the least significant bits from an IP
>    address, replacing them with zeroes.  In effect, it replaces a host
>    address with a network address for some fixed netblock; for IPv4
>    addresses, 8-bit truncation corresponds to replacement with a /24
>    network address.  Truncation is a non-reversible generalisation
>    scheme.  Note that while truncation is effective for making hosts
>    non-identifiable, it preserves information which can be used to
>    identify an organization, a geographic region, a country, or a
>    continent (or RIR region of responsibility).
>=20
>    Truncation to an address length of 0 is equivalent to black-marker
>    anonymisation.  Complete removal of IP address information is only
>    recommended for analysis tasks which have no need to separate flow
>    data by host or network; e.g. as a first stage to per-application
>    (port) or time-series total volume analyses.
>=20
>=20
>=20
>=20
>=20
>=20
> Boschi & Trammell        Expires August 19, 2010               [Page =
10]
> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>=20
>=20
> 4.1.2.  Reverse Truncation
>=20
>    Reverse truncation removes "n" of the most significant bits from an
>    IP address, replacing them with zeroes.  Reverse truncation is a =
non-
>    reversible generalisation scheme.  Reverse truncation is effective
>    for making networks unidentifiable, partially or completely =
removing
>    information which can be used to identify an organization, a
>    geographic region, a country, or a continent (or RIR region of
>    responsibility).  However, it may cause ambiguity when applied to
>    data collected from more than one network, since it treats all the
>    hosts with the same address on different networks as if they are =
the
>    same host.  It is not particularly useful when publishing data =
where
>    the network of origin is known or can be easily guessed by virtue =
of
>    the identity of the publisher.
>=20
>    Like truncation, reverse truncation to an address length of 0 is
>    equivalent to black-marker anonymisation.
>=20
> 4.1.3.  Permutation
>=20
>    Permutation is a direct substitution technique, replacing each IP
>    address with an address selected from the set of possible IP
>    addresses, guaranteeing that each anonymised address represents a
>    unique original address.  The selection function is often random,
>    though it is not necessarily so.  Permutation does not preserve any
>    structural information about a network, but it does preserve the
>    unique count of IP addresses.  Any application that requires more
>    structure than host-uniqueness will not be able to use permuted IP
>    addresses.
>=20
> 4.1.4.  Prefix-preserving Pseudonymisation
>=20
>    Prefix-preserving pseudonymisation is a direct substitution
>    technique, like permutation but further restricted such that the
>    structure of subnets is preserved at each level while anonymising =
IP
>    addresses.  If two real IP addresses match on a prefix of "n" bits,
>    the two anonymised IP addresses will match on a prefix of "n" bits =
as
>    well.  This is useful when relationships among networks must be
>    preserved for a given analysis task, but introduces structure into
>    the anonymised data which can be exploited in attacks against the
>    anonymisation technique.
>=20
>    Scanning in Internet background traffic can cause particular =
problems
>    with this technique: if a scanner uses a predictable and known
>    sequence of addresses, this information can be used to reverse the
>    substitution.  The low order portion of the address can be left
>    unanonymized as a partial defense against this attack.
>=20
>=20
>=20
>=20
> Boschi & Trammell        Expires August 19, 2010               [Page =
11]
> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>=20
>=20
> 4.2.  Hardware Address Anonymisation
>=20
>    Flow data containing sub-IP information can also contain =
identifying
>    information in the form of the hardware (MAC) address.  While
>    hardware address information cannot be used to locate a node within =
a
>    network, it can be used to directly uniquely identify a specific
>    device.  Vendors or organizations within the supply chain may then
>    have the information necessary to identify the entity or individual
>    that purchased the device.
>=20
>    Hardware address information is not as structured as IP address
>    information.  EUI-48 and EUI-64 hardware addresses contain an
>    Organizational Unique Identifier in the three most significant =
bytes
>    of the address; this OUI additionally contains bits noting whether
>    the address is locally or globally administered.  Beyond this, the
>    address is unstructured, and there is no particular relationship
>    among the OUIs assigned to a given vendor.
>=20
>    Note that hardware address information also appear within IPv6
>    addresses, as the EAP-64 address, or EAP-48 address encoded as an
>    EAP-64 address, is used as the least significant 64 bits of the =
IPv6
>    address in the case of link local addressing or stateless
>    autoconfiguration; the considerations and techniques in this =
section
>    may then apply to such IPv6 addresses as well.
>=20
>            +-----------------------------+---------------------+
>            | Scheme                      | Action              |
>            +-----------------------------+---------------------+
>            | Reverse Truncation          | Generalisation      |
>            | Permutation                 | Direct Substitution |
>            | Structured Pseudonymisation | Direct Substitution |
>            +-----------------------------+---------------------+
>=20
> 4.2.1.  Reverse Truncation
>=20
>    Reverse truncation removes "n" of the most significant bits from an
>    MAC address, replacing them with zeroes.  Reverse truncation is a
>    non-reversible generalisation scheme.  This has the effect of
>    removing bits of the OUI, which identify manufacturers, before
>    removing the least significant bits.  Reverse truncation of 24 bits
>    zeroes out the OUI.
>=20
>    Reverse truncation is effective for making device manufacturers
>    partially or completely unidentifiable within a dataset.  However, =
it
>    may cause ambiguity by introducing the possibility of truncated MAC
>    address collision.  Also note that the utility or removing
>    manufacturer information is dubious, and not particularly well-
>    covered by the literature.
>=20
>=20
>=20
> Boschi & Trammell        Expires August 19, 2010               [Page =
12]
> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>=20
>=20
>    Reverse truncation to an address length of 0 is equivalent to =
black-
>    marker anonymisation.
>=20
> 4.2.2.  Permutation
>=20
>    Permutation is a direct substitution technique, replacing each MAC
>    address with an address selected from the set of possible MAC
>    addresses, guaranteeing that each anonymised address represents a
>    unique original address.  The selection function is often random,
>    though it is not necessarily so.  Permutation does not preserve any
>    structural information about a network, but it does preserve the
>    unique count of devices on the network.  Any application that
>    requires more structure than host-uniqueness will not be able to =
use
>    permuted MAC addresses.
>=20
> 4.2.3.  Structured Pseudonymisation
>=20
>    Structured pseudonymisation for MAC addresses is a direct
>    substitution technique, like permutation, but restricted such that
>    the OUI (the most significant three bytes) is permuted separately
>    from the node identifier, the remainder.  This is useful when the
>    uniqueness of OUIs must be preserved for a given analysis task, but
>    introduces structure into the anonymised data which can be =
exploited
>    in attacks against the anonymisation technique.
>=20
> 4.3.  Timestamp Anonymisation
>=20
>    The particular time at which a flow began or ended is not
>    particularly identifiable information, but it can be used as part =
of
>    attacks against other anonymisation techniques or for user =
profiling.
>    Presice timestamps can be used in injected-traffic fingerprinting
>    attacks as well as to identify certain activity by response delay =
and
>    size fingerprinting.  Therefore, timestamp information may be
>    anonymised in order to ensure the protection of the entire dataset.
>=20
>           +-----------------------+----------------------------+
>           | Scheme                | Action                     |
>           +-----------------------+----------------------------+
>           | Precision Degradation | Generalisation             |
>           | Enumeration           | Direct or Set Substitution |
>           | Random Shifts         | Direct Substitution        |
>           +-----------------------+----------------------------+
>=20
> 4.3.1.  Precision Degradation
>=20
>    Precision Degradation is a generalisation technique that removes =
the
>    most precise components of a timestamp, accounting all events
>    occurring in each given interval (e.g. one millisecond for
>=20
>=20
>=20
> Boschi & Trammell        Expires August 19, 2010               [Page =
13]
> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>=20
>=20
>    millisecond level degradation) as simultaneous.  This has the =
effect
>    of potentially collapsing many timestamps into one.  With this
>    technique time precision is reduced, and sequencing may be lost, =
but
>    the information at which time the event occurred is preserved.  The
>    anonymised data may not be generally useful for applications which
>    require strict sequencing of flows.
>=20
>    Note that flow meters with low time precision (e.g. second =
precision,
>    or millisecond precision on high-capacity networks) perform the
>    equivalent of precision degradation anonymisation by their design.
>=20
>    Note also that degradation to a very low precision (e.g. on the =
order
>    of minutes, hours, or days) is commonly used in analyses operating =
on
>    time-series aggregated data, and may also be described as binning;
>    though the time scales are longer and applicability more =
restricted,
>    this is in principle the same operation.
>=20
>    Precision degradation to infinitely low precision is equivalent to
>    black-marker anonymisation.  Removal of timestamp information is =
only
>    recommended for analysis tasks which have no need to separate flows
>    in time, for example for counting total volumes or unique =
occurrences
>    of other flow keys in an entire dataset.
>=20
> 4.3.2.  Enumeration
>=20
>    Enumeration is a substitution function that retains the =
chronological
>    order in which events occurred while eliminating time information.
>    Timestamps are substituted by equidistant timestamps (or numbers)
>    starting from a randomly chosen start value.  The resulting data is
>    useful for applications requiring strict sequencing, but not for
>    those requiring good timing information (e.g. delay- or jitter-
>    measurement for QoS applications or SLA validation).
>=20
> 4.3.3.  Random Shifts
>=20
>    Random time shifts add a random offset to every timestamp within a
>    dataset.  This reversible substitution technique therefore retains
>    duration and inter-event interval information as well as
>    chronological order of flows.  It is primarily intended to defeat
>    traffic injection fingerprinting attacks.
>=20
> 4.4.  Counter Anonymisation
>=20
>    Counters (such as packet and octet volumes per flow) are subject to
>    fingerprinting and injection attacks against anonymisation, or for
>    user profiling as timestamps are.  Counter anonymisation can help
>    defeat these attacks, but are only usable for analysis tasks for
>    which relative or imprecise magnitudes of activity are useful.
>=20
>=20
>=20
> Boschi & Trammell        Expires August 19, 2010               [Page =
14]
> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>=20
>=20
>    Counter information can also be completely removed, but this is =
only
>    recommended for analysis tasks which have no need to evaluate the
>    removed counter, for example for counting only unique occurrences =
of
>    other flow keys.
>=20
>           +-----------------------+----------------------------+
>           | Scheme                | Action                     |
>           +-----------------------+----------------------------+
>           | Precision Degradation | Generalisation             |
>           | Binning               | Generalisation             |
>           | Random noise addition | Direct or Set Substitution |
>           +-----------------------+----------------------------+
>=20
> 4.4.1.  Precision Degradation
>=20
>    As with precision degradation in timestamps, precision degradation =
of
>    counters removes lower-order bits of the counters, treating all the
>    counters in a given range as having the same value.  Depending on =
the
>    precision reduction, this loses information about the relationships
>    between sizes of similarly-sized flows, but keeps relative =
magnitude
>    information.  Precision degradation to an infinitely low precision =
is
>    equivalent to black-marker anonymisation.
>=20
> 4.4.2.  Binning
>=20
>    Binning can be seen as a special case of precision degradation; the
>    operation is identical, except for in precision degradation the
>    counter ranges are uniform, and in binning they need not be.  For
>    example, a common counter binning scheme for packet counters could =
be
>    to bin values 1-2 together, and 3-infinity together, thereby
>    separating potentially completely-opened TCP connections from
>    unopened ones.  Binning schemes are generally chosen to keep
>    precisely the amount of information required in a counter for a =
given
>    analysis task.  Note that, also unlike precision degradation, the =
bin
>    label need not be within the bin's range.  Binning counters to a
>    single bin is equivalent to black-marker anonymisation.
>=20
> 4.4.3.  Random Noise Addition
>=20
>    Random noise addition adds a random amount to a counter in each =
flow;
>    this is used to keep relative magnitude information and minimize =
the
>    disruption to size relationship information while avoiding
>    fingerprinting attacks against anonymisation.  Note that there is =
no
>    guarantee that random noise addition will maintain ranking order by =
a
>    counter among members of a set.  Random noise addition is
>    particularly useful when the derived analysis data will not be
>    presented in such a way as to require the lower-order bits of the
>    counters.

I have problems understand the purpose of this anonymisation technique. =
How can it "keep relative magnitude information" but gives "no guarantee =
that random noise will maintain ranking order"? Isn't this a =
contradiction?

I have problems understanding the following:

What is the difference between 4.4.3 and 4.4.1 with respect of the =
information that is contained in the lower-order bits of the counters? =
As far as I understand both anonymisation techniques remove the complete =
information from the lower-order bits. Did I get this wrong?

>=20
> Boschi & Trammell        Expires August 19, 2010               [Page =
15]
> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>=20
>=20
> 4.5.  Anonymisation of Other Flow Fields
>=20
>    Other fields, particularly port numbers and protocol numbers, can =
be
>    used to partially identify the applications that generated the
>    traffic in a a given flow trace.  This information can be used in
>    fingerprinting attacks, and may be of interest on its own (e.g., to
>    reveal that a certain application with suspected vulnerabilities is
>    running on a given network).  These fields are generally anonymised
>    using one of two techniques.
>=20
>                    +-------------+---------------------+
>                    | Scheme      | Action              |
>                    +-------------+---------------------+
>                    | Binning     | Generalisation      |
>                    | Permutation | Direct Substitution |
>                    +-------------+---------------------+
>=20
> 4.5.1.  Binning
>=20
>    Binning is a generalisation technique mapping a set of potentially
>    non-uniform ranges into a set of arbitrarily labeled bins.  Common
>    bin arrangements depend on the field type and the analysis
>    application.  For example, an IP protocol bin arrangement may
>    preserve 1, 6, and 17 for ICMP, UDP, and TCP traffic, and bin all
>    other protocols into a single bin, to mitigate the use of uncommon
>    protocols in fingerprinting attacks.  Another example arrangement =
may
>    bin source and destination ports into low (0-1023) and high (1024-
>    65535) bins in order to tell service from ephemeral ports without
>    identifying individual applications.
>=20
>    Binning other flow key fields to a single bin is equivalent to =
black-
>    marker anonymisation.  Removal of other flow key information is =
only
>    recommended for analysis tasks which have no need to differentiate
>    flows on the removed keys, for example for total traffic counts or
>    unique counts of other flow keys.
>=20
> 4.5.2.  Permutation
>=20
>    Permutation is a direct substitution technique, replacing each =
value
>    with an value selected from the set of possible range, guaranteeing
>    that each anonymised value represents a unique original value.  =
This
>    is used to preserve the count of unique values without preserving
>    information about, or the ordering of, the values themselves.
>=20
>=20
> 5.  Parameters for the Description of Anonymisation Techniques
>=20
>    This section details the abstract parameters used to describe the
>=20
>=20
>=20
> Boschi & Trammell        Expires August 19, 2010               [Page =
16]
> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>=20
>=20
>    anonymisation techniques examined in the previous section, on a =
per-
>    parameter basis.  These parameters and their export safety inform =
the
>    design of the IPFIX anonymisation metadata export specified in the
>    following section.
>=20
> 5.1.  Stability
>=20
>    Any given anonymisation technique may be applied with a varying =
range
>    of stability.  Stability is important for assessing the =
comparability
>    of anonymised information in different data sets, or in the same =
data
>    set over different time periods.  In general, stability ranges from
>    completely stable to completely unstable; however, note that the
>    completely unstable case is indistinguishable from black-marker
>    anonymisation.  A completely stable anonymisation will always map a
>    given value in the real space to the same value in the anonymised
>    space.  In practice, an anonymisation may also be stable for every
>    data set published by an a particular producer to a particular
>    consumer, stable for a stated time period within a dataset or =
across
>    datasets, or stable only for a single data set.

I had troubles understanding this (and had to read this section several =
times before I got it). Maybe one could first define stability and than =
talk about the varying range:

	A complete stable anonymisation will always map a given value
	in the real space to the same value in the anonymised space. A
	completely unstable case is indistinguishable from black-marker=20=

	anonymisation. Any given anonymisation technique may be ....

Could this be a way to start the section?=20


>    If no information about stability is available, users of anonymised
>    data may assume that the techniques used are stable across the =
entire
>    dataset, but unstable across datasets. =20

Is this a definition or "common practice"?

> Note that stability presents
>    a risk-utility tradeoff, as completely stable anonymisation can be
>    used for longer-term trend analysis tasks but also presents more =
risk
>    of attack given the stable mapping.
>=20
> 5.2.  Truncation Length
>=20
>    Truncation and precision degradation are described by the =
truncation
>    length, or the amount of data still remaining in the anonymised =
field
>    after anonymisation.
>=20
>    Truncation length can be inferred from a given data set, and need =
not
>    be specially exported or protected.
>=20
> 5.3.  Bin Map
>=20
>    Binning is described by the specification of a bin mapping =
function.
>    This function can be generally expressed in terms of an associative
>    array that maps each point in the original space to a bin, although
>    from an implementation standpoint most bin functions are much =
simpler
>    and more efficient.
>=20
>    Since knowledge of the bin mapping function can be used to =
partially
>    deanonymise binned data, depending on the degree of generalisation,
>    no information about the bin mapping function should be exported.
>=20
>=20
>=20
>=20
> Boschi & Trammell        Expires August 19, 2010               [Page =
17]
> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>=20
>=20
> 5.4.  Permutation
>=20
>    Like binning, permutation is described by the specification of a
>    permutation function.  In the general case, this can be expressed =
in
>    terms of an associative array that maps each point in the original
>    space to a point in the anonymised space.  Unlike binning, each =
point
>    in the anonymised space must correspond to a single, unique point =
in
>    the original space.
>=20
>    Since knowledge of the permutation function can be used to =
completely
>    deanonymise permuted data, no information about the permutation
>    function or its parameters should be exported.
>=20
> 5.5.  Shift Amount
>=20
>    Shifting requires an amount to shift each value by.  Since the =
shift
>    amount can be used to deanonymise data protected by shifting, no
>    information about the shift amount should be exported.
>=20
>=20
> 6.  Anonymisation Export Support in IPFIX
>=20
>    Anonymised data exported via IPFIX SHOULD be annotated with
>    anonymisation metadata, which details which fields described by =
which
>    Templates are anonymised, and provides appropriate information on =
the
>    anonymisation techniques used.  This metadata SHOULD be exported in
>    Data Records described by the recommended Options Templates =
described
>    in this section; these Options Templates use the additional
>    Information Elements described in the following subsection.
>=20
>    Note that fields anonymised using the black-marker (removal)
>    technique do not require any special metadata support.  =
Black-marker
>    anonymised fields SHOULD NOT be exported at all; the absence of the
>    field in a given Data Set is implicitly declared by not including =
the
>    corresponding Information Element in the Template describing that
>    Data Set.
>=20
> 6.1.  Anonymisation Options Template
>=20
>    The Anonymisation Options Template describes anonymisation records,
>    which allow anonymisation metadata to be exported inline over IPFIX
>    or stored in an IPFIX File, by binding information about
>    anonymisation techniques to Information Elements within defined
>    Templates.  IPFIX Exporting Processes SHOULD export anonymisation
>    records for any Template describing exported anonymised Data =
Records;
>    IPFIX Collecting Processes and processes downstream from them MAY =
use
>    anonymisation records to treat anonymised data differently =
depending
>    on the applied technique.
>=20
>=20
>=20
> Boschi & Trammell        Expires August 19, 2010               [Page =
18]
> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>=20
>=20
>    An Exporting Process SHOULD export anonymisation records after the
>    Templates they describe have been exported, and SHOULD export
>    anonymisation records reliably.
>=20
>    Anonymisation records, like Templates, MUST be handled by =
Collecting
>    Processes as scoped to the Transport Session in which they are =
sent.


>    While the Stability Class within the anonymisationFlags IE can be
>    used to declare that a given anonymisation technique's mapping will
>    remain stable across multiple sessions, each session MUST re-export
>    the anonymisation Records along with the templates.

I got totally confused about the Stability Class and the =
anonymisationFlag (they where not introduced at this point) and had no =
clue what you wanted to say with that. A clear case of too much =
information. Maybe this could be made easier like

	If a session is re-exported, all anonymisation records MUST be =
re-exported along with the templates.

(You might want to turn that into proper English)

>    =
+-------------------------+-----------------------------------------+
>    | IE                      | Description                             =
|
>    =
+-------------------------+-----------------------------------------+
>    | templateId [scope]      | The Template ID of the Template         =
|
>    |                         | containing the Information Element      =
|
>    |                         | described by this anonymisation record. =
|
>    |                         | This Information Element MUST be        =
|
>    |                         | defined as a Scope Field.               =
|
>    | informationElementId    | The Information Element identifier of   =
|
>    | [scope]                 | the Information Element described by    =
|
>    |                         | this anonymisation record.  This        =
|
>    |                         | Information Element MUST be defined as  =
|
>    |                         | a Scope Field.                          =
|
>    | informationElementId    | The Private Enterprise Number of the    =
|
>    | [scope] [optional]      | enterprise-specific Information Element =
|
>    |                         | described by this anonymisation record. =
|
>    |                         | This Information Element MUST be        =
|
>    |                         | defined as a Scope Field if present.    =
|
>    | informationElementIndex | The Information Element index of the    =
|
>    | [scope] [optional]      | instance of the Information Element     =
|
>    |                         | described by this anonymisation record  =
|
>    |                         | identified by the informationElementId  =
|
>    |                         | within the Template.  Optional; need    =
|
>    |                         | only be present when describing         =
|
>    |                         | Templates that have multiple instances  =
|
>    |                         | of the same Information Element.  This  =
|
>    |                         | Information Element MUST be defined as  =
|
>    |                         | a Scope Field if present.  This         =
|
>    |                         | Information Element is defined in       =
|
>    |                         | Section 6.2, below.                     =
|
>    | anonymisationFlags      | Flags describing the mapping stability  =
|
>    |                         | and specialized modifications to the    =
|
>    |                         | Anonymisation Technique in use.  SHOULD =
|
>    |                         | be present.  This Information Element   =
|
>    |                         | is defined in Section 6.2, below.       =
|
>=20
>=20
>=20
>=20
>=20
> Boschi & Trammell        Expires August 19, 2010               [Page =
19]
> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>=20
>=20
>    | anonymisationTechnique  | The technique used to anonymise the     =
|
>    |                         | data.  MUST be present.  This           =
|
>    |                         | Information Element is defined in       =
|
>    |                         | Section 6.2, below.                     =
|
>    =
+-------------------------+-----------------------------------------+
>=20
> 6.2.  Recommended Information Elements for Anonymisation Metadata
>=20
> 6.2.1.  informationElementIndex
>=20
>    Description:   A zero-based index of an Information Element
>       referenced by informationElementId within a Template referenced =
by
>       templateId; used to disambiguate scope for templates containing
>       multiple identical Information Elements.
>=20
>    Abstract Data Type:   unsigned16
>=20
>    ElementId:   TBD3

Is this meant to be "To Be Decided"?

>    Status:   Proposed
>=20
> 6.2.2.  anonymisationFlags
>=20
>    Description:   A flag word describing specialized modifications to
>       the anonymisation policy in effect for the anonymisation =
technique
>       applied to a referenced Information Element within a referenced
>       Template.  When flags are clear (0), the normal policy (as
>       described by anonymisationTechnique) applies without =
modification.
>=20
>       MSB   14  13  12  11  10   9   8   7   6   5   4   3   2   1  =
LSB
>       =
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
>       |                Reserved                       |LOR|PmA|   SC  =
|
>       =
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
>=20
>                             anonymisationFlags IE
>=20
>    =
+--------+----------+-----------------------------------------------+
>    | bit(s) | name     | description                                   =
|
>    | (LSB =3D |          |                                             =
  |
>    | 0)     |          |                                               =
|
>    =
+--------+----------+-----------------------------------------------+
>    | 0-1    | SC       | Stability Class: see the Stability Class      =
|
>    |        |          | table below, and section Section 5.1.         =
|
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
> Boschi & Trammell        Expires August 19, 2010               [Page =
20]
> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>=20
>=20
>    | 2      | PmA      | Perimeter Anonymisation: when set (1), source =
|

By the time reading this, I had no clue what Petrimeter Anonymisation =
could mean (until reading 7.2.2). Maybe a reference to rfc 5103 could =
help?=10

>    |        |          | address Information Elements are interpreted  =
|
>    |        |          | as external addresses, and destination        =
|
>    |        |          | address Information Elements are interpreted  =
|
>    |        |          | as internal addresses, for the purposes of    =
|
>    |        |          | associating anonymisationTechnique to         =
|
>    |        |          | Information Elements.  MUST NOT be set when   =
|
>    |        |          | associated with a non-endpoint (i.e., source- =
|
>    |        |          | or destination-) Information Element.  SHOULD =
|
>    |        |          | be consistent within a record (i.e., if a     =
|
>    |        |          | source- Information Element has this flag     =
|
>    |        |          | set, the corresponding destination- element   =
|
>    |        |          | SHOULD have this flag set, and vice-versa.)   =
|
>    | 3      | LOR      | Low-Order Unchanged: when set (1), the        =
|
>    |        |          | low-order bits of the anonymised Information  =
|
>    |        |          | Element contain real data.  This modification =
|
>    |        |          | is intended for the anonymisation of          =
|
>    |        |          | network-level addresses while leaving         =
|
>    |        |          | host-level addresses intact in order to       =
|
>    |        |          | preserve host level-structure, which could    =
|
>    |        |          | otherwise be used to reverse anonymisation.   =
|
>    |        |          | MUST NOT be set when associated with a        =
|
>    |        |          | truncation-based anonymisationTechnique.      =
|
>    | 4-15   | Reserved | Reserved for future use: SHOULD be cleared    =
|
>    |        |          | (0) by the Exporting Process and MUST be      =
|
>    |        |          | ignored by the Collecting Process.            =
|
>    =
+--------+----------+-----------------------------------------------+
>=20
>       The Stability Class portion of this flags word describes the
>       stability class of the anonymisation technique applied to a
>       referenced Information Element within a referenced Template.
>       Stability classes refer to the stability of the parameters of =
the
>       anonymisation technique, and therefore the comparability of the
>       mapping between the real and anonymised values over time.  This
>       determines which anonymised datasets may be compared with each
>       other.  Values are as follows:
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
> Boschi & Trammell        Expires August 19, 2010               [Page =
21]
> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>=20
>=20
>    =
+-----+-----+-------------------------------------------------------+
>    | Bit | Bit | Description                                           =
|
>    | 1   | 0   |                                                       =
|
>    =
+-----+-----+-------------------------------------------------------+
>    | 0   | 0   | Undefined: the Exporting Process makes no             =
|
>    |     |     | representation as to how stable the mapping is, or    =
|
>    |     |     | over what time period values of this field will       =
|
>    |     |     | remain comparable; while the Collecting Process MAY   =
|
>    |     |     | assume Session level stability, Session level         =
|
>    |     |     | stability is not guaranteed.  Processes SHOULD assume =
|
>    |     |     | this is the case in the absence of stability class    =
|
>    |     |     | information; this is the default stability class.     =
|
>    | 0   | 1   | Session: the Exporting Process will ensure that the   =
|
>    |     |     | parameters of the anonymisation technique are stable  =
|
>    |     |     | during the Transport Session.  All the values of the  =
|
>    |     |     | described Information Element for each Record         =
|
>    |     |     | described by the referenced Template within the       =
|
>    |     |     | Transport Session are comparable.  The Exporting      =
|
>    |     |     | Process SHOULD endeavour to ensure at least this      =
|
>    |     |     | stability class.                                      =
|
>    | 1   | 0   | Exporter-Collector Pair: the Exporting Process will   =
|
>    |     |     | ensure that the parameters of the anonymisation       =
|
>    |     |     | technique are stable across Transport Sessions over   =
|
>    |     |     | time with the given Collecting Process, but may use   =
|
>    |     |     | different parameters for different Collecting         =
|
>    |     |     | Processes.  Data exported to different Collecting     =
|
>    |     |     | Processes is not comparable.                          =
|
>    | 1   | 1   | Stable: the Exporting Process will ensure that the    =
|
>    |     |     | parameters of the anonymisation technique are stable  =
|
>    |     |     | across Transport Sessions over time, regardless of    =
|
>    |     |     | the Collecting Process to which it is sent.           =
|
>    =
+-----+-----+-------------------------------------------------------+
>=20
>    Abstract Data Type:   unsigned16
>=20
>    ElementId:   TBD1
>=20
>    Status:   Proposed
>=20
> 6.2.3.  anonymisationTechnique
>=20
>    Description:   A description of the anonymisation technique applied
>       to a referenced Information Element within a referenced =
Template.
>       Each technique may be applicable only to certain Information
>       Elements and recommended only for certain Infomation Elements;
>       these restrictions are noted in the table below.
>=20
>=20
>=20
>=20
>=20
> Boschi & Trammell        Expires August 19, 2010               [Page =
22]
> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>=20
>=20
>    =
+-------+---------------------------+-----------------+-------------+
>    | Value | Description               | Applicable to   | Recommended =
|
>    |       |                           |                 | for         =
|
>    =
+-------+---------------------------+-----------------+-------------+
>    | 0     | Undefined: the Exporting  | all             | all         =
|
>    |       | Process makes no          |                 |             =
|
>    |       | representation as to      |                 |             =
|
>    |       | whether the defined field |                 |             =
|
>    |       | is anonymised or not.     |                 |             =
|
>    |       | While the Collecting      |                 |             =
|
>    |       | Process MAY assume that   |                 |             =
|
>    |       | the field is not          |                 |             =
|
>    |       | anonymised, it is not     |                 |             =
|
>    |       | guaranteed not to be.     |                 |             =
|
>    |       | This is the default       |                 |             =
|
>    |       | anonymisation technique.  |                 |             =
|
>    | 1     | None: the values exported | all             | all         =
|
>    |       | are real.                 |                 |             =
|
>    | 2     | Precision                 | all             | all         =
|
>    |       | Degradation/Truncation:   |                 |             =
|
>    |       | the values exported are   |                 |             =
|
>    |       | anonymised using simple   |                 |             =
|
>    |       | precision degradation or  |                 |             =
|
>    |       | truncation.  The new      |                 |             =
|
>    |       | precision or number of    |                 |             =
|
>    |       | truncated bits is         |                 |             =
|
>    |       | implicit in the exported  |                 |             =
|
>    |       | data, and can be deduced  |                 |             =
|
>    |       | by the Collecting         |                 |             =
|
>    |       | Process.                  |                 |             =
|

How can I deduce that as a collector? What is implicit? Is an =
IPv4-Adress encoded in 3 bytes instead of 4 if I do 8-bit truncation?

>    | 3     | Binning: the values       | all             | all         =
|
>    |       | exported are anonymised   |                 |             =
|
>    |       | into bins.                |                 |             =
|
>    | 4     | Enumeration: the values   | all             | timestamps  =
|
>    |       | exported are anonymised   |                 |             =
|
>    |       | by enumeration.           |                 |             =
|
>    | 5     | Permutation: the values   | all             | identifiers =
|
>    |       | exported are anonymised   |                 |             =
|
>    |       | by random permutation.    |                 |             =
|
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
> Boschi & Trammell        Expires August 19, 2010               [Page =
23]
> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>=20
>=20
>    | 6     | Structured Permutation:   | addresses       |             =
|
>    |       | the values exported are   |                 |             =
|
>    |       | anonymised by random      |                 |             =
|
>    |       | permutation, preserving   |                 |             =
|
>    |       | bit-level structure as    |                 |             =
|
>    |       | appropriate; this         |                 |             =
|
>    |       | represents                |                 |             =
|
>    |       | prefix-preserving IP      |                 |             =
|
>    |       | address anonymisation or  |                 |             =
|
>    |       | structured MAC address    |                 |             =
|
>    |       | anonymisation.            |                 |             =
|
>    | 7     | Reverse Truncation: the   | addresses       |             =
|
>    |       | values exported are       |                 |             =
|
>    |       | anonymised using reverse  |                 |             =
|
>    |       | truncation.  The number   |                 |             =
|
>    |       | of truncated bits is      |                 |             =
|
>    |       | implicit in the exported  |                 |             =
|
>    |       | data, and can be deduced  |                 |             =
|
>    |       | by the Collecting         |                 |             =
|
>    |       | Process.

Again. How do I do this as a collector?=20

>                  |                 |             |
>    | 8     | Noise: the values         | non-identifiers | counters    =
|
>    |       | exported are anonymised   |                 |             =
|
>    |       | by adding random noise to |                 |             =
|
>    |       | each value.               |                 |             =
|
>    | 9     | Offset: the values        | all             | timestamps  =
|
>    |       | exported are anonymised   |                 |             =
|
>    |       | by adding a single offset |                 |             =
|
>    |       | to all values.            |                 |             =
|
>    =
+-------+---------------------------+-----------------+-------------+
>=20
>    Abstract Data Type:   unsigned16
>=20
>    ElementId:   TBD2
>=20
>    Status:   Proposed
>=20
>=20
> 7.  Applying Anonymisation Techniques to IPFIX Export and Storage
>=20
>    When exporting or storing anonymised flow data using IPFIX, certain
>    interactions between the IPFIX Protocol and the anonymisation
>    techniques in use must be considered; these are treated in the
>    subsections below.
>=20
> 7.1.  Arrangement of Processes in IPFIX Anonymisation
>=20
>    Anonymisation may be applied to IPFIX data at three stages within =
the
>    collection infrastructure: on initial export, at a mediator, or =
after
>=20
>=20
>=20
> Boschi & Trammell        Expires August 19, 2010               [Page =
24]
> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>=20
>=20
>    collection, as shown in Figure 2.  Each of these locations has
>    specific considerations and applicability.
>=20
>                +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+
>                | Exporting Process                        |
>                +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+
>                  |                                      |
>                  |    (Anonymised at Original Exporter) |
>                  V                                      |
>                +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+          |
>                | Mediator                    |          |
>                +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+          |
>                  |                                      |
>                  | (Anonymising Mediator)               |
>                  V                                      V
>                +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+
>                | Collecting Process                       |
>                +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+
>                        |
>                        | (Anonymising CP/File Writer)
>                        V
>                +--------------------+
>                | IPFIX File Storage |
>                +--------------------+
>=20
>                 Figure 2: Potential Anonymisa
> tion Locations
>=20
>    Anonymisation is generally performed before the wider dissemination
>    or repurposing of a flow data set, e.g., adapting operational
>    measurement data for research.  Therefore, direct anonymisation of
>    flow data on initial export is only applicable in certain =
restricted
>    circumstances: when the Exporting Process is "publishing" data to a
>    Collecting Process directly, and the Exporting Process and =
Collecting
>    Process are operated by different entities.  Note that certain
>    guidelines in Section 7.2.3 with respect to timestamp anonymisation
>    may not apply in this case, as the Collecting Process may be able =
to
>    deduce certain timing information from the time at which each =
Message
>    is received.
>=20
>    A much more flexible arrangement is to anonymise data within a
>    Mediator [I-D.ietf-ipfix-mediators-framework].  Here, original data
>    is sent to a Mediator, which performs the anonymisation function =
and
>    re-exports the anonymised data.  Such a Mediator could be located =
at
>    the administrative domain boundary of the initial Exporting Process
>    operator, exporting anonymised data to other consumers outside the
>    organisation.  In this case, the original Exporter SHOULD use TLS =
as
>    specified in [RFC5101] to secure the channel to the Mediator, and =
the
>    Mediator should follow the guidelines in Section 7.2, to mitigate =
the
>=20
>=20
>=20
> Boschi & Trammell        Expires August 19, 2010               [Page =
25]
> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>=20
>=20
>    risk of original data disclosure.
>=20
>    When data is to be published as an anonymised data set in an IPFIX
>    File [RFC5655], the anonymisation may be done at the final =
Collecting
>    Process before storage and dissemination, as well.  In this case, =
the
>    Collector should follow the guidelines in Section 7.2, especially =
as
>    regards File-specific Options in Section 7.2.4
>=20
>    In each of these data flows, the anonymisation of records is
>    undertaken by an Intermediate Anonymisation Process (IAP); the data
>    flows into and out of this IAP are shown in Figure 3 below.
>=20
>    packets --+                     +- IPFIX Messages -+
>              |                     |                  |
>              V                     V                  V
>    +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ =
+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ =
+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+
>    | Metering Process | | Collecting Process | | File Reader |
>    +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ =
+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ =
+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+
>              |      Non-anonymised | Records          |
>              V                     V                  V
>    +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D+
>    |          Intermediate Anonymisation Process (IAP)       |
>    +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D+
>              | Anonymised     ^            Anonymised |
>              | Records        |               Records |
>              V                |                       V
>    +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+    =
Anonymisation      +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+
>    | Exporting Process |<--- Parameters ------>| File Writer |
>    +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+         =
              +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+
>              |                                        |
>              +------------> IPFIX Messages <----------+
>=20
>           Figure 3: Data flows through the anonymisation process
>=20
>    Anonymisation parameters must also be available to the Exporting
>    Process and/or File Writer in order to ensure header data is also
>    appropriately anonymised as in Section 7.2.3.
>=20
>    Following each of the data flows through the IAP, we describe five
>    basic types of anonymisation arrangements within this framework in
>    Figure 4.  In addition to the three arrangements described in =
detail
>    above, anonymisation can also be done at a collocated Metering
>    Process and File Writer (see section 7.3.2 of [RFC5655]), or at a
>    file manipulator (see section 7.3.7 of [RFC5655]).
>=20
>=20
>=20
>=20
>=20
>=20
>=20
> Boschi & Trammell        Expires August 19, 2010               [Page =
26]
> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>=20
>=20
>          +----+  +-----+  +----+
>  pkts -> | MP |->| IAP |->| EP |-> anonymisation on Original Exporter
>          +----+  +-----+  +----+
>          +----+  +-----+  +----+
>  pkts -> | MP |->| IAP |->| FW |-> Anonymising collocated MP/File =
Writer
>          +----+  +-----+  +----+
>          +----+  +-----+  +----+
> IPFIX -> | CP |->| IAP |->| EP |-> Anonymising Mediator (Masquerading =
Proxy)
>          +----+  +-----+  +----+
>          +----+  +-----+  +----+
> IPFIX -> | CP |->| IAP |->| FW |-> Anonymising collocated CP/File =
Writer
>          +----+  +-----+  +----+
>          +----+  +-----+  +----+
> IPFIX -> | FR |->| IAP |->| FW |-> Anonymising file manipulator
>  File    +----+  +-----+  +----+
>=20
>         Figure 4: Possible anonymisation arrangements in the IPFIX
>                                architecture
>=20
>    Note that anonymisation may occur at more than one location within =
a
>    given collection infrastructure, to provide varying levels of
>    anonymisation, disclosure risk, or data utility for specific
>    purposes.
>=20
> 7.2.  IPFIX-Specific Anonymisation Guidelines
>=20
>    In implementing and deploying the anonymisation techniques =
described
>    in this document, implementors should note that IPFIX already
>    provides features that support anonymised data export, and use =
these
>    where appropriate.  Care must also be taken that data structures
>    supporting the operation of the protocol itself do not leak data =
that
>    could be used to reverse the anonymisation applied to the flow =
data.
>    Such data structures may appear in the header, or within the data
>    stream itself, especially as options data.  Each of these and their
>    impact on specific anonymisation techniques is noted in a separate
>    subsection below.
>=20
> 7.2.1.  Appropriate Use of Information Elements for Anonymised Data
>=20
>    Note, as in Section 6 above, that black-marker anonymised fields
>    SHOULD NOT be exported at all; the absence of the field in a given
>    Data Set is implicitly declared by not including the corresponding
>    Information Element in the Template describing that Data Set.
>=20
>    When using precision degradation of timestamps, Exporting Processes
>    SHOULD export timing information using Information Elements of an
>    appropriate precision, as explained in Section 4.5 of [RFC5153].  =
For
>    example, timestamps measured in millisecond-level precision and
>=20
>=20
>=20
> Boschi & Trammell        Expires August 19, 2010               [Page =
27]
> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>=20
>=20
>    degraded to second-level precision should use flowStartSeconds and
>    flowEndSeconds, not flowStartMilliseconds and flowEndMilliseconds.
>=20
>    When exporting anonymised data and anonymisation metadata, =
Exporting
>    Processes SHOULD ensure that the combination of Information Element
>    and declared anonymisation technique are compatible.  Specifically,
>    the applicable and recommended Information Element types and
>    semantics for each technique are noted in the description of the
>    anonymisationTechnique Information Element in Section 6.2.3.  In =
this
>    description, a timestamp is an Information Element with the data =
type
>    dateTimeSeconds, dataTimeMilliseconds, dateTimeMicroseconds, or
>    dateTimeNanoseconds; an address is an Information Element with the
>    data type ipv4Address, ipv6Address, or macAddress; and an =
identifier
>    is an Information Element with identifier data type semantics.
>    Exporting Process MUST NOT export Anonymisation Options records
>    binding techniques to Information Elements to which they are not
>    applicable, and SHOULD NOT export Anonymisation Options records
>    binding techniques to Information Elements for which they are not
>    recommended.
>=20
> 7.2.2.  Export of Perimeter-Based Anonymisation Policies
>=20
>    Data collected from a single network may require different
>    anonymisation policies for addresses internal and external to the
>    network.  For example, internal addresses could be subject to =
simple
>    permutation, while external addresses could be aggregated into
>    networks by truncation.  When exporting anonymised perimeter
>    bidirectional flow (biflow) data as in section 5.2 of [RFC5103], =
this
>    arrangement may be easily represented by specifying one technique =
for
>    source endpoint information (which represents the external endpoint
>    in a perimeter biflow) and one technique for destination endpoint
>    information (which represents the internal address in a perimeter
>    biflow).
>=20
>    However, it can also be useful to represent perimeter-based
>    anonymisation policies with unidirectional flow (uniflow), or non-
>    perimeter biflow data.  In this case, the Perimeter Anonymisation =
bit
>    (bit 2) in the anonymisationFlags Information Element describing =
the
>    anonymised address Information Elements can be set to change the
>    meaning of "source" and "destination" of Information Elements to =
mean
>    "external" and "internal" as with perimeter biflows, but only with
>    respect to anonymisation policies.
>=20
> 7.2.3.  Anonymisation of Header Data
>=20
>    Each IPFIX Message contains a Message Header; within this Message
>    Header are contained two fields which may be used to break certain
>    anonymisation techniques: the Export Time, and the Observation =
Domain
>=20
>=20
>=20
> Boschi & Trammell        Expires August 19, 2010               [Page =
28]
> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>=20
>=20
>    ID
>=20
>    Export of IPFIX Messages containing anonymised timestamp data where
>    the original Export Time Message header has some relationship to =
the
>    anonymised timestamps SHOULD anonymise the Export Time header field
>    using an equivalent technique, if possible.  Otherwise, =
relationships
>    between export and flow time could be used to partially or totally
>    reverse timestamp anonymisation.

Hmm, just thinking ...

Could anonymising the Export Time reveal more information than not =
anonymising this in some cases?

Let ET be the original Export Time, AET be the anonymized Export Time, =
CT the time at with the collector received the Message and RTT the =
round-trip time

ET =3D CT - RTT

If the RTT is nearly stable, the original ET can be easily decoded =
revealing one time mapping If you anonymize timestamps by adding a fixed =
value, you revealed the anonymisation.

Could the draft recommend to use different anonymization techniques for =
the flow data and the export time?


>    The similarity in size between an Observation Domain ID and an IPv4
>    address (32 bits) may lead to a temptation to use an IPv4 interface
>    address on the Metering or Exporting Process as the Observation
>    Domain ID.  If this address bears some relation to the IP addresses
>    in the flow data (e.g., shares a network prefix with internal
>    addresses) and the IP addresses in the flow data are anonymised in =
a
>    structure-preserving way, then the Observation Domain ID may be =
used
>    to break the IP address anonymisation.  Use of an IPv4 interface
>    address on the Metering or Exporting Process as the Observation
>    Domain ID is NOT RECOMMENDED in this case.
>
> 7.2.4.  Anonymisation of Options Data
>=20
>    IPFIX uses the Options mechanism to export, among other things,
>    metadata about exported flows and the flow collection =
infrastructure.
>    As with the IPFIX Message Header, certain Options recommended in
>    [RFC5101] and [RFC5655] containing flow timestamps and network
>    addresses of Exporting and Collecting Processes may be used to =
break
>    certain anonymisation techniques; care should be taken while using
>    them with anonymised data export and storage.
>=20
>    The Exporting Process Reliability Statistics Options Template,
>    recommended in [RFC5101], contains an Exporting Process ID field,
>    which may be an exportingProcessIPv4Address Information Element or =
an
>    exportingProcessIPv6Address Information Element.  If the Exporting
>    Process address bears some relation to the IP addresses in the flow
>    data (e.g., shares a network prefix with internal addresses) and =
the
>    IP addresses in the flow data are anonymised in a structure-
>    preserving way, then the Exporting Process address may be used to
>    break the IP address anonymisation.  Exporting Processes exporting
>    anonymised data in this situation SHOULD mitigate the risk of =
attack
>    either by omitting Options described by the Exporting Process
>    Reliability Statistics Options Template, or by anonymising the
>    Exporting Process address using a similar technique to that used to
>    anonymise the IP addresses in the exported data.
>=20
>    Similarly, the Export Session Details Options Template and Message
>    Details Options Template specified for the IPFIX File Format
>    [RFC5655] may contain the exportingProcessIPv4Address Information
>=20
>=20
>=20
> Boschi & Trammell        Expires August 19, 2010               [Page =
29]
> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>=20
>=20
>    Element or the exportingProcessIPv6Address Information Element to
>    identify an Exporting Process from which a flow record was =
received,
>    and the collectingProcessIPv4Address Information Element or the
>    collectingProcessIPv6Address Information Element to identify the
>    Collecting Process which received it.  If the Exporting Process or
>    Collecting Process address bears some relation to the IP addresses =
in
>    the flow data (e.g., shares a network prefix with internal =
addresses)
>    and the IP addresses in the flow data are anonymised in a =
structure-
>    preserving way, then the Exporting Process or Collecting Process
>    address may be used to break the IP address anonymisation.  Since
>    these Options Templates are primarily intended for storing IPFIX
>    Transport Session data for auditing, replay, and testing purposes, =
it
>    is NOT RECOMMENDED that storage of anonymised data include these
>    Options Templates in order to mitigate the risk of attack.
>=20
>    The Message Details Options Template specified for the IPFIX File
>    Format [RFC5655] also contains the collectionTimeMilliseconds
>    Information Element.  As with the Export Time Message Header field,
>    if the exported flow data contains anonymised timestamp =
information,
>    and the collectionTimeMilliseconds Information Element in a given
>    Message has some relationship to the anonymised timestamp
>    information, then this relationship can be exploited to reverse the
>    timestamp anonymisation.  Since this Options Template is primarily
>    intended for storing IPFIX Transport Session data for auditing,
>    replay, and testing purposes, it is NOT RECOMMENDED that storage of
>    anonymised data include this Options Template in order to mitigate
>    the risk of attack.
>=20
>    Since the Time Window Options Template specified for the IPFIX File
>    Format [RFC5655] refers to the timestamps within the flow data to
>    provide partial table of contents information for an IPFIX File, =
care
>    must be taken to ensure that Options described by this template are
>    written using the anonymised timestamps instead of the original =
ones.
>=20
> 7.2.5.  Special-Use Address Space Considerations
>=20
>    When anonymising data for transport or storage using IPFIX =
containing
>    anonymised IP addresses, and the analysis purpose permits doing so,
>    it is recommended to filter out or leave unanonymised data =
containing
>    the special-use IPv4 addresses enumerated in [RFC3330] or the
>    special-use IPv6 addresses enumerated in [RFC5153].  Data =
containing
>    these addresses (e.g. 0.0.0.0 and 169.254.0.0/16 for link-local
>    autoconfiguration in IPv4 space) are often associated with =
specific,
>    well-known behavioral patterns.  Detection of these patterns in
>    anonymised data can lead to deanonymisation of these special-use
>    addresses, which increases the chance of a complete reversal of
>    anonymisation by an attacker, especially of prefix-preserving
>    techniques.
>=20
>=20
>=20
> Boschi & Trammell        Expires August 19, 2010               [Page =
30]
> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>=20
>=20
> 8.  Examples
>=20
>    In this example, consider the export or storage of an anonymised =
IPv4
>    dataset from a single network described by a simple template
>    containing a timestamp in seconds, a five-tuple, and packet and =
octet
>    counters.  The template describing each record in this dataset is
>    shown in figure Figure 5.
>=20
>                             1                   2                   3
>         0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 =
1
>        =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        |          Set ID =3D 2           |          Length =3D  40     =
    |
>        =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        |      Template ID =3D 256        |        Field Count =3D 8    =
    |
>        =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        |0| flowStartSeconds        150 |       Field Length =3D  4     =
  |
>        =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        |0| sourceIPv4Address         8 |       Field Length =3D  4     =
  |
>        =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        |0| destinationIPv4Address   12 |       Field Length =3D  4     =
  |
>        =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        |0| sourceTransportPort       7 |       Field Length =3D  2     =
  |
>        =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        |0| destinationTransportPort 11 |       Field Length =3D  2     =
  |
>        =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        |0| packetDeltaCount          2 |       Field Length =3D  4     =
  |
>        =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        |0| octetDeltaCount           1 |       Field Length =3D  4     =
  |
>        =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        |0| protocolIdentifier        4 |       Field Length =3D  1     =
  |
>        =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>=20
>                       Figure 5: Example Flow Template
>=20
>    Suppose that this dataset is anonymised according to the following
>    policy:
>=20
>    o  IP addresses within the network are protected by reverse
>       truncation.
>=20
>    o  IP addresses outside the network are protected by prefix-
>       preserving anonymisation.
>=20
>    o  Octet counts are exported using degraded precision in order to
>       provide minimal protection against fingerprinting attacks.
>=20
>    o  All other fields are exported unanonymised.
>=20
>=20
>=20
>=20
> Boschi & Trammell        Expires August 19, 2010               [Page =
31]
> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>=20
>=20
>    In order to export anonymisation records for this template and
>    policy, first, the Anonymisation Options Template shown in figure
>    Figure 6 is exported.  For this example, the optional
>    privateEnterpriseNumber and informationElementIndex Information
>    Elements are omitted, because they are not used.
>=20
>                               1                   2                   =
3
>           0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 =
0 1
>          =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>          |          Set ID =3D 3           |          Length =3D  26   =
      |
>          =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>          |      Template ID =3D 257        |        Field Count =3D 4  =
      |
>          =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>          |    Scope Field Count =3D 2      |0| templateID              =
346 |
>          =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Why is the templateID 346? Shouldn't it be 256?

>          |       Field Length =3D 2        |0| informationElementId    =
303 |
>          =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

informationElementID  =3D=3D TBD3?

>          |       Field Length =3D 2        |0| anonymisationFlags      =
339 |
>          =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

anonymisationFlgas =3D=3D TBD1?

>          |       Field Length =3D 2        |0| anonymisationTechnique  =
344 |
>          =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

anonymisationTechnique =3D TBD2?

>          |       Field Length =3D 2        |
>          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>=20
>              Figure 6: Example Anonymisation Options Template
>=20
>    Following the Anonymisation Options Template comes a Data Set
>    containing Anonymisation Records.  This data set has an entry for
>    each Information Element Specifier in Template 256 describing the
>    flow records.  This Data Set is shown in figure Figure 7.  Note =
that
>    sourceIPv4Address and destinationIPv4Address have the Perimeter
>    Anonymisation (0x0004) flag set in anonymisationFlags, meaning that
>    source address should be treated as network-external, and the
>    destination address as network-internal.
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
> Boschi & Trammell        Expires August 19, 2010               [Page =
32]
> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>=20
>=20
>                             1                   2                   3
>         0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 =
1
>        =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        |          Set ID =3D 257         |          Length =3D  68     =
    |
>        =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        |          Template 256         | flowStartSeconds       IE 150 =
|
>        =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        | no flags               0x0000 | Not Anonymised              1 =
|
>        =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        |          Template 256         | sourceIPv4Address        IE 8 =
|
>        =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        | Perimeter, Session SC 0x0005  | Structured Permutation      6 =
|
>        =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        |          Template 256         | destinationIPv4Address  IE 12 =
|
>        =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        | Perimeter, Stable     0x0005  | Reverse Truncation          7 =
|
>        =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        |          Template 256         | sourceTransportPort      IE 7 =
|
>        =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        | no flags               0x0000 | Not Anonymised              1 =
|
>        =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        |          Template 256         | dest.TransportPort      IE 11 =
|
>        =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        | no flags               0x0000 | Not Anonymised              1 =
|
>        =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        |          Template 256         | packetDeltaCount         IE 2 =
|
>        =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        | no flags               0x0000 | Not Anonymised              1 =
|
>        =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        |          Template 256         | octetDeltaCount          IE 1 =
|
>        =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        | Stable                 0x0003 | Precision Degradation       2 =
|
>        =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        |          Template 256         | protocolIdentifier      IE 4  =
|
>        =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>        | no flags               0x0000 | Not Anonymised              1 =
|
>        =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>=20
>                   Figure 7: Example Anonymisation Records
>=20
>    Following the Anonymisation Records come the data sets containing =
the
>    anonymised data, exported according to the template in figure
>    Figure 5
>=20
>=20
> 9.  Security Considerations
>=20
>    This document provides guidelines for exporting metadata about
>=20
>=20
>=20
> Boschi & Trammell        Expires August 19, 2010               [Page =
33]
> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>=20
>=20
>    anonymised data in IPFIX, or storing metadata about anonymised data
>    in IPFIX Files.  It is not intended as a general statement on the
>    applicability of specific flow data anonymisation techniques.
>    Exporters or publishers of anonymised data must take care that the
>    applied anonymisation technique is appropriate for the data source,
>    the purpose, and the risk of deanonymisation of a given =
application.
>=20
>    We note specifically that anonymisation is not a replacement for
>    encryption for confidentiality.  It is only appropriate for
>    protecting identifying information in data to be used for purposes =
in
>    which the protected data is irrelevant.  Confidentiality in export =
is
>    best served by using TLS or DTLS as in the Security Considerations
>    section of [RFC5101], and in long-term storage by implementation-
>    specific protection applied as in the Security Considerations =
section
>    of [RFC5655].  Indeed, confidentiality and anonymisation are not
>    mutually exclusive, as encryption for confidentiality may be =
applied
>    to anonymised data export or storage, as well, when the anonymised
>    data is not intended for public release.
>=20
>    When using pseudonymisation techniques that have a mutable mapping,
>    there is an inherent tradeoff in the stability of the map between
>    long-term comparability and security of the dataset against
>    deanonymisation.  In general, deanonymisation attacks are more
>    effective given more information, so the longer a given mapping is
>    valid, the more information can be applied to deanonymisation.  The
>    specific details of this are technique-dependent and therefore out =
of
>    the scope of this document.
>=20
>    When releasing anonymised data, publishers need to ensure that data
>    that could be used in deanonymisation is not leaked through the
>    export protocol; guidelines for addressing this risk are provided =
in
>    Section 7.2.
>=20
>    Note as well that the Security Considerations section of [RFC5101]
>    applies as well to the export of anonymised data, and the Security
>    Considerations section of [RFC5655] to the storage of anonymised
>    data, or the publication of anonymised traces.
>=20
>=20
> 10.  IANA Considerations
>=20
>    This document specifies the creation of several new IPFIX =
Information
>    Elements in the IPFIX Information Element registry located at
>    http://www.iana.org/assignments/ipfix, as defined in Section 6.2
>    above.  IANA has assigned the following Information Element numbers
>    for their respective Information Elements as specified below:
>=20
>=20
>=20
>=20
>=20
> Boschi & Trammell        Expires August 19, 2010               [Page =
34]
> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>=20
>=20
>    o  Information Element number TBD1 for the anonymisationFlags
>       Information Element.
>=20
>    o  Information Element number TBD2 for the anonymisationTechnique
>       Information Element.
>=20
>    o  Information Element number TBD3 for the informationElementIndex
>       Information Element.
>=20
>    [NOTE for IANA: The text TBDn should be replaced with the =
respective
>    assigned Information Element numbers where they appear in this
>    document.]
>=20
>=20
> 11.  Acknowledgments
>=20
>    We thank Paul Aitken and John McHugh for their comments and =
insight,
>    and Carsten Schmoll for his review.  Special thanks to the =
ICT-PRISM
>    project for its material support of this work.
>=20
>=20
> 12.  References
>=20
> 12.1.  Normative References
>=20
>    [RFC5101]  Claise, B., "Specification of the IP Flow Information
>               Export (IPFIX) Protocol for the Exchange of IP Traffic
>               Flow Information", RFC 5101, January 2008.
>=20
>    [RFC5102]  Quittek, J., Bryant, S., Claise, B., Aitken, P., and J.
>               Meyer, "Information Model for IP Flow Information =
Export",
>               RFC 5102, January 2008.
>=20
>    [RFC5610]  Boschi, E., Trammell, B., Mark, L., and T. Zseby,
>               "Exporting Type Information for IP Flow Information =
Export
>               (IPFIX) Information Elements", RFC 5610, July 2009.
>=20
>    [RFC5655]  Trammell, B., Boschi, E., Mark, L., Zseby, T., and A.
>               Wagner, "Specification of the IP Flow Information Export
>               (IPFIX) File Format", RFC 5655, October 2009.
>=20
>    [RFC3330]  IANA, "Special-Use IPv4 Addresses", RFC 3330,
>               September 2002.
>=20
> 12.2.  Informative References
>=20
>    [RFC5103]  Trammell, B. and E. Boschi, "Bidirectional Flow Export
>               Using IP Flow Information Export (IPFIX)", RFC 5103,
>=20
>=20
>=20
> Boschi & Trammell        Expires August 19, 2010               [Page =
35]
> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>=20
>=20
>               January 2008.
>=20
>    [RFC5472]  Zseby, T., Boschi, E., Brownlee, N., and B. Claise, "IP
>               Flow Information Export (IPFIX) Applicability", RFC =
5472,
>               March 2009.
>=20
>    [RFC5470]  Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek,
>               "Architecture for IP Flow Information Export", RFC 5470,
>               March 2009.
>=20
>    [I-D.ietf-ipfix-mediators-framework]
>               Kobayashi, A., Claise, B., and K. Ishibashi, "IPFIX
>               Mediation: Framework",
>               draft-ietf-ipfix-mediators-framework-04 (work in
>               progress), October 2009.
>=20
>    [I-D.ietf-ipfix-mediators-problem-statement]
>               Kobayashi, A., Claise, B., Nishida, H., Sommer, C.,
>               Dressler, F., and E. Stephan, "IPFIX Mediation: Problem
>               Statement",
>               draft-ietf-ipfix-mediators-problem-statement-07 (work in
>               progress), December 2009.
>=20
>    [RFC5153]  Boschi, E., Mark, L., Quittek, J., Stiemerling, M., and =
P.
>               Aitken, "IP Flow Information Export (IPFIX) =
Implementation
>               Guidelines", RFC 5153, April 2008.
>=20
>    [RFC3917]  Quittek, J., Zseby, T., Claise, B., and S. Zander,
>               "Requirements for IP Flow Information Export (IPFIX)",
>               RFC 3917, October 2004.
>=20
>    [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
>               Requirement Levels", BCP 14, RFC 2119, March 1997.
>=20
>=20
> Authors' Addresses
>=20
>    Elisa Boschi
>    Hitachi Europe
>    c/o ETH Zurich
>    Gloriastrasse 35
>    8092 Zurich
>    Switzerland
>=20
>    Phone: +41 44 632 70 57
>    Email: elisa.boschi@hitachi-eu.com
>=20
>=20
>=20
>=20
>=20
> Boschi & Trammell        Expires August 19, 2010               [Page =
36]
> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>=20
>=20
>    Brian Trammell
>    Hitachi Europe
>    c/o ETH Zurich
>    Gloriastrasse 35
>    8092 Zurich
>    Switzerland
>=20
>    Phone: +41 44 632 70 13
>    Email: brian.trammell@hitachi-eu.com
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
> Boschi & Trammell        Expires August 19, 2010               [Page =
37]


--
Lothar Braun
Chair for Network Architectures and Services (I8)
Department of Informatics
Technische Universit=E4t M=FCnchen
Boltzmannstr. 3, 85748 Garching bei M=FCnchen, Germany
Phone:  +49 89 289-18010       Fax: +49 89 289-18033
E-mail: braun@net.in.tum.de=20







From trammell@tik.ee.ethz.ch  Mon Apr 19 11:06:13 2010
Return-Path: 
X-Original-To: ipfix@core3.amsl.com
Delivered-To: ipfix@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 8EE0F3A6836 for ; Mon, 19 Apr 2010 11:06:13 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.299
X-Spam-Level: 
X-Spam-Status: No, score=-5.299 tagged_above=-999 required=5 tests=[AWL=1.300,  BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0MQCvNI5ohDO for ; Mon, 19 Apr 2010 11:06:11 -0700 (PDT)
Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by core3.amsl.com (Postfix) with ESMTP id 05E323A68C4 for ; Mon, 19 Apr 2010 11:06:10 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 62D8DD9349; Mon, 19 Apr 2010 20:06:00 +0200 (MEST)
X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch
Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id AGn-kkIuyiLA; Mon, 19 Apr 2010 20:06:00 +0200 (MEST)
Received: from [192.168.0.195] (77-57-164-35.dclient.hispeed.ch [77.57.164.35]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id 7E50DD9309; Mon, 19 Apr 2010 20:05:59 +0200 (MEST)
Mime-Version: 1.0 (Apple Message framework v1078)
Content-Type: text/plain; charset=us-ascii
From: Brian Trammell 
In-Reply-To: 
Date: Mon, 19 Apr 2010 20:05:58 +0200
Content-Transfer-Encoding: quoted-printable
Message-Id: <11413CE7-4BE3-43D2-A2D7-5B63C9BFC9C0@tik.ee.ethz.ch>
References: 
To: Lothar Braun 
X-Mailer: Apple Mail (2.1078)
Cc: IETF IPFIX Working Group 
Subject: Re: [IPFIX] WG last call on draft-ietf-ipfix-anon-02.txt
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Mon, 19 Apr 2010 18:06:13 -0000

Hi, Lothar!

Thanks for the comments! As follows tradition, comments thereon inline =
below. Will carry these comments into a subsequent anon version, =
sometime this week...

Cheers,

Brian

On Apr 19, 2010, at 3:46 PM, Lothar Braun wrote:

> Hi,
>=20
> I finally found some time to review the draft. Please find my comments =
inline.
>=20
> Best regards,
>  Lothar=20
>=20
>>=20
>>=20
>> 3.  Categorisation of Anonymisation Techniques
>>=20
>>=20
 
>>=20
>>   Countability compares the dimension of the anonymised space (N) to
>>   the dimension of the real space (M), and denotes how the count of
>>   unique values is preserved by the anonymisation function.  If the
>>   anonymised space is smaller than the real space, then the function =
is
>>   said to generalise the input, mapping more than one input point to
>>   each anonymous value (e.g., as with aggregation).  By definition,
>>   generalisation is not recoverable.
>=20
> It might be possible to recover the original generalised values by =
combining the anonymised data with other data. Example: The =
autoconfigured IPv6 address are truncated by 48 bits, but the flows =
contain the original MAC addresses.
>=20
> Maybe this should be stated here?

This is why we chose to call this operation "recovery" as opposed to =
"deanonymisation"... Recoverability refers only to whether information =
is destroyed by the transformation or not.

>=20
>>   If the dimensions of the anonymised and real spaces are the same,
>>   such that the count of unique values is preserved, then the =
function
>>   is said to be a direct substitution function.  If the dimension of
>>   the anonymised space is larger, such that each real value maps to a
>>   set of anonymised values, then the function is said to be a set
>>   substitution function.  Note that with set substitution functions,
>>   the sets of anonymised values are not necessarily disjoint.  Either
>>   direct or set substitution functions are said to be one-way if =
there
>>   exists no practical method for recovering the real data point from =
an
>>   anonymised one.

Okay, point. Here we talk about "practical methods". It turns out that =
if you're attacking an entire record, there are a lot of practical =
methods. However, we should be more exact that we're talking about =
single values of single fields in isolation...

>>   This classification is summarised in the table below.
>>=20
>>   =
+------------------------+-----------------+------------------------+
>>   | Recoverability /       | Recoverable     | Non-recoverable        =
|
>>   | Countability           |                 |                        =
|
>>   =
+------------------------+-----------------+------------------------+
>>   | N < M                  | N.A.            | Generalisation         =
|
>>   | N =3D M                  | Direct          | One-way Direct       =
  |
>>   |                        | Substitution    | Substitution           =
|
>>   | N > M                  | Set             | One-way Set            =
|
>>   |                        | Substitution    | Substitution           =
|
>>   =
+------------------------+-----------------+------------------------+
>>=20
>>=20
>> 4.  Anonymisation of IP Flow Data
>>=20
>>   Due to the restricted semantics of IP flow data, there is a
>>   relatively limited set of specific anonymisation techniques =
available
>>   on flow data, though each falls into the broad categories above.
>>   Each type of field that may commonly appear in a flow record may =
have
>>   its own applicable specific techniques.
>>=20
>>   While anonymisation is generally applied at the resolution of =
single
>>   fields within a flow record, attacks against anonymisation use =
entire
>>   flows and relationships between hosts and flows within a given data
>>=20
>>=20
>>=20
>> Boschi & Trammell        Expires August 19, 2010                [Page =
8]
>> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>>=20
>>=20
>>   set.  Therefore, fields which may not necessarily be identifying by
>>   themselves may be anonymised in order to increase the anonymity of
>>   the data set as a whole.
>>=20
>>   Of all the fields in an IP flow record, IP addresses are the most
>>   likely to be used to directly identify entities in the real world.
>>   Each IP address is associated with an interface on a network host,
>>   and can potentially be identified with a single user.  =
Additionally,
>>   IP addresses are structured identifiers; that is, partial IP =
address
>>   prefixes may be used to identify networks just as full IP addresses
>>   identify hosts.  This makes anonymisation of IP addresses
>>   particularly important.
>>=20
>>   Hardware addresses uniquely identify devices on the network; while
>>   they are not often available in traffic data collected at Layer 3,
>>   and cannot be used to locate devices within the network, some =
traces
>>   may contain sub-IP data including hardware address data.  Hardware
>>   addresses may be mappable to device serial numbers, and to the
>>   entities or individuals who purchased the devices, when combined =
with
>>   external databases.  They may also leak via IPv6 addresses in =
certain
>>   circumstances.  Therefore, hardware address anonymisation is also
>>   important.
>=20
> The other way around could be even worse (see my example above).

Point. Will mention.

>>   Port numbers identify abstract entities (applications) as opposed =
to
>>   real-world entities, but they can be used to classify hosts and =
user
>>   behavior.  Passive port fingerprinting, both of well-known and
>>   ephemeral ports, can be used to determine the operating system
>>   running on a host.  Relative data volumes by port can also be used =
to
>>   determine the host's function (workstation, web server, etc.); this
>>   information can be used to identify hosts and users.
>>=20
>>   While not identifiers in and of themselves, timestamps and counters
>>   can reveal the behavior of the hosts and users on a network.  Any
>>   given network activity is recognizable by a pattern of relative =
time
>>   differences and data volumes in the associated sequence of flows,
>>   even without host address information.  They can therefore be used =
to
>>   identify hosts and users.  Timestamps and counters are also
>>   vulnerable to traffic injection attacks, where traffic with a known
>>   pattern is injected into a network under measurement, and this
>>   pattern is later identified in the anonymised data set.
>>=20
>>   The simplest and most extreme form of anonymisation, which can be
>>   applied to any field of a flow record, is black-marker =
anonymisation,
>>   or complete deletion of a given field.  Note that black-marker
>>   anonymisation is equivalent to simply not exporting the field(s) in
>>   question.
>>=20
>>   While black-marker anonymisation completely protects the data in =
the
>>=20
>>=20
>>=20
>> Boschi & Trammell        Expires August 19, 2010                [Page =
9]
>> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>>=20
>>=20
>>   deleted fields from the risk of disclosure, it also reduces the
>>   utility of the anonymised data set as a whole.  Techniques that
>>   retain some information while reducing (though not eliminating) the
>>   disclosure risk will be extensively discussed in the following
>>   sections; note that the techniques specifically applicable to IP
>>   addresses, timestamps, ports, and counters will be discussed in
>>   separate sections.



>> 4.4.3.  Random Noise Addition
>>=20
>>   Random noise addition adds a random amount to a counter in each =
flow;
>>   this is used to keep relative magnitude information and minimize =
the
>>   disruption to size relationship information while avoiding
>>   fingerprinting attacks against anonymisation.  Note that there is =
no
>>   guarantee that random noise addition will maintain ranking order by =
a
>>   counter among members of a set.  Random noise addition is
>>   particularly useful when the derived analysis data will not be
>>   presented in such a way as to require the lower-order bits of the
>>   counters.
>=20
> I have problems understand the purpose of this anonymisation =
technique. How can it "keep relative magnitude information" but gives =
"no guarantee that random noise will maintain ranking order"? Isn't this =
a contradiction?

No. Say you have data points in the 10^7 range and other data points in =
the 5x10^6 range. Random noise of magnitude on the order of 10^4 keeps =
each group together without guaranteeing order of individual points; =
however, it completely destroys information in the 10^2 range.

> I have problems understanding the following:
>=20
> What is the difference between 4.4.3 and 4.4.1 with respect of the =
information that is contained in the lower-order bits of the counters? =
As far as I understand both anonymisation techniques remove the complete =
information from the lower-order bits. Did I get this wrong?

When the random noise magnitude is clipped to a bit boundary, these are =
more or less equivalent. When the random noise is normally distributed, =
it may cause higher order bits to change as well.=20

Practically, though, you're right, there's not much difference.


>>=20
>> 5.1.  Stability
>>=20
>>   Any given anonymisation technique may be applied with a varying =
range
>>   of stability.  Stability is important for assessing the =
comparability
>>   of anonymised information in different data sets, or in the same =
data
>>   set over different time periods.  In general, stability ranges from
>>   completely stable to completely unstable; however, note that the
>>   completely unstable case is indistinguishable from black-marker
>>   anonymisation.  A completely stable anonymisation will always map a
>>   given value in the real space to the same value in the anonymised
>>   space.  In practice, an anonymisation may also be stable for every
>>   data set published by an a particular producer to a particular
>>   consumer, stable for a stated time period within a dataset or =
across
>>   datasets, or stable only for a single data set.
>=20
> I had troubles understanding this (and had to read this section =
several times before I got it). Maybe one could first define stability =
and than talk about the varying range:
>=20
> 	A complete stable anonymisation will always map a given value
> 	in the real space to the same value in the anonymised space. A
> 	completely unstable case is indistinguishable from black-marker=20=

> 	anonymisation. Any given anonymisation technique may be ....
>=20
> Could this be a way to start the section?=20

Something like it, yes. THanks!

>=20
>>   If no information about stability is available, users of anonymised
>>   data may assume that the techniques used are stable across the =
entire
>>   dataset, but unstable across datasets. =20
>=20
> Is this a definition or "common practice"?

Here, a definition derived from common practice. So it should be a MAY, =
yes.



>>   While the Stability Class within the anonymisationFlags IE can be
>>   used to declare that a given anonymisation technique's mapping will
>>   remain stable across multiple sessions, each session MUST re-export
>>   the anonymisation Records along with the templates.
>=20
> I got totally confused about the Stability Class and the =
anonymisationFlag (they where not introduced at this point) and had no =
clue what you wanted to say with that. A clear case of too much =
information. Maybe this could be made easier like
>=20
> 	If a session is re-exported, all anonymisation records MUST be =
re-exported along with the templates.
>=20
> (You might want to turn that into proper English)

Yep. Confusing. Will do.



>>=20
>> 6.2.  Recommended Information Elements for Anonymisation Metadata
>>=20
>> 6.2.1.  informationElementIndex
>>=20
>>   Description:   A zero-based index of an Information Element
>>      referenced by informationElementId within a Template referenced =
by
>>      templateId; used to disambiguate scope for templates containing
>>      multiple identical Information Elements.
>>=20
>>   Abstract Data Type:   unsigned16
>>=20
>>   ElementId:   TBD3
>=20
> Is this meant to be "To Be Decided"?

To be defined; see IANA considerations.

>=20
>>   Status:   Proposed
>>=20
>> 6.2.2.  anonymisationFlags
>>=20
>>   Description:   A flag word describing specialized modifications to
>>      the anonymisation policy in effect for the anonymisation =
technique
>>      applied to a referenced Information Element within a referenced
>>      Template.  When flags are clear (0), the normal policy (as
>>      described by anonymisationTechnique) applies without =
modification.
>>=20
>>      MSB   14  13  12  11  10   9   8   7   6   5   4   3   2   1  =
LSB
>>      =
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
>>      |                Reserved                       |LOR|PmA|   SC  =
|
>>      =
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
>>=20
>>                            anonymisationFlags IE
>>=20
>>   =
+--------+----------+-----------------------------------------------+
>>   | bit(s) | name     | description                                   =
|
>>   | (LSB =3D |          |                                             =
  |
>>   | 0)     |          |                                               =
|
>>   =
+--------+----------+-----------------------------------------------+
>>   | 0-1    | SC       | Stability Class: see the Stability Class      =
|
>>   |        |          | table below, and section Section 5.1.         =
|
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>> Boschi & Trammell        Expires August 19, 2010               [Page =
20]
>> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>>=20
>>=20
>>   | 2      | PmA      | Perimeter Anonymisation: when set (1), source =
|
>=20
> By the time reading this, I had no clue what Petrimeter Anonymisation =
could mean (until reading 7.2.2). Maybe a reference to rfc 5103 could =
help?=10

It's in a subsequent section; I believe I've already fixed this in the =
working copy, but if not, will do....


>>   |        |          | address Information Elements are interpreted  =
|
>>   |        |          | as external addresses, and destination        =
|
>>   |        |          | address Information Elements are interpreted  =
|
>>   |        |          | as internal addresses, for the purposes of    =
|
>>   |        |          | associating anonymisationTechnique to         =
|
>>   |        |          | Information Elements.  MUST NOT be set when   =
|
>>   |        |          | associated with a non-endpoint (i.e., source- =
|
>>   |        |          | or destination-) Information Element.  SHOULD =
|
>>   |        |          | be consistent within a record (i.e., if a     =
|
>>   |        |          | source- Information Element has this flag     =
|
>>   |        |          | set, the corresponding destination- element   =
|
>>   |        |          | SHOULD have this flag set, and vice-versa.)   =
|
>>   | 3      | LOR      | Low-Order Unchanged: when set (1), the        =
|
>>   |        |          | low-order bits of the anonymised Information  =
|
>>   |        |          | Element contain real data.  This modification =
|
>>   |        |          | is intended for the anonymisation of          =
|
>>   |        |          | network-level addresses while leaving         =
|
>>   |        |          | host-level addresses intact in order to       =
|
>>   |        |          | preserve host level-structure, which could    =
|
>>   |        |          | otherwise be used to reverse anonymisation.   =
|
>>   |        |          | MUST NOT be set when associated with a        =
|
>>   |        |          | truncation-based anonymisationTechnique.      =
|
>>   | 4-15   | Reserved | Reserved for future use: SHOULD be cleared    =
|
>>   |        |          | (0) by the Exporting Process and MUST be      =
|
>>   |        |          | ignored by the Collecting Process.            =
|
>>   =
+--------+----------+-----------------------------------------------+
>>=20
>>      The Stability Class portion of this flags word describes the
>>      stability class of the anonymisation technique applied to a
>>      referenced Information Element within a referenced Template.
>>      Stability classes refer to the stability of the parameters of =
the
>>      anonymisation technique, and therefore the comparability of the
>>      mapping between the real and anonymised values over time.  This
>>      determines which anonymised datasets may be compared with each
>>      other.  Values are as follows:
>>=20



>> 6.2.3.  anonymisationTechnique
>>=20
>>   Description:   A description of the anonymisation technique applied
>>      to a referenced Information Element within a referenced =
Template.
>>      Each technique may be applicable only to certain Information
>>      Elements and recommended only for certain Infomation Elements;
>>      these restrictions are noted in the table below.
>>=20
>>=20
>>=20
>>=20
>>=20
>> Boschi & Trammell        Expires August 19, 2010               [Page =
22]
>> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>>=20
>>=20
>>   =
+-------+---------------------------+-----------------+-------------+
>>   | Value | Description               | Applicable to   | Recommended =
|
>>   |       |                           |                 | for         =
|
>>   =
+-------+---------------------------+-----------------+-------------+
>>   | 0     | Undefined: the Exporting  | all             | all         =
|
>>   |       | Process makes no          |                 |             =
|
>>   |       | representation as to      |                 |             =
|
>>   |       | whether the defined field |                 |             =
|
>>   |       | is anonymised or not.     |                 |             =
|
>>   |       | While the Collecting      |                 |             =
|
>>   |       | Process MAY assume that   |                 |             =
|
>>   |       | the field is not          |                 |             =
|
>>   |       | anonymised, it is not     |                 |             =
|
>>   |       | guaranteed not to be.     |                 |             =
|
>>   |       | This is the default       |                 |             =
|
>>   |       | anonymisation technique.  |                 |             =
|
>>   | 1     | None: the values exported | all             | all         =
|
>>   |       | are real.                 |                 |             =
|
>>   | 2     | Precision                 | all             | all         =
|
>>   |       | Degradation/Truncation:   |                 |             =
|
>>   |       | the values exported are   |                 |             =
|
>>   |       | anonymised using simple   |                 |             =
|
>>   |       | precision degradation or  |                 |             =
|
>>   |       | truncation.  The new      |                 |             =
|
>>   |       | precision or number of    |                 |             =
|
>>   |       | truncated bits is         |                 |             =
|
>>   |       | implicit in the exported  |                 |             =
|
>>   |       | data, and can be deduced  |                 |             =
|
>>   |       | by the Collecting         |                 |             =
|
>>   |       | Process.                  |                 |             =
|
>=20
> How can I deduce that as a collector? What is implicit? Is an =
IPv4-Adress encoded in 3 bytes instead of 4 if I do 8-bit truncation?

No, but the CP or something downstream can notice how many low order =
bits are zero. Usually, there's a human analyst or researcher who will =
notice this.=20



>>=20
>> 7.2.3.  Anonymisation of Header Data
>>=20
>>   Each IPFIX Message contains a Message Header; within this Message
>>   Header are contained two fields which may be used to break certain
>>   anonymisation techniques: the Export Time, and the Observation =
Domain
>>=20
>>=20
>>=20
>> Boschi & Trammell        Expires August 19, 2010               [Page =
28]
>> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>>=20
>>=20
>>   ID
>>=20
>>   Export of IPFIX Messages containing anonymised timestamp data where
>>   the original Export Time Message header has some relationship to =
the
>>   anonymised timestamps SHOULD anonymise the Export Time header field
>>   using an equivalent technique, if possible.  Otherwise, =
relationships
>>   between export and flow time could be used to partially or totally
>>   reverse timestamp anonymisation.
>=20
> Hmm, just thinking ...
>=20
> Could anonymising the Export Time reveal more information than not =
anonymising this in some cases?
>=20
> Let ET be the original Export Time, AET be the anonymized Export Time, =
CT the time at with the collector received the Message and RTT the =
round-trip time
>=20
> ET =3D CT - RTT
>=20
> If the RTT is nearly stable, the original ET can be easily decoded =
revealing one time mapping If you anonymize timestamps by adding a fixed =
value, you revealed the anonymisation.
>=20
> Could the draft recommend to use different anonymization techniques =
for the flow data and the export time?

Good point. Will cover this point...



>>  In order to export anonymisation records for this template and
>>   policy, first, the Anonymisation Options Template shown in figure
>>   Figure 6 is exported.  For this example, the optional
>>   privateEnterpriseNumber and informationElementIndex Information
>>   Elements are omitted, because they are not used.
>>=20
>>                              1                   2                   =
3
>>          0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 =
0 1
>>         =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>>         |          Set ID =3D 3           |          Length =3D  26   =
      |
>>         =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>>         |      Template ID =3D 257        |        Field Count =3D 4  =
      |
>>         =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>>         |    Scope Field Count =3D 2      |0| templateID              =
346 |
>>         =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>=20
> Why is the templateID 346? Shouldn't it be 256?

The information element number of information element templateID is 346.

>=20
>>         |       Field Length =3D 2        |0| informationElementId    =
303 |
>>         =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>=20
> informationElementID  =3D=3D TBD3?
>=20
>>         |       Field Length =3D 2        |0| anonymisationFlags      =
339 |
>>         =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>=20
> anonymisationFlgas =3D=3D TBD1?
>=20
>>         |       Field Length =3D 2        |0| anonymisationTechnique  =
344 |
>>         =
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>=20
> anonymisationTechnique =3D TBD2?

These are fixed in the current working copy...



From braun@net.in.tum.de  Mon Apr 19 23:14:39 2010
Return-Path: 
X-Original-To: ipfix@core3.amsl.com
Delivered-To: ipfix@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 442583A6AA4 for ; Mon, 19 Apr 2010 23:14:39 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: 0.351
X-Spam-Level: 
X-Spam-Status: No, score=0.351 tagged_above=-999 required=5 tests=[BAYES_50=0.001, HELO_EQ_DE=0.35]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id j8+MjC77Vhx8 for ; Mon, 19 Apr 2010 23:14:38 -0700 (PDT)
Received: from mail-out1.informatik.tu-muenchen.de (mail-out1.informatik.tu-muenchen.de [131.159.0.8]) by core3.amsl.com (Postfix) with ESMTP id BADDA3A6A92 for ; Mon, 19 Apr 2010 23:14:36 -0700 (PDT)
Received: from [192.168.0.4] (e181074102.adsl.alicedsl.de [85.181.74.102]) by mail.net.in.tum.de (Postfix) with ESMTPSA id 8686C206492E for ; Tue, 20 Apr 2010 08:15:20 +0200 (CEST)
Content-Type: text/plain; charset=iso-8859-1
Mime-Version: 1.0 (Apple Message framework v1078)
From: Lothar Braun 
In-Reply-To: <11413CE7-4BE3-43D2-A2D7-5B63C9BFC9C0@tik.ee.ethz.ch>
Date: Tue, 20 Apr 2010 08:14:20 +0200
Content-Transfer-Encoding: quoted-printable
Message-Id: <73DE58B6-50D4-4691-8D37-C282FF0CE349@net.in.tum.de>
References:  <11413CE7-4BE3-43D2-A2D7-5B63C9BFC9C0@tik.ee.ethz.ch>
To: IETF IPFIX Working Group 
X-Mailer: Apple Mail (2.1078)
X-Virus-Scanned: ClamAV using ClamSMTP
Subject: Re: [IPFIX] WG last call on draft-ietf-ipfix-anon-02.txt
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Tue, 20 Apr 2010 06:14:39 -0000

Hi Brian,

please find my comments inline, irrelevant parts snipped.

Best regards,
  Lothar

On Apr 19, 2010, at 8:05 PM, Brian Trammell wrote:
> 
>=20
>>> 4.4.3.  Random Noise Addition
>>>=20
>>>  Random noise addition adds a random amount to a counter in each =
flow;
>>>  this is used to keep relative magnitude information and minimize =
the
>>>  disruption to size relationship information while avoiding
>>>  fingerprinting attacks against anonymisation.  Note that there is =
no
>>>  guarantee that random noise addition will maintain ranking order by =
a
>>>  counter among members of a set.  Random noise addition is
>>>  particularly useful when the derived analysis data will not be
>>>  presented in such a way as to require the lower-order bits of the
>>>  counters.
>>=20
>> I have problems understand the purpose of this anonymisation =
technique. How can it "keep relative magnitude information" but gives =
"no guarantee that random noise will maintain ranking order"? Isn't this =
a contradiction?
>=20
> No. Say you have data points in the 10^7 range and other data points =
in the 5x10^6 range. Random noise of magnitude on the order of 10^4 =
keeps each group together without guaranteeing order of individual =
points; however, it completely destroys information in the 10^2 range.
>=20
>> I have problems understanding the following:
>>=20
>> What is the difference between 4.4.3 and 4.4.1 with respect of the =
information that is contained in the lower-order bits of the counters? =
As far as I understand both anonymisation techniques remove the complete =
information from the lower-order bits. Did I get this wrong?
>=20
> When the random noise magnitude is clipped to a bit boundary, these =
are more or less equivalent. When the random noise is normally =
distributed, it may cause higher order bits to change as well.=20
>=20
> Practically, though, you're right, there's not much difference.

I see, thank you for explaining.=20

> 
>=20
>>> 6.2.3.  anonymisationTechnique
>>>=20
>>>  Description:   A description of the anonymisation technique applied
>>>     to a referenced Information Element within a referenced =
Template.
>>>     Each technique may be applicable only to certain Information
>>>     Elements and recommended only for certain Infomation Elements;
>>>     these restrictions are noted in the table below.
>>>=20
>>>=20
>>>=20
>>>=20
>>>=20
>>> Boschi & Trammell        Expires August 19, 2010               [Page =
22]
>>> Internet-Draft        IP Flow Anonymisation Support        February =
2010
>>>=20
>>>=20
>>>  =
+-------+---------------------------+-----------------+-------------+
>>>  | Value | Description               | Applicable to   | Recommended =
|
>>>  |       |                           |                 | for         =
|
>>>  =
+-------+---------------------------+-----------------+-------------+
>>>  | 0     | Undefined: the Exporting  | all             | all         =
|
>>>  |       | Process makes no          |                 |             =
|
>>>  |       | representation as to      |                 |             =
|
>>>  |       | whether the defined field |                 |             =
|
>>>  |       | is anonymised or not.     |                 |             =
|
>>>  |       | While the Collecting      |                 |             =
|
>>>  |       | Process MAY assume that   |                 |             =
|
>>>  |       | the field is not          |                 |             =
|
>>>  |       | anonymised, it is not     |                 |             =
|
>>>  |       | guaranteed not to be.     |                 |             =
|
>>>  |       | This is the default       |                 |             =
|
>>>  |       | anonymisation technique.  |                 |             =
|
>>>  | 1     | None: the values exported | all             | all         =
|
>>>  |       | are real.                 |                 |             =
|
>>>  | 2     | Precision                 | all             | all         =
|
>>>  |       | Degradation/Truncation:   |                 |             =
|
>>>  |       | the values exported are   |                 |             =
|
>>>  |       | anonymised using simple   |                 |             =
|
>>>  |       | precision degradation or  |                 |             =
|
>>>  |       | truncation.  The new      |                 |             =
|
>>>  |       | precision or number of    |                 |             =
|
>>>  |       | truncated bits is         |                 |             =
|
>>>  |       | implicit in the exported  |                 |             =
|
>>>  |       | data, and can be deduced  |                 |             =
|
>>>  |       | by the Collecting         |                 |             =
|
>>>  |       | Process.                  |                 |             =
|
>>=20
>> How can I deduce that as a collector? What is implicit? Is an =
IPv4-Adress encoded in 3 bytes instead of 4 if I do 8-bit truncation?
>=20
> No, but the CP or something downstream can notice how many low order =
bits are zero. Usually, there's a human analyst or researcher who will =
notice this.=20

Hmm, maybe this should be noted somewhere. Especially the human analyst =
or researcher part.

Having an automatic implementation that counts the zero low order bits =
may fail (even humans may).
For example if you have an anonymisation process that truncates IPv4 =
addresses by 8 bit.=20

Lets than say we have the flows

flow 1: 10.0.0.1-> 10.0.0.2
flow 2: 192.168.0.1 -> 192.168.0.2

results in=20

anonymized flow 1: 10.0.0.0 -> 10.0.0.0
anonymized flow 2: 192.168.0.0 -> 192.168.0.0

Parsing both flow records for zeros results in wrong truncation bit =
count.

I'm wondering if there is a really reliable way for anyone to be sure =
that the correct truncation length has been determined correctly, if the =
truncation bit count is larger (or equal?) to 1 bit. Do I overlook =
something?

Best regards,
  Lothar

--
Lothar Braun
Chair for Network Architectures and Services (I8)
Department of Informatics
Technische Universit=E4t M=FCnchen
Boltzmannstr. 3, 85748 Garching bei M=FCnchen, Germany
Phone:  +49 89 289-18010       Fax: +49 89 289-18033
E-mail: braun@net.in.tum.de=20







From trammell@tik.ee.ethz.ch  Tue Apr 20 01:49:56 2010
Return-Path: 
X-Original-To: ipfix@core3.amsl.com
Delivered-To: ipfix@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 817683A6968 for ; Tue, 20 Apr 2010 01:49:56 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -4.149
X-Spam-Level: 
X-Spam-Status: No, score=-4.149 tagged_above=-999 required=5 tests=[AWL=0.150,  BAYES_00=-2.599, MANGLED_SAVELE=2.3, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4wxcHL4mtBXU for ; Tue, 20 Apr 2010 01:49:55 -0700 (PDT)
Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by core3.amsl.com (Postfix) with ESMTP id 557DC3A677D for ; Tue, 20 Apr 2010 01:49:54 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 77D11D9377; Tue, 20 Apr 2010 10:49:44 +0200 (MEST)
X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch
Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id Xv7jFX3oOa2l; Tue, 20 Apr 2010 10:49:44 +0200 (MEST)
Received: from pb-10072.ethz.ch (pb-10072.ethz.ch [82.130.103.195]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id 473B2D9367; Tue, 20 Apr 2010 10:49:44 +0200 (MEST)
Mime-Version: 1.0 (Apple Message framework v1078)
Content-Type: text/plain; charset=us-ascii
From: Brian Trammell 
In-Reply-To: <73DE58B6-50D4-4691-8D37-C282FF0CE349@net.in.tum.de>
Date: Tue, 20 Apr 2010 10:49:43 +0200
Content-Transfer-Encoding: quoted-printable
Message-Id: <744A3FB6-7EC8-4A89-B147-BB6386E4D2E0@tik.ee.ethz.ch>
References:  <11413CE7-4BE3-43D2-A2D7-5B63C9BFC9C0@tik.ee.ethz.ch> <73DE58B6-50D4-4691-8D37-C282FF0CE349@net.in.tum.de>
To: Lothar Braun 
X-Mailer: Apple Mail (2.1078)
Cc: IETF IPFIX Working Group 
Subject: Re: [IPFIX] WG last call on draft-ietf-ipfix-anon-02.txt
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Tue, 20 Apr 2010 08:49:56 -0000

Hi, Lothar,

One answer, inline, below,

Cheers,

Brian

On Apr 20, 2010, at 8:14 AM, Lothar Braun wrote:

> Hi Brian,
>=20
> please find my comments inline, irrelevant parts snipped.
>=20
> Best regards,
>  Lothar
>=20
> On Apr 19, 2010, at 8:05 PM, Brian Trammell wrote:
>>>> | 2     | Precision                 | all             | all         =
|
>>>> |       | Degradation/Truncation:   |                 |             =
|
>>>> |       | the values exported are   |                 |             =
|
>>>> |       | anonymised using simple   |                 |             =
|
>>>> |       | precision degradation or  |                 |             =
|
>>>> |       | truncation.  The new      |                 |             =
|
>>>> |       | precision or number of    |                 |             =
|
>>>> |       | truncated bits is         |                 |             =
|
>>>> |       | implicit in the exported  |                 |             =
|
>>>> |       | data, and can be deduced  |                 |             =
|
>>>> |       | by the Collecting         |                 |             =
|
>>>> |       | Process.                  |                 |             =
|
>>>=20
>>> How can I deduce that as a collector? What is implicit? Is an =
IPv4-Adress encoded in 3 bytes instead of 4 if I do 8-bit truncation?
>>=20
>> No, but the CP or something downstream can notice how many low order =
bits are zero. Usually, there's a human analyst or researcher who will =
notice this.=20
>=20
> Hmm, maybe this should be noted somewhere. Especially the human =
analyst or researcher part.

Will do.

> Having an automatic implementation that counts the zero low order bits =
may fail (even humans may).
> For example if you have an anonymisation process that truncates IPv4 =
addresses by 8 bit.=20
>=20
> Lets than say we have the flows
>=20
> flow 1: 10.0.0.1-> 10.0.0.2
> flow 2: 192.168.0.1 -> 192.168.0.2
>=20
> results in=20
>=20
> anonymized flow 1: 10.0.0.0 -> 10.0.0.0
> anonymized flow 2: 192.168.0.0 -> 192.168.0.0
>=20
> Parsing both flow records for zeros results in wrong truncation bit =
count.
>=20
> I'm wondering if there is a really reliable way for anyone to be sure =
that the correct truncation length has been determined correctly, if the =
truncation bit count is larger (or equal?) to 1 bit. Do I overlook =
something?

If you're dealing with a truncated data set, basically, the truncation =
for an IE happens at or below the lowest set bit in each instance of the =
set. In real data sets, covering networks with more than one host, with =
common truncations (people like /8, /16, and /24), this will tend to be =
obvious. However, from a data analysis perspective, does it really =
matter where the truncation happens?=20

Let's take your data set. It contains only two flows. It's missing its =
last 19 bits de facto (it's a /13). While I will treat this differently =
knowing whether it's truncated or not, what am I going to do differently =
whether it's a /13 (what the last-bit algorithm says), a /16 (what I as =
a human operator would expect based on that truncation), or a /24 (what =
you say you actually did at the truncation point)?

Perhaps something of this discussion should be carried into the draft.

Cheers,

Brian=

From root@core3.amsl.com  Tue Apr 20 08:30:02 2010
Return-Path: 
X-Original-To: ipfix@ietf.org
Delivered-To: ipfix@core3.amsl.com
Received: by core3.amsl.com (Postfix, from userid 0) id 70F513A69EB; Tue, 20 Apr 2010 08:30:01 -0700 (PDT)
From: Internet-Drafts@ietf.org
To: i-d-announce@ietf.org
Content-Type: Multipart/Mixed; Boundary="NextPart"
Mime-Version: 1.0
Message-Id: <20100420153002.70F513A69EB@core3.amsl.com>
Date: Tue, 20 Apr 2010 08:30:02 -0700 (PDT)
Cc: ipfix@ietf.org
Subject: [IPFIX] I-D Action:draft-ietf-ipfix-mediators-framework-06.txt
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Tue, 20 Apr 2010 15:30:02 -0000

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the IP Flow Information Export Working Group of the IETF.


	Title           : IPFIX Mediation: Framework
	Author(s)       : A. Kobayashi, et al.
	Filename        : draft-ietf-ipfix-mediators-framework-06.txt
	Pages           : 33
	Date            : 2010-04-20

This document describes a framework for IPFIX Mediation.  This
framework extends the IPFIX reference model by defining the IPFIX
Mediator components.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-ipfix-mediators-framework-06.txt

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Message/External-body;
	name="draft-ietf-ipfix-mediators-framework-06.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID: <2010-04-20082657.I-D@ietf.org>


--NextPart--

From akoba@nttv6.net  Tue Apr 20 09:02:39 2010
Return-Path: 
X-Original-To: ipfix@core3.amsl.com
Delivered-To: ipfix@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 832A03A69DF for ; Tue, 20 Apr 2010 09:02:39 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.188
X-Spam-Level: 
X-Spam-Status: No, score=-2.188 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HOST_MISMATCH_COM=0.311, RDNS_DYNAMIC=0.1]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id j356z2PQOb9s for ; Tue, 20 Apr 2010 09:02:38 -0700 (PDT)
Received: from mail.nttv6.net (mail.nttv6.net [IPv6:2001:fa8::25]) by core3.amsl.com (Postfix) with ESMTP id 1A2E13A6AE8 for ; Tue, 20 Apr 2010 09:02:34 -0700 (PDT)
Received: from [127.0.0.1] (dhcp-3-152.nttv6.com [192.47.163.152]) by mail.nttv6.net (8.14.4/8.14.3) with ESMTP id o3KG2OSx049340 for ; Wed, 21 Apr 2010 01:02:25 +0900 (JST) (envelope-from akoba@nttv6.net)
Date: Wed, 21 Apr 2010 00:55:45 +0900
From: Atsushi Kobayashi 
To: ipfix@ietf.org
In-Reply-To: <20100420153002.70F513A69EB@core3.amsl.com>
References: <20100420153002.70F513A69EB@core3.amsl.com>
Message-Id: <20100421005502.40FB.17391CF2@nttv6.net>
MIME-Version: 1.0
Content-Type: text/plain; charset="US-ASCII"
Content-Transfer-Encoding: 7bit
X-Mailer: Becky! ver. 2.50.05 [ja] (Unregistered)
X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.3 (mail.nttv6.net [192.16.178.5]); Wed, 21 Apr 2010 01:02:25 +0900 (JST)
Subject: Re: [IPFIX] I-D Action:draft-ietf-ipfix-mediators-framework-06.txt
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Tue, 20 Apr 2010 16:02:39 -0000

Hi all,

I submitted both of new mediation drafts after Anaheim meeting.
Problem statements draft is already a new version, and then it can go to
IESG review. In framework draft, we can start WG last call.

IPFIX chairs, could you please call it?

Regards,
Atsushi

On Tue, 20 Apr 2010 08:30:02 -0700 (PDT)
Internet-Drafts@ietf.org wrote:

> A New Internet-Draft is available from the on-line Internet-Drafts directories.
> This draft is a work item of the IP Flow Information Export Working Group of the IETF.
> 
> 
> 	Title           : IPFIX Mediation: Framework
> 	Author(s)       : A. Kobayashi, et al.
> 	Filename        : draft-ietf-ipfix-mediators-framework-06.txt
> 	Pages           : 33
> 	Date            : 2010-04-20
> 
> This document describes a framework for IPFIX Mediation.  This
> framework extends the IPFIX reference model by defining the IPFIX
> Mediator components.
> 
> A URL for this Internet-Draft is:
> http://www.ietf.org/internet-drafts/draft-ietf-ipfix-mediators-framework-06.txt
> 
> Internet-Drafts are also available by anonymous FTP at:
> ftp://ftp.ietf.org/internet-drafts/
> 
> Below is the data which will enable a MIME compliant mail reader
> implementation to automatically retrieve the ASCII version of the
> Internet-Draft.

--- 
Atsushi KOBAYASHI  
NTT Information Sharing Platform Lab.
tel:+81-(0)422-59-3978 fax:+81-(0)422-59-5637



From wwwrun@rfc-editor.org  Tue Apr 20 16:57:13 2010
Return-Path: 
X-Original-To: ipfix@core3.amsl.com
Delivered-To: ipfix@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 2C8993A68B6; Tue, 20 Apr 2010 16:57:13 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.057
X-Spam-Level: 
X-Spam-Status: No, score=-2.057 tagged_above=-999 required=5 tests=[AWL=-0.057, BAYES_00=-2.599, J_CHICKENPOX_93=0.6, NO_RELAYS=-0.001]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id x5FS4hFaS4Pr; Tue, 20 Apr 2010 16:57:12 -0700 (PDT)
Received: from rfc-editor.org (rfc-editor.org [IPv6:2001:1890:1112:1::2f]) by core3.amsl.com (Postfix) with ESMTP id 832B13A689C; Tue, 20 Apr 2010 16:57:12 -0700 (PDT)
Received: by rfc-editor.org (Postfix, from userid 30) id D822CE07D3; Tue, 20 Apr 2010 16:57:03 -0700 (PDT)
To: ietf-announce@ietf.org, rfc-dist@rfc-editor.org
From: rfc-editor@rfc-editor.org
Message-Id: <20100420235703.D822CE07D3@rfc-editor.org>
Date: Tue, 20 Apr 2010 16:57:03 -0700 (PDT)
Cc: ipfix@ietf.org, rfc-editor@rfc-editor.org
Subject: [IPFIX] RFC 5815 on Definitions of Managed Objects for IP Flow Information Export
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Tue, 20 Apr 2010 23:57:13 -0000

A new Request for Comments is now available in online RFC libraries.

        
        RFC 5815

        Title:      Definitions of Managed Objects for 
                    IP Flow Information Export 
        Author:     T. Dietz, Ed.,
                    A. Kobayashi, B. Claise,
                    G. Muenz
        Status:     Standards Track
        Stream:     IETF
        Date:       April 2010
        Mailbox:    Thomas.Dietz@nw.neclab.eu, 
                    akoba@nttv6.net, 
                    bclaise@cisco.com,  muenz@net.in.tum.de
        Pages:      64
        Characters: 131258
        Updates/Obsoletes/SeeAlso:   None

        I-D Tag:    draft-ietf-ipfix-mib-10.txt

        URL:        http://www.rfc-editor.org/rfc/rfc5815.txt

This document defines managed objects for IP Flow Information eXport
(IPFIX).  These objects provide information for monitoring IPFIX
Exporters and IPFIX Collectors including the basic configuration
information. [STANDARDS TRACK]

This document is a product of the IP Flow Information Export Working Group of the IETF.

This is now a Proposed Standard Protocol.

STANDARDS TRACK: This document specifies an Internet standards track
protocol for the Internet community,and requests discussion and suggestions
for improvements.  Please refer to the current edition of the Internet
Official Protocol Standards (STD 1) for the standardization state and
status of this protocol.  Distribution of this memo is unlimited.

This announcement is sent to the IETF-Announce and rfc-dist lists.
To subscribe or unsubscribe, see
  http://www.ietf.org/mailman/listinfo/ietf-announce
  http://mailman.rfc-editor.org/mailman/listinfo/rfc-dist

For searching the RFC series, see http://www.rfc-editor.org/rfcsearch.html.
For downloading RFCs, see http://www.rfc-editor.org/rfc.html.

Requests for special distribution should be addressed to either the
author of the RFC in question, or to rfc-editor@rfc-editor.org.  Unless
specifically noted otherwise on the RFC itself, all RFCs are for
unlimited distribution.


The RFC Editor Team
Association Management Solutions, LLC



From braun@net.in.tum.de  Wed Apr 21 03:36:30 2010
Return-Path: 
X-Original-To: ipfix@core3.amsl.com
Delivered-To: ipfix@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 0BDA33A6358 for ; Wed, 21 Apr 2010 03:36:30 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: 1.823
X-Spam-Level: *
X-Spam-Status: No, score=1.823 tagged_above=-999 required=5 tests=[AWL=-0.872,  BAYES_50=0.001, DATE_IN_PAST_03_06=0.044, HELO_EQ_DE=0.35, MANGLED_SAVELE=2.3]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RqVcbxBwiZxp for ; Wed, 21 Apr 2010 03:36:27 -0700 (PDT)
Received: from mail-out1.informatik.tu-muenchen.de (mail-out1.informatik.tu-muenchen.de [131.159.0.8]) by core3.amsl.com (Postfix) with ESMTP id 431893A65A6 for ; Wed, 21 Apr 2010 03:36:26 -0700 (PDT)
Received: from honshu.net.in.tum.de (honshu.net.in.tum.de [131.159.20.100]) by mail.net.in.tum.de (Postfix) with ESMTPSA id 2BA2F2243D0C for ; Wed, 21 Apr 2010 12:36:34 +0200 (CEST)
Content-Type: text/plain; charset=iso-8859-1
Mime-Version: 1.0 (Apple Message framework v1078)
From: Lothar Braun 
In-Reply-To: <744A3FB6-7EC8-4A89-B147-BB6386E4D2E0@tik.ee.ethz.ch>
Date: Wed, 21 Apr 2010 09:32:55 +0200
Content-Transfer-Encoding: quoted-printable
Message-Id: 
References:  <11413CE7-4BE3-43D2-A2D7-5B63C9BFC9C0@tik.ee.ethz.ch> <73DE58B6-50D4-4691-8D37-C282FF0CE349@net.in.tum.de> <744A3FB6-7EC8-4A89-B147-BB6386E4D2E0@tik.ee.ethz.ch>
To: IETF IPFIX Working Group 
X-Mailer: Apple Mail (2.1078)
X-Virus-Scanned: ClamAV using ClamSMTP
Subject: Re: [IPFIX] WG last call on draft-ietf-ipfix-anon-02.txt
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Wed, 21 Apr 2010 10:36:30 -0000

Hi Brian,

On Apr 20, 2010, at 10:49 AM, Brian Trammell wrote:
>> Having an automatic implementation that counts the zero low order =
bits may fail (even humans may).
>> For example if you have an anonymisation process that truncates IPv4 =
addresses by 8 bit.=20
>>=20
>> Lets than say we have the flows
>>=20
>> flow 1: 10.0.0.1-> 10.0.0.2
>> flow 2: 192.168.0.1 -> 192.168.0.2
>>=20
>> results in=20
>>=20
>> anonymized flow 1: 10.0.0.0 -> 10.0.0.0
>> anonymized flow 2: 192.168.0.0 -> 192.168.0.0
>>=20
>> Parsing both flow records for zeros results in wrong truncation bit =
count.
>>=20
>> I'm wondering if there is a really reliable way for anyone to be sure =
that the correct truncation length has been determined correctly, if the =
truncation bit count is larger (or equal?) to 1 bit. Do I overlook =
something?
>=20
> If you're dealing with a truncated data set, basically, the truncation =
for an IE happens at or below the lowest set bit in each instance of the =
set. In real data sets, covering networks with more than one host, with =
common truncations (people like /8, /16, and /24), this will tend to be =
obvious. However, from a data analysis perspective, does it really =
matter where the truncation happens?=20
>=20
> Let's take your data set. It contains only two flows. It's missing its =
last 19 bits de facto (it's a /13). While I will treat this differently =
knowing whether it's truncated or not, what am I going to do differently =
whether it's a /13 (what the last-bit algorithm says), a /16 (what I as =
a human operator would expect based on that truncation), or a /24 (what =
you say you actually did at the truncation point)?

Hmm, I think there are two different points we have to look at:

1.) Can the number of truncated bits be derived from the data?

I think it will probably work in most of the cases because people will =
probably stick with /8, /16, /24 etc.=20
But it is not possible to do this in every case (e.g. the example I gave =
you) and you can almost never be sure if your guess on the truncation =
length is correct.=20

If you try to guestimate the number of truncated bits, you should have a =
look at the complete trace in order to minimize the possibility of =
guessing a wrong value.

2.) Is the number of truncated bits important or useful for any task?

I have no idea but as a user of an anonymized trace, I'd like to have as =
much information about the anonymisation as I can have about it.

Random brainstorming: Someone could have the idea to insert IP addresses =
into the blacked-out address space (e.g. because his/her analysis =
requires having full IP addresses). Knowing how many bits where =
truncated could be important in this case.

I would be fine with either of

1.) The draft says the number of truncated bits can be guestimated from =
the data. But there is no guarantee that the estimation is correct
2.) The draft introduces a way of explicitly encoding the number of =
truncated bits.

Best regards,
  Lothar

--
Lothar Braun
Chair for Network Architectures and Services (I8)
Department of Informatics
Technische Universit=E4t M=FCnchen
Boltzmannstr. 3, 85748 Garching bei M=FCnchen, Germany
Phone:  +49 89 289-18010       Fax: +49 89 289-18033
E-mail: braun@net.in.tum.de=20







From trammell@tik.ee.ethz.ch  Wed Apr 21 23:28:32 2010
Return-Path: 
X-Original-To: ipfix@core3.amsl.com
Delivered-To: ipfix@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 2F5B33A69D5 for ; Wed, 21 Apr 2010 23:28:32 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.999
X-Spam-Level: 
X-Spam-Status: No, score=-3.999 tagged_above=-999 required=5 tests=[BAYES_50=0.001, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GnsktM-xFSer for ; Wed, 21 Apr 2010 23:28:31 -0700 (PDT)
Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by core3.amsl.com (Postfix) with ESMTP id B3FD23A6990 for ; Wed, 21 Apr 2010 23:28:21 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id E7E1CD9384; Thu, 22 Apr 2010 08:28:09 +0200 (MEST)
X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch
Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 9Erx+yirBUj0; Thu, 22 Apr 2010 08:28:09 +0200 (MEST)
Received: from [10.0.1.11] (80-218-65-112.dclient.hispeed.ch [80.218.65.112]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id 7616DD9382; Thu, 22 Apr 2010 08:28:09 +0200 (MEST)
Mime-Version: 1.0 (Apple Message framework v1078)
Content-Type: text/plain; charset=us-ascii
From: Brian Trammell 
In-Reply-To: 
Date: Thu, 22 Apr 2010 08:28:08 +0200
Content-Transfer-Encoding: quoted-printable
Message-Id: <596D46DA-914F-4308-9778-2ED5F3E2FB7B@tik.ee.ethz.ch>
References:  <11413CE7-4BE3-43D2-A2D7-5B63C9BFC9C0@tik.ee.ethz.ch> <73DE58B6-50D4-4691-8D37-C282FF0CE349@net.in.tum.de> <744A3FB6-7EC8-4A89-B147-BB6386E4D2E0@tik.ee.ethz.ch> 
To: Lothar Braun 
X-Mailer: Apple Mail (2.1078)
Cc: IETF IPFIX Working Group 
Subject: Re: [IPFIX] WG last call on draft-ietf-ipfix-anon-02.txt
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Thu, 22 Apr 2010 06:28:32 -0000

Hi, Lothar,

Good points. I'll reply to them backwards...

On Apr 21, 2010, at 9:32 AM, Lothar Braun wrote:

> 2.) Is the number of truncated bits important or useful for any task?
>=20
> I have no idea but as a user of an anonymized trace, I'd like to have =
as much information about the anonymisation as I can have about it.

As the user of an anonymised trace, I'd like to have the original data. =
;)=20

> Random brainstorming: Someone could have the idea to insert IP =
addresses into the blacked-out address space (e.g. because his/her =
analysis requires having full IP addresses).

This would of course introduce errors into the data, unless a fair =
amount of work was done to differentiate hosts in the anonymised trace. =
However, truncation is only done in cases in which the publisher of the =
data does not want downstream users to be able to distinguish hosts =
(otherwise, the addresses would be reverse truncated, or permuted, or =
something). So, your example here is in essence an attack against =
truncation.

> Knowing how many bits where truncated could be important in this case.


By the nature of the data, you know (mostly) how many bits were =
truncated. The difference between how much you know and what you need to =
know to do whatever you're going to do, though, is negligible.

> Hmm, I think there are two different points we have to look at:
>=20
> 1.) Can the number of truncated bits be derived from the data?
>=20
> I think it will probably work in most of the cases because people will =
probably stick with /8, /16, /24 etc.=20
> But it is not possible to do this in every case (e.g. the example I =
gave you) and you can almost never be sure if your guess on the =
truncation length is correct.=20
>=20
> If you try to guestimate the number of truncated bits, you should have =
a look at the complete trace in order to minimize the possibility of =
guessing a wrong value.
> I would be fine with either of
>=20
> 1.) The draft says the number of truncated bits can be guestimated =
from the data. But there is no guarantee that the estimation is correct
> 2.) The draft introduces a way of explicitly encoding the number of =
truncated bits.

How about both:

NEW (in working copy of -03):

5.2.  Truncation Length

   Truncation and precision degradation are described by the truncation
   length, or the amount of data still remaining in the anonymised field
   after anonymisation.

   Truncation length can generally be inferred from a given data set,
   and need not be specially exported or protected.  For bit-level
   truncation, the truncated bits are generally inferable by the least =
significant
   bit set for an instance of an Information Element described by a
   given Template (or the most significant bit set, in the case of
   reverse truncation).  For precision degradation, the truncation is
   inferable from the maximum precision given. Note that while this
   inference method is generally applicable, it is data-dependent: there =
is
   no guarantee that it will recover the exact truncation length used to
   prepare the data.

   In the special case of IP address export with variable (per-record)
   truncation, the truncation MAY be expressed by exporting the prefix
   length alongside the address.

(This discussion has raised another issue: what if each truncation is =
done with a separate mask? In this case, we already have an appropriate =
answer: we can use the existing prefix length IEs.)

Cheers,

Brian



From braun@net.in.tum.de  Thu Apr 22 01:23:11 2010
Return-Path: 
X-Original-To: ipfix@core3.amsl.com
Delivered-To: ipfix@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id BF1BF3A6A42 for ; Thu, 22 Apr 2010 01:23:11 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: 0.717
X-Spam-Level: 
X-Spam-Status: No, score=0.717 tagged_above=-999 required=5 tests=[AWL=1.107,  BAYES_20=-0.74, HELO_EQ_DE=0.35]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1Ypdr2ClYAP2 for ; Thu, 22 Apr 2010 01:23:09 -0700 (PDT)
Received: from mail-out1.informatik.tu-muenchen.de (mail-out1.informatik.tu-muenchen.de [131.159.0.8]) by core3.amsl.com (Postfix) with ESMTP id 114933A69EF for ; Thu, 22 Apr 2010 01:23:07 -0700 (PDT)
Received: from honshu.net.in.tum.de (honshu.net.in.tum.de [131.159.20.100]) by mail.net.in.tum.de (Postfix) with ESMTPSA id 7D92C206491C; Thu, 22 Apr 2010 10:22:44 +0200 (CEST)
Mime-Version: 1.0 (Apple Message framework v1078)
Content-Type: text/plain; charset=iso-8859-1
From: Lothar Braun 
In-Reply-To: <596D46DA-914F-4308-9778-2ED5F3E2FB7B@tik.ee.ethz.ch>
Date: Thu, 22 Apr 2010 10:22:43 +0200
Content-Transfer-Encoding: quoted-printable
Message-Id: <7FB974DC-1F80-4EE7-B8AE-BE41249BDFBC@net.in.tum.de>
References:  <11413CE7-4BE3-43D2-A2D7-5B63C9BFC9C0@tik.ee.ethz.ch> <73DE58B6-50D4-4691-8D37-C282FF0CE349@net.in.tum.de> <744A3FB6-7EC8-4A89-B147-BB6386E4D2E0@tik.ee.ethz.ch>  <596D46DA-914F-4308-9778-2ED5F3E2FB7B@tik.ee.ethz.ch>
To: Brian Trammell 
X-Mailer: Apple Mail (2.1078)
X-Virus-Scanned: ClamAV using ClamSMTP
Cc: IETF IPFIX Working Group 
Subject: Re: [IPFIX] WG last call on draft-ietf-ipfix-anon-02.txt
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Thu, 22 Apr 2010 08:23:11 -0000

Hi Brian,

On Apr 22, 2010, at 8:28 AM, Brian Trammell wrote:
> On Apr 21, 2010, at 9:32 AM, Lothar Braun wrote:
>> Hmm, I think there are two different points we have to look at:
>>=20
>> 1.) Can the number of truncated bits be derived from the data?
>>=20
>> I think it will probably work in most of the cases because people =
will probably stick with /8, /16, /24 etc.=20
>> But it is not possible to do this in every case (e.g. the example I =
gave you) and you can almost never be sure if your guess on the =
truncation length is correct.=20
>>=20
>> If you try to guestimate the number of truncated bits, you should =
have a look at the complete trace in order to minimize the possibility =
of guessing a wrong value.
>> I would be fine with either of
>>=20
>> 1.) The draft says the number of truncated bits can be guestimated =
from the data. But there is no guarantee that the estimation is correct
>> 2.) The draft introduces a way of explicitly encoding the number of =
truncated bits.
>=20
> How about both:
>=20
> NEW (in working copy of -03):
>=20
> 5.2.  Truncation Length
>=20
>   Truncation and precision degradation are described by the truncation
>   length, or the amount of data still remaining in the anonymised =
field
>   after anonymisation.
>=20
>   Truncation length can generally be inferred from a given data set,
>   and need not be specially exported or protected.  For bit-level
>   truncation, the truncated bits are generally inferable by the least =
significant
>   bit set for an instance of an Information Element described by a
>   given Template (or the most significant bit set, in the case of
>   reverse truncation).  For precision degradation, the truncation is
>   inferable from the maximum precision given. Note that while this
>   inference method is generally applicable, it is data-dependent: =
there is
>   no guarantee that it will recover the exact truncation length used =
to
>   prepare the data.
>=20
>   In the special case of IP address export with variable (per-record)
>   truncation, the truncation MAY be expressed by exporting the prefix
>   length alongside the address.


Great, I'm fine with that.

Best regards,
  Lothar=20

--
Lothar Braun
Chair for Network Architectures and Services (I8)
Department of Informatics
Technische Universit=E4t M=FCnchen
Boltzmannstr. 3, 85748 Garching bei M=FCnchen, Germany
Phone:  +49 89 289-18010       Fax: +49 89 289-18033
E-mail: braun@net.in.tum.de=20







From root@core3.amsl.com  Thu Apr 22 08:45:01 2010
Return-Path: 
X-Original-To: ipfix@ietf.org
Delivered-To: ipfix@core3.amsl.com
Received: by core3.amsl.com (Postfix, from userid 0) id 9E77E3A6A8B; Thu, 22 Apr 2010 08:45:01 -0700 (PDT)
From: Internet-Drafts@ietf.org
To: i-d-announce@ietf.org
Content-Type: Multipart/Mixed; Boundary="NextPart"
Mime-Version: 1.0
Message-Id: <20100422154501.9E77E3A6A8B@core3.amsl.com>
Date: Thu, 22 Apr 2010 08:45:01 -0700 (PDT)
Cc: ipfix@ietf.org
Subject: [IPFIX] I-D Action:draft-ietf-ipfix-anon-03.txt
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Thu, 22 Apr 2010 15:45:01 -0000

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the IP Flow Information Export Working Group of the IETF.


	Title           : IP Flow Anonymisation Support
	Author(s)       : E. Boschi, B. Trammell
	Filename        : draft-ietf-ipfix-anon-03.txt
	Pages           : 41
	Date            : 2010-04-22

This document describes anonymisation techniques for IP flow data and
the export of anonymised data using the IPFIX protocol.  It
categorizes common anonymisation schemes and defines the parameters
needed to describe them.  It provides guidelines for the
implementation of anonymised data export and storage over IPFIX, and
describes an information model and Options-based method for
anonymisation metadata export within the IPFIX protocol or storage in
IPFIX Files.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-ipfix-anon-03.txt

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Message/External-body;
	name="draft-ietf-ipfix-anon-03.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID: <2010-04-22083818.I-D@ietf.org>


--NextPart--

From trammell@tik.ee.ethz.ch  Thu Apr 22 08:52:27 2010
Return-Path: 
X-Original-To: ipfix@core3.amsl.com
Delivered-To: ipfix@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 5FAAA3A6BC2 for ; Thu, 22 Apr 2010 08:52:27 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.374
X-Spam-Level: 
X-Spam-Status: No, score=-5.374 tagged_above=-999 required=5 tests=[AWL=1.225,  BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VcHwIr70y9Z0 for ; Thu, 22 Apr 2010 08:52:26 -0700 (PDT)
Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by core3.amsl.com (Postfix) with ESMTP id CCDAB3A6BA1 for ; Thu, 22 Apr 2010 08:52:07 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 4765FD93DE for ; Thu, 22 Apr 2010 17:51:56 +0200 (MEST)
X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch
Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id w8ChqOeJR96v for ; Thu, 22 Apr 2010 17:51:56 +0200 (MEST)
Received: from pb-10072.ethz.ch (pb-10072.ethz.ch [82.130.103.195]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id 0EEA2D9397 for ; Thu, 22 Apr 2010 17:51:56 +0200 (MEST)
From: Brian Trammell 
Content-Type: multipart/mixed; boundary=Apple-Mail-5--560944019
Date: Thu, 22 Apr 2010 17:51:55 +0200
References: <20100422154501.9E77E3A6A8B@core3.amsl.com>
To: IETF IPFIX Working Group 
Message-Id: <7D6A6449-FA7C-42F0-9974-2F3E29F0C632@tik.ee.ethz.ch>
Mime-Version: 1.0 (Apple Message framework v1078)
X-Mailer: Apple Mail (2.1078)
Subject: [IPFIX] Fwd:  I-D Action:draft-ietf-ipfix-anon-03.txt
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Thu, 22 Apr 2010 15:52:27 -0000

--Apple-Mail-5--560944019
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

Greetings, all,

We've just posted -03 of the anonymisation draft; we believe this to =
satisfactorily address all comments received during WGLC, and as such, =
it's ready for the IESG.

Cheers,

Brian

Begin forwarded message:

> From: Internet-Drafts@ietf.org
> Date: April 22, 2010 5:45:01 PM GMT+02:00
> To: i-d-announce@ietf.org
> Cc: ipfix@ietf.org
> Subject: [IPFIX] I-D Action:draft-ietf-ipfix-anon-03.txt
>=20
> A New Internet-Draft is available from the on-line Internet-Drafts =
directories.
> This draft is a work item of the IP Flow Information Export Working =
Group of the IETF.
>=20
>=20
> 	Title           : IP Flow Anonymisation Support
> 	Author(s)       : E. Boschi, B. Trammell
> 	Filename        : draft-ietf-ipfix-anon-03.txt
> 	Pages           : 41
> 	Date            : 2010-04-22
>=20
> This document describes anonymisation techniques for IP flow data and
> the export of anonymised data using the IPFIX protocol.  It
> categorizes common anonymisation schemes and defines the parameters
> needed to describe them.  It provides guidelines for the
> implementation of anonymised data export and storage over IPFIX, and
> describes an information model and Options-based method for
> anonymisation metadata export within the IPFIX protocol or storage in
> IPFIX Files.
>=20
> A URL for this Internet-Draft is:
> http://www.ietf.org/internet-drafts/draft-ietf-ipfix-anon-03.txt
>=20
> Internet-Drafts are also available by anonymous FTP at:
> ftp://ftp.ietf.org/internet-drafts/
>=20
> Below is the data which will enable a MIME compliant mail reader
> implementation to automatically retrieve the ASCII version of the
> Internet-Draft.

--Apple-Mail-5--560944019
Content-Disposition: attachment;
	filename="Mail Attachment"
Content-Type: message/external-body;
	name="Mail Attachment"
Content-Transfer-Encoding: 7bit

Content-Type: text/plain
Content-ID: <2010-04-22083818.I-D@ietf.org>




--Apple-Mail-5--560944019
Content-Transfer-Encoding: 7bit
Content-Type: text/plain;
	charset=us-ascii

> _______________________________________________
> IPFIX mailing list
> IPFIX@ietf.org
> https://www.ietf.org/mailman/listinfo/ipfix


--Apple-Mail-5--560944019--

From sujay.ietf@gmail.com  Fri Apr 23 12:56:19 2010
Return-Path: 
X-Original-To: ipfix@core3.amsl.com
Delivered-To: ipfix@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id BC5C228C12F for ; Fri, 23 Apr 2010 12:56:19 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.599
X-Spam-Level: 
X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QlvYC90oRxkJ for ; Fri, 23 Apr 2010 12:56:18 -0700 (PDT)
Received: from mail-gw0-f44.google.com (mail-gw0-f44.google.com [74.125.83.44]) by core3.amsl.com (Postfix) with ESMTP id 9032F3A6B2A for ; Fri, 23 Apr 2010 12:55:15 -0700 (PDT)
Received: by gwaa12 with SMTP id a12so1868173gwa.31 for ; Fri, 23 Apr 2010 12:55:02 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=atgkUa7ophJB2FU1nzqkg/Otb6F6XkShjc+ggezcMqU=; b=gp+uxI9S5we78/dpnVSrZF71gi2SpVYVrIFM7N7jHHuGaGf+pJoVSWUOn1LpopGoBz OD+VnzgVw1bLw+Hl9p/vYcQqlxgvXjRwgAm2E0jyxwo4uZwiXD/fKVGgfqDz6mZtO0J+ LqHJYF8xmrTMsbaIEL3fLIB9kSZ2Hd2v1DgpE=
DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=VOvkhLAIMieDr7TIOl3gSwO8J2ZPKFj1AtMqS9+X4AE9Il0U1yRDZD2ebtan6pkl9X jr0VnbMlezsDYegtKp4QcDWba4SPRsM4nClKLwgnstcfdL/FQg9FdGibgZqeIVrNAEt3 K4CuXPmFoeoIMR8NcF/mm/0q5TGTPMLVBVQZg=
MIME-Version: 1.0
Received: by 10.90.219.1 with SMTP id r1mr244342agg.118.1272052502003; Fri, 23  Apr 2010 12:55:02 -0700 (PDT)
Received: by 10.90.78.2 with HTTP; Fri, 23 Apr 2010 12:55:01 -0700 (PDT)
In-Reply-To: <7D6A6449-FA7C-42F0-9974-2F3E29F0C632@tik.ee.ethz.ch>
References: <20100422154501.9E77E3A6A8B@core3.amsl.com> <7D6A6449-FA7C-42F0-9974-2F3E29F0C632@tik.ee.ethz.ch>
Date: Fri, 23 Apr 2010 15:55:01 -0400
Message-ID: 
From: sujay gupta 
To: Brian Trammell 
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Cc: IETF IPFIX Working Group 
Subject: Re: [IPFIX] Fwd: I-D Action:draft-ietf-ipfix-anon-03.txt
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Fri, 23 Apr 2010 19:56:19 -0000

HI Brian,

A very dumb question,

Why don't we use any sort of encryption mechanism's to protect data?
Is it for compatibility reasons to the base protocol?

Thanks,
-Sujay

On Thu, Apr 22, 2010 at 11:51 AM, Brian Trammell
 wrote:
> Greetings, all,
>
> We've just posted -03 of the anonymisation draft; we believe this to sati=
sfactorily address all comments received during WGLC, and as such, it's rea=
dy for the IESG.
>
> Cheers,
>
> Brian
>
> Begin forwarded message:
>
>> From: Internet-Drafts@ietf.org
>> Date: April 22, 2010 5:45:01 PM GMT+02:00
>> To: i-d-announce@ietf.org
>> Cc: ipfix@ietf.org
>> Subject: [IPFIX] I-D Action:draft-ietf-ipfix-anon-03.txt
>>
>> A New Internet-Draft is available from the on-line Internet-Drafts direc=
tories.
>> This draft is a work item of the IP Flow Information Export Working Grou=
p of the IETF.
>>
>>
>> =A0 =A0 =A0 Title =A0 =A0 =A0 =A0 =A0 : IP Flow Anonymisation Support
>> =A0 =A0 =A0 Author(s) =A0 =A0 =A0 : E. Boschi, B. Trammell
>> =A0 =A0 =A0 Filename =A0 =A0 =A0 =A0: draft-ietf-ipfix-anon-03.txt
>> =A0 =A0 =A0 Pages =A0 =A0 =A0 =A0 =A0 : 41
>> =A0 =A0 =A0 Date =A0 =A0 =A0 =A0 =A0 =A0: 2010-04-22
>>
>> This document describes anonymisation techniques for IP flow data and
>> the export of anonymised data using the IPFIX protocol. =A0It
>> categorizes common anonymisation schemes and defines the parameters
>> needed to describe them. =A0It provides guidelines for the
>> implementation of anonymised data export and storage over IPFIX, and
>> describes an information model and Options-based method for
>> anonymisation metadata export within the IPFIX protocol or storage in
>> IPFIX Files.
>>
>> A URL for this Internet-Draft is:
>> http://www.ietf.org/internet-drafts/draft-ietf-ipfix-anon-03.txt
>>
>> Internet-Drafts are also available by anonymous FTP at:
>> ftp://ftp.ietf.org/internet-drafts/
>>
>> Below is the data which will enable a MIME compliant mail reader
>> implementation to automatically retrieve the ASCII version of the
>> Internet-Draft.
>
>> _______________________________________________
>> IPFIX mailing list
>> IPFIX@ietf.org
>> https://www.ietf.org/mailman/listinfo/ipfix
>
>
> _______________________________________________
> IPFIX mailing list
> IPFIX@ietf.org
> https://www.ietf.org/mailman/listinfo/ipfix
>
>

From vdharani@gmail.com  Fri Apr 23 17:28:12 2010
Return-Path: 
X-Original-To: ipfix@core3.amsl.com
Delivered-To: ipfix@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 4F8D73A67B3 for ; Fri, 23 Apr 2010 17:28:12 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.599
X-Spam-Level: 
X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vtWtjWt8rA8U for ; Fri, 23 Apr 2010 17:28:10 -0700 (PDT)
Received: from mail-qy0-f181.google.com (mail-qy0-f181.google.com [209.85.221.181]) by core3.amsl.com (Postfix) with ESMTP id B15FA3A6783 for ; Fri, 23 Apr 2010 17:28:10 -0700 (PDT)
Received: by qyk11 with SMTP id 11so12217982qyk.13 for ; Fri, 23 Apr 2010 17:27:57 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=U/sr5cTDkLnSg+aYXXm7n3tA9F1tNWummTeuE42iy9w=; b=IcuPj/km6v2XkXV9AdTceOiRcttu54dhLGPJLxSCaE09YL5ohFt/mAFuYWaiUr/0bv vfsHGA5fRvsmBRUAAms7NUYZZ2UIFiKG1C2ANJtG/MfFLtZS2bIY5AK57PTOEesibaVz Rdme686bnUMmXQeHwYzmvSblAlCi6hhzDnCI8=
DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=o4sm0Mjap43JIPcIK057nXP8U0ZwM+uhEQMaLbQUFVWAI28sweh6VQWFHU1bwDTiMP mADf+zeKxJ06yTGAralLFK1iKtep26jlXt/Za+BneE+SaDWKNQi1rnOCVDwjK1ywQW9o VbCGNPvyg9pUe+cuy62TLoE8q1A97mF9QmeOc=
MIME-Version: 1.0
Received: by 10.229.217.1 with SMTP id hk1mr962250qcb.81.1272068877307; Fri,  23 Apr 2010 17:27:57 -0700 (PDT)
Received: by 10.229.227.135 with HTTP; Fri, 23 Apr 2010 17:27:57 -0700 (PDT)
In-Reply-To: 
References: <20100422154501.9E77E3A6A8B@core3.amsl.com> <7D6A6449-FA7C-42F0-9974-2F3E29F0C632@tik.ee.ethz.ch> 
Date: Fri, 23 Apr 2010 17:27:57 -0700
Message-ID: 
From: Tharaneedharan Vilwanathan 
To: sujay gupta 
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Cc: IETF IPFIX Working Group 
Subject: Re: [IPFIX] Fwd: I-D Action:draft-ietf-ipfix-anon-03.txt
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Sat, 24 Apr 2010 00:28:12 -0000

Hi Sujay,

Are you asking this question with respect to anonymisation? If so,
anonymisation is not about transferring the data securely. It is about
hiding specifics but still provide data sufficient to arrive at the
result. For example, we could hide the ip addresses but provide L4
port details of the flows to find out most active dest ports.

Hope this helps. Someone please correct me if I am wrong.

Thanks
dharani

On Fri, Apr 23, 2010 at 12:55 PM, sujay gupta  wrote:
> HI Brian,
>
> A very dumb question,
>
> Why don't we use any sort of encryption mechanism's to protect data?
> Is it for compatibility reasons to the base protocol?
>
> Thanks,
> -Sujay
>
> On Thu, Apr 22, 2010 at 11:51 AM, Brian Trammell
>  wrote:
>> Greetings, all,
>>
>> We've just posted -03 of the anonymisation draft; we believe this to sat=
isfactorily address all comments received during WGLC, and as such, it's re=
ady for the IESG.
>>
>> Cheers,
>>
>> Brian
>>
>> Begin forwarded message:
>>
>>> From: Internet-Drafts@ietf.org
>>> Date: April 22, 2010 5:45:01 PM GMT+02:00
>>> To: i-d-announce@ietf.org
>>> Cc: ipfix@ietf.org
>>> Subject: [IPFIX] I-D Action:draft-ietf-ipfix-anon-03.txt
>>>
>>> A New Internet-Draft is available from the on-line Internet-Drafts dire=
ctories.
>>> This draft is a work item of the IP Flow Information Export Working Gro=
up of the IETF.
>>>
>>>
>>> =A0 =A0 =A0 Title =A0 =A0 =A0 =A0 =A0 : IP Flow Anonymisation Support
>>> =A0 =A0 =A0 Author(s) =A0 =A0 =A0 : E. Boschi, B. Trammell
>>> =A0 =A0 =A0 Filename =A0 =A0 =A0 =A0: draft-ietf-ipfix-anon-03.txt
>>> =A0 =A0 =A0 Pages =A0 =A0 =A0 =A0 =A0 : 41
>>> =A0 =A0 =A0 Date =A0 =A0 =A0 =A0 =A0 =A0: 2010-04-22
>>>
>>> This document describes anonymisation techniques for IP flow data and
>>> the export of anonymised data using the IPFIX protocol. =A0It
>>> categorizes common anonymisation schemes and defines the parameters
>>> needed to describe them. =A0It provides guidelines for the
>>> implementation of anonymised data export and storage over IPFIX, and
>>> describes an information model and Options-based method for
>>> anonymisation metadata export within the IPFIX protocol or storage in
>>> IPFIX Files.
>>>
>>> A URL for this Internet-Draft is:
>>> http://www.ietf.org/internet-drafts/draft-ietf-ipfix-anon-03.txt
>>>
>>> Internet-Drafts are also available by anonymous FTP at:
>>> ftp://ftp.ietf.org/internet-drafts/
>>>
>>> Below is the data which will enable a MIME compliant mail reader
>>> implementation to automatically retrieve the ASCII version of the
>>> Internet-Draft.
>>
>>> _______________________________________________
>>> IPFIX mailing list
>>> IPFIX@ietf.org
>>> https://www.ietf.org/mailman/listinfo/ipfix
>>
>>
>> _______________________________________________
>> IPFIX mailing list
>> IPFIX@ietf.org
>> https://www.ietf.org/mailman/listinfo/ipfix
>>
>>
> _______________________________________________
> IPFIX mailing list
> IPFIX@ietf.org
> https://www.ietf.org/mailman/listinfo/ipfix
>

From trammell@tik.ee.ethz.ch  Mon Apr 26 00:58:42 2010
Return-Path: 
X-Original-To: ipfix@core3.amsl.com
Delivered-To: ipfix@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 8FA193A68F9 for ; Mon, 26 Apr 2010 00:58:42 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.782
X-Spam-Level: 
X-Spam-Status: No, score=-5.782 tagged_above=-999 required=5 tests=[AWL=0.817,  BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kYj7Lhw9F276 for ; Mon, 26 Apr 2010 00:58:41 -0700 (PDT)
Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by core3.amsl.com (Postfix) with ESMTP id 5CD0B3A68BF for ; Mon, 26 Apr 2010 00:58:40 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 33F40D93A9; Mon, 26 Apr 2010 09:58:25 +0200 (MEST)
X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch
Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id yHAGTSTLJS0R; Mon, 26 Apr 2010 09:58:24 +0200 (MEST)
Received: from pb-10072.ethz.ch (pb-10072.ethz.ch [82.130.103.195]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id E21C3D960E; Mon, 26 Apr 2010 09:57:14 +0200 (MEST)
Mime-Version: 1.0 (Apple Message framework v1078)
Content-Type: text/plain; charset=us-ascii
From: Brian Trammell 
In-Reply-To: 
Date: Mon, 26 Apr 2010 09:57:13 +0200
Content-Transfer-Encoding: quoted-printable
Message-Id: 
References: <20100422154501.9E77E3A6A8B@core3.amsl.com> <7D6A6449-FA7C-42F0-9974-2F3E29F0C632@tik.ee.ethz.ch>  
To: Tharaneedharan Vilwanathan 
X-Mailer: Apple Mail (2.1078)
Cc: IETF IPFIX Working Group 
Subject: Re: [IPFIX] Fwd: I-D Action:draft-ietf-ipfix-anon-03.txt
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Mon, 26 Apr 2010 07:58:42 -0000

hi, Dharani, Sujay,

Yep, you're right. Encryption is only useful for data protection if the =
recipient of the data should have full access to it; we talk about the =
difference between anonymisation and encryption in the security =
considerations section. Anonymisation destroys some part of the =
information that is less useful for a specific analysis, preserving =
enough information that the analysis can proceed while protecting the =
identities of the participants in any given exchange.=20

Cheers,

Brian

On Apr 24, 2010, at 2:27 AM, Tharaneedharan Vilwanathan wrote:

> Hi Sujay,
>=20
> Are you asking this question with respect to anonymisation? If so,
> anonymisation is not about transferring the data securely. It is about
> hiding specifics but still provide data sufficient to arrive at the
> result. For example, we could hide the ip addresses but provide L4
> port details of the flows to find out most active dest ports.
>=20
> Hope this helps. Someone please correct me if I am wrong.
>=20
> Thanks
> dharani
>=20
> On Fri, Apr 23, 2010 at 12:55 PM, sujay gupta  =
wrote:
>> HI Brian,
>>=20
>> A very dumb question,
>>=20
>> Why don't we use any sort of encryption mechanism's to protect data?
>> Is it for compatibility reasons to the base protocol?
>>=20
>> Thanks,
>> -Sujay
>>=20
>> On Thu, Apr 22, 2010 at 11:51 AM, Brian Trammell
>>  wrote:
>>> Greetings, all,
>>>=20
>>> We've just posted -03 of the anonymisation draft; we believe this to =
satisfactorily address all comments received during WGLC, and as such, =
it's ready for the IESG.
>>>=20
>>> Cheers,
>>>=20
>>> Brian
>>>=20
>>> Begin forwarded message:
>>>=20
>>>> From: Internet-Drafts@ietf.org
>>>> Date: April 22, 2010 5:45:01 PM GMT+02:00
>>>> To: i-d-announce@ietf.org
>>>> Cc: ipfix@ietf.org
>>>> Subject: [IPFIX] I-D Action:draft-ietf-ipfix-anon-03.txt
>>>>=20
>>>> A New Internet-Draft is available from the on-line Internet-Drafts =
directories.
>>>> This draft is a work item of the IP Flow Information Export Working =
Group of the IETF.
>>>>=20
>>>>=20
>>>>       Title           : IP Flow Anonymisation Support
>>>>       Author(s)       : E. Boschi, B. Trammell
>>>>       Filename        : draft-ietf-ipfix-anon-03.txt
>>>>       Pages           : 41
>>>>       Date            : 2010-04-22
>>>>=20
>>>> This document describes anonymisation techniques for IP flow data =
and
>>>> the export of anonymised data using the IPFIX protocol.  It
>>>> categorizes common anonymisation schemes and defines the parameters
>>>> needed to describe them.  It provides guidelines for the
>>>> implementation of anonymised data export and storage over IPFIX, =
and
>>>> describes an information model and Options-based method for
>>>> anonymisation metadata export within the IPFIX protocol or storage =
in
>>>> IPFIX Files.
>>>>=20
>>>> A URL for this Internet-Draft is:
>>>> http://www.ietf.org/internet-drafts/draft-ietf-ipfix-anon-03.txt
>>>>=20
>>>> Internet-Drafts are also available by anonymous FTP at:
>>>> ftp://ftp.ietf.org/internet-drafts/
>>>>=20
>>>> Below is the data which will enable a MIME compliant mail reader
>>>> implementation to automatically retrieve the ASCII version of the
>>>> Internet-Draft.
>>>=20
>>>> _______________________________________________
>>>> IPFIX mailing list
>>>> IPFIX@ietf.org
>>>> https://www.ietf.org/mailman/listinfo/ipfix
>>>=20
>>>=20
>>> _______________________________________________
>>> IPFIX mailing list
>>> IPFIX@ietf.org
>>> https://www.ietf.org/mailman/listinfo/ipfix
>>>=20
>>>=20
>> _______________________________________________
>> IPFIX mailing list
>> IPFIX@ietf.org
>> https://www.ietf.org/mailman/listinfo/ipfix
>>=20